Performance Benchmarking for NoSQL Database Management Systems

  • C.-F. Andor Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania
DOI: https://doi.org/10.24193/subbi.2021.1.02

Abstract

NoSQL database management systems are very diverse and are known to evolve very fast. With so many NoSQL database options available nowadays, it is getting harder to make the right choice for certain use cases. Also, even for a given NoSQL database management system, performance may vary significantly between versions. Database performance benchmarking shows the actual performance for different scenarios on different hardware configurations in a straightforward and precise manner. This paper presents a NoSQL database performance study in which two of the most popular NoSQL database management systems (MongoDB and Apache Cassandra) are compared, and the analyzed metric is throughput. Results show that Apache Cassandra outperformes MongoDB in an update heavy scenario only when the number of operations is high. Also, for a read intensive scenario, Apache Cassandra outperformes MongoDB only when both number of operations and degree of parallelism are high.

References

[1] C.-F. Andor and B. Pˆarv. NoSQL Database Performance Benchmarking - A Case Study. Studia Informatica, LXIII(1):80–93, 2018.
[2] Apache Cassandra. http://cassandra.apache.org/. Accessed: 2021-02-14.
[3] Performance Analysis: Benchmarking a NoSQL Database on Bare-Metal and Virtualized Public Cloud - Aerospike NoSQL Database on Internap Bare Metal, Amazon EC2 and Rackspace Cloud. http://pages.aerospike.com/rs/229-XUE-318/images/Internap_CloudSpectatorAerospike.pdf. Accessed: 2021-03-24.
[4] B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM symposium on Cloud computing, pages 143–154, 2010.
[5] Fixstars. GridDB and Cassandra Performance and Scalability. A YCSB Performance Comparison on Microsoft Azure. Technical report, Fixstars Solutions, 2016.
[6] A. Gandini, M. Gribaudo, W. J. Knottenbelt, R. Osman, and P. Piazzolla. Performance Evaluation of NoSQL Databases. EPEW 2014: Computer Performance Engineering, Lecture Notes in Computer Science, 8721:16–29, 2014.
[7] J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham, and C. Matser. Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, pages 5–10, 2015.
[8] MongoDB. https://www.mongodb.com/. Accessed: 2021-02-14.
[9] NoSQLBench. https://github.com/nosqlbench/nosqlbench. Accessed: 2021-03-24.
[10] Stress Test for Couchbase Client and Cluster. https://docs.couchbase.com/sdk-api/couchbase-c-client/md_doc_cbc-pillowfight.html. Accessed: 2021-03-21.
[11] The cassandra-stress tool. https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsCStress.html. Accessed: 2021-03-21.
[12] YCSB MongoDB Performance Testing. https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb. Accessed: 2021-03-24.
Published
2021-07-01
How to Cite
ANDOR, C.-F.. Performance Benchmarking for NoSQL Database Management Systems. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 66, n. 1, p. 23-36, july 2021. ISSN 2065-9601. Available at: <https://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/62>. Date accessed: 08 feb. 2025. doi: https://doi.org/10.24193/subbi.2021.1.02.
Citation Formats
Issue
Vol 66 No 1 (2021)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Transfer of copyright agreement: When the article is accepted for publication, I, as the author and the representative of the coauthors, hereby agree to transfer to Studia Universitatis Babes-Bolyai, Series Informatica, all rights, including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the authors specifically retain: the authors can use the material however they want as long as it fits the NC ND terms of the license. The authors have all rights for reuse according to the below license.