Performance Benchmarking for NoSQL Database Management Systems

  • C.-F. Andor Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania

Abstract

NoSQL database management systems are very diverse and are known to evolve very fast. With so many NoSQL database options available nowadays, it is getting harder to make the right choice for certain use cases. Also, even for a given NoSQL database management system, performance may vary significantly between versions. Database performance benchmarking shows the actual performance for different scenarios on different hardware configurations in a straightforward and precise manner. This paper presents a NoSQL database performance study in which two of the most popular NoSQL database management systems (MongoDB and Apache Cassandra) are compared, and the analyzed metric is throughput. Results show that Apache Cassandra outperformes MongoDB in an update heavy scenario only when the number of operations is high. Also, for a read intensive scenario, Apache Cassandra outperformes MongoDB only when both number of operations and degree of parallelism are high.

References

[1] C.-F. Andor and B. Pˆarv. NoSQL Database Performance Benchmarking - A Case Study. Studia Informatica, LXIII(1):80–93, 2018.
[2] Apache Cassandra. http://cassandra.apache.org/. Accessed: 2021-02-14.
[3] Performance Analysis: Benchmarking a NoSQL Database on Bare-Metal and Virtualized Public Cloud - Aerospike NoSQL Database on Internap Bare Metal, Amazon EC2 and Rackspace Cloud. http://pages.aerospike.com/rs/229-XUE-318/images/Internap_CloudSpectatorAerospike.pdf. Accessed: 2021-03-24.
[4] B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM symposium on Cloud computing, pages 143–154, 2010.
[5] Fixstars. GridDB and Cassandra Performance and Scalability. A YCSB Performance Comparison on Microsoft Azure. Technical report, Fixstars Solutions, 2016.
[6] A. Gandini, M. Gribaudo, W. J. Knottenbelt, R. Osman, and P. Piazzolla. Performance Evaluation of NoSQL Databases. EPEW 2014: Computer Performance Engineering, Lecture Notes in Computer Science, 8721:16–29, 2014.
[7] J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham, and C. Matser. Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, pages 5–10, 2015.
[8] MongoDB. https://www.mongodb.com/. Accessed: 2021-02-14.
[9] NoSQLBench. https://github.com/nosqlbench/nosqlbench. Accessed: 2021-03-24.
[10] Stress Test for Couchbase Client and Cluster. https://docs.couchbase.com/sdk-api/couchbase-c-client/md_doc_cbc-pillowfight.html. Accessed: 2021-03-21.
[11] The cassandra-stress tool. https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsCStress.html. Accessed: 2021-03-21.
[12] YCSB MongoDB Performance Testing. https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb. Accessed: 2021-03-24.
Published
2021-07-01
How to Cite
ANDOR, C.-F.. Performance Benchmarking for NoSQL Database Management Systems. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 66, n. 1, p. 23-36, july 2021. ISSN 2065-9601. Available at: <https://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/62>. Date accessed: 05 dec. 2021. doi: https://doi.org/10.24193/subbi.2021.1.02.
Section
Articles