Runtime Performance Benchmarking for NoSQL Databases

  • C.-F. Andor Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania
  • B. Pârv Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania

Abstract

An experimental study regarding the performance of NoSQL database management systems is presented in this paper. In this study, two NoSQL database management systems (MongoDB and Cassandra) were compared, and the following factors were considered: degree of parallelism and workload. Two distinct workloads (mostly read and update heavy) were used, and various numbers of client threads. The measured results refer to total runtime and they confirm that MongoDB outperforms Cassandra in almost all tests considered.

References

[1] Apache Cassandra. http://cassandra.apache.org/. Accessed: 2017-09-25.
[2] BaseX. http://basex.org/. Accessed: 2018-11-27.
[3] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A Distributed Storage System for Structured Data. OSDI ’06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, 7, 2006.
[4] B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM Symposium on Cloud Computing, pages 143–154, 2010.
[5] Couchbase. https://www.couchbase.com/. Accessed: 2019-01-22.
[6] CouchDB. http://couchdb.apache.org/. Accessed: 2017-09-25.
[7] G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s Highly Available Key-value Store. Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles, oct 2007.
[8] Fixstars. GridDB and Cassandra Performance and Scalability. A YCSB Performance Comparison on Microsoft Azure. Technical report, Fixstars Solutions, 2016.
[9] A. Gandini, M. Gribaudo, W. J. Knottenbelt, R. Osman, and P. Piazzolla. Performance Evaluation of NoSQL Databases. EPEW 2014: Computer Performance Engineering, Lecture Notes in Computer Science, 8721:16–29, 2014.
[10] A. Ghazal, T. Rabl, M. Hu, F. Raab, M. Poess, A. Crolotte, and H.-A. Jacobsen. Big-Bench: Towards an Industry Standard Benchmark for Big Data Analytics. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pages 1197–1208, 2013.
[11] HBase. https://hbase.apache.org/. Accessed: 2017-09-25.
[12] Hypertable. http://www.hypertable.org/. Accessed: 2018-11-27.
[13] JSON. https://www.json.org/. Accessed: 2018-03-16.
[14] J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham, and C. Matser. Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, pages 5–10, 2015.
[15] A. Lakshman and P. Malik. Cassandra: A Decentralized Structured Storage System. ACM SIGOPS Operating Systems Review, 44:35–40, 2010.
[16] MongoDB. https://www.mongodb.com/. Accessed: 2017-09-25.
[17] R. O. Nambiar and M. Poess. The Making of TPC-DS. VLDB ’06 Proceedings of the 32nd International Conference on Very Large Data Bases, pages 1049–1058, 2006.
[18] R Statistics Package. https://www.r-project.org/. Accessed: 2017-09-25.
[19] P. J. Sadalage and M. Fowler. NoSQL distilled : a brief guide to the emerging world of polyglot persistence. Addison-Wesley Professional, 2012.
[20] Stress Test for Couchbase Client and Cluster. http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.8/md_doc_cbc-pillowfight.html. Accessed: 2019-01-03.
[21] The cassandra-stress tool. https://docs.datastax.com/en/cassandra/3.0/cassandra/tools/toolsCStress.html. Accessed: 2019-01-03.
[22] The YCSB Core Workloads. https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads. Accessed: 2017-09-25.
[23] XML. https://www.w3.org/TR/2008/REC-xml-20081126/. Accessed: 2018-03-16.
[24] YAML. http://yaml.org/. Accessed: 2018-03-16.
[25] YCSB Github Wiki. https://github.com/brianfrankcooper/YCSB/wiki. Accessed: 2017-09-25.
Published
2019-06-17
How to Cite
ANDOR, C.-F.; PÂRV, B.. Runtime Performance Benchmarking for NoSQL Databases. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 64, n. 1, p. 77-90, june 2019. ISSN 2065-9601. Available at: <http://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/38>. Date accessed: 29 nov. 2020. doi: https://doi.org/10.24193/subbi.2019.1.07.
Section
Articles