Runtime Performance Benchmarking for NoSQL Databases

  • C.-F. Andor Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania
  • B. Pârv Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania
DOI: https://doi.org/10.24193/subbi.2019.1.07

Abstract

An experimental study regarding the performance of NoSQL database management systems is presented in this paper. In this study, two NoSQL database management systems (MongoDB and Cassandra) were compared, and the following factors were considered: degree of parallelism and workload. Two distinct workloads (mostly read and update heavy) were used, and various numbers of client threads. The measured results refer to total runtime and they confirm that MongoDB outperforms Cassandra in almost all tests considered.

References

[1] Apache Cassandra. http://cassandra.apache.org/. Accessed: 2017-09-25.
[2] BaseX. http://basex.org/. Accessed: 2018-11-27.
[3] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A Distributed Storage System for Structured Data. OSDI ’06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, 7, 2006.
[4] B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM Symposium on Cloud Computing, pages 143–154, 2010.
[5] Couchbase. https://www.couchbase.com/. Accessed: 2019-01-22.
[6] CouchDB. http://couchdb.apache.org/. Accessed: 2017-09-25.
[7] G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s Highly Available Key-value Store. Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles, oct 2007.
[8] Fixstars. GridDB and Cassandra Performance and Scalability. A YCSB Performance Comparison on Microsoft Azure. Technical report, Fixstars Solutions, 2016.
[9] A. Gandini, M. Gribaudo, W. J. Knottenbelt, R. Osman, and P. Piazzolla. Performance Evaluation of NoSQL Databases. EPEW 2014: Computer Performance Engineering, Lecture Notes in Computer Science, 8721:16–29, 2014.
[10] A. Ghazal, T. Rabl, M. Hu, F. Raab, M. Poess, A. Crolotte, and H.-A. Jacobsen. Big-Bench: Towards an Industry Standard Benchmark for Big Data Analytics. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pages 1197–1208, 2013.
[11] HBase. https://hbase.apache.org/. Accessed: 2017-09-25.
[12] Hypertable. http://www.hypertable.org/. Accessed: 2018-11-27.
[13] JSON. https://www.json.org/. Accessed: 2018-03-16.
[14] J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham, and C. Matser. Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, pages 5–10, 2015.
[15] A. Lakshman and P. Malik. Cassandra: A Decentralized Structured Storage System. ACM SIGOPS Operating Systems Review, 44:35–40, 2010.
[16] MongoDB. https://www.mongodb.com/. Accessed: 2017-09-25.
[17] R. O. Nambiar and M. Poess. The Making of TPC-DS. VLDB ’06 Proceedings of the 32nd International Conference on Very Large Data Bases, pages 1049–1058, 2006.
[18] R Statistics Package. https://www.r-project.org/. Accessed: 2017-09-25.
[19] P. J. Sadalage and M. Fowler. NoSQL distilled : a brief guide to the emerging world of polyglot persistence. Addison-Wesley Professional, 2012.
[20] Stress Test for Couchbase Client and Cluster. http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.8/md_doc_cbc-pillowfight.html. Accessed: 2019-01-03.
[21] The cassandra-stress tool. https://docs.datastax.com/en/cassandra/3.0/cassandra/tools/toolsCStress.html. Accessed: 2019-01-03.
[22] The YCSB Core Workloads. https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads. Accessed: 2017-09-25.
[23] XML. https://www.w3.org/TR/2008/REC-xml-20081126/. Accessed: 2018-03-16.
[24] YAML. http://yaml.org/. Accessed: 2018-03-16.
[25] YCSB Github Wiki. https://github.com/brianfrankcooper/YCSB/wiki. Accessed: 2017-09-25.
Published
2019-06-17
How to Cite
ANDOR, C.-F.; PÂRV, B.. Runtime Performance Benchmarking for NoSQL Databases. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 64, n. 1, p. 77-90, june 2019. ISSN 2065-9601. Available at: <https://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/38>. Date accessed: 23 apr. 2024. doi: https://doi.org/10.24193/subbi.2019.1.07.
Citation Formats
Issue
Vol 64 No 1 (2019)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

When the article is accepted for publication, I, as the author and representative of the coauthors, hereby agree to transfer to Studia Universitatis Babes-Bolyai, Series Informatica, all rights, including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author specifically retain: the right to make further copies of all or part of the published article for my use in classroom teaching; the right to reuse all or part of this material in a review or in a textbook of which I am the author; the right to make copies of the published work for internal distribution within the institution that employs me.