Log Replication in Raft vs Kafka

  • M. Petrescu Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, 1 Kogalniceanu St., 400084 Cluj-Napoca, Romania
  • R. Petrescu Montran Corporation, Romania

Abstract

The implementation of a fault-tolerant system requires some type of consensus algorithm for correct operation. From Paxos to View-stamped Replication and Raft multiple algorithms have been developed to handle this problem. This paper presents and compares the Raft algorithm and Apache Kafka, a distributed messaging system which, although at a higher level, implements many concepts present in Raft (strong leadership, append-only log, log compaction, etc.).
This shows that mechanisms conceived to handle one class of problems (consensus algorithms) are very useful to handle a larger category in the context of distributed systems.

References

[1] Apache Kafka. http://kafka.apache.org
[2] C. McCabe, Kafka improvements proposals” (having status accepted 2020, Retrieved from https://cwiki.apache.org/confluence/display/KAFKA/KIP-500
[3] D. Ongaro, J. Ousterhout (2013), In Search of an Understandable Consensus Algorithm, USENIX ATC 14, Proceedings of the 2014 USENIX Annual Technical Conference, June 2014, Pages 305-320, Retrieved from https://web.stanford.edu/ ouster/cgi-bin/papers/raft-atc14
[4] J. Kreps, Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines), Retrieved from https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines, Retrieved
[5] J. Ousterhout, Lecture for the Raft User Study, March 2013, Retrieved from https://raft.github.io/slides/raftuserstudy2013.pdf
[6] L. Lamport, The Part-Time Parliament, ACM Transactions on Computer Systems, 16 (2): 133-169. DOI:10.1145/279227.279229.
[7] T. D Chandra, R. Griesemer, J.Redstone, Paxos made live: an engineering perspective. In Proc. PODC’07, ACM Symposium on Principles of Distributed Computing (2007), ACM, pp. 398-407.
[8] W. Guozhang, J. Koshy, S. Subramanian, K. Paramasivam, M. Zadeh, N. Narkhede, J. Rao, J. Kreps, J. Stei, Building a Replicated Logging System with Apache Kafka, Proceedings of the VLDB Endowment, Retrieved from http://www.vldb.org/pvldb/vol8/p1654-wang.pdf, DOI:10.14778/2824032.2824063
[9] W. Han Z. Shang, K. Wolter, Performance Prediction for the Apache Kafka Messaging System, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City, DOI: 10.1109/HPCC/SmartCity/DSS.2019.00036
[10] B. Reed, F. Junqueira, A simple totally ordered broadcast protocol, Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware, September 2008, Article No 2, Pages 1-6, doi:10.1145/1529974.1529978,
[11] A. Medeiros, ZooKeeper’s atomic broadcast protocol: Theory and practice, Helsinki University of Technology, 2012, Retrieved from https://www.semanticscholar.org/paper/ZooKeeper-
[12] F. P. Junqueira, B. C. Reed and M. Serafini, Zab: High-performance broadcast for primary-backup systems, 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks (DSN), Hong Kong, 2011, pp. 245-256, DOI:10.1109/DSN.2011.5958223
[13] P. Hunt, M. Konar, F. Junqueira, B. Reed, ZooKeeper: wait-free coordination for internet-scale systems, USENIXATC’10: Proceedings of the 2010 USENIX conference on USENIX annual technical conference, June 2010, Pages 11
[14] H. Yusuf1, I Surjandari. (2020). Comparison of Performance Between Kafka and Raft as Ordering Service Nodes Implementation in Hyperledger Fabric. International Journal of Advanced Science and Technology, 29(7s), 3549-3554. Retrieved from http://sersc.org/journals/index.php/IJAST/article/view/17652
[15] H. Yusuf, I Surjandari, Comparison of Performance Between Kafka and Raft as Ordering Service Nodes Implementation in Hyperledger Fabric, International Journal of Advanced Science and Technology Vol. 29, No. 7s, (2020), pp. 3549-3554, ISSN: 2005-4238 IJAST
[16] C. Wang, X. Chu, Performance Characterization and Bottleneck Analysis of Hyper-ledger Fabric, arXiv:2008.05946v1 [cs.DC]
[17] C. McCabe, Kafka improvements proposals (having status accepted), 2020, https://cwiki.apache.org/confluence/display/KAFKA/KIP500
Published
2020-12-12
How to Cite
PETRESCU, M.; PETRESCU, R.. Log Replication in Raft vs Kafka. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 65, n. 2, p. 66-77, dec. 2020. ISSN 2065-9601. Available at: <https://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/59>. Date accessed: 23 oct. 2021. doi: https://doi.org/10.24193/subbi.2020.2.05.
Section
Articles