Kafka is an open source message publishing and subscribing system of LinkedIn in February 2065438+00, which is mainly used for processing active streaming data and big data.
1) In terms of architecture model,
RabbitMQ follows AMQP protocol, and the agent of RabbitMQ consists of exchange, binding and queue, in which exchange and binding constitute the routing key of the message. The client producer communicates with the server through the connection channel, and the consumer obtains the message from the queue for consumption (long connection, the message in the queue will be pushed to the consumer, and the consumer will read the data from the input stream circularly). RabbitMQ is broker-centric; There is a message confirmation mechanism.
Kafka follows the general MQ structure and takes producers, brokers and consumers as the center. The consumption information of the message is stored in the client consumer, and the consumer pulls data from the agent in batches according to the consumption point. There is no message confirmation mechanism.
2) In terms of throughput,
Kafka has high throughput, and adopts the mechanism of message batch processing and zero copy internally. The storage and acquisition of data are sequential batch operations on the local disk, with the complexity of O( 1) and high message processing efficiency.
RabbitMQ is slightly inferior to kafka in throughput, and their starting points are different. RabbitMQ supports reliable message delivery, transaction and batch operation. According to the requirements of storage reliability, memory or hard disk can be used for storage.
3) In terms of usability,
RabbitMQ supports miror's queue, the main queue is invalid, and miror's queue takes over.
Kafka's agent supports active and standby modes.
4) In the aspect of cluster load balancing,
Kafka uses zookeeper to manage broker and consumer in the cluster, and can register topics with zookeeper. Through the coordination mechanism of zookeeper, producer saves the broker information corresponding to the topic, which can be sent to the broker randomly or by polling. And producer can specify fragments based on semantics, and messages are sent to fragments of broker.
The load balancing of rabbitMQ needs the support of a separate load balancer.
So regarding these two choices, we still understand these four general differences. Kafka is chosen for the specific scene analysis of Qualcomm Quantity and our team's log. Of course, the design concept is different. RabbitMQ is used for reliable messaging and intelligent things, and does not support batch operations. Availability is similar, but implementation is different. Kafka is better than others in clustering. It registers zookeeper through the theme and calls the mechanism to realize semantic specified segmentation. However, the load of rabbitMQ needs a separate load balancer to support it.
————————————————
Copyright statement: This article is the original article of CSDN blogger "Dazhuang vip", which follows the CC 4.0 BY-SA copyright agreement. Please attach the original source link and this statement.
Original link: /QQ _ 33792843/ Articles/Details /757279 1 1