Apache Kafka: Building Intelligent Systems

Category: Technology

Apache Kafka is a distributed cluster of brokers. A cluster can have several topics. Topics can be considered as tables/directories, basically used to group all related information. Each topic contains multiple partitions. Each partition should fit into one broker, a topic can have several partitions. Producers (telematics devices) can write to their respective topics. Producers, in turn, can write to multiple partitions, thus balancing the workload. Consumers can eventually consume data by subscribing to topics. Consumers belonging to various consumer groups can subscribe to each topic. This helps various teams inside organization to utilize the data more meaningfully.