Apache Kafka was originally developed to complement stream processing engines such as Samza and Spark, but it may soon end up displacing them in many organizations if Confluent Inc. has its way. The startup, which was founded in 2014 by the original creators of the message broker, today released a real-time analytics module for the project that removes the need to use an external data crunching framework.

The operational benefits are considerable, at least on paper. Not having to set up a separate analytics component can potentially drastically reduce the amount of time it takes to implement a stream processing cluster while easing maintenance from there onwards. The latter selling point is made especially appealing by the fact that frameworks such as Spark and Samza are notoriously difficult to manage. Not only do they require a lot of complex scaffolding to operate at a large scale, but putting the pieces together also necessities specialized talent that is both difficult and expensive to find.

Confluent’s stream processing library should be particularly appealing to smaller organizations that potentially have a lot to gain from being able to analyze their data faster, but couldn’t do so before due to the difficulty of the task. The startup hopes to monetize the functionality by bundling it into its open-source distribution of Kafka, which is also receiving a new paid management module today that promises to ease the maintenance of large-scale implementations.

The Confluent Control Center enables administrators to track every last piece of data passing through their environments, as well as high-level details about any issues that may be hindering the flow of records. That includes everything from performance bottlenecks to delivery problems preventing important information from reaching its destination. Lastly, the module doubles as a configuration console where message streams can be easily added or modified when an organization’s analytic requirements change.

Image via Pixabay