It’s little wonder the open-source ecosystem has developed a reputation for moving fast. Merely three weeks after the latest release of Apache Kafka hit general availability, Confluent Inc. is rolling out an updated version of its commercial distribution that implements all the major features introduced in the new iteration. Standing out in particular are the security mechanisms that have been added to help block unauthorized access to large-scale deployments.

As the central nervous system that connects an organization’s disparate data silos and analytic applications, Kafka is a natural target for any hacker hoping to steal sensitive business records. The implementation that powers LinkedIn Inc., where the information distribution framework was first hatched, alone processes over 1.1 trillion messages per day, and it’s only one of the several thousand out there. The transmissions flowing through those deployments can now be encrypted using SSL to reduce the risk of interception en route to their destination.

Confluent sees organizations only using the feature selectively to protect important transmissions that have to travel over potentially insecure network paths to reach the analytics backend, like diagnostics data from sensors installed in heavy machinery at a remote factory. Less critical information that is exposed to similar conditions can be encrypted beforehand to avoid the added overhead of using SSL.

Coupled with the new authentication options added in conjunction, the feature removes the need for much the bolt-on functionality organizations have been using until now to protect data as it travels through their Kafka deployments. Confluent hopes to provide the same convenience for integrating its distribution of the framework into the other components of an analytics environment with the other major addition introduced as part of the update.

Kafka Connect automates the most difficult tasks involved in linking the project to third party data processing software, such as ensuring that the integration doesn’t go offline in the event one of the supporting servers fails and monitoring for traffic anomalies, to reduce the burden on developers. Confluent hopes that making it easier to implement the framework in heterogeneous environments will broaden its appeal to more organizations and thus speed adoption, which has already increased sevenfold since the beginning of the year according to internal figures.

The startup plans to follow up the addition with a series of pre-implemented integrations for popular databases and Hadoop that will cut the amount of work involved in the process even further, freeing up organizations to focus on more important tasks. Like setting up the new security features, for instance.

Image via jeferrb