The latest player to jump aboard the Apache Spark bandwagon is bound to turn some heads in the upstream ecosystem. Lucidworks Inc. has added integration with the speedy data crunching framework in the new version of its flagship enterprise search engine that debuted this morning as part of an effort to catch up with the changing requirements of CIOs embarking on analytics projects.

The update represents the latest sign of the rekindled industry interest in Spark spurred by IBM Corp.’s recent pledge to commit a billion dollars and 4,500 engineers towards improving the capabilities of the project. Monumental in and of itself, the investment is made even more significant by the fact that the technology giant has a long history of helping to popularize open-source software.

The most notable fruit of its efforts is Linux, which catapulted from enterprise obscurity to becoming the dominant operating system in the data center thanks to a similar 10-figure commitment at the turn of the millennium. Lucidworks now hopes to take that disruption power to its advantage through Spark, which faces an equally bright future as Linux did over a decade ago.

That’s thanks mainly to its combination of speed and versatility, which Fusion 2.0 harnesses to try and provide more accurate results for queries run against unstructured data. More specifically, the software – a commercial implementation of Solr, one of the most popular open-source search engines for Hadoop, in turn the de facto storage backend for Spark – uses the project’s native machine learning component to help hone the relevance of results.

As it happens, Spark ML is also one of the main focal points of IBM’s billion-dollar development push, but that is not to say Lucidworks focused all of its efforts on that front with the new release of Fusion. Also introduced in the update is an improved interface that allows users to customize what information is retrieved by the search engine based on their work.

A marketer, for instance, could configure Fusion to only return certain performance metrics for an advertising campaign while a financial professional would be able to focus on the associated costs. That emphasis on fine-grained  also extends to the new data visualization capabilities included in the reworked interface, which allow users to incorporate their results into reports and dashboards that can be shared with colleagues.

The upgrade should put Lucidworks in a much better position against rival Elastic B.V., which has gained a tremendous amount of momentum in recent years with its namesake search engine, an implementation of the likewise Solr-based Lucene project. But it remains to be seen whether the company will be able to effectively put that new competitive potential to use.

Photo via Lucidworks