Hortonworks Inc. has announced the availability of support subscriptions for its advanced stream processing platform DataFlow, following a quiet launch late last week.

DataFlow, which is based on the NSA-developed Apache NiFi project, was first introduced to an audience of oil and gas companies during a webinar last week as the solution to the processing needs of the Internet-of-Anything (IoAT). Apache NiFi is an open-source real-time data streaming and processing system originally known as Niagarafiles. Following the NSA’s decision to open-source the software, a group of ex-NSA developers quickly founded Onyara Inc. to commercialize it. Seeing its potential, Hortonworks moved to acquired Onyara so it can integrate NiFi with its own Hadoop distribution.

Hortonworks DataFlow (HDF) is an ideal fit for the IoAT thanks to NiFi’s ability to stream data from multiple sources, Hortonworks claims. The data that flows into HDF can be “multidirectional” and “point-to-point”, which allows users to both interact with the data and reach out to the data source, right down to the sensor or device. Hortonworks says HDF is complementary to the Hortonworks Data Platform (its Hadoop distro), with HDF streaming data-in-motion and HDP garnering those all-important business insights from data at rest.

With the availability of support subscriptions, Hortonworks says HDF with HDP will give customers a complete set of secure solutions to manage and find value in the increasing volume of new and legacy data types.

“HDF simplifies the process of collecting, conducting and curating the vast amounts of data currently being generated via the Internet of Anything,” said Tim Hall, vice president of product management at Hortonworks. “By flowing that data into HDP, our customers are able to rapidly bring these new data elements under management in a completely secure and purely open way.”

Interestingly, it’s worth pointing out that NiFi, and by extension, DataFlow, is not just for the IoAT. Hortonworks says the platform can be used for a variety of real-time data processing tasks, including Big Data ingest, fraud detection, predictive analytics, resource evaluation and more.

HDF will likely be well received among enterprise Hadoop users, but there could be one caveat – As Wikibon analyst George Gilbert noted in a recent report, Hortonworks’ decision to combine HDF with HDP could well signal the beginning of fragmentation in the Hadoop environment as vendors go beyond differentiating themselves on manageability alone, introducing core components specific to their own distributions.

Image credit: prochazka567 via pixabay.com