The tech landscape is shifting, with developers at the forefront of the revolution. The convergence of cloud computing, real-time data processing, vector databases and generative AI is reshaping the future of application development.

With a future looming where app development is more open-ended and less programmatic, what does this mean for the developers tasked with architecting digital landscapes?

“There are so many moving pieces, and they’re so novel,” said Tim Faulkes (pictured), chief developer advocate of Aerospike Inc. “How do they hang together? It’s almost frustrating for experienced developers. You hear all these terms, [but] how do you put them together? How do you get a generative AI application that’s going to work at scale when you don’t understand what large language models and vectors and things like that are? We’re not used to doing fuzzy things; we’re used to getting an answer out of a computer.”

Faulkes spoke with theCUBE’s John Furrier ahead of the Real-time Data Summit, during an exclusive conversation from SiliconANGLE Media’s studio in Palo Alto, California. They discussed the new era in technology, where the integration of real-time data, vector databases and generative AI redefines the future of application development. (* Disclosure below.)

Vector databases: The backbone of AI applications

The modern imperative for developers is integrating AI technologies into traditional apps, moving from deterministic programming to working with probabilistic outputs. As a result, new tooling, re-platforming and comprehensive AI strategies are required, according to Faulkes.

“It’s not the answer that everyone’s looking for,” he said. “Fraud detection and things like recommendation engines really have great benefits in the use of AI. It’s not just the generative side of things; it’s the whole AI ecosystem. And plugging that into your traditional applications and having the two liaised together in the right manner so you get the best application results, to me as a developer, that’s really what I find exciting.”

Vector databases are crucial in managing unstructured data at scale. They transform inputs — from images to audio and text — into a series of numbers (vectors) that can be processed efficiently. These vectors allow for high-speed, approximate searches across expansive datasets, making them ideal for real-time data-intensive processing, according to Faulkes.

“If you’re doing a recommendation engine, you want the business objects that represent your data in there,” he said. “It’s a combination of all these factors. You’ve got to use a large language model to do something useful with it. But without having some context for those large language models, then they’re most likely to hallucinate a lot. They’re prone to these hallucinations unless you can give it context.”

Understanding vector databases is essential for developers as they underpin many AI applications. They facilitate the move from traditional data storage and retrieval methods to more dynamic, scalable solutions. The ability to handle multimodal inputs and perform fuzzy searches significantly enhances the accuracy and efficiency of AI models, Faulkes noted.

“Vectors are these mystical concepts, but they’re really easy — it’s just a set of numbers,” he said. “It loses information and turns it into a smaller representation of the same thing. That’s exactly what a vector does. It takes whatever input it has, be it an audio file or a question you give it and it turns it into a series of numbers. And that’s all it is.”

Aerospike is a leader in real-time data management that provides expertise in real-time data processing, which is increasingly vital as the ecosystem shifts toward generative AI, according to Faulkes. The company’s technology enables seamless integration of large language models and multimodal capabilities, ensuring efficient data handling and real-time processing. This supports developers in creating sophisticated, data-driven applications.

Here’s theCUBE’s complete video interview with Tim Faulkes:

(* Disclosure: Aerospike Inc. sponsored this segment of theCUBE. Neither Aerospike nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE