Anyone who has worked in the data industry for several decades can speak to its continued evolution. Today, the focus has shifted to artificial intelligence, emphasizing AI data cleansing and governance, as clean data is essential for effective AI implementation.

That’s an emphasis that many have been pushing forward in this new era. Data must be cleansed for analytics to be more reliable, according to Nusrath Mohammed (pictured), data practice leader at Tata Consultancy Services.

Tata Consultancy Services’ Nusrath Mohammed talks to theCUBE about AI data cleansing.

“Now, because of AI, everybody wants to jump on that bandwagon. But then we realize data is not cleansed and then we have to go backwards and start cleaning that first,” Mohammed said. “Now, we can jump on that bandwagon.”

Mohammed spoke with theCUBE Research’s Dave Vellante at the CDOIQ Symposium, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the importance of AI data cleansing and the evolving challenges in data governance.

From AI data cleansing to optimizing

There are a lot of tools available in the generative AI world right now. But certain tools, such as Microsoft Corp.’s Purview, still pose plenty of questions, according to Mohammed.

“It’s a beautiful tool. But then I’m asking the question, that tool can be utilized only at the top layer, once we have the data lake has been built,” she said. “It’s at the top layer. But what about at the layer of the data inception?”

When one investigates more about how generative AI can be utilized, the answer is that one must implement the technology at all levels. That clicked, according to Mohammed.

“During the data inception, when your data is coming into your source systems, then apply gen AI there,” she said. “Detect and give us a report up there and say, ‘OK, your addresses are not matching their accurate addresses.’”

It could even be implemented when a user is creating data, as help arrives to inform that user that they may have entered an address incorrectly. Then, what’s taking place at a change level is also taking place at an interface level, according to Mohammed.

“You bring it at the top level. Now, you want to bring your insights,” she said. “You’re going to have much more better outputs too … this is how I see gen AI helping and supporting what we are doing.”

Advances in technology doesn’t mean that embedded generative AI is set to solve every problem that a company faces. Machine learning skills, data science and other skills are all still required after generative AI has provided feedback, according to Mohammed.

“Maybe I still need to apply the traditional tools to get to an actual solution. Those are some of the challenges or the talk which I’m not hearing and which I would like to hear,” she said. “It’s like, ‘This is not my silver bullet. This is at least aiding me.’”

AI copilots might one day be equipped to solve very difficult problems. But for now, copilots still have to learn what humans are doing, according to Mohammed.

“The copilots might help, but not right now,” she said. “It will learn from me what I’m doing on a day-to-day basis, and then it’ll give me much more intelligent advice at the end. But I don’t think it is giving me intelligent advice at this moment.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the CDOIQ Symposium:

Photo: SiliconANGLE