ChatGPT has been hailed as everything from the future of search to the end of term papers, but it’s far from a sentient being. While OpenAI LLC’s chatbot is a wonder at explaining the infield fly rule in Shakespearean English, detractors have noted that its answers can be overly simplistic, confusing or just plain wrong.

That isn’t stopping business leaders from imagining how generative artificial intelligence systems such as ChatGPT, a type of AI that can create new data based on existing patterns and information, can improve efficiency, reduce manual tasks and improve customer experience. Indeed, the technology has recently galvanized tech giants such as Microsoft Corp. and Google LLC to leverage the technology in their products and services.

SiliconANGLE contacted executives at a few companies that are likely to be affected by the technology to find out where they think the greatest potential lies as well as areas in which more work needs to be done.

Jonathan Rosenberg called the smart bot a “step change improvement,” meaning that it will move the bar permanently higher for AI developers, who no longer need to train their own large language models, a subset of generative AI trained using deep learning to produce humanlike text responses. Rosenberg, who is the chief technology officer at cloud call center service provider Five9 Inc., said the use of GPT-3 and succeeding models will let developers shift their attention to tasks with greater business impact.

Moving up the stack

Five9’s Rosenberg sees ChatGPT as a “step change improvement” in AI development. Photo: SiliconANGLE

“It used to be that all natural language systems needed training data,” he said. “Any other AI will be able to skip that step.” Rosenberg likened the scenario to the impact that low-cost speech-to-text application program interfaces have had on the adoption of speech recognition. “A long time ago you had to train speech recognition with a bespoke language model,” he said. “Nobody does that anymore. The off-the-shelf models work great.”

Nithya Thadani finds two aspects of ChatGPT interesting. “One is that it is actually generating [an answer] and not just pulling from a link,” said Thadani, who is chief executive of Rain LLC, a developer of conversational technology based on deep learning. “The second is that it is truly conversational, like a very natural human interaction.”

But the human element can be deceptive, she said. “It is incredible that this chatbot can give you an articulate human answer to anything, but you have no idea where it’s coming from or if it’s even true,” she said.

The accuracy element is what’s most concerning about general-purpose generative AI, experts said. “It will tell you very convincingly what it ‘thinks’ is right, no matter how wrong it is,” said Henrik Roth, co-founder and chief marketing officer of neuroflash GmbH, a Hamburg, Germany-based developer of an AI text generator. That makes ChatGPT a great tool for creative writing and advertising, he said, but “if facts matter more — such as in journalism and science — one should fact-check every claim.”

Imminent threat?

Scientist and author Gary Marcus said as much in a recent post on Substack in which he called systems like ChatGPT “a real and imminent threat to the fabric of society.” Noting that the bot can be easily coaxed into giving wrong answers at large scale, generative AI is “on a path to reducing the cost of generating disinformation to zero,” he wrote.

Rain’s Thadani sees big potential in domain-specific applications of ChatGPT. Photo: Rain

The system’s directness and disarming familiarity can be misleading, said Pieter Buteneers, director of engineering for AI and machine learning at mobile messaging vendor Sinch AB. “ChatGPT will always respond to your question, even if it is unsure if it is correct,” he said in emailed comments. “It will come up with a grammatically correct answer that could fool an outsider but prove to be complete nonsense to those with domain knowledge.”

One possible solution would be for the bot to display a confidence score for its answers. That’s technically possible no small task to implement. “OpenAI has not integrated confidence scores in its ChatGPT model because it is a challenging task to accurately estimate the confidence in a model’s output, especially in a conversational context where the model is generating text based on previous context and conversational history,” said ChatGPT in response to a query. “It requires a lot of additional training data and fine-tuning to estimate confidence scores in a manner that is consistent with human expectations.”

The usefulness of a response is influenced by the quality of the training data, which consisted of roughly 300 billion words supplemented by supervised and reinforcement learning, as well as how questions are asked, said Wayne Butterfield, partner at global technology research and advisory firm Information Services Group Inc. “‘Garbage in, garbage out’ springs to mind,” he said.

Confuse a bot

Source: ChatGPT

Scores of articles have appeared in major publications detailing how the bot can be confused by ambiguous questions or asked to create an argument in favor of an absurd point of view. One Reddit user created a “jailbreak” that commands ChatGPT to create an alter ego that will respond to questions that fall outside the ethical guidelines OpenAI established. Even ChatGPT itself admits that “you can try to trick it by asking questions that are unclear, ambiguous, or have multiple possible answers.”

Butterfield said that even when answers are correct, they often aren’t useful to anyone who has more than a basic understanding of a topic. “ChatGPT is still in its infancy with regards to creating content that does not need any form of human fine-tuning, especially if you are already an expert in your field,” he said.

Experts agree that response quality will improve over time as language models are customized to different use case scenarios. “Right now [OpenAI is] trying to be everything to everyone; from coding and writing poetry and essays to paralegal work,” Thadani said. Narrowing generative AI to a specific domain can “be pretty transformational,” she said, but the open beta test isn’t intended to showcase that kind of expertise. “It’s like they’re serving as the foundation for a lot of other growth in technology,” she said.

Targeted expertise

Thadani pointed to Rain’s market as one that can benefit from more targeted expertise. The company’s conversational technology is aimed at giving auto mechanics factual answers to a limited domain of likely questions using Hearst Business Publishing Inc.’s Motor database as a source.

“There’s nothing subjective about the responses,” she said, “and so we have a lot of confidence that our answers are correct.” On the other hand, diagnosing the cause of a problem is much tougher to solve with AI because answers are necessarily subjective.

Dan Abelon, a partner at Two Sigma Investments LP, expects “many fine-tuned models” to emerge. “These will vary from models tailor-made for an individual’s writing style to ones trained on all the documents in a company so employees can surface any kernel of insight at a moment’s notice,” he said in e-mailed comments.

Five9’s Rosenberg sees applications to his business in the costly and time-consuming process of transcribing and interpreting voice conversations. Many contact centers record calls and employ a staff of people to assess the quality of interactions between customers and agents.

“That’s an area where this technology will help,” he said. “We can drop it in to allow people to do a lot more with less time.”

All talk, no action

Humans will still be needed to take action, at least for now, Rosenberg noted. Existing large language models are intended to answer questions but not transfer money or cancel an account. “It can speak but it can’t act,” he said.

That likely will change, though, potentially with application programming interfaces or other software modifications. “ChatGPT can be modified to perform specific actions by integrating it with appropriate APIs or other software systems,” the bot said in response to a query.

ISGs Butterfield: Technologies like ChatGPT “will augment search but are very unlikely to eradicate it.” Photo: ISG

One topic upon which nearly everyone seems to agree is that ChatGPT won’t replace search engines. Google and its competitors have trained users to expect multiple answers to a question and to do their own legwork vetting the source. By most estimates, Google’s “I’m Feeling Lucky” button, which serves up a single answer to a query, is used on fewer than 1% of searches.

“Technologies like this will augment search but are very unlikely to eradicate it,” said ISG’s Butterfield. “Most searches at present are not purely interested in a single response.”

Neuroflash’s Roth said generative AI can effectively address questions that require answers, such as “When was the battle of Gettysburg fought?” However, “if you’re looking for an answer from a reputable source, like health advice, other search options are superior,” he said.

Beyond the hype

No one expects the current fascination with ChatGPT’s uncanny conversational capabilities to last. OpenAI CEO Sam Altman said as much in a recent interview with Tech Monitor. “One of the strange things about these technologies is they are impressive but not robust,” he said. “Use them in a demo, you think ‘good to go’ but use them longer-term and you see the weaknesses.”

But that doesn’t mean the technology won’t be game-changing in many contexts. Thadani said it has the potential to “level the uneven playing field in education. A kid in a rural or an underserved neighborhood that can ask ChatGPT about an academic subject can get the same experience they might get in a classroom.”

The success of the public beta test has lit a fire under competitors and unleashed an “overwhelming” flood of new business plans, said Jim Whitehurst, former IBM Corp. president and now a special adviser at Silver Lake Partners LP.

Large language models will be adapted to a massive number of new uses and increasingly integrated into everyday productivity applications and search engines, Rosenberg said. “It’s what Clippy should have been,” he said, referring to Microsoft Corp.’s much-derided virtual assistant. “Anybody whose job it is to write a document will have tools to do more with less. For Word, this is a deep no-brainer.”

Photo by Unsplash