Anthropic, an artificial intelligence research startup that aims to build trustworthy AI models to rival OpenAI LP’s ChatGPT, today released an upgraded version of its Claude chatbot with greatly improved safety and coding capabilities.

Claude 2, an upgrade over Claude 1.3, is now available as a beta chat experience for users in the United States and the United Kingdom. The chatbot is also available for business access through an application programming interface at the same price as the previous model.

Claude is a generative AI chatbot that can understand conversational natural language and produce responses that include commentary, research assistance, poetry, analysis, summarizing long documents and even coding assistance. “Think of Claude as a friendly, enthusiastic colleague or personal assistant who can be instructed in natural language to help you with many tasks,” the Anthropic team said in a blog post.

Among the improvements, Claude 2’s training data includes updates in information from 2022 and early 2023, so it has a lot more recent context about recent events than before. However, users should be aware that it may still produce errors or generally be confused about topics that it might not be aware of. As a result, the team warned that Claude might still be prone to “confabulations” or “hallucinations” as chatbots can sometimes generate.

The model incorporates a much longer context window that the company expanded earlier this year from 9,000 to 100,000 tokens, which corresponds to about 75,000 words. This means that users can submit extremely large documents to the chatbot for it to ingest and analyze. After that, it can summarize parts of it or answer questions about it.

The model can also produce coherent responses of about 3,000 words, which is much longer than Claude 1.3’s limit of about 400 words. As a result, Claude can write much longer documents in response, including memos, letters and stories, all in one shot.

The new Claude 2 chatbot incorporates numerous improvements over previous models making it better at coding, math and reasoning. It was also trained to better produce correctly formatted code in structured data and languages such as JSON, XML, YAML and other markup languages — which is useful for transforming data from one format to another.

As for coding skills in general, Claude 2 scored 71.2%, up from 56%, on the CodexHumanEval Python coding test, showing that it has improved significantly. In math, it improved some with a score of 88%, up from 85.2% on the GSM8k, a large set of grade-school math problems.

Alongside working on making Claude 2 smarter, the Anthropic team has been working on making the model less harmful by improving its underlying safety, making it more difficult to prompt it to produce offensive or dangerous outputs.

“We have an internal red-teaming evaluation that scores our models on a large representative set of harmful prompts, using an automated test while we also regularly check the results manually,” the Anthropic team said. “In this evaluation, Claude 2 was 2x better at giving harmless responses compared to Claude 1.3.”

In order to help produce these less harmful results, Anthropic came up with a concept it calls “Constitutional AI,” a learning model for AI that imbues the AI system with a set of values that it should follow. The aim is to make it less toxic or likely to become harmful by having another AI help supervise its responses and also revise its own based on those values.

The company claims it’s currently working with thousands of businesses that are currently using the Claude API to power their own AI apps. Among them include Salesforce Inc.’s Slack, which uses Claude to summarize conversations and draft documents, and Zoom Video Communications Inc., which uses Claude to help its contact center agents to respond to customer queries more quickly.

Sourcegraph, a code search and intelligence tool, uses Claude 2 in its AI-enabled coding assistant Cody which allows developers to prototype code and fix errors rapidly. It also takes advantage of the model’s improvements and larger context window, meaning that it can read larger codebases at once and produce larger snippets of code. And with the addition of recent data to its knowledge base, Cody knows about new frameworks.

“When it comes to AI coding, devs need fast and reliable access to context about their unique codebase and a powerful LLM with a large context window and strong general reasoning capabilities,” said Quinn Slack, co-founder and chief executive of Sourcegraph, which makes a code search and intelligence tool. “The slowest and most frustrating parts of the dev workflow are becoming faster and more enjoyable. “

Claude 2 currently powers Anthropic’s free web-based chat experience, but it’s available only in the U.S. and U.K. regions. The company said it’s working to roll it out globally in the coming months.

Image: Anthropic