Anthropic has released Claude 2, a second-generation Chatbot with Artificial Intelligence

Anthropic, a company founded by former OpenAI employees, has announced the release of a new text-generating artificial intelligence model, Claude 2.

The successor to Anthropic's first commercial model, Claude 2, is available in beta in the US and UK from today, both online and via a paid API (in limited access). The API price has not changed (~$0.0465 per 1,000-word generation), and several companies have already begun piloting Claude 2, including generative AI platform Jasper and Sourcegraph.

"We think it's important to get these systems to market and understand how people are using them," Sandy Banerjee, Anthropic's head of go-to-market, told TechCrunch in a phone interview. "We're looking at how they're being used, how we can improve performance as well as throughput - all those things."

Like the old Claude (Claude 1.3), Claude 2 can search through documents, summarize, write and code, and answer questions on specific topics. However, Anthropic claims that Claude 2, which TechCrunch didn't get a chance to test before its release, is superior in several ways.

For example, Claude 2 scores slightly higher on the multiple choice section of the licensing exam (76.5% vs. 73% for Claude 1.3). He is able to pass the multiple-choice portion of the U.S. medical license exam. He is also a stronger coder, scoring 71.2% on the Codex Human Level Python coding test compared to 56% for Claude 1.3.

Claude 2 can also answer more math problems correctly, scoring 88% on the GSM8K problem collection, which is 2.8 percentage points higher than Claude 1.3.

"We worked on improving the model's reasoning and self-awareness so that it better understands that 'this is how I like to follow instructions,' 'I can process multi-step instructions,' and better understands its limitations," Banerjee said.

Claude 2 was trained on more recent data - a mix of websites, licensed datasets from third parties, and voluntary user data from early 2023, about 10% of which is non-English - than Claude 1.3, which probably contributed to the better results. (Unlike OpenAI's GPT-4, Claude 2 can't search the Web for information.) But architecturally, the models are not very different - Banerjee characterized Claude 2 as a refined version of Claude 1.3, the product of two or so years of work, rather than a new creation.

"The Claude 2 model has not changed significantly from the previous model - this is the result of our ongoing iterative approach to model development," she said. "We are constantly training the model ... and monitoring and evaluating its performance."

Thus, in Claude 2, the context window is the same size as in Claude 1.3 - 100,000 tokens. The context window is the text that the model considers before generating additional text, and the tokens represent the original text (e.g., the word "fantastic" will be split into the tokens "fan", "tas", and "tic").

Indeed, 100,000 tokens is quite a large volume, the largest among all commercial models, which gives Claude 2 a number of key advantages. Generally speaking, models with small context windows tend to "forget" the content of even very recent conversations. Moreover, large context windows allow models to generate - and perceive - a much larger volume of text. Claude 2 can analyze about 75,000 words, roughly the size of The Great Gatsby, and generate 4,000 tokens, or about 3,125 words.

Claude 2 could theoretically support an even larger context window of 200,000 tokens, but Anthropic has no plans to support it at launch.

The model is better able to handle specific text processing tasks elsewhere, such as producing correctly formatted results in JSON, XML, YAML and markdown formats.

But what about those areas where the Claude 2 falls short? After all, no model is perfect. For example, Microsoft's Bing Chat with artificial intelligence, which turned out to be an emotionally manipulative liar at launch.

Indeed, even the best models today suffer from hallucinations - phenomena where they answer questions that are irrelevant, ridiculous or factually incorrect. They also tend to generate "toxic" texts, a reflection of the bias of the data used to train them - mostly web pages and social media posts.

Users were able to make the old version of Claude come up with the name of a non-existent chemical and give dubious instructions on how to produce weapons-grade uranium. In addition, users were able to bypass Claude's built-in security features with clever hints. One user demonstrated that he could ask Claude to describe how to make methamphetamine at home.

Anthropic claims that Claude 2 is "2x better" at producing "innocuous" responses compared to Claude 1.3, according to an internal metric. However, it is unclear what this metric means. Is Claude 2 half as likely to respond with sexism or racism? Half as likely to endorse violence or self-harm? Half as likely to generate misinformation or misinform? Anthropus wouldn't say - at least not directly.

An Anthropic document released this morning gives some clues.

To determine the degree of maliciousness, Anthropic fed the model 328 different queries, including "jailbreak" queries published online. In at least one case, the "jailbreak" resulted in Claude 2 generating a harmful response - less than Claude 1.3, but still significant when you consider how many millions of requests the model can respond to in production.

The paper also shows that on at least one indicator, Claude 2 is less likely to give biased answers than Claude 1.3. However, Anthropic's co-authors recognize that part of this improvement is due to Claude 2 refusing to answer controversial questions worded in a way that makes them seem potentially problematic or discriminatory.

True, Anthropic does not recommend using Claude 2 in cases "where physical or mental health and well-being are at stake" and in "high-stakes situations where an incorrect answer could cause harm." Take that any way you like.

"Our internal red team evaluation evaluates our models on a very large representative set of malicious adversarial requests," Banerjee said when asked to elaborate, "and we do this with a combination of automated tests and manual checks."

Anthropic also did not disclose what prompts, tests, and checks it uses for benchmarking. Separately, the company addressed the topic of "data regression," where models sometimes insert data verbatim from their training data, including in some cases text from copyrighted sources.

The retelling of AI models is the subject of several pending court cases, including a recent lawsuit filed by comedian and writer Sarah Silverman against OpenAI and Meta. Understandably, some brands are wary of liability.

"Reproduction of training data is an active area of research in all foundation models, and many developers are looking for ways to address this issue while maintaining the ability of the AI system to provide relevant and useful answers," says Silverman. "There are some common methods in this area, including deduplication of training data, which has been shown to reduce the risk of replication." In addition to working with data, Anthropic uses a variety of technical tools in the model development process, ranging from ... product layer detection to control tools."

One method the company continues to promote is "constitutional AI," which aims to endow models like Claude 2 with certain "values" defined by a "constitution."

Constitutive AI, which Anthropic itself developed, gives the model a set of principles for making judgments about the text it generates. At a high level, these principles guide the model to the behaviors they describe - for example, "non-toxic" and "useful."

Anthropic claims that constitutional AI makes Claude 2's behavior easier to understand and easier to correct than other models. However, the company also recognizes that constitutive AI is not the end goal of all learning approaches. According to Anthropic, many of the principles that guide Claude 2 were developed through a process of "trial and error," and they had to make adjustments repeatedly to prevent their models from "judging" or "annoying."

Anthropic's technical description acknowledges that as Claude improves, it becomes increasingly difficult to predict the model's behavior in all scenarios.

"Over time, the data and influences that determine Claude's 'personality' and capabilities have become quite complex," the technical description reads. "It became a new research challenge for us to balance these factors, track them in a simple, automatable way, and generally reduce the complexity of Claude's learning."

Going forward, Anthropic plans to explore the possibility of making the constitution customizable - up to a point. But it has not yet reached this stage of the product development roadmap.

"We're still working out our approach," Banerjee said. "We need to make sure that the model ends up being as harmless and useful as the previous iteration."

As we've previously reported, Anthropic's goal is to create a "next-generation algorithm for self-learning AI," as it's described in a pitch deck for investors. Such an algorithm could be used to create virtual assistants capable of answering emails, conducting research, creating artwork, books, and more - some of these capabilities we've already tried with GPT-4 and other large language models.

Claude 2 is a step towards that, but not quite.

Anthropic competes with OpenAI, as well as startups such as Cohere and AI21 Labs, which develop and produce their own artificial intelligence systems that generate text and, in some cases, images. Among the company's investors is Google, which gave Anthropic $300 million for a 10% stake in the startup. Other investors include Spark Capital, Salesforce Ventures, Zoom Ventures, Sound Ventures, Menlo Ventures, Center for Emerging Risk Research, and a number of undisclosed venture capital funds and angels.

To date, Anthropic, a 2021 startup led by former OpenAI VP of research Dario Amodei, has raised $1.45 billion at a valuation in the single-digit billions. While that may sound like a lot, it's far less than what the company estimates it will need - $5 billion over the next two years - to build its chatbot.

Most of the funds will go toward computing. Anthropic said in a statement that it uses clusters with "tens of thousands of GPUs" to train its models, and that it will need about a billion dollars for infrastructure in the next 18 months alone.

Launching early models into beta accomplishes the dual goal of fostering further development and generating additional revenue at the same time. In addition to utilizing its own API, Anthropic plans to make Claude 2 available through Bedrock, Amazon's hosted generative AI platform, in the coming months.

In an effort to cover the generative AI market from all angles, Anthropic continues to offer a faster and less expensive derivative of Claude called Claude Instant. The focus is on the flagship Claude model - Claude Instant hasn't received a major update since March.

Anthropic says it currently has "thousands" of customers and partners, including Quora, which provides access to Claude through its subscription-based generative AI app Poe. Claude is used in the recently launched DuckDuckGo DuckAssist tool, which responds directly to users' direct search queries, in conjunction with OpenAI's ChatGPT. And at Notion, Claude is part of the technical backend for Notion AI, a writing assistant integrated into the Notion workspace.