Big-data and machine learning software provider Databricks Inc. today released Dolly 2.0, the next iteration of the company’s open-source generative artificial intelligence model that has ChatGPT-like capabilities.

Databricks released Dolly two weeks ago as a response to the multitude of large language model generative AIs on the market today that are mostly inaccessible to researchers and businesses because they are locked behind paywalls and held by centralized services.

Dolly 2.0, like its predecessor, uses a smaller dataset than most LLMs, which makes it particularly lightweight. Dolly’s model was trained on 6 billion parameters, compared to OpenAI LP’s GPT-3’s 175 billion, whereas Dolly 2.0 features double that at 12 billion parameters. It has also been fine-tuned on a high-quality instruction-following dataset, which was crowdsourced from Databricks employees.

The AI uses its training set to generate coherent sentences and responses, which is the hallmark of generative AI, when asked questions by users. It can do this even though its training data is much smaller than OpenAI’s models, which makes it possible to run on a company’s internal servers without needing to share data with a third party.

“We believe models like Dolly will help democratize LLMs, transforming them from something very few companies can afford into a commodity every company can own and customize to improve their products,” the company said when it released Dolly.

Considering this, Databricks has fully open-sourced Dolly 2.0, including its training code and dataset for commercial use. The dataset included with Dolly 2.0 is the “databricks-dolly-15k” dataset, which contains 15,000 high-quality human-generated prompt and response pairs that anyone can use, modify and extend under a Creative Commons license.

“When we first released Dolly two weeks ago, we were immediately flooded with requests from people who wanted to try it out. The number one question was ‘can I use this commercially?'” Ali Ghodsi, chief executive and co-founder of Databricks, told SiliconANGLE in an email interview.

However, because Dolly had been trained using a dataset built by the Stanford Alpaca team using the OpenAI API, it could not be use commercially — as the company’s terms of service forbid anyone from creating a model that competes with it. “The answer was unfortunately ‘no,'” Ghodsi said.

To get around that issue, Databricks set out to create a new model all its own by using only employee responses, and not a training set “tainted” by a commercial license.

To get the new model trained, Databricks set up a contest among its 5,000 employees to answer original questions. Tasks included following instructions such as open questions and answers such as opinions as to “Why do people like comedy movies?” or extracting and summarizing information from Wikipedia. It also required them to do brainstorming and creative writing tasks such as producing love letters, poetry or songs.

According to Databricks, Dolly 2.0 is the only model currently that doesn’t suffer from this limitation. Other examples include Alpaca, Koala, GPT4All and Vicuna, which cannot be used commercially because of their training data.

“Unlike previous models, Dolly 2.0 provides a license that permits commercial use,” said Ghodsi. “That means enterprises can tune, develop, and use Dolly 2.0 using their proprietary datasets. They don’t have to give those datasets to a third-party vendor, and they can monetize the model in a way that otherwise is prohibited by existing models. Of course, we hope people will train their models on Databricks, but these models and datasets can be used elsewhere. ”

Researchers and developers can download the Dolly 2.0 model weights from Databricks starting today and the dataset is also available on GitHub.

Image: Pixabay