Artificial intelligence startup Rabbit Inc., the maker of a low-cost AI consumer mobile device that fits in a pocket, today released a way for users to teach its AI assistant to take actions on their behalf on digital interfaces, starting with websites.

The new system, called “teach mode,” is being released in beta testing. It will allow the AI device to navigate websites, check information and operate a variety of different digital systems.

Rabbit has produced a low-cost mobile device called the r1, launched in early 2024, that looks like a small red box designed to fit in a user’s pocket. Powered by Rabbit OS, it uses an AI model that allows users to interact with apps using their voice and natural language commands. For example, they can press a button on the device to purchase plane tickets or buy groceries.

Under the hood, the company said, it developed a specialized AI called a large action model, which can comprehend human intent and transform that into actions similar to the way that a human might perform the same task. At launch, r1 could perform searches, reply to questions, use its camera to transcribe text, assist with bidirectional transition and assist with buying items.

Now, users will be able to teach the r1 to perform even more actions using teach mode.

“All the best car manufacturers compete over their engines, but when electric cars came out, they didn’t even need an engine to run,” said founder and Chief Executive Jesse Lyu. “We shouldn’t carry the burden of previous operating systems into the current systems. A developer ecosystem is crucial to the success of an operating system, and teach mode is that missing link, giving people the power to create their own custom agents.” 

Users do not need to have any coding experience or advanced AI or developer knowledge to use the teach mode. The new mode was first announced alongside the r1 at CES 2024. In September the alpha testing program was launched and more than 400 “lessons” were created by a group of 20 testers.

Using teach mode, a user essentially shows the r1 how to execute a series of steps by going through the motions. As the AI model follows along, it watches and links each step and the user can annotate the steps with text to explain why the step was done. If a misstep is made, the user can delete it and backtrack.

Examples include sending a custom tweet or adding an item to an Amazon.com shopping cart. By having users go through the motions of executing the action with one product or tweet, the AI can learn how to do it again for any number of other products or tweets. Through framework of how it worked once, the Rabbit LAM agent can automate the task again and again when it’s called up in the future after having a demonstration of the steps and the reasoning behind them.

“A major challenge with AI products is that companies need to directly work with customers to learn their behavior and create the experience from the ground up because AI hardware is new and there are no predecessors,” said Lyu. “We are fortunate to have one of the most engaged communities for emerging technologies. With their support, we are one of the first and only companies in the world to deliver a useful general agent to consumers at scale.”

According to Rabbit, the initial test period for teach mode enabled the company to expedite the public beta release and significant improvements that allowed the company to support more complex and dynamic websites.

On Oct. 1 Rabbit launched LAM playground, a vision language model-driven agent, for all r1 users. Unlike teach mode, LAM playground is an experimental agent system that performs tasks using AI by visually examining websites with a VLM and having users tell it what they want it to do without any prior demonstration.

Image: Rabbit