In a major advanced for AI, Anthropic’s Claude can now operate your computer, including “looking at a screen, moving a cursor, clicking buttons, and typing text.”

Claude has emerged as one of the leading AI models, rivaling and often surpassing rival ChatGPT. Anthropic’s latest innovation adds the ability for Claude to interact directly with a computer, using all the same tools a person would use.

The company describes its goals in a blog post:

With computer use, we’re trying something fundamentally new. Instead of making specific tools to help Claude complete individual tasks, we’re teaching it general computer skills—allowing it to use a wide range of standard tools and software programs designed for people. Developers can use this nascent capability to automate repetitive processes, build and test software, and conduct open-ended tasks like research.

Anthropic has developed an API so developers can tap into this new ability.

To make these general skills possible, we’ve built an API that allows Claude to perceive and interact with computer interfaces. Developers can integrate this API to enable Claude to translate instructions (e.g., “use data from my computer and online to fill out this form”) into computer commands (e.g. check a spreadsheet; move the cursor to open a web browser; navigate to the relevant web pages; fill out a form with the data from those pages; and so on). On OSWorld, which evaluates AI models’ ability to use computers like people do, Claude 3.5 Sonnet scored 14.9% in the screenshot-only category—notably better than the next-best AI system’s score of 7.8%. When afforded more steps to complete the task, Claude scored 22.0%.

The company is also working to ensure the new capabilities are rolled out safely, especially since an AI using a computer could represent new attack vectors.

Because computer use may provide a new vector for more familiar threats such as spam, misinformation, or fraud, we’re taking a proactive approach to promote its safe deployment. We’ve developed new classifiers that can identify when computer use is being used and whether harm is occurring. You can read more about the research process behind this new skill, along with further discussion of safety measures, in our post on developing computer use.

Using a computer and performing general computer tasks is a significant step forward and likely represents the next major iteration of AI advancement.