Announcing new computer use in Microsoft Copilot Studio for UI automation

Announcing computer use in Copilot Studio! This new feature allows your Copilot Studio agents to interact directly with websites and desktop applications. Want to join the limited preview? Read on.

AI innovation is accelerating at an unprecedented pace, and Microsoft Copilot Studio is at the forefront-integrating the best AI advancements into a platform built to solve business challenges at scale. Last month, we introduced deep reasoning capabilities for agents, support for model context protocol (MCP), and the general availability of agent flows in Copilot Studio.

Today, we are excited to announce that computer use is coming to Copilot Studio through an early access research preview. This new capability allows your Copilot Studio agents to treat websites and desktop applications as tools. With computer use, agents can now interact with any system that has a graphical user interface!

Achieve new efficiencies with computer use

Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen. This allows agents to handle tasks even when there is no API available to connect to the system directly. If a person can use the app, the agent can too.

Computer use adapts to changes in apps and websites automatically. It adjusts in real time using built-in reasoning to fix issues on its own, so work continues without interruption. It is also built on Copilot Studio's robust security measures and governance frameworks, to help ensure compliance with organizational and industry standards.

With computer use in Copilot Studio, makers can build agents that automate tasks on user interfaces across both desktop and browser applications, including Edge, Chrome, and Firefox. Additionally, computer use runs on Microsoft-hosted infrastructure, meaning organizations don't need to manage their own servers. Enterprise data stays within Microsoft Cloud boundaries and is not used to train the Frontier model. This helps your organization accelerate deployment, reduce maintenance, and lower infrastructure costs.

Unlock new value with agentic and automation scenarios

To bring this technology to life, consider the following high value use cases:

• Automated data entry: Imagine a scenario where an enterprise needs to input large volumes of data from various sources into a centralized system. Computer use can automate this process, reducing manual effort and minimizing errors.
• Market research: Marketing teams can leverage the tool to automate the collection of market data from various online sources for analysis, providing valuable insights without the need for manual intervention.
• Invoice processing: For finance departments, the tool can automate the extraction of data from invoices and input it into accounting systems, streamlining the entire invoicing process and reducing manual errors.

Reimagining robotic process automation (RPA)

Computer use agents are transforming robotic process automation (RPA). They overcome traditional limitations, like the fragility of UI elements, and can handle complex dynamic interfaces. This makes automation accessible to people beyond professional RPA developers.

In Copilot Studio, computer use addresses common RPA challenges by making automation smarter and more intuitive:

• It responds to changes in real time: When buttons or screens change, the tool keeps working without breaking your flow.
• It is easy to use: You can describe what you want in natural language, no coding needed, and test and refine the prompt with real-time side-by-side video of the computer use reasoning chain and the planned UI automation.
• It is built with intelligence: The agent sees what is on the screen and makes smart decisions in real time, even in complex or constantly changing environments.
• It comes with full visibility: Makers can view a history of computer use activity at will, including captured screenshots and reasoning steps.

The future of innovation with Copilot Studio

Copilot Studio is the end-to-end agent platform designed to help organizations achieve their AI and operational goals. We want to empower you to streamline processes, enhance productivity, and drive innovation.

If you are interested in exploring the new computer use capability, we would love to hear from you! Please fill out this form to let us know you would like to participate.

We will also share more about this new announcement at Microsoft Build in May 2025-register here to join us!