AI agents — what they are, and how they'll change the way we work

Agents are not new. Microsoft has done extensive research in the area and even created a multi-agent library last year for developers around the world, work that helped shape what agents can do today. They're getting more attention now because recent advances in large language models (LLMs) help anyone - even outside the developer community - communicate with AI. That agent-LLM duo makes AI tools more tangibly useful.

"People expect AI to do things for them," not to just generate language, says Ece Kamar, the managing director of Microsoft's AI Frontiers Lab. "If you want to have a system that can really solve real world problems and help people, that system has to have a good understanding of the world we live in, and when something happens, that system has to perceive that change and take action accordingly."

Agents are like layers on top of the language models that observe and collect information, provide input to the model and together generate an action plan and communicate that to the user - or even act on their own, if permitted. So both agents and models are equally important pieces of the puzzle, as far as generative AI tools go.

Agents will become more useful and able to have more autonomy with innovations in their three necessary elements: memory, entitlements and tools.

Memory helps provide continuity so that each time you ask for something, it isn't like starting from scratch.

"To be autonomous you have to carry context through a bunch of actions, but the models are very disconnected and don't have continuity the way we do, so every prompt is in a vacuum and it might pull the wrong memory out," says Sam Schillace, Microsoft's deputy chief technology officer. "It's like you're watching a stop-motion animation, one isolated frame after another, and your mind puts it into motion. The clay model doesn't move on its own."

To build up the memory infrastructure to address this, Schillace and his team are working on a process of chunking and chaining. That's essentially what it sounds like: They're experimenting with dividing up interactions in bits that can be stored and linked together by relevance for faster access, akin to a memory - like grouping conversations about a certain project so an agent can recall those details when you ask for a status update and not have to search through its entire database.

The work with entitlements and tools is making sure agents have secure access to, or are entitled to, information they need in order to accomplish things for you, with your permission - like who your boss is, for example - and to the computer programs they need to take action on your behalf, like Teams and PowerPoint.

How to use and build agents for work

You can already create and publish agents in Microsoft 365 Copilot that can help you in your daily work as easily as you'd create a spreadsheet or presentation - no coding skills required.

You don't need to be a developer to build agents using Copilot Studio, either. Anyone can connect them to relevant business data such as emails, reports and customer management systems so they can perform tasks and provide insights.

And you'll soon be able to enlist new agents in Microsoft 365 to help with common workflows and tasks. Interpreter in Teams will provide real-time speech-to-speech translation during meetings, for example, and you can opt to have it simulate your voice. The Employee Self-Service Agent will simplify human resource and IT help desk-related tasks like helping workers resolve a laptop issue or find out if they've maxed out certain benefits, and it can connect to company systems for further customization in Copilot Studio.

Microsoft Dynamics 365 will have agents as well for a range of common business workflows across sales, supply chain, finance and customer service functions.

And every SharePoint site will soon come equipped with an agent tailored to your organization's content that allows employees to quickly tap into these vast knowledge bases and find exactly what they need in seconds, whether it's project details buried in a workback schedule or a summary of a recent product memo.

Developers have even more options. With the new Azure AI Agent Service, you'll be able to choose from small or large language models to orchestrate, develop and scale agent-powered apps to streamline and automate complex workflows like order processing and customer data synchronization. It provides a software development kit with tools for developing agents, allowing you to efficiently integrate agent capabilities using Visual Studio Code and GitHub.

One type of model, OpenAI's recently announced o1 series, can bring more advanced reasoning capabilities to agents, allowing them to take on more complicated tasks by breaking them down into steps - like getting the information someone on an IT help desk would need to solve a problem, factoring in solutions they've tried and coming up with a plan.

You can also use the power of agents in LinkedIn; the platform's first agent can help recruiters with hiring.

Assessing risk for autonomous action

There are extra safety considerations with agents that can act autonomously, and Microsoft is focused on making sure agents only access what you want them to, says Sarah Bird, the company's chief product officer of Responsible AI.

"Agents certainly up the stakes from a responsible AI point of view," Bird says. "So we have to have much, much lower error rates. And there's many more nuanced ways in which something could be an error. This is the big challenge with agents."

But the same responsible AI foundational playbook for other AI applications can be used to assess and mitigate risk with agents, she says.

The new Copilot Control System helps IT departments manage Copilot and agents with data access and governance, management and security controls, as well as measurement reports and tools to track adoption and business value.

Many agents, like those created for Microsoft 365 and Dynamics 365, include "human in the loop" approvals, where people are required to take the final step of reviewing and sending an email the Sales Order Agent wrote, for example. And for agents developed in Copilot Studio, authors can review the records to see which actions the agent took and why.

The key is to focus on testing and moderating to ensure accuracy, Bird says, and for organizations to choose the right starting point for their needs.

"We will of course make progress by building on the foundation we already have, so we're starting the journey from a strong place," Bird says.

Looking back - and into the future

Technologists have long been excited by the idea of autonomous systems working side-by-side with people to help them, says Kamar, who has been working on AI agents since 2005 and even wrote her Ph.D. thesis on the topic in 2010. The hurdle was that "we lacked that general problem-solving power" on the back end, she says.

With LLMs, "we finally have this missing component," she says. "Now we can bring back a lot of the ideas from our decades of research."

Going forward, Kamar envisions a new ecosystem or marketplace of agents, sort of like how apps empower people to do more with their smartphones.

Agents already have "the basic building blocks of what it takes to complete a task," she says. "Like observing, 'I can see your meeting is taking longer; I should delay the next meeting.'"

They're getting more helpful as they gain autonomy through the innovations in memory and entitlements. They're relieving pain points for employees by helping with things like expense reporting, project management and meeting facilitation. And they're driving exponential impact for businesses by taking on duties like alerting supply chain managers to low inventory and then automatically reordering to help drive sales and keep customers satisfied.

Agents matter because they "open up a whole set of opportunities for working with people for getting tasks done, and that's what we expect from AI systems," Kamar says. "AI agents are not only a way to get more value for people but are going to be a paradigm shift in terms of how work gets done."

And this is just the beginning. Copilot is set to evolve with new capabilities like Copilot Actions, designed to handle routine tasks that can bog down employees like summarizing emails missed during time off, compiling agenda items and generating monthly reports. More capabilities like these are coming over the next year to lift the weight of work for employees and teams.

"Copilot will empower every employee to do their best work in less time, and focus on more meaningful tasks," Spataro says. "And agents created in Copilot Studio will transform every business process, helping companies streamline operations, enhance collaboration and drive innovation at scale."

Illustrations by Michał Bednarski / Makeshift Studios

Public link

Please use the above public link if you want to share this noodl on another website.