I, Agent: What Happens to Edtech When the User Isn't Human

I, Agent: What Happens to Edtech When the User Isn't Human

by InstructureCast

• Agentic AI changes what interoperability optimizes for, moving the priority toward semantic understanding of course content rather than identical schemas across systems.
• New protocols build on existing standards instead of replacing them, so prior investments in open APIs and LTI still matter.
• Keeping a human in control sometimes costs a little speed, and both guests argue that maximum speed should not be the goal of an educational AI system.
• Transparency is non-negotiable: if AI touches a grade, students should know, and they should be able to question the result.

What is Educast 3000?

Ah, education…a world filled with mysterious marvels. From K12 to Higher Ed, educational change and innovation are everywhere. And with that comes a few lessons, too.

Each episode, EduCast3000 hosts, Melissa Loble and Ryan Lufkin, will break down the fourth wall and reflect on what's happening in education - the good, the bad, and, in some cases, the just plain chaotic. This is the most transformative time in the history of education, so if you're passionate about the educational system and want some timely and honest commentary on what's happening in the industry, this is your show.

Subscribe wherever you listen to your podcasts and join the conversation! If you have a question, comment, or topic to add, drop us a line using your favorite social media platform.

• I, Agent: What Happens to Edtech When the User Isn't Human
  Welcome to Educast three thousand. It's the most transformative time in the history of education. So join us as we break down the fourth wall and reflect on what's happening. The good, the bad, and even the chaotic. Here's your hosts, Melissa Lobel and Ryan Lufkin.
  
  Hello and welcome to another episode of Educast three thousand. I am your cohost, Ryan Lufkin.
  
  And I'm your cohost, Melissa Lobel. Today, we're going technical. Just to warn you all, this is exciting. Our featured guest is Mike Nast, principal group program manager for Microsoft Education, where he leads EdTech product integrations and partnerships.
  
  Mike serves on the board of One EdTech with me. He's amazing. And this is the consortium behind many of the open standards that have quietly connected tools and created a really incredible infrastructure for ed tech environments. He was recently honored at One Ed Tech with a Learning Impact Award for all the work and contributions he's done.
  
  And he spent his whole career building that connective tissue I've talked about in educational technology. So he is an expert. This is so exciting. Mike, we are thrilled to have you here.
  
  Thank you. I'm very happy to be here.
  
  And because we wanted to get properly in the weeds today, we brought a second super smart fellow that you may have listened to before here on the Educast three thousand podcast, Zach Pendleton, who is our chief architect at Instructure. Zach has spent more than a decade in education technology, working to keep learning platforms open and extensible, and his research these days lands right in our topic area: generative AI in education, digital credentialing, and edge computing. So, Zach, welcome to the podcast.
  
  You and Mike are actually going to be asking and answering each other's questions. So consider this a techie to techie conversation with Melissa and I chiming in every once in a while and essentially the live studio audience.
  
  And absorbing it all and eating popcorn. Always.
  
  Always. Make you guys smarter. Right?
  
  Oh, it is always, always a pleasure to be able to be on the podcast. I think I would I would debate the description of me as super smart, but otherwise, very happy to be here and and looking forward to it.
  
  I'll double down on that, Zach. You know that. But before we dive in, Mike, would you share a little bit more about yourself so our audience can get to know you a little bit better?
  
  Sure. I think Melissa did a great job hitting the highlights here. So I work in the Microsoft Education Product Team. So that is the team that builds the extensions and adaptations for Windows and M three sixty five, specifically in teaching and learning.
  
  And I've been at Microsoft for about eight years in various roles, but the one constant has been the work that I do with the EdTech community. So I've been focused on partnerships where our partners are integrating with our platform, building on our platform, or where we are integrating into other platforms.
  
  Certainly the nature of that has changed over the years and recently there's been a very strong focus on AI and agentic AI and what that means for integration and interoperability for Microsoft in teaching and learning and across ed tech in general.
  
  As Melissa mentioned, I'm a member of the board of directors of One Ed Tech. Microsoft is a contributing member of One Ed Tech. We're very active in the community, and we've certainly been trying to apply a lot of pressure within the organization to really confront what interoperability means in the world of AI, generative AI, and agentic AI. So as we're pushing that forward, I'm working here in my home in Michigan. I've been a remote worker since about the year two thousand. So I had a good couple decades of practice before the pandemic hit. And it's been an interesting ride.
  
  You tend to be the same places that I think Melissa and Zach and I. I see you at all the shows and, you know, we've known you for a long time. So it's great to have you on the show.
  
  Yeah. Thank you.
  
  And Zach, I know you've been on our podcast before, but so how about you share just a quick version of what is being a chief architect today at Instructure mean? And sort of where's your attention perhaps, particularly as we think about generative AI?
  
  Yeah. Absolutely. So, you know, mean, architect is a technical role, but I think a lot of my work starts by visiting schools, visiting universities, visiting class rooms, and understanding what problems teachers and students are having right now, and then trying to map that back to how we develop software at Instructure. So understanding what we need to build, but then really working with our engineering team to make sure that we're building it in a way that is prepared for the future.
  
  So that means, you know, we've talked already a lot about openness here. I hope that continues on this podcast. I think that has been critical in ed tech, and it's been such an important piece of the success of education technology. And so I spent a lot of time making sure that, you know, we as a team continue to invest there and and build in in ways that are certainly open, but also are scalable, are secure, and are I think a really important one is adaptive.
  
  Right? I mean, things are changing so quickly right now, I think, for everybody, and and technology is certainly not an exception there. And and we need to make sure that we're prepared not just for what's coming right now, but what's going to be coming in the next five or ten years.
  
  Thanks, Zach. That makes so much sense.
  
  And a lot of what you just shared, we're gonna be asking both of you questions about. So thank you for that setup. But before we get in, one of the things that we always do on this podcast is ask our guests a favorite learning moment. And that can be one where you were the learner, it can be one where you were the teacher, it can be something you've observed in your family or friends, anything related to learning.
  
  But we always like to do this because it starts to peel back this. How do you think about learning as we get deeper into the conversation? So Mike, I'm gonna start with you. Would you mind sharing a favorite learning moment with our audience?
  
  I don't know if this counts as a moment, but a book that always stuck with me and sort of changed the way that I view myself and and other people is Malcolm Gladwell's Outliers. I think it's just this amazing exploration of all of the different factors that can contribute to an individual's success right up to and including the day they were born.
  
  And with education, I think it also provides a great illustration of how early advantages or disadvantages can have this cumulative effect over the lifetime of a learner or or person. It really has changed my perspective on on so many things that I'd highly recommend it.
  
  Yeah. It's it's on the bookshelf back there and yeah. I love it.
  
  I love that. And a book absolutely counts as a learning moment.
  
  How about you, Zach? How about a learning moment you'd like to share?
  
  Yeah. You know, a recent one for me that I'm just thrilled about. So I have a ten year old son, and, you know, his hobbies and his interests change about every six months. And we're trying to adapt to that as parents, but, you know, recently, he's decided he wants to be a stuntman.
  
  And so he was really, really interested in building a bike ramp. So it gave me a chance, you know, to go to the hardware store with him, buy all the wood, buy all the the screws, the fasteners, and everything else, and and actually, you know, teach him a little bit about how to use these tools and then how to build things in a safe way. And he was he was very motivated because, you know, that first jump, he was really afraid that the bike ramp was gonna collapse underneath him. And so it was a lot of fun to get to work with him and teach him a little bit about something that he was interested in.
  
  I love those. You got engineering. You got physics.
  
  You got, you know, death There's some math in there.
  
  In there. For sure. Yeah. It's amazing.
  
  Oh, yeah. Absolutely. You should have also bought a first egg kit.
  
  Yeah. That's that's the next lesson, Mike. Yeah. We're waiting for that one.
  
  Some biology.
  
  Yeah. Basic first aid.
  
  I love that story, though. That's awesome. And I hope as we have you on future podcasts, we can hear updates on his his progression towards this new career path.
  
  Also, have a buddy from college who was a stuntman. If you want somebody to talk him out of it, I'll line that up too.
  
  Oh my goodness. Yeah. We're we're hoping that he just forgets about this one soon.
  
  Six months. You've got six months.
  
  Alright. We've got two incredibly smart technologists in the room. So I want to start at the foundation. When people say interoperability in ed tech, what's the layer that actually matters in twenty twenty six? And how has that answer changed now that AI agents and other tooling are entering into the stack? Perhaps, Mike, you start and then I'll pass it over to Zach.
  
  Sure. I think as we have talked about interoperability and standards, in the past, the focus has very necessarily been on making sure that two systems have the exact same model of the world. You know, that they share the same vocabulary, perhaps the same data schema, maybe the same rigid API definitions. And I think what really changes when you introduce AI and agents into the mix is when you have an agent as a client, the agent is able to learn new vocabularies.
  
  It's able to reason over schemas. It's able to interact with different APIs if given the appropriate instructions. And I think that changes what we need to focus on when we're thinking about interoperability. It becomes far less important for every system out there to have the same rigid conformance to all of these different mechanisms.
  
  And it becomes far more important that they're able to describe to the agent how to interact with those particular systems.
  
  And when we look at say protocols like MCP, that's exactly what they're doing. And I see the protocol becoming much much more important going forward. So now that I have an agent, I don't care if I want to ask a learning management system what are my courses. It doesn't matter if that requires the query to be GraphQL or if it requires some custom API.
  
  Doesn't matter if it calls an entity, a course offering or a section. As long as that can be described to the agent, the agent can understand how to transact with that system, put it into my context, and figure out what what I need.
  
  Yeah. I think that is it's a good point, Mike. I I mean, that is such a an interesting twist, and I I think provides a lot of opportunity for us as system builders because it means that integration, I think, becomes in a lot of ways more flexible and then a lot easier. I think, you know, something that I hear people talk about quite a bit is this idea that everything is changing.
  
  And I do think interoperability today and some of these new tools like MCP or as we think about agents, it's not an or between those systems and all of the investments we've made in ed tech, you know, and certainly in in my career. I think it's an and that these things provide new opportunities, but they build on top of the great work that we all have already done. So, you know, if you've got an open extensible API, if you've got good LTI support, all of those things feed into the possibilities of the future and expand those. And without those, I think we're still constrained by these new tools and what we can do.
  
  So you do get things like Drift, and we're trying to solve that end by end integration problem. Right? As we talk about the LMS, the SIS, Gradebook, all these different systems, it's a pretty complex, you know, ecosystem we're trying to pull together. So do agents end up speaking LTI in one roster? Do we need something new? Like, what's what is the next step to create that consistency, that still common standard, even though we have that flexibility with these agents?
  
  When you look at LTI or OneRoster, the specifications are made for very specific use cases, right? You know, LTI is more about how learning tools interact in in a platform, how they integrate into a user's workflow and learning platform. One roster is more about moving data from one system to another.
  
  The introduction of AgenTic AI, it doesn't change those use cases, it doesn't diminish those standards, those are still needed. But and there will be overlaps. One example of an overlap, you know, if you look at LTI. LTI has given us a world now where a lot of the content in the learning management system might not be stored directly in learning management system, but it's links out to learning tools.
  
  And those links are made to take a user from the learning platform into the learning tool. But now if an AI agent looks at that link, what can it do with it? It really can't do anything with it. It can't launch it.
  
  You know, maybe if it's an agentic browser, but that's not what we're talking about here. Generally, it's gonna see this link, it's not going to understand what's the source, it's not going to understand any context around what's supposed to be behind that link.
  
  Is it a meeting tool? Is it a bit of content? Is it an assessment tool? He has no idea. So we need to start thinking about how that intersects with LTI, what sort of extensions might be needed around LTI to enable agents. I don't see agents speaking LTI.
  
  You look at one roster and it's like, sure, I could wrap one roster at MCP and an agent could use it. But it'd have to be a highly privileged agent because one roster doesn't have delegated scopes, doesn't have the concept of user access.
  
  It may or may not be useful, and you know perhaps we'll do it, maybe not. But I do think something new is needed, you know, where our attention really should be are on things like semantic understanding of the course. You know, for an agent to really help a user, to help an educator create materials, to help a student navigate a course or learn, it needs to have a semantic understanding of the learning environment. It needs to understand what's in the course. It needs to understand all the systems that surround it and the content that's stored throughout the ecosystem.
  
  And that implies that we need something like semantic search. The ability to find course objects based on meaning. The ability to find learning content based on meaning.
  
  And there aren't many systems that have really or many providers who've even started to think about this. I would say to Instructure's credit, you're one of the few actually. I think your your Smart Search API, which has been in beta for a long time now, was really forward looking and a great example of this. It's exactly the sort of thing that an agent needs.
  
  You know if a student's trying to learn about photosynthesis, that's an API that can actually go and find the related pages and assignments and items from the course and bring that to the agent, give them snippets of the content and allow it to reason over the actual content. I think those are the sorts of things that I would start to focus on when it comes to standard. I would worry less about the vocabularies that we're the we're using or or schemas and things start to focus more on what are the common services that we really need to enable AI use cases. And I think semantic search is a clear one.
  
  And, you know, if instructor wanted to jump out there and lead a working group on it, I think that would be fantastic.
  
  That seems like an invitation, Zach.
  
  Oh, man. Let's let's talk about that after. I'm curious, Mike. There are a couple things you mentioned I I want to pull the thread on a little bit because, you know, on the one hand, I I think you're right. We've got this world where semantics and context become increasingly important to be able to steer these agents. And we have to reckon with the fact that that content and context may be scattered across a lot of different systems.
  
  And access to those systems was really designed for kind of humans. And that becomes a little problematic when we think about kind of agentic or or automated access because on the other hand, that means we may not have all of the right kind of identity scope and delegation tools in place for these. Right? And I I think as we think about agents, something that that keeps me up at night a little bit is that the actions that are taking place become increasingly distant from the human who expressed the intent or set the goal.
  
  Right? That it's I can't ever, in this agentic world, really trust that a human was standing over the machine and pushed the button. Do you have thoughts about how we ought to be thinking about things like delegated authorization and and how we keep I mean, I hate to say human in the loop because I it's important, but I I think it is a little reductive. But how do we maintain human authority over these systems and these workflows moving forward?
  
  There are two cases here. You know, one, you could have an autonomous agent that has its own identity, and that I would treat just like a human interacting with the system. And the other case is you have an agent that is acting on behalf of the user, which is exactly what you were getting at. And when you look at a protocol like MCP, there are requirements in the specification for consent, right?
  
  The agent needs the user's consent to connect to a system. The agent needs the user's consent to take action on behalf of the user. And it's using OAuth, so it's not the agent taking the user's credentials and logging in. It's the user actually enter logging into the system and then letting the agent do what it does.
  
  And in my mind that isn't very different from what we've had, you know, with users logging in with applications. And to your point about the possibility of the actions getting very distant from the user's intent, that's not a new one because I think of that in the exact same terms that I would think of say malicious actors hijacking tokens, stealing credentials or malicious applications getting into the system. We have had to already put up guardrails against that. So I think you're talking about the exact same security principles in terms of things like least privileged access and token scoping.
  
  And certainly in terms of your auditing, know every transaction that happens with your system you know you need to have an audit trail, you need to understand what the user's doing, what they did. You know, maybe there are some things that we need to think about what we're capturing in that audit trail, you know, in addition to you know, the user account that's logged in, certainly you wanna capture things like the user agent, the application ID. So you have a better idea of where that originated from and a better idea of whether it was, you know, been a specific application versus an autonomous agent.
  
  But yeah, don't think it keeps me up as night quite as much as it does you because I think we've faced these exact same threats at a very different context.
  
  That makes a ton of sense. And I actually want to maybe lean in on an output of this perhaps if we wanna think it that way. And I am nowhere near as technical as the two of you, and so I'm probably not saying this quite right. But we're also seeing AI.
  
  We're seeing this debate between small models and big frontier models. And where is AI located? Is it running on a local machine for an end user? Is it in the cloud?
  
  Like, Zach, talk me through your thoughts on sort of those shifts and how that may relate to this larger how are we ensuring those human agency in all of this?
  
  Yeah. Great. So I you know, that first generation of large language models that really hit the zeitgeist in late twenty twenty two, early twenty three, I mean, we're just they were so big. At the frontier, they've only gotten bigger such that the the conversation about what could we run on the edge or what could we run locally was really moot because it just was not an option. You know? You needed a server farm to run these things.
  
  I think what we started to see in, you know, probably kind of twenty four in earnest and and into twenty five was kind of the progress at the frontier slowed a little bit where it was just the best models were not doubling in capability every six months like they had been. And so there was a lot of investment placed in in smaller models that could be run on smaller hardware in the cloud, you know, at the edge, or could even be run locally, you know, if you had a a modern laptop or or desktop. And so I think that does provide us a a lot of choice in what we run where. Now I I think it's not still something that I would maybe say is consumer grade for most people.
  
  You know? I I think it's still for for technologists, but, you know, for me, I think the future I want is one where I don't have to go out to the cloud for every LLM request, that I can have a a smaller model that runs locally that I can trust with more sensitive information or with more private details to do things on my behalf. So, I mean, I can tell you, I'm I'm already doing this today. I know a lot of engineers and Instructure are where we're when we're programming with large language models, we're using a large frontier model in the cloud to kinda plan what we want to have happen and to kind of chunk that work in the way that we would even chunk it for an engineering team and then using smaller local models to execute that word for us.
  
  Now I think that, you know, that provides, you know, really clear privacy benefits. I think it provides clear environmental benefits, but it also provides real cost benefit, which I know continues to be a challenge in education technology. Right? Large language models at the frontier are just very expensive.
  
  And I talked about this this push towards smaller models. I think we're we're seeing that flip a little bit now, whereas as LLM providers are looking to IPO, costs are coming back up because, you know, they've gotta get those margins right. You know, I think there's a lot of possibility here, but I think it's going to require all of us kind of pushing on this and and making sure this is a priority for providers. And I, you know, I will shout out Microsoft.
  
  I think, you know, they just recently released their own family of models, and it's really heartening to see them release models of various capabilities and sizes because I think it provides us as technologists, but I think also us as educators and as people kind of choice in how we engage with these tools and can we can deploy what makes the most sense where it makes the most sense. I don't know, Mike. Have you seen anything similar here? What are your thoughts about the kind of small model versus large model or or local inference?
  
  I think I'm too old. I just kinda see this as the same pattern that we've been watching for ages, you know, from from server to client server, from there to web, then to mobile, and you know, as you saw in mobile things start to settle into this hybrid world where you have a mixture of local and cloud capabilities. And that's exactly where we're gonna end up with AI. You're going to have a hybrid world. You're going to have a world where there's going to be AI on the device where you need latency, where you need very low latency, where you want certain privacy characteristics, where you want lower cost, you want quick response, where it's close to the hardware. I think this is gonna be things like, you know, like real time translation, voice recognition, video processing, any sort of screen understanding.
  
  And you know, you're talking about language models, you can see this today in products that Microsoft is delivering. We have a product called, Learning Zone.
  
  And it is a tool where it can run learning activities, interactive activities on the device using the small models, they're very fast, very responsive.
  
  But when it needs to do more intense generation, needs access to larger amounts of data, that's sort of thing that will get delegated to the cloud. I think you're going to see more and more of that. There's a place for on device AI, but when it comes to some of the broader workloads where you need access to broad array of enterprise data, or where you need deeper reasoning or creativity of the frontier models that will get delegated to the cloud. But yeah, absolutely more and more is going to run on the device. And I think it can be transformative for some of the applications out there in the industry. When I think of areas like classroom management or student safety, know a lot of the monitoring and filtering and types of things can be done now on the device and protecting the student privacy and making sure that only appropriate things and signals go up to the cloud where they're needed for the enterprise monitoring.
  
  Yeah. I mean, besides the AI washing that Zach talked about that we see, I see a lot of kind of oversimplification of some of these processes. Like, people will be like, Oh, you can just vibe code a tool for that, right? And I'm like, Oh my God, do you know, like, the requirements?
  
  For my agents are really only as good as the information they're grounded on, right? So, schools are sitting on rosters, outcomes, activity streams with Caliper events, OneRaster data, case competency frameworks. How do we expose all of that to AI in a structured, governed way without turning every institution's data into a free for all, right? So we're not having to just buy code access to this protected data.
  
  I don't think there's a magical answer there.
  
  You're seeing a pattern here, I keep looking to the past, right? When you think of an agent that's maybe a teacher assistant or an assistant for a student, if it's helping the student learn something in class, chances are it's going to want access very similar to what the student has in the learning environment. Right? Maybe it's looking at it's looking at the course, it's looking at the contents of the course, maybe it's gonna look at the student's grades, it's gonna look for content related.
  
  It's going to be very transactional. It's gonna have the same sort of transactions that that student might have had as a user with those source systems. I think most of the agentic use cases in the classroom for educators and for learners are going to fall into that model of transactional access. And there you already have your answer because the platforms are in control.
  
  They have the model for transactional access, they have the model for security, they understand licensing, access control, auditing, all of that.
  
  When you get into the broader institutional use cases where you're looking at large data sets that cross the entire organization and you're trying to you know, assess outcomes at an institutional level, That's where you're getting into the models that we've been using in analytics for for years. Know, it went from data warehouses to data lakes, you're looking at data pipelines and the the assembly of that information.
  
  Those tend to be bespoke implementations, the exact governance model depends on the data that's being moved and the use case being implemented. I don't think there's a simple answer there, but we've been doing this for years and you know, there are a lot of organizations out there that have a lot of experience in building out these systems. I don't think that swapping out, you know, some analytics use case for an AI use case makes a big difference.
  
  It's the same treatment of of the processes.
  
  Yeah.
  
  Yeah. Another question for you, Mike. I I know you've got a a long history in open standards and in integrations work, and I I think that every time there's a new technology, it feels like there's this kind of reflexive pull from providers back towards walled gardens or or closed systems. And so for every one MCP, you know, we're getting four or five proprietary kind of closed integration tools and being told, oh, things are different this time because it's a new technology.
  
  But I feel like the principles of openness probably matter just as much now, if not more than ever. And I I would love to hear your thoughts about what needs to happen at the consortium level and and what we all should be thinking about to ensure that these new AI tools still kinda bias to open? Right? And does one ed tech have a role in that?
  
  Are they the right venue?
  
  Yeah. You know, so first, I think that that reflexive closed approach is somewhat necessary. What we've seen over the last couple years is every vendor racing to understand what AI means to their products, to their users.
  
  And the easiest thing in the world to implement with a technology that just is fueled by data is doing that in your own ecosystem. Right? So we've all been working in our own gardens. But I think the good news is that we're all going to feel very natural pressure to work with the world, reach beyond our walled gardens.
  
  For a learning management system like Canvas, we already talked about LTI. A lot of your course content, lot of the scope and sequence that's stored in a course may just be links to the outside world and you know, your AI has no idea what's out there unless the vendors open up and allow for you to retrieve context around those links. And likewise, everybody surrounding you, you know, they have no insight into what the structure of the classroom is unless you open up and give access to us. So I think there's just this natural pressure that is going to cause us all to need interoperability.
  
  So my biggest concern right now is accidental interoperability, or inoperability. What would happen if everyone develops their own MCP servers and their own clients and without talking to each other and just throwing them out into the market? We are probably going to end up with a mix of solutions that may or may not work together. Yeah and maybe that's because the tools didn't put on the right annotations or you know maybe it's some oddity in the way that authentication is handled by the server.
  
  There are so many things that can go wrong if we don't align and don't work together before we put these solutions into market. And I think that's where the consortium can really really help, know getting the vendors together and the institutions together to talk through the mechanics around things like how we're implementing MCP clients and MCP servers. I think the hard thing honestly is getting them to focus and getting them to focus on sort of this immediate problem that can have very large short term returns rather than sort of the typical moonshots that they like to consider, theoretical things like what does learning context mean.
  
  It's like we don't need to know that right now. What we need to do is make sure that we don't all throw a whole bunch of solutions in the market that don't work together.
  
  So yeah, I think the OneNET Tech Consortium is a great place to solve this. I I do believe we're getting a working group started on this very problem right now to focus in on these mechanics around MCP, and I think that can be, yeah, very, very impactful. You know, Zach, just to throw one back to you, as we start talking about this and we start talking about this world of MCP where, say, agents become first class users of a system, you know, how are you thinking about that as an architect? You know, what are some of the architectural assumptions that you think are probably now less important, and what is important today?
  
  Oh my goodness. Yeah. So, you know, I think you spoke earlier to how we have a lot of of concepts and tools in place to police access to systems. You know, we we've got privileges or at least access and and things like this that that we've been looking at as security tools primarily, but I think those matter a lot more as we start looking at agents because I think that, you know, a lot of our systems were designed you know, the user experience assumed that the user was a rational actor, that it was a human who was going to come in.
  
  They knew what they wanted to do, and they knew how to do it. And and so we had a lot of flows, you know, where we just said, well, a human would never do that. Right? It's it's against their self interest to go, you know, delete their course roster or to get rid of the assignments in a course or something.
  
  But I think with agents, we don't have all of those same guarantees. Right? There's the I think this the story now that that went a little viral about the engineering manager at Google who, you know, connected an autonomous agent to her inbox, and it it texted her one day that it was cleaning out her mail by deleting everything. You know?
  
  And she said, I'm I'm frantically, like, running back to my laptop to throw it out the window while texting it. Stop. Stop. Stop.
  
  You know? It just saw the the clearest path to inbox management is to not having any email. And so I think that for me, that's the one where I I think we still have gotta wrestle a little bit with how these tools that we have can be applied in a way that not just protects against bad actors, but protects against irrational action. You know?
  
  And I think keeps a human in control of that process even if that costs us maybe a little bit of efficiency. Right? I I don't know that maximum efficiency ought to be the goal of of any LLM system.
  
  A lot of, really scary movies start with maximum efficiency as the goal, I think.
  
  Yeah. Yeah. And I think irrational action and bad actors often go hand in hand. Yeah. They're sometimes the same thing.
  
  It's so true. Well, I'm curious. I'm gonna lean in a little bit more to what you just were sharing, Zach. So I think from a nontechnical perspective, there's a lot of educators out there excited about having an agent act on behalf of them.
  
  So getting to a place where it can grade, that, you know, that agent can provide thoughtful feedback, that agent can do actions and they're going to be rational. We're not there yet. I think we all know this. I've seen countless number of tools be built meaningfully and purposefully with still that human authorizing a final step action.
  
  Zach, do you think we'll get to a future? How are you thinking about this? And then I'm gonna pass it to you after that, Mike, around. Will we get to a place where there is more high risk automation or a high risk agent to system work that can happen, or will we always have humans be in that loop?
  
  Yeah. So there are two questions here. The first is what the technology can do, and then the second question is what we want to have happen. I think the technology will support this at some point.
  
  You know? I I think it could do some things today. We'll only see the capability of these systems to act independently and on a autonomously grow and grow. You know, to your question about what do we want here, I think Mike's earlier point about kind of auditability and monitoring is critical.
  
  Right? I I think we do need really clear value or kind of transactional chains and logging so that we always know what happened even if we can't always explain why a large language model did something. You know? It makes it easier to revert it and to solve.
  
  The other thing I would say here is that I you know, there's something I I keep coming back to as we roll these systems out into more mission critical or sensitive areas, which is asking who bears the burden here if the system goes wrong. Right? Now I think that exposes a lot of inequities and a lot of challenges in current tools. Right?
  
  It's something like an automated grading assistant is used by an instructor, but when it goes wrong, the student bears the burden. You know? And the same goes for AI detection tools today. I think that inequity creates real problems.
  
  And so I I like that question because it asks, it helps steer me towards problems I ought to be focusing on and things I I may need to wait to address.
  
  I don't know, Mike.
  
  What are your thoughts?
  
  I agree with you. You know, in terms of the control and auditing, know, obviously, the point of transaction is where we we have a choke point. We can capture things, we can record them and potentially have the ability to roll back. Then getting to your other point about the impact that an agent's decision could have on a user, yeah, it's really on us as the agent builders to make sure that we have the the right safeguards in place and the right controls for the agent. And I think you know some really good guidelines and principles if you look at say Microsoft's principles of responsible AI. I think they're really good guides on how to implement systems like that improperly and effectively with fairness, with trust, transparency. I think it's a a great guide to look to.
  
  Something I have seen here work in some classrooms is that if you want to experiment in your institution or in your class with these higher risk use cases is, you know, not using them to replace what a human's doing today but to do more of it. So I don't know that today, I would use AI to grade an assignment that I'm already giving students, but I may use it to provide more opportunities for feedback to students. Right? So instead of having one assignment, now maybe I have two, and I can candidly tell students this second assignment is graded by AI. And it's really think of it as an opportunity for more kind of formative assessment or an opportunity to test yourself. And I think that provides students more opportunities for learning, but it doesn't replace my responsibility as an instructor, or I think introduce undue risk.
  
  Yeah. Absolutely.
  
  That's part of the transparency. You know, if something is being graded by AI, students should absolutely know it.
  
  But even from Microsoft's perspective, we're not even going there. We're not automating grading, but we will provide the educator assistance. We will help draft feedback for them that they could choose to modify and pass on if they wanted to. We will will help make them more efficient. Having the human there, I think, is is very important. And, you know, if it's not, yeah, you you definitely need that transparency so that, you know, somebody can know to challenge it, to question the results.
  
  So, know, Mike, I think to move this conversation, I guess, to continue it, I have a question for you. Because as I think about software architecture, just saying it works on my machine or, like, oh, I I used it, you know, and I I didn't have a problem is something that, you know, engineers famously already do, but we know that's not enough. And I think that becomes exacerbated as we look at systems like this because it may work once with a set of inputs, but then given those same inputs, it may not work the second time. Right?
  
  We we've got this nondeterminism now. And so it's not just about kind of asserting that it worked once now. I I need to know that it works most of the time, and I can catch where it fails and how it fails and how it drifts. Like, what are your thoughts about quality assurance with large language models, and how should we all be thinking about evaluating these systems in education and ensuring their correctness?
  
  So first, I think we've always had nondeterministic elements in our system. Right? We just called them users. Humans are about as nondeterministic as you can get, And they're going to make mistakes.
  
  Yeah, they make mistakes when grading, they'll make mistakes when assessing. So this is no different. So I think it could be interesting to think about how have we treated quality control of human workflows in the past. You know, things auditing or peer review.
  
  Maybe if you have a sensitive workflow, you may want peer agents to review the work of your other agent. Maybe you wanna go so far as having them using different models so they don't have the same biases. You know I think we need to really look at how we've handled things in the past and apply them to this new paradigm.
  
  That makes a lot of sense that I even just think about some of the research you've done Zach too. You've done research in this way where you've compared models, compared approaches, compared human to models, right, even some of the rubric research.
  
  Yeah. Absolutely. You know? And I I think it is you know, it's interesting because there certainly are places where humans prefer the outputs of models because they are more consistent maybe than than a group of humans, and then there are places where humans are are clear winners.
  
  So, yeah, that that's such an interesting thing. Right? That, yeah, we were the original nondeterministic system. I I love that.
  
  That's a great point. I love that.
  
  I love that. Well, let me ask a last question for both of you. You've got a magic wand and you get to set a single standard or design principle or some concept or theory to this whole field so that we can see the benefit of AI and education realized. Okay. So what would that be?
  
  And, like, what would hold how would that happen? What's gonna hold that together so we can see that come to fruition over the next five to ten years? And maybe I'll start with you, Zach, and then I'll we'll we'll have Mike share sort of the closing inspiration.
  
  Perfect. Yeah. I I think I may know Mike's answer to this question, so I'm gonna go in a different direction. I'll say, I spoke about this a little bit earlier.
  
  I think the concept of human in the loop, it is well intentioned. I think that as systems continue to move towards more agenic flows and more autonomy, I think the idea of human in the loop as we all understood it three years ago is fundamentally flawed. So, you know, as I think of standards and design principles, I think it's building regulatory systems and building kind of design frameworks that keep humans in control no matter where that action is is taking place. And I I think, you know, the the challenge there, but the opportunity is that I I don't know that starts with technology.
  
  Right? I think that starts with with educators and students asking how they wanna teach and how they wanna learn, and then holding technology providers accountable to delivering those experiences.
  
  That's great.
  
  K. Magic wan time, Mike.
  
  Oh, we'll see if Zach was right about his guess. But for me, it is treat agents as users. Right? So when you think of accessibility in your application, think of accessibility to GenTick users.
  
  What do they need? When you think of everything else, you know, security and auditing, all those things, again, think of what are your needs when agents are interacting with your platform. I don't know what holds us all together over the next you know, ten years or so. In a world of exponential progress, I can't even imagine what the world's going to look like at the end of it.
  
  But I do know that this is going to create some very interesting things that we'll need to tackle in the next few years. When we think of AI, I don't think we should be thinking of it as another layer in our enterprise platform stack. We need to be thinking about outside actors as well. You know, students, educators bringing their own personal assistants that are going to be there with them throughout their life.
  
  And this is going to apply in work too. And I don't think we've driven any really substantive discussions of what that means. Is the interface between say consumer AI and enterprise AI? I think we need to start talking about that now because it is coming very, very quickly.
  
  And I mean, honestly, it's already here. It's just being worked around by copy paste and all sorts of things that are probably violating privacy laws and other Awesome.
  
  Well, Mike, Zach, this is exactly the deep dive we had hoped for. Thanks for getting in the weeds with us. Again, you know, this is Melissa and I always talk about the podcast is, you know, even if nobody listened, it's selfishly, we get smarter by listening to you all.
  
  So thank you so much.
  
  Yeah. Yeah.
  
  Thank you all so much.
  
  Thank you both so much.
  
  Thank you.
  
  Don't forget to like, subscribe, and drop us a review on your favorite podcast players so you don't miss an episode. If you have a topic you'd like us to explore more, please email us at InstructureCast at Instructure dot com, or you can drop us a line on any of the socials. You can find more contact info in the show notes. Thanks for listening, and we'll catch you on the next episode of Educast three thousand.