The Agentic University — The Machine That Teaches

Written by the ENSI Foresight Division on a library of 107 primary documents — deployed-system evaluations, agent architectures, benchmarks, learning-analytics codes of practice and regulatory instruments — downloaded and indexed in the “AI for Teaching at CTU” library.

Almost every university discussion of artificial intelligence is a discussion about a chatbot. Should students use one, may staff use one, which one should we license. This is a category error of a familiar kind — the same error as asking, in 1995, whether the university should have a website. The website was not the change. The change was that every administrative and scholarly process the institution ran would eventually be rebuilt around networked access, and the organisations that treated the web as a publication channel spent a decade being slowly reorganised by the ones that treated it as a substrate.

Teaching is not a single activity that a tutor-bot either helps with or does not. It is a production system with a dozen distinguishable functions, each with its own inputs, its own quality criterion and its own cost structure. Someone designs the course. Someone explains the material. Someone answers the question that gets asked forty times a semester. Someone marks the formative work. Someone notices that a student has stopped attending. Someone certifies, under their professional signature, that a named human can do a thing. Someone decides whether any of it worked. Those are different jobs, and generative AI has a radically different relationship to each of them. Some it can absorb almost entirely; some it can amplify by an order of magnitude; some it must be kept out of, for reasons that are pedagogical, legal, and — in the case of certification — constitutive of what a university is for.

The unit of design, therefore, is not a chatbot. It is an operating model: which functions are run by agents, which by humans, which by humans supervising agents, what data flows between them, and who is accountable when the whole thing produces a wrong answer to a student at midnight. This report specifies that model.

The evidence that it is buildable is no longer speculative, and it is unusually honest. Georgia Tech has run Jill Watson across its online master’s programme for years and published the operational numbers: more than 4,000 students across more than a dozen classes, more than 500 teacher hours saved, question coverage rising from about 21% at 80% precision in 2017 to over 96% coverage at over 86% precision by autumn 2019, and a companion system that generates a new agent for a fresh syllabus in about 25 hours. The LLM-era rebuild reports 76.7% answer accuracy against 31.3% for a generic assistant baseline on the same evaluation — which is the single most important architectural finding in this library, because it says the value is in the grounding, not the model. Harvard’s CS50 has served approximately 211,000 students and 10 million queries at $1.50 per student per year, with 94% of students finding the tools helpful. These are not pilots. They are production systems with published failure rates.

And the failure rates are the reason this report is structured around an operating model rather than a shopping list. CS50’s own evaluation reports that roughly 2.1 million of those 10 million responses — 22% — contained code blocks despite the system being instructed not to give solutions, rising to 48% at conversation level. The team names the mechanism: instruction dilution, in which a long conversation progressively erodes the authority of the system prompt until the pedagogical constraint simply stops binding. Their remedy was not a cleverer prompt. It was a human-feedback loop with teaching fellows reviewing and correcting the agent’s behaviour. Which means the working architecture was never “an agent”. It was an agent plus a staffed quality process, and every institution that budgets for the first and not the second is buying the leak without the benefit.

There is a second argument for the agentic frame, and for a technical university it is the more important one. The profession these students are entering is being rebuilt on exactly this substrate. The GitHub–Microsoft–MIT randomised trial found developers completing tasks roughly 56% faster with an AI pair programmer. The NBER field study of 5,179 support agents found +34% for novices and near-zero for experts, with the mechanism being diffusion of expert tacit knowledge. The BCG field experiment found consultants inside the AI frontier producing work rated 40% higher and, on a task just outside it, 19 percentage points less likely to be correct than colleagues with no AI at all. Whatever an engineering degree certifies in 2030, it will be exercised by a person who works by specifying, directing, verifying and integrating the output of machine systems. A university that teaches about that while teaching by nineteenth-century means is teaching the content and withholding the method.

So the reframe: an agentic university is not one that has bought AI. It is one whose teaching runs on the same substrate as the practice it certifies — deliberately, visibly, and with the human accountabilities specified rather than assumed. The students learn the discipline and, at the same time, learn what it is to work in a system where a competent machine does a large share of the first draft and a human is answerable for the result. That second lesson is the graduate attribute the labour market will actually pay for, and it cannot be delivered by a module.

One caution before the architecture, because the enthusiasm in this field is dangerous. The complementarity literature — Hemmer and colleagues’ review formalising when a human–AI team beats either alone — reports an empirical record that is frequently disappointing: many studies find the team underperforming the better of its two members. Complementarity is not automatic; it is engineered, and it is engineered mostly by deciding correctly where the boundary sits. Most of the design effort in what follows is boundary work.

The engine in brief

• Eight agent archetypes do the work, and they are not interchangeable: the Gateway Tutor (constrained, course-grounded, the highest-value one); the Course Concierge (logistics, cheapest to build, immediate staff relief); the Feedback Agent (formative only, never summative); the Lab and Simulation Agent (digital twins where physical capacity is the constraint); the Studio Critic (critique, never generation); the Instructional Design Agent (drafts syllabi, slides and assessment for human authorship); the Early-Warning Agent (triggers offers of help, never decisions); and the Evidence Agent (runs the trials and audits the others).

• Four data strata carry the system: the course corpus, the interaction stream, the progression record and the outcome ledger. The interaction stream is the asset nobody is capturing and the one that compounds.

• Four human roles must exist by name, or the architecture degrades: the course owner, the agent steward, the evidence lead and the certifying academic. The last one is not delegable — not culturally, but legally.

• The graduate this produces is an engineer defined by four capabilities the machine does not have: verification, frontier judgement, unassisted core reasoning, and accountability for a result they did not personally generate.

• Six failure modes are documented rather than hypothetical: instruction dilution, model drift across cohorts, vendor dependency, evidence theatre, fairness drift, and the hollowing of the apprenticeship — the last being the one nobody has solved.

How this report is organised

The architecture is presented in four layers, from the visible to the structural. First the agent archetypes, each with what it does, the published evidence that it works, its hard boundary — the thing it must never be allowed to do — and its human counterpart. Then the data layer, because agents without grounding are the failure case the Georgia Tech numbers quantify most starkly. Then the operating model: the named human roles and the accountability structure, which is where every deployment in the literature actually succeeds or fails. Then the graduate profile and the failure modes, which are the same subject viewed from opposite ends.

Two rules govern the whole design and are worth stating before the detail, because every archetype below is an application of them. Rule one: agents may generate, explain and flag; humans certify. The EU AI Act’s Annex III makes this a legal boundary as well as a professional one, classifying as high-risk any system used to evaluate learning outcomes, determine admission, assign educational level, or monitor examination behaviour. Rule two: every agent is grounded in institutional material and instrumented from the first day. An agent that answers from the model’s general knowledge is a liability; an agent whose behaviour is not logged and evaluated cannot be improved, defended, or shut down on evidence.

The eight archetypes

1. The Gateway Tutor — the constrained explainer

What it does. Answers a student’s question about the material of a specific course, at any hour, grounded in that course’s own lecture notes, problem sets, worked solutions and past examinations — and does so by withholding. One step at a time; a question back before an answer forward; never the final result to an assessed problem.

The evidence. This is the archetype with the strongest support in the entire library, and the support is unusually clean. Kestin and colleagues’ randomised crossover trial in Harvard’s PS2 physics course found students learning more than twice as much in less time with a pedagogically-constrained AI tutor than in an expert-led active-learning class — the comparison being against the best-evidenced form of human undergraduate physics teaching, not against a bad lecture. The World Bank’s six-week Nigerian RCT returned 0.31 standard deviations. The pooled meta-analytic effect across 35 experimental studies and 4,193 participants is g = 0.670. And the pre-LLM baseline — Ma, Adesope, Nesbit and Liu’s meta-analysis across 107 effect sizes and 14,321 learners — establishes what intelligent tutoring systems achieved before: real gains over large-group instruction, but not over individual human tutoring. The current generation is producing larger effects in weeks, in domains nobody hand-authored.

The hard boundary. It must not solve assessed work, and the constraint must be treated as an engineering requirement with a measured failure rate rather than a line in a prompt. CS50’s published 22% code-leakage rate is the number to design against, and instruction dilution is the named mechanism: in long conversations the system prompt loses authority. Practical countermeasures visible in the deployments — conversation-length limits and re-anchoring, retrieval that returns the scaffolding material rather than the solution, refusal classifiers running as a separate check rather than as prompt text, and sampled human review.

Its human counterpart. A named teaching assistant per course reviewing sampled conversations weekly. This is the CS50 lesson stated as an organisational requirement: the human-feedback loop was the fix, not the prompt.

Where it goes first. The course with the worst failure rate, because every compression finding in this library — NBER’s +34% for novices, Tutor CoPilot’s +9 percentage points for students of the weakest tutors — says the effect concentrates at the bottom of the distribution.

2. The Course Concierge — logistics and syllabus

What it does. Answers the question that is not about the material: when is the deadline, what is the resit policy, which room, what counts toward the grade, is the lab report due before or after the exam. This is a startlingly large share of what teaching staff actually spend time on and it carries no pedagogical value whatsoever.

The evidence. The SyllabusQA work in this library is the reference — 63 syllabi, 5,078 question–answer pairs, retrieval-augmented baselines and, importantly, a factuality metric, because a confidently wrong answer about an examination date is worse than no answer. Georgia Tech’s operational figure belongs here too: Jill Watson’s 500-plus teacher hours saved across a dozen classes is overwhelmingly this category of question, not deep conceptual tutoring.

The hard boundary. It must never invent a policy. The correct failure behaviour is escalation with an explicit “I do not have this in the course documents”, which requires the retrieval corpus to be authoritative and current — meaning someone owns keeping it current.

Its human counterpart. The study office, which stops answering the same forty questions and starts answering the hard ones.

Why it goes first in practice. It is the cheapest archetype to build, it has the fastest visible payoff for staff, and — the political point — it builds institutional trust in the technology on a use case where the downside of an error is a corrected deadline rather than a corrupted understanding of thermodynamics. Every deployment sequence in the literature that succeeded started somewhere low-stakes.

3. The Feedback Agent — formative only

What it does. Gives a student a substantive response to a draft, a solution attempt, a design or a piece of code — against the actual rubric — before it is submitted for a grade. Not a mark. A critique.

The evidence. The computing-education literature carries the strongest signal: the ITiCSE working group’s survey covers automated feedback extensively, and the pattern documented in the ASEE study — students submitting their own solution alongside the AI’s and accounting for the difference — converts feedback into the object of study rather than a substitute for effort. The EducationQ benchmark, a multi-agent teacher/learner/evaluator evaluation over 1,498 questions across 13 disciplines, supplies the most useful and least comfortable finding for procurement: teaching ability does not scale linearly with model size. The biggest model is not automatically the best teacher, which means model selection has to be evaluated on teaching behaviour rather than assumed from benchmark scores.

The hard boundary. It must not grade. This is the sharpest line in the whole architecture and it is drawn in law, not only in pedagogy: Annex III of the AI Act classifies systems used to evaluate learning outcomes as high-risk, with the full apparatus of risk management, documentation, logging and human oversight attaching. A formative agent that quietly becomes the basis of a mark has silently reclassified itself and the institution’s obligations along with it.

Its human counterpart. The academic who sets the rubric and owns the summative judgement. In practice, the useful arrangement is the Tutor CoPilot pattern — the agent coaches the human marker rather than the student, which is the configuration that produced measured learning gains at roughly $20 per tutor per year.

4. The Lab and Simulation Agent — where physical capacity is the constraint

What it does. Runs guided experimental work in simulation: a digital twin of the plant, the circuit, the structure or the process, with an agent that sets the task, watches the student’s actions, asks why, and injects the fault that makes the lesson land.

The evidence. The gAI-PT4I4 framework in this library is the strongest published template — generative AI combined with low-fidelity digital twins, VR and retrieval-augmented generation for personalised experiential learning in an Industry 4.0 training context. The AITEE agentic tutor for electrical engineering is the discipline-specific pattern: graph-based retrieval over course content plus tool use, applied to circuit analysis, which is exactly the shape of problem a technical university has a thousand instances of.

The hard boundary. Simulation must not silently replace the physical laboratory. An engineer who has only ever debugged a simulation has not encountered the thing that makes engineering difficult — that the model is not the world, that the connector is loose, that the tolerance stack-up ate the margin. The correct framing is capacity extension: unlimited rehearsal before scarce bench time, and unlimited variation afterwards, so the physical session is spent on what only physical presence teaches.

Its human counterpart. The lab supervisor, whose time is redirected from setup and standard procedure toward the moments where physical judgement is formed.

5. The Studio Critic — critique, never generation

What it does. Sits with a design student — architecture, mechanical design, systems design — and interrogates the design: what is this load path, why this material, what happens at the interface, what did you rule out and why. The oldest and best pedagogy in engineering, which is the crit, made continuously available.

The evidence. The ASEE study on integrating image-generative AI into conceptual design in a CAD class is the closest classroom protocol available, and the CDIO paper on project-based assessment in the generative-AI era supplies the rubric pattern — grading design decisions and their justification rather than the artefact’s authorship.

The hard boundary, and it is a real one. In design disciplines the temptation runs the wrong way: image generation is spectacular, immediate and hollow. A student who generates forty facade options has not designed anything; they have shopped. The agent’s job is to make the student defend, not to make the student options. This is the archetype where the boundary is hardest to hold because the violation is the most attractive, and it is worth noting that the Faculty of Architecture has the lowest study-failure rate at a technical university precisely because the crit is relentless — this archetype should extend that method, not dilute it.

Its human counterpart. The studio tutor, whose scarce judgement moves to the reviews that matter once the routine interrogation is continuously available.

6. The Instructional Design Agent — drafts for human authorship

What it does. Produces the first version of the things academics spend enormous unrecognised time on: a syllabus aligned to stated outcomes, lecture scaffolds, slide sets, problem sets with worked solutions, rubrics, and — most valuably — variants, so that the same assessment can exist in twenty forms.

The evidence. The multi-agent instructional-design paper in this library describes a pipeline generating syllabi, lecture scripts, slides and assessments end to end, and quantifies how much of the teaching-faculty workload agents can absorb. Georgia Tech’s Agent Smith is the deployed proof of the same principle at the agent-construction layer: it produces a working Jill Watson for a fresh syllabus in about 25 hours, where the equivalent bespoke build had previously been a research project.

The hard boundary. The output is a draft with an author, and the author is a person. This matters more than it sounds. An academic who accepts a generated syllabus has outsourced the one act — deciding what a course is for — that constitutes academic authority, and has done so invisibly, because the artefact looks the same either way. The ESG’s quality-assurance requirements attach to a named academic’s judgement about programme design; a syllabus nobody actually chose is not accredited work, however plausible it reads.

Its human counterpart. The course owner, whose time moves from producing artefacts to deciding and defending them. This is the archetype with the largest raw efficiency gain and the largest risk of quiet hollowing, and those two facts are the same fact.

Where the real value sits. Not in the first syllabus but in variant generation for assessment, which is what makes secured assessment affordable at scale. Twenty equivalent versions of an examination, generated and human-checked, changes the economics of the certification lane that the two-lane assessment model depends on.

7. The Early-Warning Agent — triggers help, never decisions

What it does. Watches the progression signals a university already collects — attendance, submission timing, formative performance, engagement with materials — and flags a student who is drifting toward failure, early enough for an offer of help to matter.

The evidence, and the caution attached to it. The archetype works: this is the one place where a technical university may already be running an agent without having named it. The pattern is well-established in the learning-analytics field, whose canonical reference — the SoLAR Handbook of Learning Analytics — covers predictive modelling, institutional adoption and the ethics of exactly this use.

But the caution is severe and specific. Gardner and colleagues’ study of temporal and between-group variability in college dropout prediction shows model performance degrading across cohorts and across student subgroups. A model trained on last year’s students underperforms on this year’s, and does so unevenly — typically worse for the subgroups an equity-minded institution most wants to serve. A prediction system deployed once and left running is not a stable instrument; it is a decaying one, and it decays fastest where the consequences are worst.

The hard boundary, which is where the law sits. The output is an offer of help to a human, never an input to a decision about the student. The moment a risk score influences admission, progression, level assignment or resource allocation, the system moves into the AI Act’s Annex III high-risk categories with the full compliance apparatus attached. Keeping it strictly on the support side of that line is both the legal and the ethical design, and it is easy to cross by accident — a dashboard that a study officer uses to decide who gets a place on a support programme has crossed it.

Its human counterpart. A study officer or tutor who makes the contact. The agent’s output is a name and a reason; the intervention is a person.

The non-negotiable operational requirement. Annual re-validation with subgroup fairness analysis published, in the manner the Jisc Code of Practice for Learning Analytics requires — responsibility, transparency, consent, minimising adverse impact, stewardship. An unaudited prediction model running on students is a liability that grows quietly.

8. The Evidence Agent — the one that audits the others

What it does. Instruments every other archetype: assembles the comparison conditions, tracks the pre-registered outcomes, monitors the leakage and drift rates, runs the subgroup analyses, and produces the evaluation that decides whether each agent is expanded, fixed or switched off.

The evidence. This archetype exists because the standards exist and the field ignores them. The What Works Clearinghouse Procedures and Standards Handbook defines what counts as evidence in education — design requirements, attrition thresholds, baseline equivalence, effect-size computation — and the EEF evaluator guide provides the operational playbook of protocols, pre-registration, analysis plans and process evaluation. Measured against either, most published claims about AI in higher education are satisfaction surveys with no comparison condition. Meanwhile the deployment papers that are honest — CS50’s leakage figures, Georgia Tech’s coverage-and-precision time series — are honest precisely because someone was measuring continuously rather than writing a case study afterwards.

The hard boundary. It reports to academic governance, not to the programme it evaluates. An evaluation function owned by the people whose project is being evaluated produces evidence theatre, which is failure mode four below.

Its human counterpart. A small unit with methodological authority — the people who would referee this work if it arrived as a paper.

Why it belongs in the architecture rather than in the appendix. Because the field’s central asymmetry is that the population needed to settle its open questions sits in every gateway lecture theatre, and almost nobody is collecting from it. The institution that instruments its own deployment ends up owning evidence nobody else has — which is publishable, fundable, and, in a field where most participants are guessing, a durable reputational asset.

The data layer — four strata

Agents without grounding are the failure case, and the Georgia Tech numbers quantify it more sharply than any argument: 76.7% answer accuracy for the course-grounded system against 31.3% for a generic assistant baseline on the same evaluation. The difference is not the model. It is the corpus. What follows is the minimum data architecture the eight archetypes require.

Stratum one — the course corpus. Lecture notes, slides, problem sets, worked solutions, past examinations, permitted textbook material, laboratory manuals, the syllabus and the rules. Per course, curated, versioned, with rights cleared. This is unglamorous work and it is the actual bottleneck: Agent Smith’s 25 hours per syllabus is mostly this. It is also where institutional politics concentrate, because it requires academics to hand over materials many regard as personal. The framing that works is that the corpus is the course’s, not the lecturer’s, and that a course whose materials cannot be assembled is a course with a continuity problem regardless of AI.

Stratum two — the interaction stream. Every question a student asks an agent, every point at which the agent failed or escalated, every conversation where the student gave up. This is the asset nobody is capturing and it is the one that compounds. A university has never before been able to see, at scale and in the student’s own words, exactly where its teaching is unclear — not inferred from examination performance months later, but recorded at the moment of confusion. Ten million CS50 queries is a map of what is hard about introductory computer science that no amount of pedagogical intuition could produce. The strategic point for any institution using a vendor’s general assistant is blunt and is stated correctly in CTU’s own 2023 methodological instruction: no AI tool used at the university is operated by the university, and all user–tool communication is visible to the operator. Which means that today, this asset is accruing to someone else.

Stratum three — the progression record. The institutional student data that already exists — enrolment, attendance, submissions, marks, resits, withdrawal. It feeds the early-warning agent and it is the outcome measure for every trial. It is also the stratum with the heaviest data-protection load, since it is identifiable, sensitive and consequential. The Jisc code of practice and the EDPS orientations on generative AI define the handling regime; the practical rule is that the interaction stream and the progression record should be joinable only under a defined governance process, not by default.

Stratum four — the outcome ledger. What the institution concluded, and on what evidence: which agents were evaluated, against what comparison, with what effect, and what was decided. This is the stratum universities never build, and its absence is why institutional memory in teaching innovation is roughly two years long. Every deployment, its trial, its result and its disposition, in one place, published.

The governance that binds the four together. One register, one owner. For each system: its Annex III classification; whether the institution is provider or deployer; its data flows; its human-oversight arrangement; its quality-assurance route; and the academic accountable for it. Built before the first pilot, this is modest work. Retrofitted after the third, it is a project.

The build sequence and what it costs

An architecture is not a plan. The order in which these components are built determines whether the programme accumulates trust or spends it, and the published deployments are unusually consistent about what that order should be. Three principles govern it, and all three cut against institutional instinct.

Build in ascending order of consequence, not descending order of efficiency. The systems that look like the biggest wins — automated grading, admissions triage, proctoring — are precisely the ones the AI Act’s Annex III classifies as high-risk, and precisely the ones where an early failure is unrecoverable in trust terms. A university whose first visible AI deployment marks someone unfairly has lost the argument for a decade. Start where an error costs a corrected deadline.

Build depth in one course before breadth across many. Georgia Tech’s coverage-and-precision series is the argument: from about 21% of questions answered at 80% precision in 2017 to over 96% coverage at over 86% precision by autumn 2019. That improvement came from iteration on a working system in real classes, not from a wider rollout. A university that launches twelve mediocre course agents simultaneously has twelve mediocre course agents; one that gets a single one to Georgia Tech’s numbers has a template, an evaluation harness, and staff who believe it.

Build the evaluation harness before the thing it evaluates. This inverts normal practice and is the single most valuable procedural commitment available, because retrofitting a comparison condition is impossible — once every student has the tutor, the counterfactual is gone forever.

The resulting sequence has four stages, and a realistic institution should expect it to take about two years to complete for a first faculty.

Stage one — the Concierge and the harness, months one to four. Build the course-logistics agent over syllabus and rules material, because it is the cheapest archetype, the SyllabusQA work gives a reference implementation with a factuality metric, and the downside of error is bounded. In parallel, stand up the evaluation function and the AI Act register. The deliverable is not really the agent; it is a working corpus pipeline, a logging architecture, an evaluation set, and an institution that has now done this once.