Persistent lessons in human-centered automation

What a 1980s photocopier study reveals about the Agentic UX era

A mixed-media recreation of a field observation at Bossa Nova Robotics, where a shopper attempts to scan a container of blueberries against an autonomous inventory robot.

During my time as a User Researcher at Bossa Nova Robotics, I conducted multiple studies in grocery store aisles watching our inventory-assessing robot do its job. My objective was to identify moments when the robot had to pause because of obstructions or human interference. The goal was to find ways for the robot to better navigate the space and indicate what it was doing to nearby humans and, in doing so, optimize its efficiency.

One afternoon, I watched a woman walk up to the ~300-pound machine. It stopped. She held a container of blueberries up to its light panel. Nothing happened. She then lifted the blueberries to the top of its lidar stack. Still nothing. Frustrated, she muttered, “This thing is so stupid,” and walked away.

The irony was not lost on me. That robot was filled with high-end cameras and on-board compute running perception algorithms in real-time. The machine was cutting-edge when it came to artificial intelligence. But when it came to coordinating and understanding human intention, the woman was right, it was pretty stupid. The lesson I learned that day was that no matter how much machine intelligence is baked into a system, if it doesn’t account for the unpredictability of human behavior it will still be dumb in human intelligence.

Today, we are seeing an incredible shift in HCI as LLMs give us new ways of presenting natural-seeming interaction patterns. Looking back to the birth of HCI reveals vital lessons we must apply to modern AI interaction design.

The birth of HCI, and the lessons we learned

In the 1970s, the Computer Science Laboratory at Xerox PARC was the absolute mecca of innovation. They gave us the graphical user interface, the laser printer, and the first modern personal computers. They were pursuing a radical idea: that computing could transcend massive mainframes and become accessible, responsive tools for individual users.

Within Xerox PARC, a group affectionately known as the “artificial intelligentsia” drove a prevailing belief. They thought making a machine smarter was the exact same thing as making it useful.

The Xerox PARC Computer Science Laboratory “beanbag” conference room, c. 1970s. Image courtesy of the Computer History Museum.

However, there was a tension between system-centric and human-centric thought in those days. The system-centric camp believed human action was predictable in some way, and if an AI planning model was smart enough, it would be able to assist humans in accomplishing a task. Researchers Thomas Moran, Stuart Card, and Allen Newell even wrote a foundational book called The Psychology of Human-Computer Interaction, which modeled humans as information processors perfectly analogous to computers.

Anthropologist Lucy Suchman challenged this assumption in her famous “man vs. machine” study. She set up a camera and simply watched two users struggle to make double-sided copies of a large stack of papers with a new Xerox photocopier. When the copier jammed, it instructed the users to reload all the pages in order. The users, however, knew they were nearly done with their task and only needed to copy the last few pages again. They struggled in vain for several minutes, and became so frustrated with the machine’s rigidity that they ultimately cussed out the machine. These users weren’t just any users, however. They were some of the premier experts in computing at the time. The man vs. machine study became cemented in history as the most famous of all usability studies.

A scene from Lucy Suchman’s foundational “man vs. machine” usability study at Xerox PARC. Screenshot from YouTube

Through this study, Suchman showed us that human work is not a rigid plan. It is moment-to-moment improvisation. Human behavior can only be understood in relation to real-world situations. The metaphor she used to describe this was that human action is like navigating a canoe through rapids. You start with a plan, but you make moment to moment decisions while you are in the experience.

A few years later, technologists at PARC decided to tackle a new challenge: designing technologies to facilitate human-to-human collaboration. Researchers like Deborah Tatar (who would later become my academic advisor in graduate school) observed early collaborative systems like Colab and Cognoter. These systems relied on a rigid ‘parcel-post’ model of communication. They treated human interaction like discrete packages being mailed back and forth. But in Tatar’s evaluation of the system, she found that real communication is a messy dance of shared gazes, subtle nods, and constant adjustments based on our partner’s reactions. Tatar and others learned that when machines enforce rigid logic, human-to-human collaboration collapses, and they are ultimately unsuccessful, and unsatisfied, with such a tool.

At the end of this era, we saw the rise of HCI and CSCW as new fields of research within computing. And we walked away with a clear set of requirements for successful human-centric technology design:

• Action is Situated: Human behavior cannot be predicted by a rigid script. It must always be understood in relation to the messy, real-world context the user is currently experiencing.
• Account for the Machine Sensory Gap: Machines only “know” the limited slice of reality that their sensors and training data tell them. Humans know the actual world. Systems must be designed to bridge that gap.
• Plans are Resources, Not Blueprints: A plan is just a rough guide. Real work requires moment-to-moment improvisation to overcome unexpected hurdles.
• Interaction is Improvised: Communication is a coordinated, back-and-forth dance of shared meaning, not a simple delivery of information bits from point A to point B.
• Person-Centered Design: Systems must be designed to increase human competence and pride in work, rather than forcing humans to adapt to the limitations of the machine.

The power of context gives rise to AIX

Back then, traditional systems, AI powered or otherwise, failed those requirements. They lacked the context and flexibility to support situated action or improvised interaction. As a result, for the last 50 years UX Design has developed principles and design patterns that focus on making systems look more like discrete tools in a tool box with obvious affordances rather than monolithic be-all-end-all machines. The systems are rigid, but the design allows you to understand their utility and use them together to accomplish a number of tasks.

But now, LLMs have us rethinking UX design entirely. Today, large language models are passing some of those requirements with flying colors. AI has moved from rigid, deterministic scripts to probabilistic, flexible reasoning. LLMs possess massive context, memory, and the capacity to repair misunderstandings interactively. Instead of treating communication like a rigid ‘parcel-post’ system, LLMs can engage in the messy dance of human conversation.

This feels like a reunion between AI and human-centered design. We can coordinate shared meaning with the machine. In fact, researchers are already identifying an entirely new interaction paradigm: the Agent interaction model (AIX). Unlike traditional graphical user interfaces that rely on explicit, step-by-step commands, AIX allows humans to state an intent and let the system proactively reason through the execution. It pushes HCI forward in thrilling new ways, allowing us to collaborate with machines almost like we would with a colleague.

The illusion of complete intelligence: where LLMs still stumble

The conversational fluency of modern LLMs creates a powerful illusion. When a system communicates in polished, natural prose, our brains naturally assume it possesses general human competence. We project common sense, situational awareness, and real-world understanding onto the machine.

This assumption is dangerous. While LLMs solve the rigid syntax problem of early computing, they do not automatically solve the five requirements of human-centered design.

The machine sensory gap has not disappeared; it has simply moved behind a curtain of sophisticated language. In their landmark paper On the Dangers of Stochastic Parrots, Emily Bender, Timnit Gebru, and their co-authors pointed out that large language models are systems for stitching together linguistic forms based on statistical likelihood, without any underlying model of intent, meaning, or physical reality. An LLM operates strictly on statistical associations across textual tokens. It has no physical embodiment, no lived presence in the physical room, and no direct perception of the user’s unstated emotional state, physical distractions, or immediate workspace constraints. When an edge case occurs, the model cannot look around the room to see what went wrong. It can only generate the next most probable sequence of words.

This creates the problem of jagged capabilities. An AI agent might synthesize hundreds of pages of research in seconds, and in the very next step hallucinate a non-existent corporate policy or misinterpret a safety constraint. Because the output format remains confident and articulate throughout, users struggle to tell where the model’s true competence ends and its blind speculation begins.

The interface paradigms we use today exacerbate this issue. Traditional graphical user interfaces (GUIs) rely on visible menus, buttons, and state indicators that show users what actions are possible. Pure conversational interfaces (Language User Interfaces, or LUIs) strip away these visible affordances. Users face an empty text box, left to guess what the system can do, what context it actually retains, and where its failure boundaries lie.

Trust does not happen in a 1:1 vacuum

To build systems people can actually rely on, we have to look closely at how trust operates in the real world. Trust is not a static score or an isolated 1:1 transaction between a human and a computer. It is an ongoing negotiation that happens across networks of people working together.

When I worked on Explainable AI (XAI) services for Google Cloud, we evaluated user needs across high-stakes domains like digital pathology and clinical decision support. We observed that an AI model never operates alone in an exam room. It sits inside a complex web of relationships involving pathologists, oncologists, patients, data scientists, and regulatory auditors.

Each person in that network approaches the system with entirely different sensemaking goals:

• The Data Scientist needs technical interpretability: feature attributions, loss curves, and data drift metrics to verify the pipeline.
• The Pathologist needs clinical rationale: why the model flagged a specific tissue cluster as malignant, which cellular patterns drove the prediction, and what differential diagnoses were considered.
• The Oncologist and Patient need collaborative clarity: what the findings mean for treatment options, how certain the prognosis is, and what trade-offs exist.
• The Auditor needs provenance: an unalterable trail showing data lineage, model versioning, and compliance with clinical standards.

Different stakeholders have distinct explanatory needs, and users such as doctors and patients will coordinate action and make meaning together through the system.

These stakeholders do not just consume model outputs individually. They use the tool to coordinate decisions with one another. The oncologist discusses the findings with the patient, pointing to visual evidence on the screen to build a shared treatment plan.

Current LLM interfaces almost completely ignore this multi-stakeholder reality. They operate as private, siloed chatbots designed for a single user staring at a prompt box. We lack interaction patterns that support group sensemaking, where multiple humans can interrogate, annotate, and co-evaluate AI reasoning together.

New interaction patterns for the intervention handshake

If we want agentic systems to succeed where previous generations of automation failed, we must design for human sensemaking from the start. We need concrete interaction patterns that help people calibrate their trust, understand system state, and step in to steer the machine without friction.

Here is a list of key design patterns for the intervention handshake, and I’ll cover each in more detail below:

1. Calibrated Uncertainty & Grounded Provenance: Visually distinguish facts from speculative leaps; make confidence inspectable and anchor claims in source evidence.
2. Continuous Mixed-Initiative Steering: Enable lightweight course correction (steerable scratchpads, inline checkpoints) without resetting state.
3. Shared Mental Models: Provide collaborative surfaces where multiple stakeholders can co-interrogate and guide AI reasoning together.
4. Progressive Disclosure: Expose underlying decision trees, retrieved chunks, and alternative logic on demand during high-stakes moments.

1. Calibrated uncertainty and grounded provenance

AI systems should never present speculative inferences with the same authoritative tone they use for verified facts. Interfaces must make uncertainty visible. When an agent synthesizes information or suggests an action, it should visually distinguish between high-confidence retrieval and probabilistic extrapolation. Surfacing underlying evidence, inline citations, and inspectable source traces allows users to verify claims quickly without having to double-check every single sentence from scratch.

2. The intervention handshake (continuous mixed-initiative steering)

Because real work is improvised, human-AI collaboration cannot be an all-or-nothing handoff. In traditional automation, handing control over to a machine often meant losing situational awareness until an emergency forced an abrupt, disorienting takeover.

Agentic systems need continuous, lightweight steering mechanisms. Users should be able to pause an ongoing workflow, prune an unproductive reasoning branch, tweak a parameter, or supply missing context mid-execution without blowing away the agent’s progress. Interaction patterns like steerable scratchpads, inline checkpoints, and editable thought trails turn the system into a collaborative workspace rather than an uncontrollable black box.

3. Shared mental models and multi-user sensemaking

We need interfaces that move beyond the single-user prompt box to support collaborative analysis. This means creating shared canvas spaces where multiple team members can view the agent’s intermediate steps, challenge its assumptions, attach domain-specific constraints, and align on decisions together. The AI becomes a shared surface for group sensemaking rather than an opaque intermediary.

4. Progressive disclosure of machine logic

Different tasks require different levels of inspection. During routine operations, users need concise, actionable outputs that preserve cognitive bandwidth. But when anomalies arise or high-stakes decisions must be made, the interface must allow users to peel back layers of reasoning progressively. Exposing decision trees, retrieved context chunks, and alternative options on demand gives users the exact depth of detail they need at any given moment.

Designing for human competence over machine autonomy

Fifty years ago, Xerox PARC showed us that treating humans as rigid information processors leads to fragile, frustrating technology. The lesson remains unchanged today. Making a model larger, faster, or more articulate does not eliminate the need for human sensemaking.

The goal of AI design is not to build autonomous machines that run without human involvement. The goal is to design systems that expand human competence, foster situational awareness, and support genuine collaboration. When we design for situated action, bridge the machine sensory gap, and build clear affordances for human steering, we transform AI from an unpredictable black box into a reliable partner.

References and Recommended reading

• Plans and Situated Actions by Lucy Suchman (book) — The foundational ethnographic study demonstrating why human action is improvised and situated rather than script-driven.
• The Psychology of Human-Computer Interaction by Stuart Card, Thomas Moran, and Allen Newell (book) — The seminal text modeling humans as information processors that spurred the debate between system-centric and human-centric design.
• Design for Conversation: Lessons from Cognoter by Deborah Tatar et al.(paper) — An early Xerox PARC study on the breakdowns that occur when software enforces rigid communication packages on human collaboration.
• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? by Emily M. Bender et al. (paper) — The landmark critique explaining why large language models stitch together statistical forms without genuine comprehension or real-world grounding.
• Explaining the Unexplainable: Explainable AI (XAI) for UX by Meg Kurdziolek(article) — A practical look at designing multi-stakeholder trust and transparency in complex machine learning workflows.
• A New Human-Computer Interaction Paradigm: Agent Interaction Model (AIX) (paper) — An analysis of how intent-driven agentic systems are reshaping the boundaries of human-computer interaction.
• Guidelines for Human-AI Interaction by Microsoft Research (framework) — Industry-standard design heuristics for transparency, error recovery, and user control in AI systems.

Persistent lessons in human-centered automation was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.