Your App Just Got a Second User — and It Doesn't Need Its Own UI
Your application has a second user now: an AI agent sharing the same session as the human. It can navigate, select, fill, and act, but it should not need a second interface — or a second permission model — to do it.
Many agentic applications solve this by generating new interfaces or by exposing application functions directly to a model. Both approaches discard something valuable: the semantics, accessibility, validation, permissions, and interaction patterns already encoded in the application.
This talk presents a different model. Chat becomes an intent surface, while results and human intervention remain in the application's existing UI. Controls declare what they do, when they are available, what state they change, and whether the decision belongs to the human or the agent. The agent operates through existing application handlers, while the application remains authoritative about permissions and state.
This becomes especially important during multi-step agent execution. A button offered to an agent five seconds ago may no longer be valid. A human may complete part of a workflow while the agent is waiting. Sensitive information can remain entirely inside the application's existing controls instead of traveling through the model context.
We will walk through these interaction patterns and the runtime evidence needed to distinguish “the agent said it worked” from “the application actually changed.”
The goal is a model-independent architecture in which the AI interprets intent and plans actions, while the application continues to own its UI, state, permissions, and truth.
This is a practical architecture and interaction-design talk about integrating AI agents into applications that already have a mature UI, permissions model, and application logic.
The central idea is simple: the agent should behave like a second user of the existing application rather than receiving a separately generated interface.
The session will cover four concrete patterns:
Existing UI as the output surface
Natural language is used to express intent, but the response appears through existing application components — navigation, forms, tables, selections and state changes — instead of generated markup.Declarative application capabilities
Controls describe what they do, the state they affect, when they are currently available, and whether the decision belongs to the human, the agent, or requires human approval.Human intervention during an agent loop
Multi-step agent execution does not have to hide behind a spinner. The human can see the intermediate state, complete a required action in the real UI, or change application state while the agent is still working.
This also creates an important privacy boundary: if a person must enter sensitive information, the agent can wait for the application to report that the required condition has been satisfied without receiving the value itself.
- Verifying effects rather than trusting claims
An agent saying “I completed the action” is not evidence that the application changed. We will look at recording the relationship between offered actions, attempted actions, application handlers, and observed state transitions.
A small demo will illustrate a race condition in which an action initially available to the agent becomes invalid after the human changes application state. The important result is not that the model was warned — it is that the application itself prevents the stale action from executing.
Attendees should leave with an architecture they can apply to existing web applications without replacing their design system, duplicating their authorization model, or generating a new interface for every agent response.
Sanjay Krishna Anbalagan is a software engineer at AWS and a researcher working at the intersection of human-computer interaction, agentic AI, and developer tooling.
His work focuses on how AI agents can interact safely with existing applications without replacing the interfaces, permissions and application semantics that humans already rely on. He created the open-source AgentFootprint ecosystem, which explores typed agent actions, runtime evidence, human-agent collaboration, and explainable execution.
Sanjay holds a PhD in Computer Science from UMass Lowell and has presented and published work on conversational interfaces, enterprise generative AI, and human-centered AI systems.