Computer Use Beyond the Demo: Bringing Legacy Systems into the Agentic Era
Many high-frequency enterprise processes remain trapped in legacy desktop applications without usable APIs. This talk examines how a complex, undocumented, approximately one-hour Siebel workflow was moved into live operation inside a restricted Windows, RDP, and VPN environment.
Later production telemetry shows the deployment operating across nine parallel agent instances, with 135 tasks marked completed over five weeks. It also records failures and cancellations—highlighting why production Computer Use cannot rely on model performance alone.
Using both the successes and failure modes of this case, we will cover the architecture needed to capture human workflows as structured SOPs, combine screen interaction with state machines and targeted API or PowerShell tools, handle unexpected screens and backend outages, and provide recordings, approvals, recovery, and human intervention.
Scope and Structure (estimated 30 minutes total):
Wonderful as an Open Agentic Platform (3 minutes)
- Headless and multicloud architecture.
- Support for multiple model providers and modalities.
- Connecting agents to enterprise tools, data, and operating environments. The Legacy-System Problem (3 minutes)
- High-value workflows that exist only behind a user interface.
- Why traditional integrations are often unavailable or too expensive.
-Computer Use as the last-mile integration layer.
The Production Architecture (6 minutes)
- Connecting agents to real desktop environments.
- Supporting Windows, remote desktops, VPNs, and customer-managed infrastructure. ● Separating agent execution from the desktop being controlled.
- Designing for restricted enterprise networks.
Capturing the Human Process (4 minutes) - Recording experienced human workers.
- Using Gemini to divide recordings into phases and draft an initial SOP. ● Human annotation of actions, preconditions, reasons, and expected outcomes. ● Iteratively expanding the SOP when new branches or failures are discovered.
The Siebel Case Study (5 minutes)
- A complex, approximately one-hour back-office workflow.
- Operating a legacy Windows system without a modern API.
- Initial problems with latency, brittleness, and undocumented exceptions. ● Moving from a proof of access toward a reliable production workflow.
Reliability, Auditability, and Human Control (3 minutes)
- Live viewing and session recording.
- Approvals before irreversible actions.
- Human escalation and operational handoff.
- Recovery from unexpected screens and backend failures.
- Using execution data to improve the process over time.
Conclusion (1 minute)
- Legacy systems do not need to be rebuilt before they can participate in agentic workflows.
- Computer Use can make them accessible, but production success depends on architecture, process knowledge, observability, and control.
- The real opportunity is turning undocumented human operations into reliable and auditable software.
Topic and Relevance
Relevant for teams automating high-frequency processes in legacy systems where APIs are unavailable and operational control, auditability, and recovery are critical.
Audience Takeaways
How to make legacy systems available to AI agents—and how process documentation, hybrid interaction patterns, observability, and human escalation separate a Computer Use demonstration from production automation.
Talk Type and Approach
A technical architecture and production-engineering talk grounded in a real deployment, including its limitations, failures, and lessons learned.
Sako Arts is the CTO of Wonderful NL, drawing on a rich background with an MSc Computer Science from the TU/e, deep learning research at Philips, AI entrepreneurship, and executive leadership. As an experienced international keynote speaker, he cuts through the hype to share practical insights and navigate the real-world challenges of implementing complex AI.