The first rule of agent UX is that humans need to see what the machine is thinking.
The Summary
- Developer ships open-source tool that lets AI agents draw visual overlays on screens: arrows, boxes, text annotations
- 270+ points on Hacker News signals demand for agent-to-human communication tooling most companies aren't building
- Solves the "black box" problem: when your agent clicks something, you should see where and why
The Signal
AI agents are getting better at doing things, but they're terrible at showing their work. You send Claude or GPT to navigate a website, fill a form, or extract data, and you get results with zero visibility into how they got there. Franz Enzenhofer's big-arrow-on-the-screen tool is a direct response to this gap.
The tool does exactly what it sounds like: it gives agents an API to paint visual overlays on your screen. Big arrows pointing at UI elements. Boxes highlighting sections. Text annotations explaining what the agent is about to do or what it just found. It's open source, built for developers who are tired of debugging agent behavior by reading logs.
"When your agent clicks something, you should see where and why."
This matters because we're entering the messy middle of agent adoption. Agents work well enough that people are deploying them for real tasks, but not well enough that anyone trusts them unsupervised. The result is a new class of tooling need: human-in-the-loop infrastructure. Not just approval gates, but real-time visibility into agent decision-making.
The Hacker News discussion around this tool shows what developers actually care about:
- Debugging: How do you know when an agent misidentified a button or got confused by page layout?
- Trust calibration: Users need to see enough agent work to trust it, but not so much that monitoring becomes a full-time job
- Handoff moments: When an agent gets stuck, the visual context helps humans quickly take over
What Enzenhofer built is simple, almost obvious in hindsight. But obvious solutions to new problems are often the ones that stick. Before GUI debuggers, developers printed log statements. Before browser dev tools, they used alert boxes. Visual overlays for agents are the same category of tool: low-tech, high-utility, solves an immediate pain.
The timing is right because agents are crossing the threshold from demos to deployment. Companies aren't just testing them anymore, they're shipping them to customers. And the first question customers ask isn't "how smart is your agent," it's "can I see what it's doing."
The Implication
If you're building agent products, visual feedback isn't a nice-to-have feature, it's table stakes. Users won't adopt tools they can't inspect. The companies that win the agent economy will be the ones that make agent behavior legible, not just effective.
Watch for this pattern to repeat across agent tooling: simple, developer-first utilities that solve the "I can't see what's happening" problem. The race isn't to build the smartest agents. It's to build agents that humans can actually work with.