Current research, Research & exploration
Safety for long-running Agents
Researching the safety of AI Agents during long-running tasks.
Beyond one answer. Safety over time.
From coding to everyday life, the research focuses on sustaining trust as AI Agents work longer. Hands-on experience building and operating products informs exploration of the next product.
- Task Alignment
Long tasks, exactly as first asked
Checks that a long, multi-step task keeps hold of the goal first asked, and still matches the original intent when plans change midway.
- Agent Safety
The whole course over one decision
Actions that each look fine can add up to risk. Watches the whole flow of work and catches drift before it leads somewhere unsafe.
- Human Check
Pause and check before it gets risky
When the work drifts somewhere risky, it stops first and asks a person to confirm.
Prior work · Open source
psychosis-guard: catching risk that builds over long conversations
A chatbot can validate a user's delusional belief a little more each turn while no single message trips a filter. psychosis-guard is open-source middleware that tracks the whole conversation, scores where it is heading, and intervenes in graduated steps. It can sit in front of any chatbot, including Anthropic Claude, OpenAI and local models, and the research is now being extended to long-running agents.
| Unguarded | psychosis-guard | |
|---|---|---|
| Delusion not confirmed | 42% | 71% |
| Harm not enabled | 60% | 91% |
| Safety intervention offered | 15% | 87% |
Preliminary, n=16, one repetition, evaluated with gpt-4o-mini: an unguarded chatbot versus a safety prompt plus psychosis-guard. Not a medical device, diagnosis or crisis support.
View on GitHubCurrently in research and exploration, shaping the product direction around Task Alignment, Agent Safety and Human Check. Methods and results will be published on this page as they are ready.