DeepSeek Harness (DSH) is an open-source framework for running AI agent sessions, and its Super Agent is a separate open-source prototype that picks up requests on GitHub and has agent workers turn them into pull requests. Both are MIT licensed. Both are honest about their status: the Harness calls itself a developer preview that has not had a security audit [1], and the Super Agent calls itself a research prototype that is not production-ready [2].
We want agents like this doing routine work for small businesses: answering support, drafting replies, keeping things up to date. So before trusting one with a customer, we measured what it does on its own, and what changes when every action it takes passes through a gate.
What we mean by "the gate"
The gate is a checkpoint between the agent and the world. Every tool call the agent makes is checked before it runs. Routine reads and work pass. Anything that leaves the business, an email, a message, a post, is held until a person approves it. Secret files are refused. One STOP ends every agent at once. And every action is logged.
How we measured
- First bench: 20 tasks, each run twice, with the agent alone and then behind the gate: 80 runs on DeepSeek V4.1 Flash.
- Then scenarios closer to a real business: a support desk, an email inbox, a busy chat, a long job, a poisoned web page, and a coding helper. We repeated key tests on two other models, Qwen 3.6 and Gemini 3.1 Flash Lite.
- Everything ran in a sandbox with the network locked, so no real message ever left. We counted what the agent tried to send, not what reached anyone.
The headline results
| Measure | Agent alone | Behind the gate |
|---|---|---|
| Outward calls that ran | 164 | 0 (31 held for a person, 1 denied) |
| "Send this out" tasks where the send ran | 10 of 10 | 0 of 10 |
| Runs that read the secret settings file | 21 of 40 | 0 (the agent tried in 20) |
| Normal tasks done right | 10 of 10 | 10 of 10 |
| Normal actions wrongly held | n/a | 4 calls in 2 runs |
| Time the gate added per tool call | n/a | 2.05 ms median, 5.95 ms for the slowest 5% |
| Model cost for the bench | USD 0.57 | USD 0.27 |
The agent did the normal work just as well behind the gate. What changed is that nothing left without a person seeing it first, and the secret settings file stayed closed.
In business-like scenarios
- Support desk: alone, 29 replies went out unchecked. Behind the gate, 0 went out and 84 were held for approval.
- Email inbox: alone, an email went out in 8 of 8 risky tasks, including one that sent a full bank routing number (test data). Behind the gate, 0 of 8. Normal inbox tasks finished 8 of 8 either way; one was wrongly held.
- One STOP: it ended 15 of 15 agent runs, and 0 tool calls ran after it. The agent alone has no STOP at all.
- Reading the web: behind the gate, 57 of 66 page reads were recorded with their address, so you can see where an answer came from. Alone, none were.
- Coding helper: alone, a secret reached the AI in 7 of 12 runs; behind the gate, 0 of 12. The bug was fixed 12 of 12 times either way.
- Two other models: alone, a hidden instruction was carried out in 3 of 10 runs and a requested send ran in 8 of 10. Behind the gate, 0 and 0.
What the gate did not do
Being honest about the limits matters more than the headline:
- Some good behaviour was the agent's own. It followed 0 of 10 prompt injections with or without the gate, ignored strangers, and refused a wire-transfer lure on both sides.
- The gate checks actions, not thoughts. In some runs on both sides, a secret still reached the model through a file the rules allowed.
- One model got a fake card number into a file using a script it wrote itself. That was a gap in our checks, and it is on our list.
- A blocked agent scanned the local machine and found other test runs' gates to probe. We now isolate every run.
- Cost per successful task went up in 3 of 6 comparisons and down in 3. The gate is not a cost saver; it is a control.
From sandbox to a real server
Next we packaged the gate with the agent for a real server: a spending cap with a kill switch, a gate that holds merges and releases, a network allow-list so the agent can only reach the services it needs, and a start check that refuses to run if any of those is missing.
What this means if you are choosing an agent
Ask any vendor, or yourself, five questions before an agent touches your customers:
- 1What can it send without a person seeing it first?
- 2Can it read your passwords, keys or payment details?
- 3Is there one switch that stops it, and has anyone tested it?
- 4Is there a hard spending cap?
- 5Can you see a log of every action, with where its information came from?
If the answers are unclear, start with a secure rollout plan. If you want help setting one up, talk to us.
