Why DeepORG03Your organization as a training environment
Turn your organization into a training environment.
The machinery that grades DeepORG in public also generates checked tasks from your own systems and serves them as graded episodes. Measure an agent on your organization, or train one with reinforcement learning, instead of on a toy.
The reward
The database marks the answer. No model is asked for its opinion.
Most ways of scoring an agent ask a model whether an answer looks right. This one asks the database. Every task carries a hidden oracle: a read-only query resolved against the same world the agent works in. A fluent, confident, wrong answer earns nothing.
The oracle is written when the task is generated, and the agent never sees it. A policy trained against this reward cannot improve its score by becoming more persuasive.
- Correctness. The answer is compared with what the oracle returns, now, in this world.
- Method. A correct answer reached the wrong way does not score full marks. An answer with no work behind it scores zero.
- Behavior. Required actions taken, forbidden ones avoided, and a clear refusal where one was called for.
- Side effects. For a change, the end state must be right and nothing outside the task’s scope may have moved.
- Efficiency. Reported, without turning a slow correct answer into a failure.
There are two reward profiles. benchmark is the published contract. rl is for training, and there safety is a gate rather than a weight: an episode that caused an unsafe effect is worth nothing.
Three ways in
Start from our world, your schema, or your people’s questions.
A container that boots ready
docker run -d -p 8090:8090 --tmpfs /tmp:size=1g dosco/graphjin:env-latestNothing is mounted and nothing is generated: a demo organization and its verified tasks are inside.
| Measured | Value |
|---|---|
| Image size | 148MB |
| Healthy after | about 4 seconds from docker run |
| Pool of 2 worlds ready in | 2055ms |
| Written to the work directory | 1.3MB |
| Tasks served | 113 |
Measured 2026-08-31 on Apple silicon laptop, native arm64 image. Re-measure with make env-image-smoke.
Your schema, without your data
graphjin env clone --url https://graphjin.internal --out ./clone-acme --seed 7DeepORG learns a running server’s shape from its catalog: table and column structure, relationships, saved query names, and the closed value sets the catalog publishes. No rows are read. Synthetic records are generated locally, and every task is verified against a booted copy of the clone before it ships.
Your people’s questions, as the tests
graphjin eval mine --remote
graphjin eval run --suite eval/mined.yml --yesEvery question your people ask is answered by a database query. When that answer is one figure the database computed, the question becomes a test whose answer key is the query, re-run whenever the test runs. Mining spends no model tokens. Your company is the environment tells that story in full.
Driving it
Three ways to run an episode. One way to grade it.
- DeepORG calls your endpoint. The default. Point it at an OpenAI-compatible endpoint and
POST /episodesruns one graded episode. - You supply each completion.
--stephands you the prompt and takes back one completion at a time. It is the mode GRPO uses. - You bring the whole agent.
--externalhands you the task, an MCP endpoint and a deadline. Your agent does the rest.
Grading is the same in all three: an answer submitted without touching the database scores zero.
Each worker in the pool provisions its own copy of the project, with its own database and its own state, and an episode leases one. The pool refuses to start unless every worker reports the same dataset. A held-out split is derived from each task’s content, so two containers agree on it without talking, and the exporter refuses to build a training set from the evaluation side.
Training
We ship the environment, not the trainer.
The loop, the optimizer and the GPUs stay on your side. The environment gives you tasks, isolated worlds, a reward, and trajectories you can train on:
graphjin eval export <run-id> --split eval/suite.split.json --side train \
--stage executor --out teacher.jsonlThe order matters:
- Distil from a teacher, so the policy can complete an episode at all.
- Rejection-sample from the tuned policy for harder, on-policy data.
- GRPO, once attempts differ from each other, because that difference is the whole signal.
Same machinery
The public benchmark is this environment, published.
The DeepORG benchmark is a frozen suite of 113 tasks, each run three times, served and graded by the same code. That is why the scores are measurements, and why the same reward is something a policy can be trained on.
The best published result is 97 of 100 with 0 unsafe effects. See every result, including the ones that failed.
In the docs
Go deeper.
- Overview and quick start.
- Drive modes and the HTTP API.
- The reward: profiles, components and rescoring.
- Your own graph and your own work.
- Worlds: generating new organizations to train on.
- Training: the recipe, in order.