Expert data, made canonical.
Verified training data for coding and business-software agents.
Written by senior engineers and finance professionals. Every coding task is sandbox-verified, every grader is tested against reward hacking, and a person approves every answer.
Proof, not promises
- 50
- RL environment tasks, each verified
- 17
- sandbox-verified coding tasks
- 4
- reward hacks every environment grader must block
- 87
- sample records you can download today
See how it works
Two minutes of real product screens, with demo data: how experts are vetted, how a project runs, and how every answer is checked.
Chapters
Read the transcript
For AI teams 0:02
Canonset makes training data for AI labs, written by vetted experts. Labs can judge the data before talking to anyone. Every coding task comes with hidden tests, proven in a sandbox. And a business app as an RL environment, each task scored by a grader.
Joining as an expert 0:21
Experts apply for a role, with the pay shown up front. They sign in with an emailed code, then take a short test. Next, an ID check, with their written consent, or a video call instead. They photograph both sides of a US driver's license or state ID. Then the camera takes three pictures as they turn their head. Our own software matches the face to the ID; if in doubt, a person looks. Verified on the spot; the photos are deleted once it's decided.
The interview, three ways 0:54
Then an interview, taken one of three ways. Live, with an AI interviewer.
AI interviewer: To start, tell me briefly about your background and the kind of work you've been doing most in the last few years.
Applicant · synthetic voice: Hi! I'm a backend engineer with about eight years of experience, mostly Python and Go, building payment and data services.
A written interview, where Claude asks and follows up. Or a call with a person on our team. Claude scores each one against a rubric, and a person decides.
A paid trial task 1:26
Then a paid trial task. Here the checks fail: no test proves the fix. One more test, and they pass.
Every answer, checked 1:36
Every answer is reviewed, with an AI second opinion where the client allows it. Coding tasks show how often a frontier model solves them. Then our team approves each expert, field by field.
A project, from request to data 1:50
A client starts a project and uploads the data. Personal data is caught before it becomes a task. Experts only ever see a codename. Here, an expert picks the better answer, and says why. The client tracks quality week by week, and gets the data by API.
Running it 2:09
Behind it all: one queue for our team, weekly payouts and an audit log. Start a pilot at canonset.com/pilot.
What we deliver
Verified coding tasks
- Experts:
- Senior engineers write a problem, starting code, a reference solution and hidden tests.
- You get:
- Tasks with fail-to-pass and pass-to-pass test lists and a reference solution.
Business-software environments
- Experts:
- Finance and operations professionals design realistic tasks in simulated business apps, starting with expense approvals.
- You get:
- RL environment tasks, each with automatic scoring and a reference solution that passes every check.
Preference and grading data
- Experts:
- Compare two AI answers, or score one against a rubric, and explain why.
- You get:
- Preference pairs, raw or in DPO format, and rubric scores, each with the expert's reasoning.
Expert-written answers
- Experts:
- Practitioners write the answer they would want from an expert assistant.
- You get:
- Ideal answers for supervised fine-tuning, each approved by a senior reviewer.
How we check quality
No answer reaches you until it passes automatic and human checks.
Sandbox-verified tests
Each coding task runs in an isolated container: at least one test must fail on the starting code, and all must pass twice on the reference solution.
Graders that block reward hacking
Before an environment task ships, four known exploits, such as approving everything or doing nothing, must fail its grader.
Known-answer tasks
Gold tasks are mixed in unannounced; misses lower an expert's quality score.
Expert agreement
Up to five experts can answer each task; for preference and rubric work, their agreement is in your quality report.
A person approves every answer
Senior reviewers approve, return or reject; nobody reviews their own work.
No AI-written answers
Timing, paste, overlap and writing-pattern flags put suspect answers first in review; a confirmed case ends the expert's access.
Security and independence
- Your data stays inside the platform: no shared documents, no public links.
- Experts see only the task in front of them, never your company name.
- Submitted code runs in our own locked-down containers, not a third-party sandbox.
- Canonset is independent of every AI lab.
For experts
Paid, flexible project work that uses what you know. Apply once, see the pay before you start, get paid every Monday. No fees, ever.
