Evaluation engineer · RL environments · verifiers
Reinforcement-learning environments and coding evaluations for frontier labs, with deterministic graders, calibrated difficulty, and a written record of every task a stronger agent broke and why.
How a task is calibrated
RL environments with deterministic graders, multi-turn coding evaluations, and verification tasks over synthetic data. MuJoCo, Python, rubric design.
Oracle at exactly 1.0, a fair reference at 0.5, naive policies at 0, then an attack by the strongest available agent before any review. Difficulty is a number, not an opinion.
Frontier-model training data via Alignerr (2026), a frontier model's evaluation at Turing (2025), and a production operations platform run by a non-technical client, and my own startup, AI Securewatch. Remote, UTC+2.
Work
Each entry opens into the full account: what the task asked, how the oracle and reference were anchored, how it was attacked, what the numbers were, and what changed as a result. Tasks for lab clients are described at the level of mechanism and measurement; internals stay with the client.
A planar robot that climbs a chimney by pressing two pads against opposite walls and inch-worming upward. Friction is the only support; release both pads and it falls.
Four continuous actions: press force and lift for each pad. Physics at 2 kHz, control at 50 Hz, 22 seconds per episode. The difficulty axis is climb completeness: execute a coordinated multi-phase gait to a tall target without ever letting both pads slip at once.
The oracle is a hand-written inchworm gait that reaches the target and scores 1.0 after calibration. The reference is the same gait told to secure only the lower half, then hold; it lands at 0.500 and is rock-stable across seeds. Naive policies (push everything, greedy timer, random, lift-only) all land at 0.000, which is the honesty gate: a task where a trivial policy scores anything is mis-anchored.
Friction coefficient, body mass and gravity are drawn from hidden ranges per episode. A policy cannot pre-compute a press schedule; it has to sense slip and respond. Twelve hidden grading scenarios, bit-identical determinism, about fourteen seconds per full battery.
Calibration is piecewise-linear from measured raw scores (floor, reference, oracle), never from a guess. Build the reference from the oracle's own code with one knob turned down, so the 0.5 anchor is a legal policy under the same information, not a hand-tuned number.
A 3-D quadrotor carries a payload on a flexible multi-link cable and must thread it through narrow rings while keeping the swing quiet. Everything cleared until the strongest agent solved it outright.
Design review, adversarial review, runtime scorer and ground truth all passed. At the agent-attack stage a frontier coding agent scored 0.917 against a 0.5 ceiling. Not an exploit: it threaded 91% of rings with a mean miss of 2.5 cm.
A 313-line differential-flatness cascade: a quintic payload trajectory in the flat output, zero-vibration input shaping tuned to one compromise frequency, a geometric attitude loop, a payload-plus-drone cascaded PD, and a per-segment linear regression that learned the systematic crossing bias and pre-compensated it. It tuned its own parameters with an offline coordinate search. It even set the cable length wrong and still scored 0.917.
I re-used the agent's own captured policy as the attacker and escalated gusts. At every level that pushed the agent under 0.4, the blind reference was already near 0.18. Reference and agent share information, so the disturbance hurts the weaker controller more. No level existed where oracle 1.0, reference 0.5 and agent under 0.4 coexist. The family was closed rather than patched.
An underactuated quadrotor keeps a 0.9 m stick upright on a two-axis hinge while threading the stick's tip through nine hoops under hidden gusts. The agent's controller was better than mine.
All nine quality stages passed. The agent then scored 1.000. Running its policy through my own scorer on the 40 grading episodes: raw 0.769 against my oracle's 0.615. It threaded 7.97 hoops of 9 to my 7.22, with a worst-case miss of 9.6 cm to my 24 cm. It also arrived at the lean-reconstruction insight the task was built around without being told.
With a linear calibration from reference to oracle, an agent clears the ceiling unless the reference is at least 1.25 times weaker than the agent in raw score. If your oracle is only 4% better than what a strong agent writes in an afternoon, no scorer geometry can save the task. On an actively balanced inverted system, feedback erases trajectory history, so planning was worth nothing and the whole difficulty rested on controller quality.
To find out what survives a frontier agent, I probed a known-passing task with a real agent run and then built three look-alikes to isolate the mechanism. The original held at 0.265; all three copies broke.
A heavy ball on a beam between two rigid decks joined by a stiff, well-damped flexure, with delayed, noisy, partially observed sensing and a weakest-case cap over the grading families. I built a leak-clean public bundle, let a frontier agent write its policy from that alone, and scored it on the grader-private cases.
Headline 0.265, a miss. The agent's controller was strong: calibrated 0.702 before the cap, acing nominal tracking (0.886), plant transfer (0.897) and sensor degradation (0.864). What it could not do was recover the ball from the rail limits through the underactuation chain under delay, and the weakest-row cap made that one family decisive.
Implementation complexity is not difficulty. The only lever that held under measurement is information the policy cannot recover: partial observability and delay, applied to a recovery that is just barely achievable from observations alone. The 2026 literature agrees: delayed observation carries a provable optimality gap, and there is a computable lower bound on causal control cost that lets a reference be placed at the true optimum instead of tuned blind.
Prompts engineered so that capable models produce architecturally different solutions, then ranked with written rationales on seven fixed engineering axes. Preference pairs used directly in training.
A useful comparison needs two answers that differ in a way a reviewer can defend, not two that differ in style. That means designing the prompt around a real fork: a concurrency choice, a data-model choice, an error-handling boundary. The follow-up turns then test whether the model holds its own design under pressure.
Seven axes, fixed before any output is read: correctness, robustness, design, readability, testing, performance, and communication. Each ranking ships with a rationale that cites lines, not impressions. The rubric is locked first so the rationale cannot drift toward whichever answer I liked.
Every rejection from the client's review is traced to its cause and fed back into how the next prompt is built, so the acceptance rate is a lagging indicator of the rubric, not of effort.
Python evaluation and verification tasks over synthetic healthcare, retail, logistics and finance data with controlled messiness, calibrated to a 20 to 50% model pass band and scored automatically.
Each task combined a generated dataset with deliberate defects (duplicates, unit drift, missing keys, contradictory records) and a multi-step question whose answer could be checked programmatically. The checker verified reasoning steps and constraint satisfaction, not just a final number.
Difficulty was tuned against the target model until pass rates sat in the 20 to 50% band, so each task carried signal: hard enough to fail sometimes, fair enough to be passable from the materials given.
A startup I co-founded in 2024. First a payment-intelligence platform that finds insider-linked vendor fraud through entity resolution and graph analysis; now PayReality, decision intelligence that finds the unresolved questions behind an AI workflow's repeated escalations and turns them into tested rules.
Enterprises pay thousands of vendors and formally manage a fraction of them. The gap between who an organisation thinks it pays (the vendor record) and who actually receives the money (vendor reality) is where insider-linked fraud hides: an employee who is also a vendor's director, shell companies sharing an address, invoices split to stay under an approval threshold. No procurement or third-party-risk system looks across those records at once.
Working with early-stage customers showed a more general problem than fraud. AI workflows handle the routine cases and stall on the exceptions, and the exceptions stall for the same reason again and again: a rule that was never made explicit. Two handlers get the same damaged-item claim; one approves, one escalates, because the policy never said whether a photo was required.
PayReality looks across many cases at once, finds the recurring uncertainty (missing information, unclear rules, judgement calls), prioritises the few questions whose answers would resolve most of it, has authorised experts answer them, encodes the answers as explicit decision logic, and replays historical cases against that logic before anyone trusts it. It sits under the models, rules engines and workflow tools a company already runs rather than replacing them. It is an evaluation harness for human and AI decisions, and it is currently a hypothesis we are testing with design partners, not a product with published customer results.
Co-founder and the engineer who designed and built the backend end to end, from data model and resolution pipeline to API and deployment. I also work with my co-founders on roadmap, investor demonstrations and technical hiring.
A year-long engagement for a client who operates many instances of a third-party desktop application. I built the control plane: hardware-level input, pixel-level state reading, a Discord-based operator console, a health loop that repairs its own failures, and an evidence harness that proves every feature on video.
The client needed to run and supervise up to 24 instances of an application that was never designed to be automated, from a phone, with no engineer on call. The application takes input only from real hardware and exposes its state only on screen. Every part of the stack had to be observable and recoverable without a terminal.
The application occasionally asks a person to prove they are one. The system detects that prompt, refuses to start automated work under it, and opens a magnified remote view so the operator can respond by hand through the same hardware path. It never answers on a person's behalf. The same principle runs through the console: anything with a cost to reverse asks before it acts.
Method
The order matters. Difficulty is measured, never asserted, and the measurement is taken against the strongest attacker available, before anyone spends time on polish.
A legal policy under the task's own rules, scored by the same grader the agents face. Not a number typed into a config.
Built from the oracle's own code with one capability removed. If the reference needs privileged knowledge, the task is unfair, not hard.
Random, do-nothing, greedy, max-effort. Any of them scoring above the floor means the scorer rewards something other than the skill.
The agent's policy becomes a permanent regression test. The raw gap between it and the oracle is the task's real difficulty.
Held or cracked, the mechanism gets written down with numbers, so the next task starts from evidence rather than intuition.
Field notes
Conclusions I would not have believed without running the experiment. Each one cost at least one task.
A task designed to need a 15-state estimator, impulse detection and delay compensation was solved at 0.711. A strong agent writes the whole stack correctly when the structure is visible.
Making a hidden quantity observable, even indirectly, hands the agent the one thing the reference did not have. Hidden parameters only matter if knowing them changes the optimal policy.
On an actively stabilised inverted plant, trajectory history is overwritten every step. Difficulty from planning cannot exist there; only controller quality remains, and that is perishable.
A disturbance added to rescue a task hurts the weaker controller more. If reference and agent see the same inputs, there is often no disturbance level where 1.0 / 0.5 / under-0.4 coexist.
Delay and partial observability carry a provable optimality gap. Everything else measured so far, including contact hybridity, was solved hierarchically by a strong agent.
An agent that hit its turn limit and never submitted also scores 0.000. The trustworthy signal is a low but non-zero final score with a readable reason.
Systems
Backend and product work that pays for the research habit. Five years of Python on AWS and in distributed architectures.
The startup I co-founded. Payment-graph fraud detection, now PayReality decision intelligence. Full account in the case study above.
Technical due diligence that executes a repository's own README claims instead of reading them: extracts claims, runs them in a sandbox, and produces a per-claim verdict ledger with explicit abstention. Built for a 75-hour agentic-workflows hackathon with an honest baseline and ablations.
Clinical communication and patient-safety platform for nursing teams: structured handover, escalation ladder, dual-sign telephone orders with a tamper-evident audit trail. Zero-dependency web app, shipped as a clickable MVP.
Backend services and SQL Server query optimisation for an internal platform used daily by the investment team. Microservices and R Shiny front ends in production.
About
I trained in applied mathematics and computer science at Rhodes University and have worked as a backend engineer and independent consultant since 2019. Since late 2025 most of my time has gone into evaluation work for frontier labs: designing tasks, writing graders, and finding out, with numbers, where a model's ability actually ends.
I like work where the output can be checked. A task with a deterministic grader, an oracle that must hit 1.0, and an attacker's score on record is honest in a way a slide is not. I keep post-mortems of everything that failed because the failures are where the method came from.
I work remotely from Johannesburg, as a contractor or through an employer of record, and overlap comfortably with US East and European hours.