All chapters
06Archived

RoboForge

Tell it, simulate it, test it against reality.

2026Team project · systems, simulation, firmwareTeam OdysseyA chat-first machine builder

A chat-first machine builder: describe a rover, it designs it, simulates it, then checks the real one against the simulation and diagnoses the gap.

900+
tests without Webots or hardware
92%
RF-DETR precision
16/16
planted defects found
2
simulators: MuJoCo + Webots
RoboForge banner showing the rover, the simulation and the reality-gap loop.
Describe a machine, simulate it, measure the gap against reality.
01

The problem

Building a robot is a chain of guesses. You choose motors, a chassis, a sensor package, then you simulate it and hope the real machine behaves the same way. It usually does not. The left motor is a few percent weaker, the gyro has a bias, the floor is not the floor in the simulator.

Most toolchains treat that gap as a failure of the simulation. We treated it as the most useful measurement in the project.

02

The decision

RoboForge is chat-first. You describe the machine you want; it produces a design, runs it in MuJoCo or Webots, and then compares the simulated behaviour with the real rover's gyro data. The difference becomes a trim, a calibration or a diagnosis.

A deterministic router handles known jobs. The language model is used only for new jobs, where a plan actually needs to be invented. Every result carries a clear statement of what was not tested, so the system never pretends to more confidence than it has.

A simulation is a hypothesis. The real machine is the experiment. The gap is the result.
03

The build

The stack includes MuJoCo and Webots for simulation, an ESP32 and ESP32-CAM for the rover, and RF-DETR for vision. The test suite runs without Webots or hardware, which is why it can reach 900+ checks in ordinary CI.

The rover loop is: plan, simulate, run the real machine, measure, trim, repeat. The interface shows the plan, the simulated trace, the measured trace and the diagnosis side by side.

  • MuJoCo and Webots simulation backends
  • ESP32 firmware with an ESP32-CAM vision module
  • RF-DETR at 92% precision and 85% recall
  • Reality-gap engine driven by gyro measurements
  • 16 of 16 planted defects detected in the test rig
04

The evidence

The project has more than 900 tests that run without Webots or hardware. RF-DETR reaches 92% precision and 85% recall. The reality-gap engine found all 16 planted defects in the test rig.

The repository is archived after the hackathon, and the README is honest about the last mile: first contact with the real rover is still pending. That honesty is part of the design.

05

What I learned

I learned to design for the gap instead of pretending it away. A system that can say 'this part is simulated, this part is measured, this part is unknown' is more useful than one that reports a single confident number.

That is now a rule for me: every result should carry its own boundary.

Built with
PythonMuJoCoWebotsESP32RF-DETR
Next chapter · 07
Client Work