RoboForge
Tell it, simulate it, test it against reality.
A chat-first machine builder: describe a rover, it designs it, simulates it, then checks the real one against the simulation and diagnoses the gap.

The problem
Building a robot is a chain of guesses. You choose motors, a chassis, a sensor package, then you simulate it and hope the real machine behaves the same way. It usually does not. The left motor is a few percent weaker, the gyro has a bias, the floor is not the floor in the simulator.
Most toolchains treat that gap as a failure of the simulation. We treated it as the most useful measurement in the project.
The decision
RoboForge is chat-first. You describe the machine you want; it produces a design, runs it in MuJoCo or Webots, and then compares the simulated behaviour with the real rover's gyro data. The difference becomes a trim, a calibration or a diagnosis.
A deterministic router handles known jobs. The language model is used only for new jobs, where a plan actually needs to be invented. Every result carries a clear statement of what was not tested, so the system never pretends to more confidence than it has.
A simulation is a hypothesis. The real machine is the experiment. The gap is the result.
The build
The stack includes MuJoCo and Webots for simulation, an ESP32 and ESP32-CAM for the rover, and RF-DETR for vision. The test suite runs without Webots or hardware, which is why it can reach 900+ checks in ordinary CI.
The rover loop is: plan, simulate, run the real machine, measure, trim, repeat. The interface shows the plan, the simulated trace, the measured trace and the diagnosis side by side.
- MuJoCo and Webots simulation backends
- ESP32 firmware with an ESP32-CAM vision module
- RF-DETR at 92% precision and 85% recall
- Reality-gap engine driven by gyro measurements
- 16 of 16 planted defects detected in the test rig
The evidence
The project has more than 900 tests that run without Webots or hardware. RF-DETR reaches 92% precision and 85% recall. The reality-gap engine found all 16 planted defects in the test rig.
The repository is archived after the hackathon, and the README is honest about the last mile: first contact with the real rover is still pending. That honesty is part of the design.
What I learned
I learned to design for the gap instead of pretending it away. A system that can say 'this part is simulated, this part is measured, this part is unknown' is more useful than one that reports a single confident number.
That is now a rule for me: every result should carry its own boundary.