first steps in robot locomotion

toby and my ml racing line both start in a physics simulator. these are notes on why, and on what has and hasn’t worked so far.

why simulate first

toby is a self-balancing robot on two wheels. an unbalanced controller doesn’t fail gracefully, it falls over, and it can break the thing it’s mounted on. in mujoco i can push it, bump it, roll it up a ramp, drop it down a step and change its leg height as many times as i like, for free. the plan is to tune balance in simulation, confirm it on a stand, and only then try the floor.

the catch is that a simulator is a claim about the world, not the world. balancing in sim isn’t enough; the controller has to balance with realistic delay, noise and limits. much of the simulation work is checking whether a result would survive the move to real hardware.

this is what the tuning has bought so far, in randomized simulation only. none of it is on the robot yet.

learning to drive

the racing project goes the other way: instead of designing a controller, i train one. a ppo agent drives a simplified formula sae-style car around a waypoint track. everything the agent knows comes through two interfaces, the observation it sees and the reward it gets, and both are choices i make.

the first long run went past 600k steps and barely moved. that points at the environment and the setup, not at the number of steps. the observation might leave out something the car needs, or the reward might pay for the wrong thing. the next round is about fixing those before spending more compute. each column below is one logged episode; the dashed line is a full lap.

what i’m taking from it

a simulator is where you find out what you don’t understand. a controller that works confirms what you already believed. one that fails shows you which assumption was wrong. that’s why the failures will go up here too.