Physics-based, ML-driven locomotion promises robots that feel alive instead of rigidly cycling through hand-authored clips. But the path to an enemy that walks with intention is paved with agents that exploit physics glitches, sprint into walls with confidence, and throw themselves off the nearest ledge to escape a per-step penalty. This talk is an account of how we got there in ARC Raiders: where plain RL fell short, how imitation-based approaches gave us control over style, and the messy reality of turning research techniques into something a game team can actually ship.
– RL agents give you exactly what you asked for, which is rarely what you wanted. Many reward hacks are just assumptions you forgot to encode.
– Imitation learning gives you control over how an agent moves and achieves a given style faster than tuning rewards by hand.
– The way you frame the problem (e.g. environment perception, robot design) constrains what’s learnable as much as the algorithm does.
– RL is hard, but incredibly fun. Agents react to situations you never scripted, and even the imperfect results come out strangely believable in ways scripted animation can’t always achieve.
