How math made virtual robots play MIDI music

Turning MIDI notes into moving machinery

MIDI is not a recording of music. It is a stream of instructions: which note to play, when it begins and ends, and often how forcefully it is struck. Animusic’s software treated those instructions as cues for a three-dimensional scene. A note could set a mallet or other mechanism in motion, making the visible action correspond to the musical event rather than merely decorate it.

That required translating musical information into physical behavior. Pitch could determine which bar or string an animated striker reached; timing determined when it moved. The software also had to make the movement look as though an object with mass had acted: a striker swings, makes contact, and rebounds instead of simply appearing at its target.

Why the physics mattered

A perfectly timed but weightless motion can look artificial. Physics-based animation gives moving parts a sense of momentum and contact, helping a machine seem to cause the sound it accompanies. The challenge is that musical timing is precise, while convincing motion takes time. A mechanism must begin moving early enough to hit the right object on the beat, yet still look as if it could physically reach it.

The MIDI event therefore served as more than a visual marker. It could drive a chain of actions—movement, impact, and recovery—while keeping those actions aligned with the music. Repeating the process across many notes made the performance legible as a sequence of causes and effects.

A score becomes a performance

This approach also made the animation responsive to the structure of the music. Instead of manually placing every movement as an unrelated visual flourish, the creators could use musical data to coordinate the scene. The score became a control system for the virtual instruments: change the notes or their timing, and the machinery’s performance could change with them.

That connection is what makes the robots feel like performers rather than props. Each impact has a musical reason, and each note leaves a visible trace in the motion of the machine.