From pixels and echoes to a usable road scene
A camera supplies images; radar and lidar, when fitted, provide different clues about distance and motion. Software first has to align those readings with the car’s position and with one another. It then identifies useful objects—such as a vehicle, cyclist, lane edge, or traffic light—and tracks them across successive updates. Tracking matters because one blurry glimpse is less informative than seeing whether an object is getting closer or changing lanes.
The result is not a perfect digital copy of the street. It is a working estimate, with uncertainty: a parked shape might be a car or a shadow, and a partly hidden pedestrian may be difficult to classify. The system must make decisions from that incomplete picture rather than wait for certainty.
Forecasts become a driving plan
For each relevant road user, prediction software estimates plausible next movements. A car near an exit might continue straight or move toward the ramp; the system weighs possibilities using its position, speed, and surrounding traffic. It cannot know another driver’s intention, so a safe plan must account for more than one likely outcome.
Planning software turns those forecasts into a route through the immediate scene: keep a following gap, yield, change lanes, or stop. A separate control layer translates that plan into steering, acceleration, and braking commands. Those commands are checked against the car’s changing motion, then adjusted as fresh sensor readings arrive. The cycle repeats, rather than issuing one decision and carrying it out blindly.
Why small errors matter
The hardest cases often involve interactions, not isolated objects: a vehicle edging into a lane, a cyclist partly hidden by a parked van, or a human driver behaving unpredictably. A mistaken estimate can make a planned maneuver unsafe, so the system must continually revise its picture and choose actions that leave room for uncertainty. In practice, autonomous driving is less like following a fixed script than repeatedly asking whether the next few seconds still look safe.
