Planning the path

The car now knows where everything is and where it's likely to go. Time for the decision that actually turns the wheel: what trajectory should we drive? This is planning — choosing a specific path and speed for the next few seconds that is safe, comfortable, legal, and actually gets you where you're going.

Scroll to plan a way around a stopped car.

Score, then choose

Planning looks a lot like the ranking you've seen elsewhere in these courses: propose many options, score each, pick the best. The score comes from a cost function that balances competing goals:

• Safety — keep distance from cars, people and the occupancy map; never risk a collision across the predicted futures from the last chapter. This dominates everything.
• Comfort — no jerky steering or harsh braking; passengers hate it.
• Progress — actually move toward the destination at a reasonable speed.
• Rules — stay in lane, obey lights and signs, signal turns.

Plan, act, repeat

The planner doesn't decide once. It re-plans the whole trajectory many times a second, each time on a fresh world model and fresh predictions. The car only ever executes the very start of the chosen path before throwing it away and planning again. This constant re-planning is what lets it react smoothly when that 30% "turns across you" future suddenly becomes what's actually happening.

while driving:
  world = perceive(camera_frames)        # Ch 2-3: objects + bird's-eye map
  futures = predict(world)               # Ch 4: others' likely paths

  candidates = sample_trajectories(world)
  best = min(candidates, key=lambda t: cost(t, world, futures))

  actuate(best[0])                       # execute only the first step
  # ...then loop: re-plan from scratch next frame

Where we're headed

That's the whole real-time driving loop: see, model, predict, plan, act — thirty times a second. But how does the network get good at each of those steps, and keep improving on the weird situations no one anticipated? That comes from something only a company with millions of cars on the road has. Finally: learning from the fleet.