DAgger

What it fixes

Behaviour cloning fits a policy to the states an expert visited. Run it, and its small mistakes take the robot to states the expert never showed, where it errs more: the errors compound. DAgger (dataset aggregation) collects data where the learner goes. Run the current policy, ask the expert what it would have done in every state the robot reached, add those labels to the dataset, and refit.

The algorithm

$a^*_t$ is the expert's action at $o_t$, whoever steered. The mix $\beta_i$ lets the expert help in the early rounds, so the first drives don't all end at a wall.

The rover

It drives a ring of 12 cm lanes at a fixed speed. A policy maps 9 lidar ranges, from its right to its left, to a turn rate. The expert steers by pure pursuit on the lane's centre line using the true pose, which the policy never sees. The demonstration is one clean lap from the start line; every drive after that starts somewhere new, up to 2 cm off-centre and 34° askew. The given fit copies the expert closely on the lap and guesses wildly off it, so cloned alone it usually hits a wall.

Tools

drive(policy, beta) runs one drive from a new start and returns (obs, expert), of shapes (T, 9) and (T,). np.vstack and np.concatenate join arrays.

Your task

Implement dagger(demos, drive, fit, rounds, episodes). Fit the demonstrations, (obs, actions). In each round i = 1 … rounds, drive the latest policy episodes times with β = 0.5^i, add the data to everything so far, then refit once. Return the last policy. At least 4 of 5 new starts must go 0.8 m without touching a wall.