Modelling the cart-pole

The cart-pole

A pole stands on a free hinge on a cart, and the cart runs along a rail with end stops 20 cm either side of the centre. You can push the cart; nothing pushes the pole. Four numbers describe everything that is going on, the state:

$$\mathbf{x} = \begin{bmatrix} x & \dot x & \theta & \dot\theta \end{bmatrix}^T$$

Symbol	Unit	Meaning
$x$	m	where the cart is along the rail, 0 at the centre
$\dot x$	m/s	the cart's velocity
$\theta$	rad	the pole's lean from upright, positive towards $+x$
$\dot\theta$	rad/s	how fast it's tipping

The input $u$ is the force on the cart in newtons, positive towards $+x$. cartpole.state() returns $\mathbf{x}$ as a numpy array.

A linear model

The real motion involves $\sin\theta$ and $\cos\theta$: gravity tips a leaning pole further, and a push swings the pole the other way. Near upright, $\sin\theta \approx \theta$ and $\cos\theta \approx 1$, and over one control period of 20 ms (with $u$ held) the change becomes linear:

$$\mathbf{x}_{k+1} \approx A\,\mathbf{x}_k + B\,u_k$$

$k$ counts periods. $A$ (4×4) is how the state carries on by itself; $B$ (4×1) is what one newton adds. cartpole.linearize() returns both. Look at $B$: its $\theta$ entry is negative, because pushing the cart forwards makes the pole lean back.

Rolling out

Apply the model again and again and you have a simulator of your own, a rollout. Because it's linear, you can also turn it around and solve for the forces you need.

The pole starts 4–7° from upright. The program asks your model for two pushes, 0.1 s each, that leave the pole upright and still: it rolls out with no force and with 1 N in each push, then solves a 2×2 system. It applies them, then lets go. The white pole is your model's prediction.

Your task

1. Write predict(A, B, x, u): the next state, a 4-vector.
2. Write rollout(A, B, x, us): the state after each force in us, one row per force.

Run it. The pole stands back up, hangs for a moment, then falls. To pass, the pushes must leave it within 0.5° of upright. Then compare your model with the real pole in the Plots tab: where do they part ways, and why?