Programming simple Bellman Equation and tracking the controls
- To: mathgroup at smc.vnet.net
- Subject: [mg83631] Programming simple Bellman Equation and tracking the controls
- From: martin_schleicher at yahoo.com
- Date: Mon, 26 Nov 2007 03:48:21 -0500 (EST)
Hi everybody,
I am pretty new to Mathematica and I guess the problem I am
encountering is not difficult to solve. I am trying to solve a simple
dynamic programing problem for finite horizon - finite states
(t=0,1,2,3), where:
1) control / decision is u(t)
2) state x(t+1)=[x(t)+u(t)]^2 (the state next period depends on the
current state and the control chosen)
3) cost at each period: x(t)-u(t)
4) Objective: Minimize sum of costs across periods starting at period
t=0 and state x(0)=0.
The following function computes the total cost properly (I think!) in
recursive fashion.
J[t_, w_] := J[t, w] =
If[t >= 3, 0,
Extract[
Minimize[{(-u + w) + J[t + 1, (w + u)^2], -1 <= u <= 1 && u \
[Element] Integers}, u]
, 1]
];
J[0, 0] = -1
However, I don't know how to track the controls u[t] to use at each
period that solve the minimization. For this basic problem they should
be: u[0]=0;u[1]=0;u[2]=1. Whatever I try I just get one control u =1.
I hope someone can help me out.
Best,
Slater