Programming simple Bellman Equation and tracking the controls

• To: mathgroup at smc.vnet.net
• Subject: [mg83631] Programming simple Bellman Equation and tracking the controls
• From: martin_schleicher at yahoo.com
• Date: Mon, 26 Nov 2007 03:48:21 -0500 (EST)

```Hi everybody,

I am pretty new to Mathematica and I guess the problem I am
encountering is not difficult to solve. I am trying to solve a simple
dynamic programing problem for finite horizon - finite states
(t=0,1,2,3), where:

1) control / decision is u(t)
2) state x(t+1)=[x(t)+u(t)]^2 (the state next period depends on the
current state and the control chosen)
3) cost at each period: x(t)-u(t)
4) Objective: Minimize sum of costs across periods starting at period
t=0 and state x(0)=0.

The following function computes the total cost properly (I think!) in
recursive fashion.

J[t_, w_] := J[t, w] =
If[t >= 3, 0,
Extract[
Minimize[{(-u + w) + J[t + 1, (w + u)^2], -1 <= u <= 1 && u \
[Element] Integers}, u]
, 1]
];

J[0, 0] = -1

However, I don't know how to track the controls u[t] to use at each
period that solve the minimization. For this basic problem they should
be: u[0]=0;u[1]=0;u[2]=1. Whatever I try I just get one control u =1.

I hope someone can help me out.

Best,
Slater

```

• Prev by Date: FindInstance puzzler
• Next by Date: solving large sparse system in Mathematica
• Previous by thread: Re: FindInstance puzzler
• Next by thread: Re: Programming simple Bellman Equation and tracking the controls