Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

Programming simple Bellman Equation and tracking the controls

  • To: mathgroup at smc.vnet.net
  • Subject: [mg83631] Programming simple Bellman Equation and tracking the controls
  • From: martin_schleicher at yahoo.com
  • Date: Mon, 26 Nov 2007 03:48:21 -0500 (EST)

Hi everybody,

I am pretty new to Mathematica and I guess the problem I am
encountering is not difficult to solve. I am trying to solve a simple
dynamic programing problem for finite horizon - finite states
(t=0,1,2,3), where:

1) control / decision is u(t)
2) state x(t+1)=[x(t)+u(t)]^2 (the state next period depends on the
current state and the control chosen)
3) cost at each period: x(t)-u(t)
4) Objective: Minimize sum of costs across periods starting at period
t=0 and state x(0)=0.

The following function computes the total cost properly (I think!) in
recursive fashion.

J[t_, w_] := J[t, w] =
   If[t >= 3, 0,
    Extract[
      Minimize[{(-u + w) + J[t + 1, (w + u)^2], -1 <= u <= 1 && u \
[Element] Integers}, u]
     , 1]
    ];

J[0, 0] = -1

However, I don't know how to track the controls u[t] to use at each
period that solve the minimization. For this basic problem they should
be: u[0]=0;u[1]=0;u[2]=1. Whatever I try I just get one control u =1.

I hope someone can help me out.

Best,
Slater


  • Prev by Date: FindInstance puzzler
  • Next by Date: solving large sparse system in Mathematica
  • Previous by thread: Re: FindInstance puzzler
  • Next by thread: Re: Programming simple Bellman Equation and tracking the controls