mdp_example_rand description

MDP Toolbox for MATLAB

mdp_example_rand

Generates a random MDP problem.

Syntax

[P, R] = mdp_example_rand (S, A)
[P, R] = mdp_example_rand (S, A, is_sparse)
[P, R] = mdp_example_rand (S, A, is_sparse, mask)

Description

mdp_example_rand generates a transition probability matrix (P) and a reward matrix (R).
Optional arguments allow to define sparse matrices and pairs of states with impossible transitions.

Arguments

S : number of states.

S is an integer greater than 0.

A : number of actions.

A is an integer greater than 0.

is_sparse (optional) : used to generate sparse matrices.

is_sparse is a boolean. If it is set to true, sparse matrices are generated.
By default, it is set to false.

mask (optional) : indicates the possible transitions between states.

mask is a (SxS) matrix composed of 0 and 1 elements (0 indicates a transition probability always equal to zero).
By default, mask is only composed of 1.

Evaluations

P : transition probability array.

P is a 3 dimensions array (SxSxA) or a list (1xA), each list element containing a sparse matrix (SxS).

R : reward matrix.

R is a 3 dimensions array (SxSxA) or a list (1xA), each list element containing a sparse matrix (SxS). Elements of R are in ]-1; 1[.

Example

>> % To be able to reproduce the following example, it is necessary to init the pseudorandom number generator
>> rand('seed',0)

>> [P, R] = mdp_example_rand (2, 2, false, [1 1; 0 1])
P(:,:,1) =
   0.2439     0.7561
            0     1.0000
P(:,:,2) =
   0.6428     0.3572
            0     1.0000
R(:,:,1) =
   -0.9309     0.0594
             0     0.3423
R(:,:,2) =
   -0.9846     -0.8663
             0     -0.1650

MDP Toolbox for MATLAB

MDPtoolbox/documentation/mdp_example_rand.html
Page created on July 31, 2001. Last update on August 31, 2009.