We propose a heuristic and two stochastic approximation based
learning algorithms for finite horizon, finite state-action
constrained Markov decision models. We include models and
numerical examples arising from risk management in fund
allocation, retailer-depot product availability in a supply chain
and admission control in a simple queue,
that have to satisfy performance based constraints.