Markov decision processes (MDPs) offer an elegant mathematical framework for representing planning and decision problems in the presence of uncertainty. However, a simple textbook MDP uses discrete state, discrete time and it does not consider structure when modeling the process dynamics. Such a representation is very limited in its scope to represent real-world domains, which are often factorized, include continuous quantities (such as, temperature, speed, position, etc.) and/or imperfect observations. The aim of our research is (1) to devise MDP models that offer more natural representations of complex real-world decision problems, and (2) to develop algorithmic solutions that let us solve these problems more efficiently.
The web page is updated by milos.
Continuous and hybrid-state MDPs
Partitioned Linear Programming Approximations for MDPs
In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, Helsinki, Finland, July 2008.
Solving
Factored MDPs with Hybrid State and Action Variables.
Journal of Artificial Intelligence Research . accepted for
publication. 2006.
Learning Basis Functions in Hybrid Domains.
Proceedings of 21st National Conference on AI (AAAI-06), Boston, MA, July 2006.
Solving Factored MDPs with
Exponential-Family Transition Models.
In Proceedings of the 16th International Conference
on Planning and Scheduling, UK, June 2006.
Approximate Linear Programming
for Solving Hybrid Factored MDPs.
Proceedings of the 9th International Symposium on Artificial
Intelligence and Mathematics , Fort Lauderdale, Florida, January 2006.
An MCMC Approach to Solving Hybrid Factored MDPs.
In Proceedings of the 19th International Joint Conference on Artificial Intelligence , Edinburgh, Scotland, August 2005.
Solving Factored MDPs with
Continuous and Discrete Variables.
Proceedings of the AAAI Workshop on Learning and Planning in Markov
Processes - Advances and Challenges, pages 19-24, August 2004.
Solving Factored MDPs with
Continuous and Discrete Variables.
In Proceedings of the 20th
Conference on Uncertainty in Artificial Intelligence
, pages 235-242, July 2004.
Heuristic Refinements of Approximate
Linear Programming for Factored Continuous-State Markov Decision
Processes.
In Proceedings of the 14th International
Conference on Automated Planning and Scheduling, pages 306-314, June 2004.
Linear program approximations for
factored continuous-state Markov Decision Processes.
Advances in Neural Information Processing Systems 16 , pages 895-
902, December 2003.
Partially observable MDPs:
Value-function
approximations for partially observable Markov decision processes .
Journal of Artificial Intelligence Research, vol.13, pp. 33-94, 2000.
Planning treatment of ischemic
heart disease with partially observable Markov decision processes.
Artificial Intelligence in Medicine, vol. 18, pp. 221-244, 2000.
Planning and
control in stochastic domains with imperfect information.
PhD
dissertation, MIT-LCS-TR-738, 1997.
Incremental
methods for computing bounds in partially observable Markov decision
processes.
In Proceedings of the 14-th National Conference on
Artificial Intelligence, Providence, RI, pp. 734-739, 1997.
Hierarchical MDPs and decomposition methods
Hierarchical
solution of Markov decision processes using macro-actions.
In
Proceedings of the 14-th Conference on Uncertainty in Artificial Intelligence,
pp. 220-229, 1998.
Solving very large
weakly-coupled Markov decision processes.
In Proceedings of the 15-th
National Conference on Artificial Intelligence, Madison, WI, pp. 165-172,
1998.
Planning
with macro-actions: Effect of initial value function estimate on the convergence
rate of value iteration.
Working paper, 1998.
Applications to medicine and investments
Efficient
methods for computing investment strategies for multi-market commodity
trading.
Applied Artificial Intelligence, vol. 15, pp. 429-452,
2001.
Evaluation and optimization of management plans in
stochastic domains with imperfect information.
In Proceedings of
the Twelfth International Workshop on Principles of Diagnosis, pp. 71--78, 2001.
Computing
near optimal strategies for stochastic investment planning
problems.
In Proceedings of the 16-th International Joint
Conference on Artificial Intelligence, pp. 1310-1315, 1999.
Modeling
Treatment of Ischemic Heart Disease with Partially Observable Markov Decision
Processes.
In Proceedings of American Medical Informatics Association
annual symposium on Computer Applications in Health Care, Orlando,
Florida, pp. 538-542, 1998.
Semi-Markov models for vehicle routing optimization
Approximation strategies for routing in stochastic dynamic
networks.
In Proceedings of the Tenth International Symposium on Artificial Intelligence and Mathematics , Ft. Lauderdale, FL, January 2008.