Books and Proceedings

ICML Workshop on Machine Learning for Interactive Systems (MLIS'15): Bridging the Gap between Perception, Action and Communication, at ICML15, Montreal Canada, published as JMLR proceedings

H. Cuayahuitl, N. Dethlefs, L. Frommberger and M. van Otterlo (2014) AAAI Workshop on Machine Learning for Interactive Systems (MLIS'14): Bridging the Gap between Perception, Action and Communication, Quebec, Canada
Published in the AAAI proceedings series

H. Cuayahuitl, N. Dethlefs, L. Frommberger and M. van Otterlo (2013) IJCAI Workshop on Machine Learning for Interactive Systems (MLIS'13): Bridging the Gap between Perception, Action and Communication, Bejing, China
Published in the ACM International Conference Proceedings Series under ISBN 978-1-4503-2019-1

M.A. Wiering and M. van Otterlo (2012) Reinforcement Learning: State-of-the-Art, Springer Berlin Heidelberg (ISBN 978-3-642-27644-6) see here. A cool development is that it is now being translated (2016) in Chinese and will appear in print in that language too!

M. van Otterlo (2009) The Logic of Adaptive Behavior, IOS Press (ISBN 978-90-365-2677-7).

For more information see here. In addition, you can find some online contents there, such as a table of contents and the preface about Borges, libraries and more ... The complete book can be downloaded here

M. van Otterlo (2008) The Logic of Adaptive Behavior: Knowledge Representation and Algorithms for Adaptive Sequential Decision Making in First-Order and Relational Domains, PhD thesis, Department of Computer Science, University of Twente, The Netherlands. ISBN 978-90-365-2677-7   ---   512 pp.

Cover design by Joop Wever. Download it here. Here's a tag cloud to show what's in it:

If you're interested in a general, 10-minutes,'first-order MDPs for dummies'-kind of talk, then you can take a look at my slides.

Ramon, Vens, Driessens, van Otterlo, Vanschoren (eds.) Benelearn 2010 informal (web) proceedings 2010

K. Driessens, A. Fern and M. van Otterlo (eds.) Proceedings of the ICML'05 Workshop on Rich Representations for Reinforcement Learning 2005

M van Otterlo, M. Poel and A. Nijholt (eds) Proceedings of the Fourteenth Belgium-Dutch Conference on Machine Learning (Benelearn'05) CTIT, 2005

Recent, general (for more publications, specifically on algorithm/AI Ethics, see the other tab above)

Martijn van Otterlo (2018) From Algorithmic Black Boxes to Adaptive White Boxes: Declarative Decision-Theoretic Ethical Programs as Codes of Ethics Proceedings of the Benelux conference on Artificial Intelligence (as extended abstract and oral presentation) Download here.

Frederique van den Boogaart, Thomas Hantke and Martijn van Otterlo (2018) Predicting next week redemption in a digital frequency loyalty program, Proceedings of the Benelux conference on machine learning. Download here.

Ali El Hassouni, Mark Hoogendoorn, Martijn van Otterlo and Eduardo Barbaro (2018) Personalization of Health Interventions using Cluster-Based Reinforcement Learning, to appear in Proceedings of the International Conference on Principles and Practice of Multi-Agent Systems (see here for a previous version in the mean time).

Martijn van Otterlo (2018) From Algorithmic Black Boxes to Adaptive White Boxes: Declarative Decision-Theoretic Ethical Programs as Codes of Ethics Proceedings of the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, see on here and arXiv

Paul Ozkohen, Jelle Visser, Martijn van Otterlo and Marco Wiering (2017) Learning to Play Donkey Kong Using Neural Networks and Reinforcement Learning, proceedings of the Dutch-Belgian Conference on Artificial Intelligence (see the preproceedings and it was selected to appear in the postproceedings by Springer, to appear)

download here.
van Otterlo, M. and Warnaar, M. (2017) Towards Optimizing the Public Library: Indoor Localization in Semi-Open Spaces and Beyond, proceedings of the Benelux Conference on Machine Learning (Benelearn-2017, distinguished paper award)

P. Kordjamshidi, M. van Otterlo and M.-F. Moens (2017) Spatial Role Labeling Annotation Scheme, in Handbook of Linguistic Annotation, edited by James Pustejovsky and Nancy Ide. Springer

Structured Representations in Vision, Robotics, and Spatial Linguistics

S. vd Leemput and M. van Otterlo, DupLog: (2013) Probabilistic Logical Interpretation of Duplo Assemblies from 3D Vision, Proceedings of BNAIC-2013, accepted

Bogdan Moldovan, Martijn van Otterlo, Luc De Raedt, Plinio Moreno, and José Santos-Victor (2014) Statistical Relational Learning of Object Affordances for Robotic Manipulation. Latest Advances in Inductive Logic Programming: pp. 95-103.

M. van Otterlo (2013) Relations between Reinforcement Learning, Visual Input, Perception and Actionb> In Dagstuhl Abstract Book at http://dx.doi.org/10.4230/DagRep.3.8.1

H. Cuayahuitl, M. van Otterlo, N. Dethlefs and L. Frommberger (2013) Machine Learning for Interactive Systems and Robots: A Brief Introduction in Proceedings of MLIS at IJCAI 2013

B. Moldovan, P. Moreno, M. van Otterlo (2013) On the Use of Probabilistic Relational Affordance Models for Sequential Manipulation Tasks in Robotics, IEEE International Conference on Robotics and Automation (ICRA)

L. Antanas, M. van Otterlo, J. Oramas M., T. Tuytelaars and L. De Raedt. There are Plenty of Places like Home: Using Relational Representations in Hierarchies for Distance-based Image Understanding (2013, officially 2014) Neurocomputing journal.

B. Moldovan, P. Moreno, M. van Otterlo, J. Santos-Victor and L. De Raedt (2012) Learning Relational Affordance Models for Robots in Multi-Object Manipulation Tasks, IEEE International Conference on Robotics and Automation (ICRA), pp 4373--4378, 10.1109/ICRA.2012.6225042

P. Kordjamshidi, P. Frasconi, M. Van Otterlo, M.-F. Moens and L. De Raedt (2012) Relational Learning for Spatial Relations Extraction from Natural Language, in Proceedings of Inductive Logic Programming, Springer Lecture Notes on Artificial Intelligence (LNAI), volume 7207, pp 204--220 (10.1007/978-3-642-31951-8_20)

L. Antanas, M. van Otterlo, J. Oramas M., T. Tuytelaars and L. De Raedt. A Relational Distance-Based Framework for Hierarchical Image Understanding (2012) International Conference on Pattern Recognition (ICPRAM) (1,2) (best paper award!)

P.K. Gerke, J. Langevoort, S. Lagarde, L. Bax, T. Grootswagers, R.-J. Drenth, V. Slieker, L. Vuurpijl, P. Haselager, I. Sprinkhuizen-Kuyper, M. van Otterlo and G. de Croon (2011) BioMAV: bio-inspired intelligence for autonomous flight in Proceedings of the International Micro Air Vehicle Conference and Flight Competition (pdf version, biomav website

P. Kordjamshidi, J. Hois, M van Otterlo, M.-F. Moens (2012) Learning to Interpret Spatial Natural Language in Terms of Qualitative Spatial Relations to appear in Representing Space in Cognition by T. Tenbrink, C. Claramunt and J.M. Wiener (eds.), Oxford University Press

B. Moldovan, M. Van Otterlo, P. Moreno Lopez, J. Santos-Victor and L. De Raedt (2011) Statistical Relational Learning of Object Affordances for Robotic Manipulation (short paper) International Conference on Inductive Logic Programming (ILP)

P. Kordjamshidi, P. Frasconi, M. Van Otterlo, M-F. Moens and L. De Raedt (2011) Spatial Relation Extraction using Relational Learning (short paper) International Conference on Inductive Logic Programming (ILP)

P. Kordjamshidi, J. Hois, M van Otterlo, M.-F. Moens (2011) Learning to Interpret Spatial Natural Language in terms of Qualitative Spatial Relations (extended abstract) in Proceedings of the International conference on Spatial Information Theory (COSIT)

P. Kordjamshidi, M. van Otterlo and M.-F. Moens Spatial Role Labeling: Towards Extraction of Spatial Relations from Natural Language (2011) ACM Transactions on Speech and Language Processing Volume 8 Issue 3, December 2011 (10.1145/2050104.2050105)

L. Antanas, M. van Otterlo, J. Oramas M., T. Tuytelaars and L. De Raedt. (2010) Not Far Away from Home: a relational distance-based approach to understand images of houses in Proceedings of the international conference on Inductive Logic Programming (ILP), Lecture Notes in Computer Science 6489, Springer, pp.22--29

P. Kordjamshidi and M. van Otterlo and M.-F. Moens (2010) From Language towards Formal Spatial Calculi in Proceedings of the Workshop on Computational Models of Spatial Language at the International Conference on Spatial Cognition

P. Kordjamshidi and M. van Otterlo and M.-F. Moens (2010) Spatial Role Labeling: Task Definition and Annotation Scheme in Proceedings of International Conference on Language Resources and Evaluation (LREC) (pdf paper

L. Antanas, I. Thon, M. van Otterlo, N. Landwehr, L. De Raedt (2009) Probabilistic Logical Sequence Learning for Video (extended abstract, poster) at the International Conference on Inductive Logic Programming (ILP)(pdf paper

L. Antanas, M. van Otterlo, L. De Raedt and I. Thon (2009) Learning Probabilistic Relational Models from Sequential Video Data with Applications in Table-top and Card Games Proceedings of the Benelux conference on machine learning (Benelearn)(pdf paper)

Relational, Reinforcement Learning and Decision-theoretic Logic

G. Van den Broeck, I. Thon, M. Van Otterlo and L. De Raedt (2010) DTProbLog: A Decision-Theoretic Probabilistic Prolog AAAI Conference on Artificial Intelligence (pdf paper)

M. van Otterlo (2012) Solving Relational and First-Order Markov Decision Processes: A Survey(PDF), pp. 253--292 in M.A. Wiering and M. van Otterlo (eds.) Reinforcement Learning: State-of-the-art, Springer (10.1007/978-3-642-27645-3_8)

M. van Otterlo (2009) Intensional Dynamic Programming: A Rosetta Stone for Structured Dynamic Programming in Journal of Algorithms, volume 64, pp. 169--191, 2009, doi

M.A. Wiering and M. van Otterlo (2012) Reinforcement Learning: State-of-the-Art, Springer Berlin Heidelberg (ISBN 978-3-642-27644-6) see here

M. van Otterlo and M.A. Wiering (2012) Markov Decision Processes: Concepts and Algorithms M.A. Wiering and M. van Otterlo (eds.) Reinforcement Learning: State-of-the-art, Springer

M. van Otterlo (2009) The Logic of Adaptive Behavior, IOS Press (ISBN 978-90-365-2677-7)

M. van Otterlo Relational Reinforcement Learning: Lifting Propositional Algorithms and Size Matters OEGAI Journal, volume 27, issue 1, 2008.

M. van Otterlo and M.A. Wiering (2012), Preface of: M.A. Wiering and M. van Otterlo (eds.) Reinforcement Learning: State-of-the-art, Springer

M.A. Wiering and M. van Otterlo (2012) Conclusions and Outlook, Chapter 20 in M.A. Wiering and M. van Otterlo (eds.) Reinforcement Learning: State-of-the-art, Springer

M. van Otterlo and T. De Vuyst Evolving and Transferring Probabilistic Policies for Relational Reinforcement Learning (Proceedings of BNAIC 2009, pp 201--208)

I. Thon, B. Guttman, M. van Otterlo, N. Landwehr and L. De Raedt From Non-deterministic to Probabilistic Planning with the help of Statistical Relational Learning (ICAPS - 2009 - WS on Planning and Learning) - submitted version

M. van Otterlo A Survey of Reinforcement Learning in Relational Domains CTIT Technical Report, TR-CTIT-05-31, July 2005, 70 pp, CTIT Technical Report Series, ISSN 1381-3625)

T.J. Muller and M. van Otterlo Evolutionary Reinforcement Learning in Relational Domains Proceedings of the 7th European Workshop on Reinforcement Learning (EWRL), Napels, 2005

M. van Otterlo Reinforcement Learning for Relational MDPs Proceedings of the Annual Machine Learning Conference of Belgium and the Netherlands, 8-9 jan 2004 in Brussels, Belgium

M. van Otterlo and K. Kersting Challenges for Relational Reinforcement Learning in Proceedings of the Relational Reinforcement Learning Workshop at ICML-2004

M van Otterlo, K.Kersting and L. De Raedt Bellman Goes Relational (Extended Abstract) Proceedings of the Belgium-Dutch Conference on Artificial Intelligence (BNAIC'04)

K.Kersting, M.van Otterlo and L. De Raedt Bellman Goes Relational in Proceedings of the International Conference on Machine Learning ( ICML-2004)

M. van Otterlo Relational Representations in Reinforcement Learning: Review and Open Problems Proceedings of the ICML'02 Workshop on Development of Representations, 2002

M. van Otterlo Efficient Reinforcement Learning using Relational Aggregation Proceedings of the Sixth European Workshop on Reinforcement Learning EWRL-6, 2003

A.M. Bazen and M. van Otterlo and S.H. Gerez and M. Poel
A Reinforcement Learning Agent for Minutiae Extraction from Fingerprints Proceedings of The Belgium-Netherlands Artificial Intelligence Conference (BNAIC'01), 2001

Other and Older

Our submission for the NAO Programming competition at the international conference on social robots (ICSR-2013) got first prize!. The 4 page abstract was titled Do You like Puppies? Speed Dating with a NAO Robot and was written by M.L.E. de Bruijn, J. van Dalen, F.W. van Rijn and M. van Otterlo.

Two of my students of my master course on human-robot interaction created a robotic demo in which humans are taught how to make karate moves (using a Nao robot and Kinect camera). Check it out at the Aldebaran website

Vanwinckelen, van Otterlo, Driessens, Polin Reliable Power Control for secondary users based on distributed measurements Proceedings of Fifth IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks 2011(DySPAN)

van Otterlo, M., Wiering, M. A., Dastani, M. and Meyer, J.-J. Ch. (2007), A Characterization of Sapient Agents in: Mayorga, R. V. and Perlovsky, L. I. (eds.), Toward Computational Sapience: Principles and Systems, chapter 9, Springer.

M. van Otterlo, De Kunst van het Zoeken: Een Borgiaanse Zoektocht naar Creativiteit, Kunst en Wetenschap, 2008.

M. van Otterlo, M. Wiering, M. Dastani and J.-J. Meyer A Characterization of Sapient Agents Proceedings of the International Conference on the Integration of Knowledge Intensive Multi-Agent Systems KIMAS'03, 2003 (This text was reprinted the following year in the Sapience Monograph by Mayorga)

M. van Otterlo, M. Wiering, M. Dastani and J.-J. Meyer A Characterization of Sapient Agents Mayorga, Sapience Monograph

M. van Otterlo and E. de Jong BeNeLearn'02 BNVKI Newsletter, February 2003, Volume 20, No. 1, ISSN 1566-8266

M. van Otterlo Thesis Review of "Relational Reinforcement Learning" by Kurt Driessens (2004) in BNVKI Newsletter, 2004

M. van Otterlo Relational Reinforcement Learning Parlezine, local newsletter of the HMI group, Twente (2004)

M. van Otterlo Simulated Emotions and Emotional Simulations Parlezine, local newsletter of the HMI group, Twente (2004)

My MSc Thesis was titled This is probably it: Plausibility in a Logical Semantics of Discourse and it dealt with extending dynamic semantics of natural language discourses with probabilities.