Real-world Sequential Decision Making: Reinforcement Learning and Beyond
Real-world Sequential Decision Making Workshop @ ICML 2019
June 14, 2019 - June 15, 2019. Long Beach, USA


From conversational agents to online recommendation to search and advertising, we are already interacting with increasingly sophisticated sequential decision making systems in daily life. Traditionally, sequential decision making research has focused on balancing the exploration-exploitation trade-off, or casting the interaction paradigm under reinforcement / imitation learning dichotomy. We aim to take a holistic view and call for a collective effort to translate principled research ideas into practically relevant solutions in both existing and new domains, such as healthcare, education, safe autonomous vehicles and robots, etc.

This workshop aims to bring together researchers from industry and academia in order to describe recent advances and discuss future research directions pertaining to real-world sequential decision making, broadly construed. We aim to highlight new and emerging research opportunities for the machine learning community that arise from the evolving needs for making decision making theoretically and practically relevant for realistic applications.

Research interest in reinforcement and imitation learning has surged significantly over the past several years, with the empirical successes of self-playing in games and availability of increasingly realistic simulation environments. We believe the time is ripe for the research community to push beyond simulated domains and start exploring research directions that directly address the real-world need for optimal decision making. We are particularly interested in understanding the current theoretical and practical challenges that prevent broader adoption of current policy learning and evaluation algorithms in high-impact applications, across a broad range of domains.


Long Beach Convention Center, Long Beach
8:45 am - 2:30 pm
June 14, 2019

Session (tentative)

8:45 - 9:00 Opening Remarks
9:00 - 9:30 Invited Talk
9:30 - 10:00 Invited Talk
10:00 - 10:30 Morning Coffee Break and Poster Session
10:30 - 11:00 Invited Talk
11:00 - 11:30 Invited Talk
11:30 - 12:00 Poster Session
12:00 - 12:30 Invited Talk
12:30 - 14:00 Lunch and Closing Remarks

Keynote Speakers

Emma Brunskill

Assistant Professor
Stanford University

Miro Dudík

Principal Researcher
Microsoft Research

Andreas Krause

ETH Zürich

Suchi Saria

Assistant Professor
Johns Hopkins

Dawn Woodard

Director of Data Science

Accepted Papers

The list of accepted papers will be posted after the author notification date.

Call for Papers

Papers submitted to the workshop should be up to five pages long excluding references and appendix, and in ICML 2019 format. As the review process is not blind, authors can reveal their identity in their submissions. All inquiries may be sent to .

Submissions page: Real-world Sequential Decision Making Workshop 2019.

We invite researchers to submit both theoretical and applied work along several possible dimensions:

  • Application papers: applications of learning-based techniques to real-life sequential decision making (robotics, healthcare, education, transportation & energy, smart-grids, sustainability, NLP, social media & advertising, agriculture, manufacturing, economics and policy)

  • Method papers that address real-world desiderata and concerns: safety, reliable decision making, theoretical guarantees, verifiability & interpretability, data-efficiency, data-heterogeneity, efficient exploration, counterfactual reasoning and off-policy evaluation, cost function design, efficient implementations to large-scale systems

  • Cross-boundary papers along the theme of RL+X where X indicate areas not commonly viewed as RL in contemporary research. We would like to encourage researchers to explore the interface between traditional RL with:

    • Other related areas in machine learning including but not limited to: imitation learning, transfer learning, active learning, structured prediction, off-policy learning, fairness in ML, privacy

    • Areas outside of machine learning including but not limited to: control theory & dynamical systems, formal methods, causal inference, game-theory, operations research, systems research, human-computer interactions, human behavior modeling

Key Dates


Paper Submission Deadline: May 20, 2019, 11:59 PM PST

Author Notification: May 26, 2019, 11:59 PM PST

Final Version: June 13, 2019, 11:59 PM PST

Workshop: June 14 or 15, 2019

Workshop Organizers

Yisong Yue


Adith Swaminathan

Microsoft Research AI

Byron Boots

Georgia Tech & NVIDIA

Ching-An Cheng

Georgia Tech