Explainable and Interpretable Reinforcement Learning for Robotics

This book surveys the state of the art in explainable and interpretable reinforcement learning (RL) as relevant for robotics. While RL in general has grown in popularity and been applied to increasingly complex problems, several challenges have impeded the real-world adoption of RL algorithms for robotics and related areas. These include difficulties in preventing safety constraints from being violated and the issues faced by systems operators who desire explainable policies and actions. Robotics applications present a unique set of considerations and result in a number of opportunities related to their physical, real-world sensory input and interactions.

The authors consider classification techniques used in past surveys and papers and attempt to unify terminology across the field. The book provides an in-depth exploration of 12 attributes that can be used to classify explainable/interpretable techniques. These include whether the RL method is model-agnostic or model-specific, self-explainable or post-hoc, as well as additional analysis of the attributes of scope, when-produced, format, knowledge limits, explanation accuracy, audience, predictability, legibility, readability, and reactivity. The book is organized around a discussion of these methods broken down into 42 categories and subcategories, where each category can be classified according to some of the attributes. The authors close by identifying gaps in the current research and highlighting areas for future investigation.

Aaron M. Roth is currently Head of Autonomy Technology for Black Sea. Previously, Aaron worked as a Research Scientist in the Distributed Autonomous Systems Group in the Navy Center for Applied Research in Artificial Intelligence at the Naval Research Laboratory and as a Researcher and PhD student at the University of Maryland in the GAMMA lab. In both capacities, he led projects conducting research into autonomous robots and artificial intelligence, with specific focus on reinforcement learning, explainable/interpretable artificial intelligence, AI Safety, robot navigation, and human-robot interaction. He has also worked at several technology startups, in industries spanning healthcare, finance, and mobile consumer applications. Aaron’s published research has been reported on by news publications worldwide, and he has presented his research speaking at international conferences in the academic research community and given talks explaining science, robotics, and artificial intelligence to the general public. He is the creator of and contributor to multiple open-source software projects. He holds a B.S.E. in Electrical Engineering from University of Pennsylvania and an M.S. in Robotics from Carnegie Mellon University. Aaron was Conference Chair for the Third Annual Artificial Intelligence Safety Unconference in 2021, an international conference about AI Safety. In both paid and volunteer capacities, Aaron has provided technical mentorship to undergraduate students, graduate students, and industry career professionals across the United States, Europe, and Australia. He is also a published science fiction author. Aaron has consulted on artificial intelligence topics for creative professionals including science fiction authors and game designers.＂

Ram D. Sriram is currently the chief of the Software and Systems Division, Information Technology Laboratory, at the National Institute of Standards and Technology. Before joining the Software and Systems Division, Sriram was the leader of the Design and Process group in the Manufacturing Systems Integration Division, Manufacturing Engineering Laboratory, where he conducted research on standards for interoperability of computer-aided design systems. Prior to joining NIST, he was on the engineering faculty (1986-1994) at the Massachusetts Institute of Technology (MIT) and was instrumental in setting up the Intelligent Engineering Systems Laboratory. He has extensive experience in developing knowledge-based expert systems, natural language interfaces, machine learning, object-oriented software development, life-cycle product and process models, geometrical modelers, object-oriented databases for industrial applications, health care informatics, bioinformatics, and bioimaging. Sriram has co-authored or authored nearly 300 publications, including several books. Sriram was a founding co-editor of the International Journal for AI in Engineering. Sriram received several awards including: an NSF’s Presidential Young Investigator Award (1989); ASME Design Automation Award (2011); ASME CIE Distinguished Service Award (2014); the Washington Academy of Sciences’ Distinguished Career in Engineering Sciences Award (2015); ASME CIE division’s Lifetime Achievement Award (2016); CMU CEE Lt. Col. Christopher Raible Distinguished Public Service Award (2018); IIT Madras Distinguished Alumnus Award (2021). Sriram is a Fellow of AAIA, AIBME, ASME, AAAS, IEEE, IET, INCOSE, SMA, and Washington Academy of Sciences, a Distinguished Member (life) of ACM, a Senior Member (life) AAAI, and an Honorary Member of IISE. Sriram has a B.Tech. from IIT, Madras, India, and an M.S. and a Ph.D. from Carnegie Mellon University, Pittsburgh, USA.

Elham Tabassi is a Senior Research Scientist at the National Institute of Standards and Technology (NIST) and the Associate Director for Emerging Technologies in the Information Technology Laboratory (ITL). She also leads NIST’s Trustworthy and Responsible AI program that aims to cultivate trust in the design, development, and use of AI technologies. As the ITL’s Associate Director for Emerging Technologies, Elham assists NIST leadership and management at all levels in determining future strategic direction for research, development, standards, testing and evaluation in the areas of emerging technologies such as artificial intelligence. She also coordinates interaction related to artificial intelligence with the U.S. research community, U.S. industrial community, international standards community, and other federal agencies; and provides leadership within NIST in the use of AI to solve scientific and engineering problems arising in measurement science and related use-inspired applications of AI.Elham has been working on various machine learning and computer vision research projects with applications in biometrics evaluation and standards since she joined NIST in 1999. She is a member of the National AI Resource Research Task Force, vice-chair of OECD working party on AI Governance, Associate Editor of IEEE Transaction on Information Forensics and Security, and a fellow of Washington Academy of Sciences. In 2023, TIME magazine has named Elham Tabassi in its list of the 100 most influential people in AI.

Dinesh Manocha is the Paul Chrisman Iribe Chair in Computer Science & Electrical and Computer Engineering and Distinguished University Professor at the University of Maryland College Park. He is also the Phi Delta Theta/Matthew Mason Distinguished Professor Emeritus of Computer Science at the University of North Carolina - Chapel Hill. He has won many awards, including Alfred P. Sloan Research Fellow, the NSF Career Award, the ONR Young Investigator Award, Google Faculty Awards, Facebook Faculty Awards, Northrop Grumman faculty award, and the Hettleman Prize for scholarly achievement. His research interests include multi-agent simulation, virtual environments, artificial intelligence, and robotics. His group has developed a number of packages for multi-agent simulation, GPU computing, and physics-based simulation that have been used by millions of users and licensed to more than 60 commercial vendors. He has published more than 725 papers and supervised more than 46 PhD dissertations. His group has received 21 best paper awards at the leading conferences in virtual reality, computer graphics, multimedia, high performance computing, etc. He is an inventor of more than 15 patents, several of which have been licensed to industry. His work has been covered by the New York Times, NPR, Boston Globe, Washington Post, ZDNet, as well as DARPA Legacy Press Release. He is a Fellow of AAAI, AAAS, ACM, and IEEE, member of ACM SIGGRAPH Academy, and received Bézier Award from Solid Modeling Association. He received the Distinguished Alumni Award from IIT Delhi and the Distinguished Career in Computer Science Award from Washington Academy of Sciences. He was a co-founder of Impulsonic, a developer of physics-based audio simulation technologies, which was acquired by Valve Inc in November 2016. He recently co-founded Inception Robotics, Inc.