Leveraging Human Teachers’ Perceptions of Task Performance In Robot Learning

Our work aims to improve robots’ abilities to learn from the natural feedback that naive, human teachers provide. This is particularly important in settings where the robot is tasked to learn in real-world settings, directly from the feedback, or reward, provided by the end user. Most learning paradigms in human-robot interaction are rooted in assumptions about the reward signal, like that it is Markovian (independent of history) and stationary (objective and consistent). These assumptions are misaligned with how humans, who are influenced by their changing expectations of the learner as the interaction progresses, teach. 

Through our research, we are looking to show and model how human teachers’ perceptions of a robot learner’s performance throughout an interactive task affect explicit, evaluative feedback from the teacher. Performance related factors include sequences of successes and failures by the robot, the robot’s overall performance (e.g., improving or declining), the perceived difficulty of the task, and the magnitude of the robot’s failures and successes. With this expectation model, we hope to develop algorithms that interpret the human feedback signal, such that it leads to more efficient learning of the goals and representation of the task by the robot.

We plan on using a cognitively difficult card selection game, called Set, as the task that the human is teaching the robot. Set involves finding and choosing a collection of cards that fulfill certain criteria, based on relationships between their features. Set is an interesting task for humans to teach because even with a perfect understanding of the rules and a complete view of the state space, the difficulty of a problem will vary between boards and teachers. We expect a wide distribution of evaluation responses by the teachers, and plan to model how expectations change within and between each teachers’ interaction with the robot.