Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization

Mingde Harry Zhao, Safa Alver, Harm van Seijen, Romain Laroche, Doina Precup, Yoshua Bengio

paper / GitHub / Mila Blogpost

(work largely done during Harry, Harm and Romain’s time at MSR)

[Accepted @ICLR2024]This is our 2nd work on integrating conscious information processing behavior into Reinforcement Learning (RL) agents. Specifically, this paper seeks to address the inabilities of RL agents to generalize its learned skills in longer-term reasoning scenarios: a trained RL agent often cannot perform well in the target task due to the incompetence to handle training-evaluation discrepancy. This work builds upon our previous work on spatial abstraction (NeurIPS 2021) in planning, raising the attention of the missing flavor of spatial abstraction in the existing temporal abstraction frameworks.

The overall framework of Skipper

For more details, please check the blog post here, by Mila.