Official Review · ICLR 2024

The paper discusses the use of memory mechanisms in language processing and reinforcement learning agents. It introduces the concept of Automatic Chunking, a model that combines the TransformerXL architecture with a cyclic memory buffer and chunk selection mechanism. The model is tested on two tasks: an audio-visual instructions task and a visual corridor task with a variable distractor. The results show that Automatic Chunking with ForgetSpan improves training performance and memory efficiency. The model demonstrates the ability to learn a proper mapping between audio and visual observations and recall information after a variable distractor phase. This paper offers a pioneering approach to enhancing memory efficiency in transformers by emulating human cognitive processes. Its introduction of a modified TransformerXL architecture is both innovative and timely. The paper further stands out with its trio of novel techniques: Automatic Chunking, ForgetSpan, and Similarity based forgetting, each addressing unique memory optimization challenges. Comprehensive testing across diverse tasks not only validates the model's robustness but also underscores its relevance in real-world robotic applications. In essence, the paper shines in its blend of innovation, rigorous evaluation, and practical applicability. - Some statements require more evidence and clarification. - Introduction: "Human cognition and decision-making works on reflection on only relevant parts of memory.". Should add citations of related works to support argument. - Introduction: "Robotic agents should have similar cognition to function well in long horizon and multi-modal tasks like navigation or human-robot collaboration.". Should add citations of related works or analysis to support argument. - Lack of comparative analysis - The paper primarily compares the proposed model with the baseline TransformerXL. A more extensive comparison with other state-of-the-art models might provide a clearer picture of the model's standing in the current research landscape. - Limited discussion about result - Most discussions only state the data provided in the figure and concluded as "our method is better than baseline", and further discussion about the result is missing. - How do you tackle the overfitting issue? While the paper mentions that the model generalizes well by testing on modified versions of tasks, there might be concerns about overfitting, especially if the modifications are not significantly different from the original tasks.