Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos

Foundation models are used for many real-world applications involving language understanding and generation from temporally-ordered multimodal events. In this work, we study the ability of models to identify the most important sub-events in a video, which is a fundamental prerequisite for summarizin