Scene understanding for robot manipulation and beyond
January 1, 2026
Overview
Our research is focused on developing efficient and effective deep learning models for scene understanding, with the goal
of providing useful information to robots to perform complex manipulation tasks. We are particularly interested in enabling
robots to understand and reason about the functionality of objects in the scene, so that they can effectively interact
with them and assist humans in everyday tasks.
Key results
NeurIPS26
Learning to Evolve Scenes: Reasoning about Human Activities with Scene GraphsGiven a scene graph and a description of an activity, GLEN predicts how the scene will change, turning scene graphs into a common language for long-horizon reasoning.
SAMWISE: Infusing wisdom in SAM2 for Text-Driven Video SegmentationSegments objects in videos from a natural-language description, adding language and temporal reasoning to SAM2 with a lightweight adapter.
ERIS: Efficient Referring Image Segmentation via Lightweight Bi-Directional Multi-Modal AlignmentSegments the object described by a sentence with a lightweight model that aligns vision and language in both directions, suited to robots.
FreeREA: Training-Free Evolution-based Architecture SearchFinds efficient neural architectures without training a single candidate, combining training-free metrics with evolutionary search.
When it comes to deploying deep learning models into edge devices like robots, model footprint is a major limit. Therefore,
all the mainstream research focused on delivering the most accurate architectures leveraging foundation models is prone to fail
in robotic applications. Our research demonstrate that it is feasible to optimize the accuracy-efficiency tradeoff, delivering
scene understanding architectures which trim significantly the model computational demand (up to -70% w.r.t. mainstream solutions),
while preserving (if not slightly increasing) its accuracy. We delivered solutions for both images and videos, and developed
methods capable to reason not only about the semantics of the objects, but also their functionality and affordances, opening
interesting perspectives for robotics applications at scale.
FocoosAI: from research to the market
Efficiency is not only a research question for us. In 2022 we co-founded
FocoosAI, a spin-off of Politecnico di Torino
that brings small, efficient and green computer vision models to real-world applications. FocoosAI shares the same
goal as this line of research, making accurate vision models light enough to run where they are needed, and the
collaboration keeps going: some of our PhD students are co-supervised together with the FocoosAI team. Read more about why we started it in
this post.
