Context Matters! Relaxing Goals with LLMs for Feasible 3D Scene Planning

ICRA 2026

Emanuele Musumeci 1, Michele Brienza 1, Francesco Argenziano 1, Abdel Hakim Drid 2, Vincenzo Suriani 1, Daniele Nardi 1, Domenico Daniele Bloisi 3
1Sapienza University of Rome 2University of Biskra 3International University of Rome
Paper arXiv Code

🎥 Summary Video

Architecture

🌟 Motivations

  • Planning in real world is made difficult by the need to ground the robot's perception of the environment to planning predicates.
  • The strictness of classical planning causes failures even when the task is still achievable (up to a certain degree).
  • LLM partially overcomes this issue thanks to commonsense reasoning, but often lead to unsafe or incorrect plans.

    • 📝 Contributions

  • A novel contextual goal-relaxation formalism that reasons along two axes (functionality and feasibility) to preserve user intent while yielding executable goals.
  • A planning framework that couples LLM commonsense for goal proposal with classical planning for feasibility validation and plan synthesis.
  • A new dataset of 141 relaxation-prone tasks compatible with popular 3D environments and 3DSGs

    • Methodology

    Methodology
    Our formalism is mainly represented by two operators, the Γshift and Δrel. Γshift represents the situational shifting, namely the operator that adapts the agent's understanding of the operating environment to the core and the planning goal. Δrel represents the complexity relaxation operators that is able to produce a more general or comprehensive formulation of the goal. Suppose we have an agent which is able to explore the environment and map every object he found to the corresponding location, effectively creating a 3D Scene Graph of the environment. The 3D Scene Graph and the natural language description of the task are given into the architecture as our input. The PDDL problem file is generated by the LLM, while the domain file is either given or generated to. The plan obtained at the current time step is used to attempt grounding the scene. If this step fails, the relaxation mechanism takes place and new PDDL files are generated for the next iteration. In this way, the architecture is able to find the optimal subset of objects needed to achieve the task. Thanks to this bidimensional mechanism, the architecture is able to gradually relax the problem until the least relaxed solution is found. Effectively, what we found is the minimal traverse cost path inside our relaxation graph.

    BibTeX

    @article{musumeci2025context,
  title={Context Matters! Relaxing Goals with LLMs for Feasible 3D Scene Planning},
  author={Musumeci, Emanuele and Brienza, Michele and Argenziano, Francesco and Suriani, Vincenzo and Nardi, Daniele and Bloisi, Domenico D},
  journal={arXiv preprint arXiv:2506.15828},
  year={2025}
}