Multi-agent Planning using Visual Language Models

ECAI 2024

Place a wet sponge inside the bathtub (trial_T20190906_234735_610018)

Ground truth Single Agent with table Multi Agent with table Single Agent with image Multi Agent with image
STEP 1: Walk to the bathtub.
STEP 2: Pick up the green sponge from the bathtub.
STEP 3: Turn around and walk over to the sink.
STEP 4: Place the sponge in the sink on its side and then turn the faucet on briefly before turning it off again.
STEP 5: Walk back to the bathtub.
STEP 6: Place the sponge in the bathtub to the left of the pink rag.
Turn around and walk to the countertop.,
Take the wet sponge from the countertop.,
Turn around and walk to the bathtub.,
Place the sponge inside the bathtub.
Turn and walk to the CounterTop at position (-1.892, 0.704, -0.528).,
Take the DishSponge from the CounterTop.,
Turn and walk to the Bathtub at position (-3.706, 0.342, 0.636).,
Wet the DishSponge in the BathtubBasin.,
Place the wet DishSponge in the Bathtub.
1. Walk to the sponge located on the left side of the vanity.
2. Pick up the sponge.
3. Turn towards the bathtub located on the right side of the bathroom.
4. Walk to the bathtub.
5. Place the wet sponge inside the bathtub.
1. Walk to vanity at position in front of the toilet.
2. Pick up sponge on vanity.
3. Walk to sink on vanity.
4. Wet sponge under sink faucet.
5. Walk to bathtub next to vanity.
6. Place wet sponge inside the bathtub.
Architecture