| Ground truth | Single Agent with table | Multi Agent with table | Single Agent with image | Multi Agent with image |
|---|---|---|---|---|
| STEP 1: Turn to your right and go past the kitchen island, then turn to your left and go to the garbage bin on the floor, past the counter, on your right.
STEP 2: Pull out the tomato that is in the garbage bin on the floor. STEP 3: Turn to your right and go to the stove, then turn to your right again and go across the kitchen to the refrigerator. STEP 4: Place the tomato on the on the bottom shelf of the refrigerator, to the right of the lettuce, in the freezer, wait a moment for the tomato to freeze, then remove the tomato from the freezer. STEP 5: Turn around and go across the kitchen to the refrigerator, then turn left and go past the counter on your right, to the garbage bin on the floor to your right. STEP 6: Put the tomato back in the garbage bin. |
Turn around and walk to the countertop.,
Take the tomato from the countertop., Turn around and walk to the fridge., Place the tomato in the freezer section of the fridge and wait until it is frozen., Take the frozen tomato out of the fridge., Turn around and walk to the garbage bin., Place the frozen tomato into the garbage bin. |
Turn around and walk to the freezer.
Take the frozen tomato out of the freezer. Turn around and walk to the garbage can. Put the frozen tomato into the garbage can. |
1. Walk to the refrigerator located on the left side of the kitchen.
2. Open the refrigerator door. 3. Reach for the frozen tomato inside the refrigerator. 4. Close the refrigerator door. 5. Turn around to face the kitchen. 6. Walk to the garbage bin located on the right side of the kitchen, near the counter. 7. Open the garbage bin lid. 8. Dispose of the frozen tomato in the garbage bin. 9. Close the garbage bin lid. |
1. Walk to the island at the center of the kitchen.
2. Pick up the tomato on the island. 3. Walk to the sink on the countertop. 4. Open the cabinet under the sink to reveal the garbage bin. 5. Put the tomato above the garbage bin into the garbage bin. 6. Close the cabinet under the sink. |