about the result

[kuangxiaocheng-zz]

Hello, I have reproduced the experiments on the AlfWorld dataset using gpt-4o-mini within the Autogen framework, but both the baseline and gmemory fail to achieve the results reported in the paper. May I ask what the reason for this could be?

[bingreeky]

Thank you very much for your question and for taking the time to reproduce the experiments. Could you kindly share more details about your setup—specifically your machine environment, package versions, and the execution logs? It is possible that recent updates to the AlfWorld environment may be contributing to the discrepancy, but we cannot determine the cause without more detailed information. We greatly appreciate your assistance in helping us diagnose this issue.

[kuangxiaocheng-zz]

total_task.log

Thank you for your response! This is the experiment log. Initially, I used Alfworld version 0.3.5, but it consistently encountered bugs as shown in the figure, resulting in a lower task success rate. To address this, I changed the Alfworld version to 0.3.4, which prevented the recurrence of similar bugs. However, the task success rate still did not match the level reported in the original paper.