Questions about PersonaMem-v2 evaluation setting and LLM judge models

[Litmeb]

Hi, thank you for releasing the codes. I am trying to reproduce and understand the PersonaMem-v2 results, and I have a few questions about the evaluation protocol.

Could you clarify the following points?

1. For the PersonaMem-v2 results reported in the paper, did you evaluate it as a multiple-choice task or as an open-ended QA task? As this benchmark has 2 evaluation modes, I am not sure which is used.

2. The paper says the protocol follows MemOS and uses “GPT-4o-mini and two auxiliary judge models.” I checked the public MemOS evaluation code, and it seems that LoCoMo / LongMemEval use GPT-4o-mini as the judge, sometimes with multiple repeated runs, while PersonaMem is evaluated via multiple-choice exact match. I could not find the two auxiliary judge models in the public MemOS implementation. Could you clarify what the two auxiliary judge models are, and whether they are part of your own evaluation setup or inherited from MemOS?

3. If possible, could you point to the exact script/config used to produce the PersonaMem-v2 numbers in the paper?

This clarification would be very helpful for fair reproduction and comparison with other memory systems. Thanks!