DPG-Bench is definitely a great work in T2I generation evaluation. As for better evaluating T2I models on DPG-Bench, I notice that you recommend generating 4 images for each prompt and forming them into a 2×2 grid, so I wonder how you set the seed for generation during evaluation (The original paper seems not to mention it..). Should I use four preset fixed seed or directly use random seed considering the average operation may mitigate the variation brought by randomness? I'm not sure which one is more reasonable.
DPG-Bench is definitely a great work in T2I generation evaluation. As for better evaluating T2I models on DPG-Bench, I notice that you recommend generating 4 images for each prompt and forming them into a 2×2 grid, so I wonder how you set the seed for generation during evaluation (The original paper seems not to mention it..). Should I use four preset fixed seed or directly use random seed considering the average operation may mitigate the variation brought by randomness? I'm not sure which one is more reasonable.