minimum details required to reproduce Table 7 on paper

[acul3]

Hi @Carlofkl

Want to ask about deployment on device in your paper

1.Was your W8A8 model created using PTQ, QAT, quantization-aware distillation, or mixed precision?

2.Which layers/operations were kept above W8A8?

3. Maybe detail about how you do it?

I am interested on research quantize unet and diffusion etc

[Carlofkl]

Hi, @acul3

1. We adopt a PTQ algorithm with W8A8-INT8 per-channel symmetric quantization, and perform a clipping-value search based on 1204 calibration samples.

2. The quantization targets are restricted to GEMM and Conv operations, which means all normalization layers retain their original precision. In addition, several layers related to time embedding account for only a small portion of the runtime but are highly sensitive to quantization, offering poor cost-effectiveness; these include time_embedding (2 layers) and add_embedding (2 layers). The input and output layers of the backbone, conv_in and conv_out, are also kept unquantized.

3. For the overall quantization of the UNet backbone, we follow the MixDQ work, but extend the quantization granularity to per-channel. In fact, most current generative models can readily achieve lossless INT8 per-channel quantization.