Skip to content

checkerboard error after model INT8 PTQ #275

Description

@maGnet2C-cmd

Hi, thank you for your project.
I am trying to deploy LightStereo-S. I modified the model to output both prob and spx_pred, and I run disparity_regression and context_upsample on the CPU as part of the model post-processing. However, after INT8 PTQ, the results appear to have checkerboard error. Do you have any thoughts on this issue? Thank you.

Image

left: float model result
right: quant model result

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions