Skip to content

Optimise the generic CSV data exporter by compressing the data stream in realtime before landing it on disk #2

Description

@shaneelprasad

Databases compress stored data, which requires considerable local storage when SDF unloads a raw copy to disk, then compresses it, which typically requires around 6X the amount of local storage compared to the compressed version on the database.

The approach to reduce the local storage requirement (and possible performance improvements) is to compress the data stream as it is being unloaded from source

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions