Skip to content

Please add environment variable to play with chunksize for upload to stream  #91

Description

@kansalankit

Hi Team,

We are dealing with millions of rows and current implementation of uploading to stream fails after sometime as it process the data in smaller chunks.

Can you please allow chunksize to be defaulted to your way of implementation and also allow users to change it through environment variable.

I have done this custom implementation and making it work - however it unnecessary creates a dependency of us creating a wheel file with changed code.

Let me know if you have any doubts...

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions