Skip to content

Address size limitations #3

Description

@kelle

"The paper suggests that SQLite and astrodbkit can be scaled to arbitrary size and scope (in the SQLite file size up to 140TB and large number of collaborations). The workflow as described will hit pratical limits much sooner than this. It's fine for a very small SQLite file to be directly included in a repository, but committing hundreds of megabytes or more as a binary blob will dramatically increase the size of the repository each time data is added - not incrementally, but by the size of the full database each time. The practical limit is much smaller. Further (as described below), the workflow only works when one user makes updates to the database at one time. The chances of "collisions" when multiple users do this at the same time increases quickly as collaborators are added. Of course, many of these problems are solved by using a client/server database where the data is not located in the repository."

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions