Skip to content

Large Bulkrax export zipfiles don't open #3478

@lsat12357

Description

@lsat12357

Descriptive summary

Appears to be a problem when a user tries to export the content files with the metadata, and the size reaches > 4G. On the server, the zipfiles are located at tmp/shared/exports; testing the zip with unzip -t shows the error "bad zipfile offset." Kevin notes that the download available from the collection page (same content) works fine though; the code behind this service is here https://github.com/OregonDigital/OD2/blob/master/app/services/oregon_digital/collection_streamer.rb and here https://github.com/OregonDigital/OD2/blob/master/app/services/concerns/oregon_digital/streaming_download_behavior.rb. Is it possible to reuse this code to replace whatever bulkrax does?
The other option would be to break the zipfile into smaller packages; Bulkrax does this based on number of files, but not file size.

Expected behavior

Bulkrax users can export metadata and content files with total package size exceeding 4G

Related work

Link to related tickets or prior related work here.

Accessibility Concerns

Add any information here to indicate any known or suspected accessibility issues for this ticket

Metadata

Metadata

Assignees

No one assigned

    Labels

    Content OpsIssues related to import, export, bulkrax, and content processing.MetadataIssues related to metadata configuration, application, and representationPriority - MediumIssues that should be prioritized ahead of low but not immediately critical - bulk of work cyclesbulkrax

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions