diff --git a/CHANGELOG.md b/CHANGELOG.md index 9eb0340..3040a46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ * [#45](https://github.com/mongoid/mongoid-grid_fs/pull/45): Encode stored file bytes rather than the model description in Base64 and data URIs - [@yudho](https://github.com/yudho), [@dblock](https://github.com/dblock). * [#48](https://github.com/mongoid/mongoid-grid_fs/pull/48): Check for the specific legacy BSON classes instead of the Moped namespace - [@jeremy6d](https://github.com/jeremy6d), [@dblock](https://github.com/dblock). +* [#77](https://github.com/mongoid/mongoid-grid_fs/issues/77): Expose chainable `where` queries on the default GridFS API; document custom namespaces and storage configuration - [@dblock](https://github.com/dblock). * [#86](https://github.com/mongoid/mongoid-grid_fs/pull/86): Expand persisted GridFS coverage for streaming, upload failures, namespaces, attributes, queries, historical records without `updateDate`, and Rails responses - [@dblock](https://github.com/dblock). * [#85](https://github.com/mongoid/mongoid-grid_fs/pull/85): Report test coverage to Coveralls and update CI badges - [@dblock](https://github.com/dblock). * [#84](https://github.com/mongoid/mongoid-grid_fs/pull/84): Migrate Danger to danger-pr-comment workflows - [@dblock](https://github.com/dblock). diff --git a/README.md b/README.md index efa9199..f0a5048 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,38 @@ mongoid-grid_fs A pure Mongoid/Moped implementation of the MongoDB GridFS specification +DESCRIPTION +----------- + +mongoid_grid_fs is A pure Mongoid/Moped implementation of the MongoDB GridFS specification + +Reference: http://docs.mongodb.org/manual/reference/gridfs/ + +It has the following features: + +- implementation is on top of mongoid for portability. moped (the driver) is + barely used, so the library should be quite durable except in the face of + massive changes to mongoid itself. + +- simple, REST-like api + +- support for custom namespaces (fs.files vs. image.files, as per the spec) + +- pathnames and io-like objects can be written to the grid + +- auto-unique pathnames are generated (by default) to avoid collisions using #put + + 'path/info/a.rb' -> '$object_id/a.rb' + +- [] and []= methods which allow the grid to be used like a giant file + hash in the sky + +- support for data_uris, like a html5 boss + + ```erb + <%= image_tag :src => file.data_uri %> + ``` + INSTALL ------- @@ -33,37 +65,36 @@ g.each { |chunk| file.write(chunk) } # streaming write ``` -DESCRIPTION ------------ - -mongoid_grid_fs is A pure Mongoid/Moped implementation of the MongoDB GridFS specification - -Reference: http://docs.mongodb.org/manual/reference/gridfs/ +QUERYING AND STORAGE +-------------------- -It has the following features: +`where` returns chainable Mongoid criteria for multiple files, while `find` returns the first match: -- implementation is on top of mongoid for portability. moped (the driver) is - barely used, so the library should be quite durable except in the face of - massive changes to mongoid itself. - -- simple, REST-like api - -- support for custom namespaces (fs.files vs. image.files, as per the spec) +```ruby +Mongoid::GridFs.where(filename: 'report.pdf').each { |file| puts file.id } +Mongoid::GridFs.where(contentType: 'application/pdf').limit(10) +``` -- pathnames and io-like objects can be written to the grid +To use `attachments.files` and `attachments.chunks` instead of `fs.files` and `fs.chunks`, use a named namespace for all operations: -- auto-unique pathnames are generated (by default) to avoid collisions using #put +```ruby +attachments = Mongoid::GridFs.namespace_for(:attachments) +file = attachments.put(readable, filename: 'report.pdf') +attachments.where(filename: 'report.pdf') +attachments.get(file.id) +attachments.delete(file.id) +``` - 'path/info/a.rb' -> '$object_id/a.rb' +For a separate host on Mongoid 5+, configure a named Mongoid client (for example `gridfs`) in `mongoid.yml`, then select it on both models during application initialization, before any GridFS operations: -- [] and []= methods which allow the grid to be used like a giant file - hash in the sky +```ruby +attachments.file_model.store_in client: :gridfs +attachments.chunk_model.store_in client: :gridfs +``` -- support for data_uris, like a html5 boss +The named client supplies the hosts, database, and credentials through normal Mongoid configuration. To change only the database, use `store_in database: 'attachments_database'` on both models instead. Keep files and chunks on the same client and database. These settings are shared model configuration, not per-request switches. Mongoid 3/4 use named sessions and `store_in session: :gridfs` instead of clients. - ```erb - <%= image_tag :src => file.data_uri %> - ``` +Use `file.destroy`, `grid_fs.delete(id)`, or `namespace.clear` to remove files and associated chunks. Mongoid's callback-skipping operations such as `file.delete`, `delete_all`, and direct collection deletes do not perform dependent destruction and can leave orphaned chunks. CONTRIBUTING ------------ diff --git a/lib/mongoid/grid_fs.rb b/lib/mongoid/grid_fs.rb index 1dcf1c7..8f29d42 100644 --- a/lib/mongoid/grid_fs.rb +++ b/lib/mongoid/grid_fs.rb @@ -24,6 +24,7 @@ def init! put get delete + where find [] []= diff --git a/test/grid_fs_interfaces_test.rb b/test/grid_fs_interfaces_test.rb index fca1a64..53ef421 100644 --- a/test/grid_fs_interfaces_test.rb +++ b/test/grid_fs_interfaces_test.rb @@ -1,6 +1,7 @@ # frozen_string_literal: true require_relative 'helper' +require 'tempfile' class GridFsInterfacesTest < Minitest::Test GridFs = Mongoid::GridFs @@ -98,10 +99,11 @@ def test_normalized_upload_attributes_survive_reload end def test_base64_and_data_uri_encode_the_stored_bytes - payload = "binary\x00\xffpayload".b * 12 - file = @namespace.put(StringIO.new(payload), filename: 'encoded.bin', chunk_size: 8) + payload = "binary\x00\xffpayload".b * ((@namespace.file_model.defaults.chunkSize / 14) + 1) + file = @namespace.put(StringIO.new(payload), filename: 'encoded.bin') file = @namespace.get(file.id) + assert_operator file.chunks.count, :>, 1 assert_equal [payload].pack('m'), file.base64 assert_equal payload, file.base64.unpack1('m') assert_equal "data:application/octet-stream;base64,#{[payload].pack('m').chomp}", file.data_uri @@ -153,6 +155,47 @@ def test_default_namespace_delegation_and_cascading_clear assert_raises(Mongoid::Errors::DocumentNotFound) { @namespace.get(file.id) } end + def test_default_namespace_where_returns_chainable_multiple_results + files = [ + GridFs.put(StringIO.new('one'), filename: 'where-regression.txt'), + GridFs.put(StringIO.new('two'), filename: 'where-regression.txt') + ] + + assert_equal files.map(&:id).sort, GridFs.where('where-regression.txt').map(&:id).sort + assert_equal files.map(&:id).sort, GridFs.where(filename: 'where-regression.txt').map(&:id).sort + assert_equal [files.last.id], GridFs.where(filename: 'where-regression.txt').where(md5: files.last.md5).map(&:id) + ensure + files&.each(&:destroy) + end + + def test_destroy_removes_all_chunks_without_removing_another_files_chunks + removed = @namespace.put(StringIO.new('x' * (@namespace.file_model.defaults.chunkSize + 1))) + retained = @namespace.put(StringIO.new('retained chunks')) + removed_chunk_ids = removed.chunks.map(&:id) + retained_chunk_ids = retained.chunks.map(&:id) + assert_operator removed_chunk_ids.size, :>, 1 + + @namespace.get(removed.id).destroy + + assert_empty @namespace.file_model.where(id: removed.id).to_a + assert_empty @namespace.chunk_model.where(:id.in => removed_chunk_ids).to_a + assert_equal retained_chunk_ids.sort, @namespace.chunk_model.where(files_id: retained.id).map(&:id).sort + assert_equal 'retained chunks', @namespace.get(retained.id).data + end + + def test_binary_path_upload_preserves_non_utf8_bytes + payload = "\x89PNG\r\n\x1a\n\xff\x00".b * ((@namespace.file_model.defaults.chunkSize / 10) + 1) + Tempfile.create(['gridfs-binary', '.png']) do |io| + io.binmode + io.write(payload) + io.flush + file = @namespace.put(io.path) + + assert_operator file.chunks.count, :>, 1 + assert_equal payload, @namespace.get(file.id).data + end + end + def test_streaming_and_slicing_across_more_than_one_fetch_batch data = "#{'abcdefgh' * GridFs::File.defaults.chunkSize}end" file = @namespace.put(StringIO.new(data))