Summary
get_files(directory) does not work on either driver, in two different ways. Listing a subdirectory is the common case (get_files("backups"), get_files("audio")), and neither driver handles it.
1. S3: passing a directory always returns an empty list
def get_files(self, directory=None):
bucket = self.get_resource().Bucket(self.get_bucket())
if directory:
objects = bucket.objects.all().filter(Prefix=directory)
else:
objects = bucket.objects.all()
files = []
for my_bucket_object in objects.all():
if "/" not in my_bucket_object.key: # ← here
files.append(File(my_bucket_object, my_bucket_object.key))
return files
The filter is presumably meant to skip "subdirectories", but it tests the whole key, not the part below the prefix. Any key matched by a non-empty Prefix necessarily contains the prefix followed by /, so it is always discarded:
keys in bucket: ['backups/a.dump', 'backups/b.dump', 'root.txt', 'audio/x/y.mp3']
matched by Prefix='backups': ['backups/a.dump', 'backups/b.dump']
kept by `"/" not in key`: ['root.txt'] ← and root.txt is not even under the prefix
So get_files("backups") returns [] whenever the prefix is non-empty. Only a root-level listing returns anything.
Suggested fix
Strip the prefix before testing for a nested separator:
prefix = f"{directory.rstrip('/')}/" if directory else ""
for obj in objects.all():
relative = obj.key[len(prefix):]
if relative and "/" not in relative:
files.append(File(obj, obj.key))
2. Local: file contents are read from the wrong path
def get_files(self, directory=""):
file_path = self.get_path(directory)
files = []
for f in os.listdir(file_path):
if not isfile(join(file_path, f)):
continue
files.append(File(self.get(f), f)) # ← here
return files
os.listdir yields bare names, so for get_files("audio") the loop calls self.get("tts-1.mp3"). get() resolves through get_path(), which joins against the disk root, not root/audio. The file is looked up at root/tts-1.mp3, misses, and get() swallows the FileNotFoundError and returns None.
The result is a list of File objects with None content — no error, just silently empty files. It only appears to work when directory is empty.
Suggested fix
files.append(File(self.get(join(directory, f)), f))
Why it matters
The two drivers also disagree about what get_files means, which makes it unusable in disk-agnostic code: on local it lists files in a directory, on S3 it only ever lists the bucket root. Anything that needs "the most recent file under this prefix" has to drop to boto3 directly or maintain its own manifest.
A fake driver equivalent would be worth adding alongside, since FakeDriver currently has no get_files at all.
Environment
- fastapi-startkit 0.56.0
- Python 3.13.7
Summary
get_files(directory)does not work on either driver, in two different ways. Listing a subdirectory is the common case (get_files("backups"),get_files("audio")), and neither driver handles it.1. S3: passing a directory always returns an empty list
The filter is presumably meant to skip "subdirectories", but it tests the whole key, not the part below the prefix. Any key matched by a non-empty
Prefixnecessarily contains the prefix followed by/, so it is always discarded:So
get_files("backups")returns[]whenever the prefix is non-empty. Only a root-level listing returns anything.Suggested fix
Strip the prefix before testing for a nested separator:
2. Local: file contents are read from the wrong path
os.listdiryields bare names, so forget_files("audio")the loop callsself.get("tts-1.mp3").get()resolves throughget_path(), which joins against the disk root, notroot/audio. The file is looked up atroot/tts-1.mp3, misses, andget()swallows theFileNotFoundErrorand returnsNone.The result is a list of
Fileobjects withNonecontent — no error, just silently empty files. It only appears to work whendirectoryis empty.Suggested fix
Why it matters
The two drivers also disagree about what
get_filesmeans, which makes it unusable in disk-agnostic code: on local it lists files in a directory, on S3 it only ever lists the bucket root. Anything that needs "the most recent file under this prefix" has to drop toboto3directly or maintain its own manifest.A
fakedriver equivalent would be worth adding alongside, sinceFakeDrivercurrently has noget_filesat all.Environment