Skip to content

About

Generate image endpoints by posting base64 image data to a temp. url WITH DB ACCESS

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Latest commit

 

History

24 Commits

Folders and files

Repository files navigation

rust-base64-resolver-gen

Ultra-fast, standalone, storage-optimized image resolution and persistence microservice in Rust.

Overview

rust-base64-resolver-gen is a high-performance Actix-web service designed for:

  1. In-Memory LRU Caching: Fast temporary image generation and URL resolution.
  2. Semi-Cached Persistent DB/Object Storage: Storing images into PostgreSQL, MongoDB, AWS S3, or Hetzner Storage, with automatic in-memory semi-caching for instant reads.
  3. Dedicated API Tokens: Separate REST authentication tokens for cache access (cache_token) and persistent storage access (storage_token).
  4. Intelligent Image Compression: WebP conversion, metadata/EXIF stripping, dimension downscaling, and content-addressable deduplication.

Zero legacy external bot DB dependencies — works 100% standalone.


Features

  • Separate REST API Tokens:
    • cache_token: Protects posting into the in-memory LRU cache (POST /api/cache).
    • storage_token: Protects posting into persistent DB/cloud storage (POST /api/storage).
  • Semi-Cached Storage:
    • Persistent images written to DB/S3 are instantly cached in memory.
    • Read requests hit RAM first; only cache misses query the database/S3.
  • Multiple Storage Backends:
    • PostgreSQL: Standalone images table with raw binary BYTEA storage and hash deduplication.
    • MongoDB: Native BSON binary storage with unique hash index deduplication.
    • AWS S3 & Compatible: S3, MinIO, Cloudflare R2, Wasabi, Backblaze B2, Ceph.
    • Hetzner Storage Box: Direct WebDAV over HTTPS (https://uXXXXX.your-storagebox.de/).
    • Hetzner Object Storage: S3-compatible bucket in fsn1, hel1, nbg1.
    • In-Memory Cache Only: Ultra-low overhead RAM-only mode.
  • Automated Compression & Storage Savings:
    • WebP conversion (60-80% smaller than PNG/JPEG).
    • EXIF & camera metadata stripping (saves 20KB-2MB per photo).
    • Smart downscaling for oversized assets.
    • "Only If Smaller" safety guard (never bloats files).
  • Interactive Web Interface:
    • Drag-and-drop, clipboard paste, or base64 input at /upload.
    • Explorer gallery with pagination at /explore.

API Endpoints

1. Post to LRU Cache (POST /api/cache)

Requires cache_token (via Authorization: Bearer <token> or X-Cache-Token or X-API-Token).

curl -X POST http://127.0.0.1:3555/api/cache \
  -H "Authorization: Bearer YOUR_CACHE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "base64": "<base64_encoded_image_data>",
    "metaName": "temp_preview.png"
  }'

Response:

{
  "urlPath": "/image/4885fe20-5c02-4fc4-8097-9e42cbfeebf1",
  "id": "4885fe20-5c02-4fc4-8097-9e42cbfeebf1",
  "hash": "b10a8db164e0754105b7a99be72e3fe5",
  "target": "cache",
  "mimeType": "image/webp",
  "originalSizeBytes": 524288,
  "storedSizeBytes": 94210,
  "savingsPercent": "82.0%",
  "compressed": true
}

2. Post to DB Storage (POST /api/storage)

Requires storage_token (via Authorization: Bearer <token> or X-Storage-Token or X-API-Token).

curl -X POST http://127.0.0.1:3555/api/storage \
  -H "Authorization: Bearer YOUR_STORAGE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "base64": "<base64_encoded_image_data>",
    "metaName": "avatar.png"
  }'

Response:

{
  "urlPath": "/storage/b10a8db164e0754105b7a99be72e3fe5",
  "dbUrlPath": "/md5/b10a8db164e0754105b7a99be72e3fe5",
  "cacheUrlPath": "/image/9f2b8441-2a1e-457a-9db2-6d2c49c2a4f0",
  "hash": "b10a8db164e0754105b7a99be72e3fe5",
  "id": "9f2b8441-2a1e-457a-9db2-6d2c49c2a4f0",
  "target": "storage",
  "provider": "postgres",
  "mimeType": "image/webp",
  "originalSizeBytes": 524288,
  "storedSizeBytes": 94210,
  "savingsPercent": "82.0%",
  "compressed": true,
  "deduplicated": false
}

3. Unified Upload (POST /image & POST /image/multipart)

Accepts target: "cache", "storage", or "both" (default). Validates the appropriate token based on target.


4. Semi-Cached Image Retrieval

  • Retrieve by Cache ID / Temporary URL:
    curl -X GET http://127.0.0.1:3555/image/{id}
  • Retrieve by Persistent Storage Hash:
    curl -X GET http://127.0.0.1:3555/storage/{hash}
    # or
    curl -X GET http://127.0.0.1:3555/md5/{hash}
    (Served directly from memory if cached; loads from DB/cloud and warms cache on miss)

Documentation

Full guides and technical specifications are available in the docs/ folder:

  • API_DOCS.md: Complete REST API specification with endpoints, request/response JSON schemas, error codes, and curl examples.
  • CONFIG.md: Comprehensive configuration guide covering database/cloud providers, image compression, LRU caching, and authentication settings.

Provider Configuration Examples

Pre-configured .toml templates for each storage provider can be found in examples/:


Quick Start

cp example.config.toml config.toml
cargo run --release

License

MIT

About

Generate image endpoints by posting base64 image data to a temp. url WITH DB ACCESS

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages