A modern, GPU-accelerated video codec built from first principles using CUDA and C++17, replacing Discrete Cosine Transforms with Partitioned Iterated Function Systems (PIFS).
Modern video compression standards such as H.264, HEVC, and AV1 rely on Discrete Cosine Transforms (DCT) to discard imperceptible high-frequency details. While effective for everyday streaming, DCT-based codecs can suffer from severe structural degradation—known as macroblocking—under ultra-low bitrates and are fundamentally bound to fixed pixel grids.
FractalCodec revives and modernizes Fractal Image and Video Compression, an approach largely abandoned in the late 1990s due to CPU computational limitations.
Instead of storing discrete pixel values or frequency coefficients, FractalCodec encodes video frames as Iterated Function Systems (IFS)—sets of contractive affine transformations that mathematically generate the target imagery.
By offloading the exhaustive, high-dimensional block-matching search to thousands of parallel CUDA threads and utilizing on-chip shared memory hierarchies, FractalCodec aims to achieve near-real-time encoding of mathematically self-similar video streams.
Because the compressed stream is composed of mathematical functions rather than raster pixels, frames can be decoded at resolutions higher than the source input without conventional bilinear blur.
Edge fidelity is synthesized natively through fractal reconstruction.
Under aggressive compression ratios such as 500:1+, the codec is designed to avoid conventional square macroblock boundaries and instead degrade toward smooth, continuous contours.
The historically expensive O(N²) encoding bottleneck is mapped onto thousands of CUDA threads.
Domain-range block matching is parallelized across GPU thread blocks with parallel reduction strategies used to identify the best candidate transformations.
The host architecture is written in C++17 and follows RAII principles.
A custom:
CudaBuffer<T>abstraction manages GPU allocations and releases resources automatically, reducing the risk of GPU memory leaks and double frees.
Decoding is designed to be significantly cheaper than encoding.
A compressed frame can be reconstructed iteratively, starting from an initial image and applying the stored transformations for several passes.
The codec treats every video frame as the attractor of an Iterated Function System, based on the contraction principle described by the Banach Fixed-Point Theorem.
The target frame is partitioned into small, non-overlapping Range Blocks:
Rᵢ = 4 × 4 pixels
Each range block represents a region that needs to be reconstructed.
A larger set of overlapping Domain Blocks is extracted from the same frame or, depending on the encoding mode, from preceding keyframes.
Example:
Dⱼ = 8 × 8 pixels
Domain blocks provide candidate source regions that can be transformed into the corresponding range blocks.
For every Range Block, CUDA threads concurrently evaluate candidate Domain Blocks.
Each candidate can be evaluated under multiple geometric isometries, such as:
- Identity
- Horizontal reflection
- Vertical reflection
- 180° rotation
- 90° rotations
- Reflected rotations
This allows the encoder to search for self-similar structures throughout the image.
For each candidate domain block, the encoder solves for the optimal contrast and brightness parameters:
s— contrast/scaling coefficiento— brightness offset
The objective is to minimize the reconstruction error:
where:
-
$R_i$ is the target Range Block. -
$D_j$ is a candidate Domain Block. -
$\phi(D_j)$ represents a geometric transformation of the Domain Block. -
$s$ controls contrast. -
$o$ controls brightness.
After the best candidate has been identified, the encoder stores only the parameters required to reconstruct the Range Block.
A .frac stream therefore contains information such as:
Domain X/Y Offset
Isometry / Transformation Flag
Contrast (s)
Brightness (o)
Range Block Position
Conceptually:
Range Block
│
▼
Best Domain Block
│
├── Spatial Offset
├── Isometry
├── Contrast
└── Brightness
fractal-codec/
├── docs/
│ └── # Technical documentation, CLI guides, and Colab instructions
│
├── include/
│ └── fractal/
│ ├── core/
│ │ └── # Encoder, Decoder, and Fractal math data structures
│ │
│ ├── cuda/
│ │ └── # RAII VRAM wrappers and CUDA error checkers
│ │
│ └── io/
│ └── # Video container demuxers and frame extraction
│
├── src/
│ ├── core/
│ │ └── # Host-side orchestration logic
│ │
│ ├── cuda/
│ │ └── # Parallel search and reduction kernels (.cu)
│ │
│ ├── io/
│ │ └── # Video container demuxers and frame extraction implementation
│ │
│ └── main.cpp
│ └── # CLI entry point
│
├── tests/
│ └── # GoogleTest test suite with GPU auto-detection fixtures
│
├── third_party/
│ └── # Experimental tools, including Jupyter notebooks for Google Colab testing
│
├── CMakeLists.txt
│ └── # Target-based CMake configuration for NVCC and CXX
│
└── CONTRIBUTING.md
└── # Contribution workflow, clang-format, and testing rules
-
NVIDIA GPU
-
Compute Capability 7.5+
-
Supported architectures include:
- Turing
- Ampere
- Ada Lovelace
- Hopper
- Blackwell
Note: If you do not have a compatible local NVIDIA GPU, you can still test and develop the CUDA kernels using our interactive Jupyter notebook on Google Colab. See docs/COLAB_CUDA.MD for a quickstart guide.
| Dependency | Requirement |
|---|---|
| NVIDIA CUDA Toolkit | 11.8+ |
| Recommended CUDA | 12.x |
| C++ Standard | C++17 |
| GCC | 9+ |
| Clang | 11+ |
| MSVC | 2019+ |
| CMake | 3.21+ |
| OpenCV | Optional |
OpenCV is used for direct ingestion of video formats such as:
.mp4
.mkv
git clone https://github.com/your-username/fractal-codec.git
cd fractal-codeccmake -B build -DCMAKE_BUILD_TYPE=Releasecmake --build build --config Release -j$(nproc)cmake --build build --config ReleaseThe automated test suite uses GoogleTest.
GPU-dependent tests can detect the availability of NVIDIA hardware and safely skip GPU-specific assertions when an appropriate GPU is unavailable.
ctest --test-dir build --output-on-failureCompress an input video into a FractalCodec bitstream:
./build/fractal_cli encode -i sample.mp4 -o encoded.frc| Parameter | Description |
|---|---|
encode |
Subcommand to encode |
-i, --input |
Input video |
-o, --output |
Output .frc stream |
Decode an encoded bitstream back into a playable video:
./build/fractal_cli decode -i encoded.frc -o reconstructed.mp4| Parameter | Description |
|---|---|
decode |
Subcommand to decode |
-i, --input |
Input .frc stream |
-o, --output |
Reconstructed video |
Potential applications include:
- Remote sensing
- Deep-sea communications
- Space communications
- Other bandwidth-constrained environments
where maintaining structural information under extreme compression is important.
Potential applications include imagery such as:
- MRI
- CT
- Satellite imagery
- Topographical data
where smooth gradients and high-resolution inspection are important.
Fractal representations may be useful for extremely compact procedural assets in real-time 3D engines, potentially reducing the storage requirements of texture and video assets.
Contributions are welcome!
Please review CONTRIBUTING.md for information about:
- Coding standards
clang-formatconfiguration- Branch naming conventions
- Commit conventions
- Testing requirements
- Pull request workflow
FractalCodec is distributed under the MIT License.
See LICENSE for the complete license text.
FractalCodec is an experimental research-oriented codec exploring the use of Partitioned Iterated Function Systems (PIFS) and GPU-accelerated fractal block matching as an alternative approach to conventional transform-based video compression.
The project is intended to investigate the practical trade-offs between:
Compression Ratio
│
├── Encoding Complexity
│
├── Decoding Complexity
│
├── Reconstruction Quality
│
└── GPU Acceleration
rather than serving as a drop-in replacement for established codecs such as H.264, HEVC, or AV1.