attn

The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"

HydraNet is a state-of-the-art transformer architecture that combines Multi-Query Attention (MQA), Mixture of Experts (MoE), and continuous learning capabilities.

Comfy Nodes (and a CLI script) for shuffling around layers in transformer models, creating a curious confusion.

Shuffle GPT-2's layers and have it prompt an image. Node works with any model - Flux, SD3, SDXL...