Works with the AI Agents you already use
Flash Attention Tuner
Swap transformer attention over to memory-efficient kernels: enable PyTorch's fused SDPA path, benchmark speed, verify numerical parity, and configure sequence-
Free
Flash Attention Tuner
Example session with this skill installed
I'm hitting OOM on my 4096 seq length transformer. Convert my manual attention block to use fused Flash Attention kernels and benchmark the memory savings.
- Read your context and instructions
- Compiled the flash attention tuner
Replaced manual QK^T implementation with F.scaled_dot_product_attention.
Benchmark (4096 seq):
- Manual: 45.2ms | 12.4GB VRAM
- Fused: 12.8ms | 1.8GB VRAM
Max Delta: 8.4e-3 (FP16).
Verified backend: SDPBackend.FLASH_ATTENTION.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Training or inference on long sequences (512+ tokens) often triggers CUDA Out-of-Memory (OOM) errors. Manual attention implementation using explicit matrix multiplication and softmax operations is memory-intensive and creates significant bottlenecks in your GPU profile.
What it does
- Swaps manual attention blocks for PyTorch Scaled Dot Product Attention (SDPA) fused kernels.
- Forces and verifies specific backends like Flash Attention or Memory Efficient kernels.
- Benchmarks latency and peak memory usage before and after optimization.
- Performs numerical parity checks to ensure bit-level agreement between implementations.
- Configures sequence length, head dimensions, and causal masks for transformer architectures.
Frameworks & tools
PyTorch (2.2+), CUDA, NVIDIA Nsight Systems (nsys), and torch.nn.functional.
Why this beats prompting it yourself
Optimizing attention kernels requires specific knowledge of PyTorch backend constraints, such as head dimension limits and dtype requirements. This skill provides a structured verification workflow that ensures you are actually hitting the fused path rather than silently falling back to slow, memory-heavy implementations.
Use cases
- Fixing OOM errors in custom transformer architectures during training.
- Optimizing inference speed for LLMs handling long context windows.
- Extending the context length of existing models on limited VRAM.
- Auditing attention performance via automated micro-benchmarking.
Known limitations
Requires CUDA-enabled GPUs; kernels are not available for CPU-only workloads. Optimal speedups are restricted to fp16 or bf16 dtypes and head dimensions under 256.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.
Requires CUDA-capable hardware and PyTorch 2.2 or higher. Flash kernels are only available for fp16 and bf16 dtypes with head dimensions up to 256.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
3 uses
Times used to date
Free to use
No account required to browse
Trust & safety
Security scan
Security scan passed 3 months ago
- Free with an Agensi account