More screenshots
Works with the AI Agents you already use
DeepSpeed: Distributed Training That Scales
Train large models across multiple GPUs with DeepSpeed — ZeRO stages, mixed precision, and parallelism chosen by measurement, one dimension at a time.
$5
DeepSpeed: Distributed Training That Scales
Example session with this skill installed
We have a 13B-parameter model and 8× A100 80GB GPUs. Training OOMs at ZeRO-1 and crawls at ZeRO-3. Give me the configuration and the reasoning.
- Read your context and instructions
- Compiled the deepspeed
13B on 8× A100 80GB — configuration and reasoning
Sizing first: parameters in BF16 are about 26GB — they fit on one GPU. The problem is Adam optimizer states in FP32, roughly 100GB+. ZeRO-1 shards only the optimizer and still OOMs because gradients plus activations do not fit either. ZeRO-3 shards everything but the communication overhead is why it crawls — you are paying full parameter-sharding cost for a model that does not need it.
Configuration: ZeRO-2 with CPU offload of optimizer states, BF16, micro-batch 4, gradient accumulation 8 (effective batch 256), gradient clipping 1.0. ds_report run first — fused kernels and async I/O verified before the run starts.
If it still OOMs: enable gradient checkpointing before reaching for ZeRO-3 — checkpointing trades compute for activation memory, which is what sharding does not cover, and it is cheaper than ZeRO-3's communication tax.
If throughput is poor: profile communication vs compute. If communication dominates, reduce the stage or raise the micro-batch — do not add GPUs to a communication-bound run. If it diverges: check BF16 vs FP16 first, then rescale the learning rate for the new effective batch size. Rubric score: 7/8 — the remaining point is the throughput baseline, which can only be measured on the first real run.
Connects securely to your tools. The creator never sees your data.
About this skill
DeepSpeed exists because models got bigger than GPUs — but most distributed-training failures are configuration errors, not hardware limits. This skill teaches the levers in the right order: ZeRO stages (use the lowest stage that fits; each stage up trades memory for communication), mixed precision (BF16 on Ampere-and-newer, FP32 for loss and clipping, learning-rate rescaling when the effective batch changes), and parallelism (data parallel first, pipeline for deep models, tensor for wide layers — each added one at a time with throughput measured per step).
You get a minimal ZeRO-2 configuration with every choice justified, a decision table for the seven common situations (OOM, hang, throughput collapse, NaN loss), and a debugging section that starts with ds_report before any GPU-hours burn. A worked decision path sizes a 13B model on 8× A100 80GB from first principles. A rubric gates the launch: below 6 out of 8, the configuration is a guess with a JSON file.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
- It does not fix a bad training recipe — learning rates, data quality, and batch-size scaling still decide whether the model learns.
- It does not help single-GPU training meaningfully; if the model fits on one GPU, the skill says so and stops.
- It cannot abstract away the interconnect: on slow networks, no configuration recovers lost throughput.
- Throughput numbers are measured on your hardware, not promised by the skill — the decision table tells you what to measure, not what you will get.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
Permissions
No special permissions declared or detected