Works with the AI Agents you already use
Llava Vision Assistant
Use a LLaVA vision-language model for visual question answering, image captioning, OCR and region-level grounding. Covers local inference via transformers/llama
Free
Llava Vision Assistant
Example session with this skill installed
Show me how to use llama.cpp to get the bounding box of a 'Submit' button in screenshot.png using LLaVA.
- Read your context and instructions
- Compiled the llava vision assistant
Use the llama-llava-cli with the GGUF model and mmproj file. Prompt: "Output the bounding box of the Submit button as x1,y1,x2,y2 normalized to 0-1000." Result: [450, 800, 550, 850]. Use Pillow to draw this box on your source image to verify the hit.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Integrating visual intelligence into apps often forces a choice between expensive proprietary APIs or complex, undocumented local setups. Extracting text from screenshots or identifying UI elements programmatically remains brittle without a multimodal model.
What it does
- Implements visual question answering (VQA) for screenshots, photos, and charts.
- Performs OCR to transcribe text verbatim from images into raw text lines.
- Executes region-level grounding to identify object coordinates via normalized bounding boxes.
- Configures local inference via transformers or llama.cpp for fully offline processing.
- Handles batch image processing for dataset pre-labeling and validation.
Frameworks & tools
Hugging Face Transformers, llama.cpp, GGUF, Python, Pillow, and LLaVA 1.5 checkpoints.
Why this beats prompting it yourself
Prompting a vision model is only half the battle. This skill provides the exact templates for local GPU/CPU deployment and the post-processing scripts needed to turn raw model output into structured data like bounding boxes and JSON lines.
Use cases
- Describe screenshots automatically to populate bug reports or documentation.
- Extract structured data from legacy dashboard photos where no API exists.
- Automate UI testing by locating buttons and elements via spatial grounding.
- Pre-label large image datasets before manual human review.
Known limitations
Not suitable for pixel-level editing or high-stakes medical diagnosis. Dense document parsing is better handled by dedicated OCR engines.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.
Not for pixel-level editing or medical-grade certainty. Dense document parsing may require dedicated OCR engines. Local 7B models require ~5-6GB RAM.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
3 uses
Times used to date
Free to use
No account required to browse
Trust & safety
Security scan
Security scan passed 3 months ago
- Free with an Agensi account