Metasearch Aggregator

The problem

Standard search APIs are often expensive, rate-limited, or compromise user privacy through tracking. Developers needing aggregated results from multiple engines currently have to manage multiple API keys and normalize varying data formats manually.

What it does

• Deploys a local SearXNG instance using Docker for private, self-hosted search.
• Queries the SearXNG JSON API using curl and jq for structured data retrieval.
• Aggregates and deduplicates results from Google, Bing, DuckDuckGo, and other engines into a single list.
• Filters search results by specific categories like news or science and recency time ranges.
• Handles engine-specific selection to compare rankings or source variety.

Frameworks & tools

Docker, SearXNG, curl, jq, and JSON API integration.

Why this beats prompting it yourself

Writing raw prompts for search often leads to hallucinated links or reliance on a single engine's bias. This skill provides the specific technical orchestration to stand up a metasearch aggregator and parse real-time, multi-engine data into clean JSON, ensuring you get actual live URLs rather than training-data guesses.

Use cases

• Build a privacy-first search tool that doesn't track user queries.
• Create a fallback search mechanism when primary search APIs are down.
• Generate automated reports by scraping and merging results from multiple search engines.
• Monitor recent news or scientific papers using specific category and time filters.

Known limitations

Does not support downloading or scraping heavy media like images and videos directly. Not suitable for searches requiring account-scoped personalization or private user data.