cookieless nginx analytics

Most self-hosted dashboards report wrong numbers because they ignore log rotation, fail to filter bots, or incorrectly sum unique visitors over time. Adding daily unique visitors together results in massive overcounting, while simple averages often decline as the measurement period grows. This tool provides a privacy-first way to measure real human traffic directly from nginx access logs without using cookies or third-party trackers.

What it does

• Unique visitor estimation uses HyperLogLog sketches to calculate the cardinality of the union across any time window without storing IP addresses.
• Bot filtering excludes self-declared crawlers to prevent automated traffic from inflating page view counts by up to 15x.
• Data accumulation solves the 14-day log rotation limit by persisting daily summaries into a non-rotated JSON storage format.
• GDPR compliance ensures no personal data or IP addresses are retained, using non-reversible probabilistic sketches instead.
• Multi-window reporting calculates precise statistics for today, yesterday, the last 7 days, and the last 30 days.

How it works

1. Configure environment variables for the nginx log path, the target site's URL pattern, and the JSON output destination.
2. Execute the collection script to process existing logs on disk and rebuild up to 14 days of historical data.
3. Schedule an hourly cron job to ensure "today's" statistics remain current throughout the day.
4. Read the JSON output from your frontend or dashboard to display deduplicated visitor counts and page views.

Frameworks & tools

The core logic uses Python 3 with the hyperloglog library. It requires access to nginx access_log files, typically found in /var/log/nginx/.

Why this beats prompting it yourself

Prompting an AI to count visitors usually results in a simple grep and wc -l script that ignores IP deduplication logic and log rotation. This skill implements specific mathematical sketches (HyperLogLog) that allow you to merge daily data into weekly or monthly totals without overcounting returning users.

Use cases

• Replace Google Analytics with a server-side, cookie-free privacy solution.
• Fix inflated traffic reports caused by aggressive search engine crawlers.
• Audit actual site reach when monthly log totals seem mathematically impossible.
• Monitor server-side traffic for GDPR-sensitive applications.

Known limitations

This skill only measures HTTP requests recorded in nginx logs. It cannot track client-side interactions like button clicks, scroll depth, or conversion funnels that do not trigger a page load.