Cheap Log Analytics
Автор: Cardinality Cloud
Загружено: 2026-01-09
Просмотров: 72
Tired of observability vendors charging you per gigabyte? Watch me build a production-grade AWS CloudFront analytics system that handles 15 TiB of data for just $279/month ($3,348/year).
In this video, I'll show you how to set up CloudFront log analytics with ClickHouse Cloud and visualize it with Grafana - all while avoiding the per-gigabyte pricing trap.
WHAT YOU'LL LEARN
✅ Set up AWS CloudFront logging to S3
✅ Configure ClickHouse Cloud with S3Queue table engine
✅ Avoid AWS Data Transfer Out (DTO) charges completely
✅ Query latency percentiles with SQL
✅ Visualize with CLI, ClickHouse Console, GNUPlot, and Grafana
✅ Build production dashboards on a fixed budget
COST BREAKDOWN
ClickHouse Cloud 8GiB Instance: $220/month
S3 Storage (15 days staging): $8/month
ClickHouse Storage (90 days, 2 TiB): $51/month
S3 API Requests: $0.43/month
TOTAL: $279/month for 15 TiB of analytics capability
Compare that to vendors charging per gigabyte!
TIMESTAMPS
00:00 Introduction - Breaking the per-gigabyte pricing trap
01:00 The "Walk" stage of observability
02:00 Architecture overview
03:00 Budget breakdown and cost optimization
05:00 Implementation Step 1: Enable CloudFront logging
06:00 Implementation Step 2: AWS IAM roles (the hard part)
07:00 Implementation Step 3: ClickHouse S3Queue schema
09:00 Implementation Step 4: Debugging S3 access
10:00 Query latency percentiles with SQL
11:00 ClickHouse Cloud Console dashboards
12:00 Grafana Cloud integration
13:00 Tradeoffs: Latency and storage
15:00 Conclusion and call to action
RESOURCES
📚 Full Blog Post
https://cardinality.cloud/blog/aws-we...
📖 ClickHouse S3Queue Documentation
https://clickhouse.com/docs/engines/t...
🔐 Accessing S3 Data Securely
https://clickhouse.com/docs/cloud/dat...
☁️ CloudFront Logging Guide
https://docs.aws.amazon.com/AmazonClo...
📊 Grafana Cloud Free Tier
https://grafana.com/products/cloud/
🗄️ ClickHouse Cloud
https://clickhouse.com/cloud
ABOUT CARDINALITY CLOUD
I'm Jack Neely, Independent Observability Architect and owner of Cardinality Cloud, LLC - where observability is leverage, not a tax. With 25 years of SRE and DevOps experience and over a decade as an observability subject matter expert, I help teams avoid six-figure observability bills.
Need help building something like this for your team? Let's talk before your CFO asks why observability is a six-figure line item.
🌐 Website: https://cardinality.cloud/
💼 LinkedIn: / jack-neely-47316575
📧 Contact: jjneely@cardinality.cloud
#observability #aws #cloudfront #clickhouse #grafana #devops #sre #costoptimization #loganalytics
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке: