Observability and SRE
Observability is only useful when on-call can answer two questions quickly: what broke, and who is impacted. I treat logs, metrics, and traces as complementary signals with different cost profiles - not as a vendor shopping list.
My SRE writing covers production debugging workflows, alert routing that scales with the team, chaos experiments that teach something actionable, and the extra instrumentation AI workloads need when latency and token cost both matter.
The articles here are written for platform and backend engineers who own services in production, not for teams that only collect dashboards. Start with the pillar posts if you are tightening SLIs, runbooks, or incident response this quarter.
- Logs vs Metrics vs TracesLogs tell you what happened, metrics how much and how fast, traces where time went. Use all three with correlation IDs-alert on metrics, debug with logs and traces.
- How to Debug a Slow API in ProductionWhen P99 spikes but errors stay flat, binary-search the request path with metrics and traces-starting at the edge, through the app, and into MySQL, Redis, and downstream dependencies.
- Slack Alerting That Scales with On-CallAlert fatigue is a routing and enrichment problem, not a Slack problem. Tiered channels, Alertmanager as source of truth, runbook annotations, and alert budgets that scale with on-call rotations.
Related writing
AI Infrastructure
AI Agent Infrastructure and Failure Modes
AI agents combine LLM unpredictability with tool execution and long-running state - which creates failure modes traditional APIs never had. These are the ones I have seen in production and how I design around them.
- AI Agents
- Reliability
- Security
Benchmarking
API Load Benchmarking for Platform Teams
Load tests that only report RPS miss the point. Platform teams need benchmarks that answer capacity questions, find breaking points before users do, and produce numbers engineering leadership can act on.
- Benchmarking
- APIs
- Performance
Automation
Automating Runbooks with GitHub Actions
Notion runbooks and 3 AM kubectl do not scale. A GitHub Actions pattern for reusable runbook workflows, OIDC to AWS, environment approval gates, and audit trails that on-call rotations can actually trust.
- GitHub Actions
- Automation
- SRE
Reliability
Chaos Engineering for Platform Reliability
Uptime during quiet weeks is not proof of resilience. A hypothesis-driven chaos playbook-tooling choices, blast radius controls, and the path from staging game days to bounded production experiments.
- Reliability
- SRE
- Kubernetes
CI/CD
Designing GitOps With Jenkins, GitHub Actions, and Argo CD
Most teams do not need one CI/CD tool to rule them all. After running Jenkins, GitHub Actions, and Argo CD together in production, the golden rule is simple: GHA for CI, Jenkins for specialized automation, Argo CD for Kubernetes CD.
- GitOps
- GitHub Actions
- Jenkins
System Design
Designing a Highly Available API Platform
High availability is explicit failure modes with mitigations-multi-AZ compute, RDS failover, graceful degradation, and a failure matrix you actually game-day.
- AWS
- System Design
- APIs
AI Infrastructure
Designing Reliable AI API Infrastructure
AI APIs fail differently from CRUD APIs - provider rate limits, token budgets, and unpredictable latency need a dedicated reliability layer. Here is how I design inference infrastructure on AWS that survives real traffic.
- AI
- APIs
- AWS
Reliability
Designing Zero-Downtime Deployments
Zero downtime needs compatible migrations, readiness probes, and graceful shutdown-then choose rolling, blue-green, or canary based on blast radius and rollback needs.
- CI/CD
- Kubernetes
- Reliability
AI Infrastructure
Evaluating AI Coding Agents in CI
HumanEval scores do not predict whether an agent will break your migrations. A fixture-based eval harness, autonomy tiers, and CI gates that treat agent patches with the same distrust as human ones.
- AI Agents
- CI/CD
- GitHub Actions
Observability
How to Debug a Slow API in Production
When P99 spikes but errors stay flat, binary-search the request path with metrics and traces-starting at the edge, through the app, and into MySQL, Redis, and downstream dependencies.
- Observability
- SRE
- APIs
AI Infrastructure
LLM Gateway Architecture
An LLM gateway is the control plane between your applications and model providers. Here is the architecture I use to centralize routing, rate limiting, cost accounting, and observability for every inference call.
- AI
- Architecture
- APIs
Observability
Logs vs Metrics vs Traces
Logs tell you what happened, metrics how much and how fast, traces where time went. Use all three with correlation IDs-alert on metrics, debug with logs and traces.
- Observability
- SRE
- OpenTelemetry
Observability
Observability for LLM Applications
Standard APM tells you the request was slow - it does not tell you why the model hallucinated or which prompt version caused a quality regression. Here is the observability stack I use for LLM applications in production.
- Observability
- AI
- OpenTelemetry
Observability
Slack Alerting That Scales with On-Call
Alert fatigue is a routing and enrichment problem, not a Slack problem. Tiered channels, Alertmanager as source of truth, runbook annotations, and alert budgets that scale with on-call rotations.
- Observability
- SRE
- Automation
Reliability
Webhook Idempotency in Production
Payment providers like Razorpay, Stripe, and PayU will deliver the same webhook more than once. If your handler is not idempotent, you will double-charge, double-enroll, or double-ship. Here is the pattern I use in production.
- APIs
- Reliability
- Payments