Platform Engineering in the Age of AI Agents

Platform engineering has always been about making the right thing the easy thing - paved roads, golden paths, self-service with guardrails. AI agents stress every assumption that model was built on. Developers now want to deploy autonomous processes that call production APIs, spend real money on inference, and behave differently every run.

I do not think platform teams become irrelevant. I expect the opposite: the need for centralized governance, cost control, and safety infrastructure increases sharply when every product team can spin up an agent in an afternoon.

What Changes for Platform Teams

Before agents, platform teams owned CI/CD pipelines, Kubernetes clusters, observability stacks, and database provisioning. That scope still exists, but I expect three new responsibilities to become first-class: agent runtime infrastructure, AI cost governance, and tool access control.

Platform Team Scope (2023)          Platform Team Scope (expected)
─────────────────────────          ─────────────────────────────
CI/CD · K8s · Observability   →    All of the above, plus:
Database · Secrets · IAM             · Agent runtime & state stores
                                     · LLM gateway & cost controls
                                     · Tool registry & permissions
                                     · Agent eval & safety pipelines

These are not optional nice-to-haves. Without them, every product team builds its own agent stack with different security postures, cost tracking, and failure handling.

The Agent Runtime as a Platform Product

I expect platform teams to offer an agent runtime the way they offer a CI pipeline template today: a standardized execution environment with built-in step limits, state management, tool registration, and observability. Product teams write agent logic; the platform owns the loop, the limits, and the audit trail.

Concretely, this means a managed service (or well-documented Helm chart) that provides Redis-backed state, MySQL audit logs, OpenTelemetry tracing, and a tool SDK. Developers register tools through an internal registry - not by giving the agent raw HTTP access to anything.

# Developer experience (what I want platform to provide)
from platform.agents import Agent, tool

@tool(name="search_docs", permissions=["read:docs"])
def search_docs(query: str) -> list:
    ...

agent = Agent(
    name="support-bot",
    tools=[search_docs],
    max_steps=15,
    cost_budget_usd=0.50,
)
result = agent.run("Find refund policy for enterprise plans")

The platform enforces max_steps, cost_budget, and tool permissions. The developer does not configure Redis, write retry logic, or set up tracing.

AI Cost Governance Becomes a Platform Concern

When one developer can accidentally spend $500 on a looped agent run, cost control is infrastructure - not finance. I expect platform teams to own per-team, per-feature, and per-environment spend caps enforced at the LLM gateway.

This mirrors how I already manage AWS budgets: centralized visibility, decentralized ownership, hard stops at the infrastructure layer. Product teams see their AI spend in a dashboard; platform teams set the guardrails.

Tool Registry and the New IAM Problem

Traditional IAM answers "can this user do X?" Agent IAM must answer "can this agent, acting on behalf of this user, call this tool with these arguments?" That is a harder problem and I expect platform teams to own the tool registry.

Every tool gets registered with: name, description (for the model), input schema, required permissions, rate limits, and audit requirements. The agent runtime checks permissions before execution - the model never sees raw credentials.

I think the biggest mistake teams will make is treating agent tool access like API key distribution. It needs the same rigor as service-to-service IAM, with short-lived tokens and scope restrictions.

Eval Pipelines as Platform Infrastructure

You cannot unit test an agent the way you unit test a REST handler. I expect platform teams to provide eval infrastructure: a framework for golden-path tests, adversarial prompt suites, and regression scoring against held-out task sets.

This runs in CI for mocked-tool tests and nightly for real-model evals. Deploys to agent infrastructure get blocked on golden-path failures, same as any other service.

Platform teams do not write the eval cases - product teams do. But the runner, the scoring pipeline, and the result dashboard are platform products.

What Stays the Same

GitOps still applies. Agent code, tool definitions, and routing configs live in Git. Argo CD reconciles agent worker deployments. GitHub Actions runs tests and eval gates.

Observability fundamentals do not change - traces, metrics, logs, SLOs. The data model expands (per-step spans, tool audit logs, cost metrics), but the tooling (OpenTelemetry, Grafana, PagerDuty) stays.

The platform team's core job remains: reduce cognitive load for product teams while maintaining security, reliability, and cost efficiency. Agents just raise the stakes on all three.

Engineering Takeaways

• I expect platform teams to own agent runtime infrastructure, not leave every team to build their own loop and state management.
• AI cost governance belongs at the infrastructure layer - gateway caps, per-run budgets, and team spend limits.
• Tool registries with scoped permissions are the new IAM problem platform teams must solve.
• Eval pipelines should be a platform product: product teams write cases, platform provides the runner and gates.
• GitOps, observability, and paved-road principles still apply - agents extend the scope, not replace it.
• The goal remains the same: make the safe, reliable path the easiest path for developers.

Trade-offs I Would Consider

Option: Centralized agent platform (one runtime for all teams)

• Pros: Consistent security, unified cost tracking, shared eval infrastructure, faster onboarding.
• Cons: Becomes a bottleneck if the platform team is understaffed; less flexibility for edge-case agent designs.

Option: Federated agent infrastructure (teams self-serve with platform guardrails)

• Pros: Teams move faster, platform provides gateway + registry + templates without owning runtime code.
• Cons: Inconsistent runtime behavior, harder to enforce limits, audit trails vary by team.

What I Would Do in Production

Start by offering the LLM gateway and tool registry as platform products. These deliver immediate value without requiring teams to adopt a full agent runtime.

Publish a "build vs. buy vs. platform" decision tree for agent workloads. Make the platform path the default for any agent touching production data.

Include AI spend in existing FinOps dashboards. Teams that see their inference costs weekly self-regulate faster than teams that get a monthly surprise invoice.

Run an internal agent hackathon with platform guardrails in place. You will discover tool permission gaps and cost control holes faster than any design doc review.

Document agent-specific runbooks alongside existing service runbooks: loop abort, cost cap breach, tool permission denial, eval score regression.

FAQ

Q: Will platform teams need ML engineers?

Not necessarily for the infrastructure layer. Platform engineers who understand API gateways, Redis, and IAM can build agent runtime and cost controls. ML expertise helps for eval pipeline design, but I expect most platform teams to partner with ML engineers rather than hire exclusively for this.

Q: Should agents run on the same Kubernetes clusters as other services?

Same cluster, separate node pool and namespace. Agent workers have different resource profiles (long-running, bursty GPU adjacency) and should not compete with latency-sensitive API pods.

Q: How do I prioritize agent platform work against existing backlog?

I prioritize based on demand: if two or more teams are building agents independently, centralize immediately. One team experimenting can use templates and the gateway without a full runtime product.

Q: What about open-source agent frameworks like LangChain or CrewAI?

Useful for prototyping. I expect production agent runtimes to wrap or replace them with platform-controlled state, limits, and observability. Do not let product teams deploy framework defaults directly to production.

Q: How does this affect platform team headcount?

I expect a temporary increase in platform scope before tooling matures. Once gateway, registry, and runtime templates exist, the marginal cost per new agent drops significantly - similar to how Kubernetes templates reduced per-service ops burden.