{ "version": 1, "generated_at": "2026-08-23T12:03:38Z", "description": "Personal site and technical writing by Gabriel Koo, a Hong Kong-based security, cloud, DevOps, automation, and AI platform engineer.", "content_types": [ "article", "project", "talk", "anime" ], "counts": { "article": 33, "project": 33, "talk": 15, "anime": 287 }, "items": [ { "id": "article:i-used-cloudflares-app-library-to-map-every-ai-services-domains-and-why-networking-still-matters-2and", "source_type": "article", "title": "Why I Split-Tunnel My VPN for AI Services — and Let Cloudflare's Application Library Pick the Domains", "url": "https://gabrielkoo.com/blog/i-used-cloudflares-app-library-to-map-every-ai-services-domains-and-why-networking-still-matters-2and/", "canonical_url": "https://dev.to/gabrielkoo/i-used-cloudflares-app-library-to-map-every-ai-services-domains-and-why-networking-still-matters-2and", "published_at": "2026-06-19", "last_verified_at": "2026-08-23", "tags": [ "networking", "cloudflare", "vpn", "ai" ], "description": "I maintain a small set of open-source GitHub repos that hold split-tunnel VPN configs for reaching AI...", "content": "I maintain a small set of **open-source GitHub repos** that hold split-tunnel VPN configs for reaching AI services — [Tailscale app connectors](https://github.com/gabrielkoo/tailscale-config-for-ai-services), [WireGuard](https://github.com/gabrielkoo/wireguard-configs-for-ai-services), and [OpenVPN](https://github.com/gabrielkoo/openvpn-configs-for-ai-services). The single hardest part of maintaining them isn't the VPN config. It's answering one deceptively boring question:\n\n**Which domains does this AI platform actually use?**\n\nThis post is about a shortcut I lean on — Cloudflare's Application Library, queried over its REST API — and a broader point I keep coming back to: even in an era where AI can scaffold most of your code and config for you, **networking fundamentals are still worth learning**. Not because the AI can't write the config, but because the difference between a clean setup and one that gets your account flagged comes down to a judgment call — *what you route and what you deliberately don't* — and that judgment is exactly the part an AI can't make for you without understanding your intent.\n\n(One caveat before we start: this is for routing traffic you're already entitled to — not for evading restrictions that legitimately apply to you. Full legitimate-use note at the end.)\n\n## The problem: a \"domain list\" is never just one domain\n\nWhen you want to route only one app through a VPN, you need the set of hostnames (or IPs) that app actually talks to. That sounds trivial until you open DevTools on a modern web app and watch it fan out to twenty different domains: the main API, an auth provider, a CDN for static assets, telemetry, a fraud-detection pixel, a feature-flag service.\n\nRoute too little and the app half-works (login spins forever, streaming responses stall). Route too much — say, the bare `*.cloudflarestorage.com` wildcard — and you scoop up huge amounts of *unrelated* traffic, which defeats the entire point of a split tunnel and can even get your exit node rate-limited.\n\nSo you need the **narrowest accurate** set of hostnames. Where do you get it?\n\n![One app fans out to many hostnames — a single ChatGPT app talking to api.openai.com, auth0.openai.com, cdn.oaistatic.com, oaiusercontent.com and more](/assets/img/caaaed39c4d2.jpg)\n\n## The shortcut: Cloudflare's Application Library\n\nCloudflare's Zero Trust product ships an [**Application Library**](https://developers.cloudflare.com/cloudflare-one/team-and-resources/app-library) — a catalog of well-known SaaS apps, each annotated with the hostnames Cloudflare has observed it using. It's meant for admins building Access policies, but it doubles as a fantastic *domain-discovery* tool. Someone at Cloudflare already did the tedious traffic-watching for ChatGPT, Claude, and friends.\n\n![Stop sniffing traffic by hand — the Cloudflare Application Library hands you the hostnames so you don't have to map them manually](/assets/img/503230b95394.jpg)\n\nYou can browse it in the dashboard under **Zero Trust → Team & Resources → Application Library**. But I don't want to click through a UI every time a provider quietly adds a domain — I want it in code. So I query the [REST API](https://developers.cloudflare.com/api/).\n\nHere's the helper I use to pull ChatGPT's hostnames. Pure stdlib, no SDK:\n\n```python\nimport json, urllib.parse, urllib.request\n\nCF_API = \"https://api.cloudflare.com/client/v4\"\n\n\ndef cf_hostnames(token, account, app_name, search):\n url = \"%s/accounts/%s/resource-library/applications?%s\" % (\n CF_API, account, urllib.parse.urlencode({\"search\": search, \"limit\": 25}))\n req = urllib.request.Request(url, headers={\"Authorization\": \"Bearer \" + token})\n with urllib.request.urlopen(req, timeout=30) as r:\n d = json.load(r)\n for a in (d.get(\"result\") or []):\n if a.get(\"name\") == app_name:\n return sorted(set(a.get(\"hostnames\") or []))\n raise SystemExit(\"CF: application %r not found (search=%r)\" % (app_name, search))\n```\n\nA few things worth calling out:\n\n- **The endpoint is `accounts/{account_id}/resource-library/applications`.** It takes a `search` query and returns matching catalog entries, each with a `hostnames` array. I match on the exact `name` (e.g. `\"ChatGPT\"`, `\"Claude\"`) because a search can return several near-matches.\n- **The token only needs read access** to the resource library. Scope it minimally — there's no reason this token should be able to change anything.\n- **It's deterministic and CI-friendly.** No browser automation, no scraping. That matters for the next step.\n- **You don't need a paid plan.** This lives in Cloudflare's Zero Trust product, and Zero Trust has a [**free tier (up to 50 seats)**](https://blog.cloudflare.com/teams-plans/) that's plenty for personal use. The Application Library and its REST API are available on that free plan — so the entire domain-discovery pipeline costs you nothing but an API token.\n\n### What the API actually returns\n\nHere's the real output for **Claude** (`name == \"Claude\"`) as of this writing:\n\n```json\n{\n \"service\": \"claude\",\n \"application\": \"Claude\",\n \"hostnames\": [\n \"a-api.anthropic.com\",\n \"a-cdn.anthropic.com\",\n \"anthropic.com\",\n \"claude.ai\",\n \"claude.com\"\n ]\n}\n```\n\nFive hostnames, and notice they cover the things that actually matter: the web app (`claude.ai`, `claude.com`), the API (`a-api.anthropic.com`), the static asset CDN (`a-cdn.anthropic.com`), and the marketing/auth origin (`anthropic.com`). That's the *narrow accurate set* I was after — nothing extraneous to prune.\n\nAnd the same call for **ChatGPT** (App ID `1199`):\n\n```json\n{\n \"service\": \"openai\",\n \"application\": \"ChatGPT\",\n \"hostnames\": [\n \"api.openai.com\",\n \"auth.openai.com\",\n \"auth0.openai.com\",\n \"cdn.oaistatic.com\",\n \"chat.openai.com\",\n \"chatgpt.com\",\n \"oaistatic.com\",\n \"oaiusercontent.com\",\n \"openai.com\"\n ]\n}\n```\n\nNine hostnames, and the list is more revealing than it looks. Miss the Auth0-backed `auth0.openai.com` (still in the login path at time of writing) and sign-in silently hangs; miss `oaiusercontent.com` and uploads/generated files break. Try assembling that from DevTools by hand and you'll miss the auth host you only hit on first login — then spend an afternoon debugging a broken login flow.\n\nCloudflare already did the traffic-watching.\n\n## From hostnames to a routable config\n\nHostnames are perfect for **[Tailscale app connectors](https://tailscale.com/kb/1281/app-connectors)**, which match on the domain name directly — Tailscale handles the DNS-to-IP mapping for you, so the config keeps working even when the provider rotates IPs.\n\nFor **WireGuard and OpenVPN** — which you'd reach for when you need router-level enforcement, or you're on a client where Tailscale isn't an option — you can't route by hostname; those tunnels route by IP. So the pipeline becomes:\n\n1. **Pull hostnames** from the Cloudflare Application Library (the function above).\n2. **Resolve each to A + AAAA records** over DNS-over-HTTPS, so the result doesn't depend on whatever resolver the CI runner happens to use.\n3. **Aggregate** the addresses to CIDR blocks, then collapse overlapping ranges. I default to `/24` (v4) and `/48` (v6), but understand this is a *heuristic*, not precision: expanding one resolved IP to a `/24` also routes the other 255 addresses on that edge node, some of which belong to unrelated tenants. It's a deliberate trade — narrower (`/32`, `/28`) is more precise but churns more often; wider (`/20`, a supernet) means fewer entries but more collateral traffic. Pick the prefix that matches how much drift and collateral you can tolerate.\n4. **Union with a static \"floor\"** for providers that publish a stable, authoritative prefix — for Anthropic that's `160.79.104.0/21` (their own AS399230), so a single resolved `/24` never accidentally blackholes the rest of the block.\n5. **Rewrite the config files** between marker comments, and let a [scheduled GitHub Action](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/.github/workflows/update-ips.yml) open the change.\n\nThe whole thing is one stdlib Python script ([`scripts/update_ips.py`](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/scripts/update_ips.py)) that runs on a cron schedule. The configs stay fresh without me touching them.\n\n![The resolve-and-aggregate pipeline — scattered cloud hostnames funnel through resolve + aggregate into a single clean config](/assets/img/de6fa0294637.jpg)\n\nAnd here's where the two providers diverge in a way that proves the whole point. Run the resolve-and-aggregate step and **ChatGPT** collapses to a pile of Cloudflare ranges:\n\n```text\n# IPv4 (all shared Cloudflare anycast)\n104.18.32.0/23\n104.18.37.0/24\n104.18.41.0/24\n162.159.140.0/24\n172.64.146.0/24\n172.64.150.0/24\n172.64.154.0/23\n172.65.90.0/24\n172.66.0.0/24\n\n# IPv6 — 9 more Cloudflare /48s\n```\n\nEvery one of those is **shared Cloudflare anycast** — [`104.18.x`, `172.64.x`, `172.66.x` are Cloudflare's](https://www.cloudflare.com/ips/), not OpenAI's. Route them and you're routing a slice of Cloudflare's entire customer base, and the specific `/24`s will drift as Cloudflare reshuffles. That's why ChatGPT-by-IP needs the scheduled refresh, and why it's genuinely better as a Tailscale *hostname* connector.\n\n**Claude** collapses to something completely different:\n\n```text\n34.36.57.0/24 ← Google Cloud LB (shared)\n160.79.104.0/21 ← Anthropic's OWN block (AS399230)\n2607:6bc0::/32 ← Anthropic's own IPv6\n```\n\nThe `160.79.104.0/21` is Anthropic's own registered allocation — a `whois`/RDAP lookup shows the block (160.79.104.0–160.79.111.255) registered directly to Anthropic, PBC, announced via AS399230. So I can route the whole `/21` and trust *who* it belongs to, even though it's 2,048 addresses — wider than Claude strictly needs today. That's the deliberate stability-over-precision trade for a block whose owner won't suddenly hand it to someone else. The `34.36.57.0/24` is a [Google Cloud front-end](https://www.gstatic.com/ipranges/cloud.json) I let the script re-resolve each run rather than hardcode, because it *is* shared infrastructure that drifts. **Same pipeline, two completely different risk profiles — and you only know which is which if you understand who owns the address space.**\n\n![Same pipeline, two risk profiles — one resolve+aggregate funnel splits into ChatGPT on shared Cloudflare anycast that drifts, versus Claude on Anthropic's own stable 160.79.104.0/21 block](/assets/img/25e7faa7e55b.jpg)\n\n## Why this is a networking lesson, not a VPN lesson\n\nEvery step above is a networking decision, and getting them wrong has real consequences:\n\n![Hostname routing vs IP routing — hostname routing follows the name and survives IP churn; IP routing pins addresses and breaks on drift](/assets/img/be11d59d9672.jpg)\n\n- **Hostname routing vs. IP routing is a fundamental tradeoff.** DNS/SNI-based routing (Tailscale) survives IP churn and cleanly isolates one app on a shared CDN. IP-CIDR routing (WireGuard/OpenVPN) is simpler and dependency-free but brittle — it breaks when addresses drift, and it *can't* separate two services that share an anycast front end. Knowing which tool fits which provider saves hours.\n- **Shared anycast CDNs are a trap.** `chatgpt.com` sits behind Cloudflare's shared anycast — the same `/24`s serve thousands of unrelated sites. Route the whole block and you've quietly tunneled a chunk of the internet. Route too narrowly and it breaks tomorrow. You have to *understand* that the IP doesn't belong to OpenAI to make a sane call.\n\n![Route a shared /24, scoop unrelated traffic — a net catches the one IP you wanted along with 254 strangers sharing the block](/assets/img/6a15b83413ab.jpg)\n\n- **Knowing who owns an IP block matters.** `160.79.104.0/21` is Anthropic's own allocation (AS399230). That's a stable, safe thing to route wholesale. A `34.36.x` Google Cloud load-balancer front-end in front of the same service is *not* — it's shared infrastructure. A quick `whois` or a look at the AS tells you which is which.\n- **Split tunneling is precision, and precision is how you stay out of trouble.** This circles back to the disclaimer. The reason I route the *narrowest* set of hostnames isn't tidiness — it's that the less unrelated traffic I push through an exit node, the less I look like abuse, and the less I disrupt the security-sensitive services (banking, corporate SSO) that legitimately *don't* want to see me arriving from a VPN IP. Full-tunneling everything is both lazier and riskier.\n\nNone of this requires Cisco's CCNA certification. But it does require knowing what a routing table is, what a CIDR block means, what an autonomous system is, and why DNS resolution is a separate concern from packet routing. That foundation turns \"I copied a VPN config off the internet and hope it works\" into \"I know exactly what traffic goes where, and why.\"\n\n## Takeaways\n\n- **Cloudflare's Application Library is an underrated domain-discovery tool** — queryable over a simple REST endpoint, no scraping required.\n- **Pick your routing primitive to match the provider:** hostname-based for shared CDNs and IP-churning services, IP-CIDR for providers on their own stable address space.\n- **Scope narrowly on purpose.** It's better for performance, it keeps your other services unaffected, and it keeps you on the right side of a provider's abuse heuristics.\n- **And stay legitimate.** Use this for what you're already entitled to use — careful routing, continuity while traveling — not for getting around restrictions that apply to you.\n\nIf you want the runnable code, the three repos are linked at the top. The Cloudflare helper and the full resolve-and-aggregate pipeline live in [`scripts/update_ips.py`](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/scripts/update_ips.py).\n\n## One last thing: the legitimate-use note\n\nI parked this for the end so it didn't bog down the technical walkthrough, but it matters. This whole approach is for one specific, legitimate situation: **your account and home region are permitted to use the service, and you want precise control over your own traffic.** Two concrete cases:\n\n- **You want to route carefully — or deliberately NOT route — to avoid tripping a provider's fraud/abuse checks.** Many platforms run security engines (WAFs, fraud-detection, bot management) that get nervous when a logged-in session suddenly appears from a datacenter IP or a region that doesn't match your account history. This cuts both ways: the same is true of *other* services you use — banks especially — which actively distrust traffic arriving from a VPN exit IP. Precision routing is as much about *keeping the wrong traffic off the tunnel* as putting the right traffic on it.\n- **You normally reside in a region that's allowed to use the service, but you're temporarily somewhere it isn't reachable** — a short work trip, a conference, a layover. You're a legitimate user of an unrestricted region who wants continuity of access you're already entitled to.\n\nWhat this is **not** for: evading a restriction that legitimately applies to you. If your account or country of residence isn't permitted to use a service, a VPN config doesn't change that, and nothing here is an invitation to break a provider's Terms of Service or your local law. **Read the ToS, respect it, and when in doubt don't.** I keep my configs scoped as narrowly as possible precisely *because* the goal is to not look like abuse traffic.\n\n## Sources\n\n- [Cloudflare — Application Library (Cloudflare One docs)](https://developers.cloudflare.com/cloudflare-one/team-and-resources/app-library)\n- [Cloudflare — REST API reference](https://developers.cloudflare.com/api/)\n- [Tailscale — App connectors](https://tailscale.com/kb/1281/app-connectors)\n- [Cloudflare — published IP ranges](https://www.cloudflare.com/ips/) · [Google Cloud — published IP ranges](https://www.gstatic.com/ipranges/cloud.json) · [ARIN RDAP — `160.79.104.0/21` (Anthropic, PBC)](https://rdap.arin.net/registry/ip/160.79.104.0)\n- Companion repos: [Tailscale](https://github.com/gabrielkoo/tailscale-config-for-ai-services) · [WireGuard](https://github.com/gabrielkoo/wireguard-configs-for-ai-services) · [OpenVPN](https://github.com/gabrielkoo/openvpn-configs-for-ai-services)", "excerpts": [ "I maintain a small set of **open-source GitHub repos** that hold split-tunnel VPN configs for reaching AI services — [Tailscale app connectors](https://github.com/gabrielkoo/tailscale-config-for-ai-services), [WireGuard](https://github.com/gabrielkoo/wireguard-configs-for-ai-services), and [OpenVPN](https://github.com/gabrielkoo/openvpn-configs-for-ai-services). The single hardest part of maintaining them isn't the VPN config. It's answering one deceptively boring question:", "**Which domains does this AI platform actually use?**", "This post is about a shortcut I lean on — Cloudflare's Application Library, queried over its REST API — and a broader point I keep coming back to: even in an era where AI can scaffold most of your code and config for you, **networking fundamentals are still worth learning**. Not because the AI can't write the config, but because the difference between a clean setup and one that gets your account flagged comes down to a judgment call — *what you route and what you deliberately don't* — and that judgment is exactly the part an AI can't make for you without understanding your intent.", "(One caveat before we start: this is for routing traffic you're already entitled to — not for evading restrictions that legitimately apply to you. Full legitimate-use note at the end.)", "The problem: a \"domain list\" is never just one domain", "When you want to route only one app through a VPN, you need the set of hostnames (or IPs) that app actually talks to. That sounds trivial until you open DevTools on a modern web app and watch it fan out to twenty different domains: the main API, an auth provider, a CDN for static assets, telemetry, a fraud-detection pixel, a feature-flag service.", "Route too little and the app half-works (login spins forever, streaming responses stall). Route too much — say, the bare `*.cloudflarestorage.com` wildcard — and you scoop up huge amounts of *unrelated* traffic, which defeats the entire point of a split tunnel and can even get your exit node rate-limited.", "So you need the **narrowest accurate** set of hostnames. Where do you get it?", "![One app fans out to many hostnames — a single ChatGPT app talking to api.openai.com, auth0.openai.com, cdn.oaistatic.com, oaiusercontent.com and more](/assets/img/caaaed39c4d2.jpg)", "The shortcut: Cloudflare's Application Library", "Cloudflare's Zero Trust product ships an [**Application Library**](https://developers.cloudflare.com/cloudflare-one/team-and-resources/app-library) — a catalog of well-known SaaS apps, each annotated with the hostnames Cloudflare has observed it using. It's meant for admins building Access policies, but it doubles as a fantastic *domain-discovery* tool. Someone at Cloudflare already did the tedious traffic-watching for ChatGPT, Claude, and friends.", "![Stop sniffing traffic by hand — the Cloudflare Application Library hands you the hostnames so you don't have to map them manually](/assets/img/503230b95394.jpg)", "You can browse it in the dashboard under **Zero Trust → Team & Resources → Application Library**. But I don't want to click through a UI every time a provider quietly adds a domain — I want it in code. So I query the [REST API](https://developers.cloudflare.com/api/).", "Here's the helper I use to pull ChatGPT's hostnames. Pure stdlib, no SDK:", "CF_API = \"https://api.cloudflare.com/client/v4\"", "def cf_hostnames(token, account, app_name, search): url = \"%s/accounts/%s/resource-library/applications?%s\" % ( CF_API, account, urllib.parse.urlencode({\"search\": search, \"limit\": 25})) req = urllib.request.Request(url, headers={\"Authorization\": \"Bearer \" + token}) with urllib.request.urlopen(req, timeout=30) as r: d = json.load(r) for a in (d.get(\"result\") or []): if a.get(\"name\") == app_name: return sorted(set(a.get(\"hostnames\") or [])) raise SystemExit(\"CF: application %r not found (search=%r)\" % (app_name, search)) ```", "A few things worth calling out:", "- **The endpoint is `accounts/{account_id}/resource-library/applications`.** It takes a `search` query and returns matching catalog entries, each with a `hostnames` array. I match on the exact `name` (e.g. `\"ChatGPT\"`, `\"Claude\"`) because a search can return several near-matches. - **The token only needs read access** to the resource library. Scope it minimally — there's no reason this token should be able to change anything. - **It's deterministic and CI-friendly.** No browser automation, no scraping. That matters for the next step. - **You don't need a paid plan.** This lives in Cloudflare's Zero Trust product, and Zero Trust has a [**free tier (up to 50 seats)**](https://blog.cloudflare.com/teams-plans/) that's plenty for personal use. The Application Library and its REST API are available on that free plan — so the entire domain-discovery pipeline costs you nothing but an API token.", "Here's the real output for **Claude** (`name == \"Claude\"`) as of this writing:", "Five hostnames, and notice they cover the things that actually matter: the web app (`claude.ai`, `claude.com`), the API (`a-api.anthropic.com`), the static asset CDN (`a-cdn.anthropic.com`), and the marketing/auth origin (`anthropic.com`). That's the *narrow accurate set* I was after — nothing extraneous to prune.", "And the same call for **ChatGPT** (App ID `1199`):", "Nine hostnames, and the list is more revealing than it looks. Miss the Auth0-backed `auth0.openai.com` (still in the login path at time of writing) and sign-in silently hangs; miss `oaiusercontent.com` and uploads/generated files break. Try assembling that from DevTools by hand and you'll miss the auth host you only hit on first login — then spend an afternoon debugging a broken login flow.", "Cloudflare already did the traffic-watching.", "From hostnames to a routable config", "Hostnames are perfect for **[Tailscale app connectors](https://tailscale.com/kb/1281/app-connectors)**, which match on the domain name directly — Tailscale handles the DNS-to-IP mapping for you, so the config keeps working even when the provider rotates IPs.", "For **WireGuard and OpenVPN** — which you'd reach for when you need router-level enforcement, or you're on a client where Tailscale isn't an option — you can't route by hostname; those tunnels route by IP. So the pipeline becomes:", "1. **Pull hostnames** from the Cloudflare Application Library (the function above). 2. **Resolve each to A + AAAA records** over DNS-over-HTTPS, so the result doesn't depend on whatever resolver the CI runner happens to use. 3. **Aggregate** the addresses to CIDR blocks, then collapse overlapping ranges. I default to `/24` (v4) and `/48` (v6), but understand this is a *heuristic*, not precision: expanding one resolved IP to a `/24` also routes the other 255 addresses on that edge node, some of which belong to unrelated tenants. It's a deliberate trade — narrower (`/32`, `/28`) is more precise but churns more often; wider (`/20`, a supernet) means fewer entries but more collateral traffic. Pick the prefix that matches how much drift and collateral you can tolerate. 4. **Union with a static \"floor\"** for providers that publish a stable, authoritative prefix — for Anthropic that's `160.79.104.0/21` (their own AS399230), so a single resolved `/24` never accidentally blackholes the rest of the block. 5. **Rewrite the config files** between marker comments, and let a [scheduled GitHub Action](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/.github/workflows/upda", "The whole thing is one stdlib Python script ([`scripts/update_ips.py`](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/scripts/update_ips.py)) that runs on a cron schedule. The configs stay fresh without me touching them.", "![The resolve-and-aggregate pipeline — scattered cloud hostnames funnel through resolve + aggregate into a single clean config](/assets/img/de6fa0294637.jpg)", "And here's where the two providers diverge in a way that proves the whole point. Run the resolve-and-aggregate step and **ChatGPT** collapses to a pile of Cloudflare ranges:", "IPv6 — 9 more Cloudflare /48s ```", "Every one of those is **shared Cloudflare anycast** — [`104.18.x`, `172.64.x`, `172.66.x` are Cloudflare's](https://www.cloudflare.com/ips/), not OpenAI's. Route them and you're routing a slice of Cloudflare's entire customer base, and the specific `/24`s will drift as Cloudflare reshuffles. That's why ChatGPT-by-IP needs the scheduled refresh, and why it's genuinely better as a Tailscale *hostname* connector.", "**Claude** collapses to something completely different:", "The `160.79.104.0/21` is Anthropic's own registered allocation — a `whois`/RDAP lookup shows the block (160.79.104.0–160.79.111.255) registered directly to Anthropic, PBC, announced via AS399230. So I can route the whole `/21` and trust *who* it belongs to, even though it's 2,048 addresses — wider than Claude strictly needs today. That's the deliberate stability-over-precision trade for a block whose owner won't suddenly hand it to someone else. The `34.36.57.0/24` is a [Google Cloud front-end](https://www.gstatic.com/ipranges/cloud.json) I let the script re-resolve each run rather than hardcode, because it *is* shared infrastructure that drifts. **Same pipeline, two completely different risk profiles — and you only know which is which if you understand who owns the address space.**", "![Same pipeline, two risk profiles — one resolve+aggregate funnel splits into ChatGPT on shared Cloudflare anycast that drifts, versus Claude on Anthropic's own stable 160.79.104.0/21 block](/assets/img/25e7faa7e55b.jpg)", "Why this is a networking lesson, not a VPN lesson", "Every step above is a networking decision, and getting them wrong has real consequences:", "![Hostname routing vs IP routing — hostname routing follows the name and survives IP churn; IP routing pins addresses and breaks on drift](/assets/img/be11d59d9672.jpg)", "- **Hostname routing vs. IP routing is a fundamental tradeoff.** DNS/SNI-based routing (Tailscale) survives IP churn and cleanly isolates one app on a shared CDN. IP-CIDR routing (WireGuard/OpenVPN) is simpler and dependency-free but brittle — it breaks when addresses drift, and it *can't* separate two services that share an anycast front end. Knowing which tool fits which provider saves hours. - **Shared anycast CDNs are a trap.** `chatgpt.com` sits behind Cloudflare's shared anycast — the same `/24`s serve thousands of unrelated sites. Route the whole block and you've quietly tunneled a chunk of the internet. Route too narrowly and it breaks tomorrow. You have to *understand* that the IP doesn't belong to OpenAI to make a sane call.", "![Route a shared /24, scoop unrelated traffic — a net catches the one IP you wanted along with 254 strangers sharing the block](/assets/img/6a15b83413ab.jpg)", "- **Knowing who owns an IP block matters.** `160.79.104.0/21` is Anthropic's own allocation (AS399230). That's a stable, safe thing to route wholesale. A `34.36.x` Google Cloud load-balancer front-end in front of the same service is *not* — it's shared infrastructure. A quick `whois` or a look at the AS tells you which is which. - **Split tunneling is precision, and precision is how you stay out of trouble.** This circles back to the disclaimer. The reason I route the *narrowest* set of hostnames isn't tidiness — it's that the less unrelated traffic I push through an exit node, the less I look like abuse, and the less I disrupt the security-sensitive services (banking, corporate SSO) that legitimately *don't* want to see me arriving from a VPN IP. Full-tunneling everything is both lazier and riskier.", "None of this requires Cisco's CCNA certification. But it does require knowing what a routing table is, what a CIDR block means, what an autonomous system is, and why DNS resolution is a separate concern from packet routing. That foundation turns \"I copied a VPN config off the internet and hope it works\" into \"I know exactly what traffic goes where, and why.\"", "- **Cloudflare's Application Library is an underrated domain-discovery tool** — queryable over a simple REST endpoint, no scraping required. - **Pick your routing primitive to match the provider:** hostname-based for shared CDNs and IP-churning services, IP-CIDR for providers on their own stable address space. - **Scope narrowly on purpose.** It's better for performance, it keeps your other services unaffected, and it keeps you on the right side of a provider's abuse heuristics. - **And stay legitimate.** Use this for what you're already entitled to use — careful routing, continuity while traveling — not for getting around restrictions that apply to you.", "If you want the runnable code, the three repos are linked at the top. The Cloudflare helper and the full resolve-and-aggregate pipeline live in [`scripts/update_ips.py`](https://github.com/gabrielkoo/wireguard-configs-for-ai-services/blob/main/scripts/update_ips.py).", "One last thing: the legitimate-use note", "I parked this for the end so it didn't bog down the technical walkthrough, but it matters. This whole approach is for one specific, legitimate situation: **your account and home region are permitted to use the service, and you want precise control over your own traffic.** Two concrete cases:", "- **You want to route carefully — or deliberately NOT route — to avoid tripping a provider's fraud/abuse checks.** Many platforms run security engines (WAFs, fraud-detection, bot management) that get nervous when a logged-in session suddenly appears from a datacenter IP or a region that doesn't match your account history. This cuts both ways: the same is true of *other* services you use — banks especially — which actively distrust traffic arriving from a VPN exit IP. Precision routing is as much about *keeping the wrong traffic off the tunnel* as putting the right traffic on it. - **You normally reside in a region that's allowed to use the service, but you're temporarily somewhere it isn't reachable** — a short work trip, a conference, a layover. You're a legitimate user of an unrestricted region who wants continuity of access you're already entitled to.", "What this is **not** for: evading a restriction that legitimately applies to you. If your account or country of residence isn't permitted to use a service, a VPN config doesn't change that, and nothing here is an invitation to break a provider's Terms of Service or your local law. **Read the ToS, respect it, and when in doubt don't.** I keep my configs scoped as narrowly as possible precisely *because* the goal is to not look like abuse traffic.", "- [Cloudflare — Application Library (Cloudflare One docs)](https://developers.cloudflare.com/cloudflare-one/team-and-resources/app-library) - [Cloudflare — REST API reference](https://developers.cloudflare.com/api/) - [Tailscale — App connectors](https://tailscale.com/kb/1281/app-connectors) - [Cloudflare — published IP ranges](https://www.cloudflare.com/ips/) · [Google Cloud — published IP ranges](https://www.gstatic.com/ipranges/cloud.json) · [ARIN RDAP — `160.79.104.0/21` (Anthropic, PBC)](https://rdap.arin.net/registry/ip/160.79.104.0) - Companion repos: [Tailscale](https://github.com/gabrielkoo/tailscale-config-for-ai-services) · [WireGuard](https://github.com/gabrielkoo/wireguard-configs-for-ai-services) · [OpenVPN](https://github.com/gabrielkoo/openvpn-configs-for-ai-services)" ] }, { "id": "article:stop-putting-api-keys-in-mcpjson-per-user-oauth-with-amazon-cognito-aws-lambda-4h2i", "source_type": "article", "title": "Stop Putting API Keys in Your MCP Config: Per-User OAuth with Amazon Cognito + AWS Lambda", "url": "https://gabrielkoo.com/blog/stop-putting-api-keys-in-mcpjson-per-user-oauth-with-amazon-cognito-aws-lambda-4h2i/", "canonical_url": "https://dev.to/aws-builders/stop-putting-api-keys-in-mcpjson-per-user-oauth-with-amazon-cognito-aws-lambda-4h2i", "published_at": "2026-06-07", "last_verified_at": "2026-08-23", "tags": [ "aws", "security", "ai", "mcp" ], "description": "The runnable companion to my AgentCon HK 2026 talk, \"Empower Team-Wide Vibe Coding with LLM Gateway...", "content": "_The runnable companion to my AgentCon HK 2026 talk, [\"Empower Team-Wide Vibe Coding with LLM Gateway and Security-First MCPs.\"](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/) The talk argued per-user OAuth is what turns a shared god-token into safe, auditable agent access. This is the wiring — explained at the architecture level. The full `template.yaml`, Lambda, and scripts live in the public repo linked at the end._\n\n---\n\n## The gap nobody fills: *your* identity in front of a shared-key API\n\nBy 2026 most big SaaS shipped an official MCP server, and many even support OAuth. So \"my tool has no MCP\" and \"my tool has no OAuth\" are both fading problems — chasing vendors that lack one is a losing game; the list shrinks every week.\n\nTavily — the search API I use here — is actually a good citizen: its remote MCP supports both a shared key in the URL (`https://mcp.tavily.com/mcp/?tavilyApiKey=`) *and* an OAuth flow. So why wrap it at all? Because even a vendor's OAuth answers a different question than the one a security team is asking:\n\n- **Shared keys are the default, and cost is why.** Plenty of these tools price per seat. So the cheapest integration — the one teams actually ship — is one account's API key wired into a single shared MCP for the whole group. The instant that key is shared, per-user identity is gone: every call is the same caller, and \"who ran this?\" has no answer. Vendor OAuth only helps if everyone pays for their own seat, which is exactly the bill teams are dodging.\n- **Claimed ≠ enforced.** Even when a tool *does* attribute per user, it often rides on a value the client supplies — Tavily's `X-Human-Id` header is exactly this. The client asserts it, so nothing stops a caller from sending someone else's; the attribution is a courtesy, not a control. Trustworthy attribution has to come from a token *your own* gateway cryptographically verified, not a string the client typed.\n\nAnd here's the part that doesn't shrink at all: **the upstreams that matter most to you will never ship OAuth.** Your internal billing API, the legacy claims system nobody wants to touch, the compliance-mandated third-party tool with a single team API key. For every one of those, the only auth is a shared key — shared by nature. And it's not only internal systems: plenty of public vendors are the same shape — Brave Search's own official MCP server, for instance, authenticates with a single `BRAVE_API_KEY` and no OAuth at all. One key, the whole team behind it.\n\nThat's the real, durable gap: **no front door that authenticates callers as *your* identity, scopes them, and audits them — before handing off to a shared-key upstream.** That's what this post builds. Tavily is just a free, public stand-in you can actually run; mentally swap it for your own shared-key API.\n\n> 💡 **The one idea:** put a real OAuth 2.0 / OIDC authorization server bound to *your* identity (Amazon Cognito) — running the PKCE and client-credentials flows the MCP spec expects — in front of a shared-key upstream, in a single Lambda. Each caller gets their own scoped, cryptographically-verified, audited identity. The shared key never leaves the server.\n\n*Want the 30-second version first? [Walk through the interactive demo](https://gabrielkoo.github.io/tavily-oauth-mcp-wrapper/) — click through the whole OAuth flow, the scoped tool call, and the `403` when a caller reaches past its scope.*\n\n## The architecture\n\nTwo pictures tell the whole story. The shared-key path — one key for everyone, whether it sits in a URL or a config file:\n\n![Shared-key access — three callers (two engineers and an agent) all funnel through one shared key, which then hits the upstream API. No per-user scope under your control, no audit you own, the key leaks into logs, and rotating it breaks the whole team.](/assets/img/3c5b6ff475fb.png)\n\nWith the wrapper, each caller arrives as themselves — verified against your own identity provider — and the key is locked away server-side:\n\n![The OAuth wrapper — each caller logs in to its own session, hits an MCP server fronted by Amazon Cognito OAuth that enforces per-user scope and audit, and the shared key sits server-side (Secrets Manager / Parameter Store), reached only by the server before it calls the upstream. Scoped per identity, fully audited, key never leaves the server, and one session can be revoked without touching the team.](/assets/img/14d02ac3c62f.png)\n\nThree managed AWS pieces do the work, each with one clear job:\n\n**Amazon Cognito — the bouncer who issues the wristbands.** It's the OAuth 2.0 / OIDC authorization server (with PKCE — the piece OAuth 2.1 and the MCP spec lean on). A caller proves who they are *to Amazon Cognito*, which hands back a short-lived signed token stamped with a scope (here, `tavily-mcp/search`). It handles both kinds of caller from the same pool: a backend agent authenticates machine-to-machine, a human logs in through a hosted login page with PKCE. Crucially, Amazon Cognito *is* your identity layer — swapping the demo's username/password for real corporate SSO (SAML, OIDC, or social logins like Google, Microsoft, Apple) is a configuration change, not a rebuild. The tokens, scopes, and everything downstream stay identical.\n\n**API Gateway HTTP API + its JWT authorizer — the wristband scanner at the door.** Every request to the MCP endpoint — a plain JSON-RPC call over HTTP POST (the Streamable HTTP transport; this wrapper uses request/response POSTs only, not the transport's optional SSE streaming channel) — hits API Gateway first. Its [built-in JWT authorizer](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-jwt-authorizer.html) — a native feature of the HTTP API (v2) flavour; the older REST API would need a Cognito or custom Lambda authorizer instead — checks the token's signature, issuer, and expiry *at the edge*, before a single line of your code runs. No token, expired token, forged token: rejected with a 401 right there. This is pure authentication: *are you who the token says you are?* Nothing reaches your logic until that passes. (One footgun worth flagging: Cognito access tokens carry no `aud` claim, so the authorizer's audience must be set to the Cognito app client ID — a mismatch here is the most common source of silent 401s in this stack.)\n\n**Lambda — the room you're actually allowed into.** Once the token is valid, the Lambda does the *authorization*: it re-reads the scope to confirm this identity may call this specific tool, writes an audit line tying the action to that caller, then — and only then — reaches into Secrets Manager for the shared upstream key and calls Tavily. The key lives only inside this function's execution role; it is never sent back to the client.\n\n![The deployed request path — MCP client sends an Amazon Cognito Bearer token; API Gateway's JWT authorizer validates it at the edge; the Lambda re-checks scope, logs the caller, and calls Tavily with a key it reads from Secrets Manager and never returns to the client.](/assets/img/c700c0918921.png)\n\nThe split is the whole point: **API Gateway answers \"is this a real, valid token?\" and Lambda answers \"is this identity allowed to do this, and let's record that they did.\"** Authentication at the edge, authorization in your code, the secret sealed behind both.\n\n## Why `type: http`, not `type: stdio`\n\nThere's a reason this wrapper is a *remote* MCP server and not a local one — and it's the same reason the talk put OAuth-fronted remote servers at the centre of the architecture. An MCP client config gives you three transport choices, and the choice quietly decides your entire security story:\n\n- **`stdio`** — the client spawns a local process (`npx some-mcp`, a Python script) and talks to it over stdin/stdout. The catch: that process needs the upstream credential *on the developer's machine*. So the key lands in `mcp.json`, in an `env` block, in shell history, in a dotfile that gets synced to who-knows-where. Every laptop is now a copy of the shared key. This is the exact shape of the BYOAI / shared-key problem — one credential, sprayed everywhere, impossible to attribute or revoke cleanly.\n- **`sse`** — the older remote transport. Remote is the right instinct, but plain SSE was only ever a transport; it never carried an auth story of its own, so in practice people bolted a static bearer token onto it and landed right back at \"one shared secret.\"\n- **`http`** (Streamable HTTP) — a remote endpoint the client reaches over plain HTTPS, and the transport the 2025 MCP spec standardised on. Crucially, the [spec defines the MCP server as an **OAuth resource server**](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) — so authentication is a first-class part of the transport, not an afterthought. The client config holds *no secret at all*; it just points at a URL and lets OAuth do the rest. (With an OAuth-native client the browser login is automatic; clients that don't yet drive the flow themselves lean on a small local helper to do it — more on that below.)\n\nThat last line is the whole pitch. Compare the two configs a developer actually writes:\n\n```jsonc\n// stdio — the secret lives on every laptop\n{ \"tavily\": { \"type\": \"stdio\", \"command\": \"npx\",\n \"args\": [\"tavily-mcp\"],\n \"env\": { \"TAVILY_API_KEY\": \"tvly-SHARED-KEY-everyone-has-this\" } } }\n\n// http — no secret, OAuth per user\n{ \"tavily\": { \"type\": \"http\", \"url\": \"https://…/mcp\" } }\n```\n\nThe `http` version has nothing to leak. The credential never reaches the client; identity is established per-user through Cognito; and the server — not the laptop — is the only thing that ever touches the upstream key. Whenever an upstream *can* be reached as a remote OAuth-fronted `http` server, it should be. This wrapper exists precisely to turn a `stdio`-shaped shared-key tool into an `http`-shaped one.\n\n## One endpoint, two kinds of caller\n\nA backend agent and a human engineer authenticate through completely different OAuth flows — `client_credentials` for the machine, `authorization_code` + PKCE for the person. But they end up carrying the *same* scope and hitting the *same* endpoint. The server treats them identically; the only difference is what lands in the audit log — a client ID for the machine, an email for the human.\n\n![The two token origins converge on one MCP endpoint — a machine client via client_credentials and a human via authorization_code + PKCE both arrive carrying the same tavily-mcp/search scope, and the server treats them identically while logging a different caller identity.](/assets/img/f42275ee79db.png)\n\n### The last mile most \"MCP + OAuth\" posts skip\n\nOne piece of the human flow is easy to gloss over, and it's exactly where most \"remote MCP + OAuth\" walkthroughs quietly wave their hands: a static MCP client config (the `claude_desktop_config.json` kind) can't pop a browser for a Cognito login on its own. The fix is a thin local helper — registered as the client's `stdio` command — that checks for a cached token, triggers the PKCE browser login when it's missing or expired, stashes the result, and forwards each request with the `Authorization: Bearer …` header. This is exactly what tools like [`mcp-remote`](https://github.com/geelen/mcp-remote) exist to do — and the [rough edges around that login flow](https://github.com/geelen/mcp-remote/issues/251) are a recurring [source of confusion](https://www.reddit.com/r/mcp/comments/1mw09b5/how_are_you_handling_oauth_when_running_mcp/) when people first wire up a remote MCP server. It's a small shim, but without it the \"human logs in with PKCE\" line is doing a lot of unspoken work. The repo includes a runnable PKCE script you can wire in as that helper.\n\n## The honest limitation\n\n**The Lambda is the *final* enforcement point — but the security of the whole scheme rests on the entire chain: the JWT authorizer validating token integrity at the edge, and the Lambda enforcing scope behind it.** The upstream still sees one shared key; it has no idea which human is behind a call. All the per-user identity, scoping, and audit live in *your* layer. So the scheme reduces to a few disciplines:\n\n- **Lock the secret to only the Lambda's execution role.** Anyone who can read it gets full upstream access.\n- **Make sure every route to the function goes through the JWT authorizer.** A second unprotected trigger would bypass the whole thing.\n- **Machine callers blur the human behind them.** When a backend agent authenticates with `client_credentials`, the audit log names the *machine* client, not the person who prompted it. For a human-triggered agent you've narrowed \"who\" to one service identity, not one person. Closing that last gap means propagating the user's own token down to the agent (OAuth 2.0 token exchange, RFC 8693) instead of falling back to a shared machine credential — worth knowing before you lean on M2M audit lines as proof of *who* did something.\n- **Mind the 30-second ceiling.** API Gateway HTTP APIs cap each integration at 30 seconds. A search call returns in well under a second, so it's a non-issue here — but if you wrap a slower upstream (a heavy query, a multi-step scrape, a long-running agent tool), a call that overruns hits a `504` at the gateway. That's the point where you'd reach for the transport's SSE streaming channel, or an async submit-then-poll pattern, instead of a single blocking POST.\n\nGet those right and you've genuinely converted a shared god-token into per-user delegation. Get them wrong and you've just added a proxy in front of the same shared key. This is also why the *audit log* matters more than it looks: it's the only place \"who ran what\" is recoverable at all — Tavily's logs will forever show one key.\n\n## It works\n\nDeployed to a real AWS account, the end-to-end path is exactly what you'd hope. A caller fetches an Amazon Cognito token, calls the MCP tool, and a real Tavily answer comes back *through the wrapper* — the client never touches the upstream key:\n\n```text\nAnswer: The Model Context Protocol (MCP) is a standardized framework\nenabling AI models to access external data sources and tools securely…\n\n1. What is the Model Context Protocol (MCP)?\n https://www.databricks.com/blog/what-is-model-context-protocol\n```\n\nAnd the boundary holds: a request with **no token, or a forged one, gets a `401` at the edge** — rejected by API Gateway before the Lambda ever runs. Meanwhile every successful call writes an audit line naming the caller — `client 59psk…` or `demo@example.com` — something the upstream's shared-key logs physically cannot produce. (Commands and full output are in the repo.)\n\n## Why this generalizes\n\nSwap Tavily for a legacy internal API, a SaaS whose own OAuth logs into *its* identity instead of yours, or a machine credential you don't want sprayed across developer laptops — the Lambda is the only thing that changes, and the payoff is the same. A leaked Amazon Cognito access token is short-lived and expires on its own, and you cut a compromised identity off at the source by revoking its refresh token (and disabling the user/app client) so it can't mint new ones — versus a leaked shared key, which means rotate-and-redeploy for the whole team. (Worth knowing: with the edge JWT authorizer, an already-issued access token stays valid until it expires; for instant kill-switch revocation you'd add a server-side check in the Lambda.) It ties straight back to the Golden Rule from the talk — *if a human can't do it in the UI, the agent can't do it via MCP* — because the agent inherits exactly the caller's scope, nothing more. That's what makes team-wide AI-assisted coding safe to roll out: developers plug a powerful shared-key tool straight into their own IDE, and security doesn't have to say no, because there's no shared credential on the laptop to leak — just the developer's own scoped, revocable session.\n\n## What does all this cost? Almost nothing.\n\nHere's the part that makes this an easy yes: the entire control plane is serverless and pay-per-use, so for a team's real workload it rounds to a rounding error. Amazon Cognito is free up to 50,000 monthly active users. Lambda's free tier covers a million requests a month. An idle HTTP API costs nothing and is about $1 per million requests after that.\n\nThe one real decision is where the shared upstream key lives. An encrypted Lambda environment variable is free but readable by anyone with `lambda:GetFunctionConfiguration` and baked into the deployment — fine for a demo, not for production. SSM Parameter Store `SecureString` is also free and gives the same *at-rest* protection for a static key (KMS-encrypted, IAM-scoped reads) — the sweet spot for most single-key wrappers. AWS Secrets Manager costs ~$0.40/month but adds the operational layer you may actually need: automatic rotation, resource policies, cross-account sharing. I used Secrets Manager in the reference because it makes the lockdown story explicit, but Parameter Store swaps in with a few lines. Either way the boundary is identical: IAM grants read to *only* the Lambda's execution role, and the key never leaves the server.\n\nYou also get *measurement* nearly for free: because every call carries a real identity, your Lambda can log structured per-user, per-tool fields, and CloudWatch Logs Insights (or an EMF metric) turns them into per-identity rate limits, anomaly alerts, or usage you bill back to a team. The shared-key world gives you one undifferentiated blob of traffic and none of that.\n\nFull source — `template.yaml`, the Lambda, both flow scripts, and a one-command teardown — is on GitHub. Clone it, point it at your own shared-key API, and you've got per-user OAuth in an afternoon.\n\n---\n\n### Further reading\n\n- **The talk this builds on** — [\"Empower Team-Wide Vibe Coding with LLM Gateway and Security-First MCPs\"](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/) (Gabriel Koo & Rakshit Jain, AgentCon HK 2026).\n- **Interactive demo** — [click through the full OAuth flow](https://gabrielkoo.github.io/tavily-oauth-mcp-wrapper/) (login, scoped call, 403 on out-of-scope tool, and the `stdio` vs `http` config contrast).\n- **Runnable demo & full source**: [github.com/gabrielkoo/tavily-oauth-mcp-wrapper](https://github.com/gabrielkoo/tavily-oauth-mcp-wrapper)", "excerpts": [ "_The runnable companion to my AgentCon HK 2026 talk, [\"Empower Team-Wide Vibe Coding with LLM Gateway and Security-First MCPs.\"](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/) The talk argued per-user OAuth is what turns a shared god-token into safe, auditable agent access. This is the wiring — explained at the architecture level. The full `template.yaml`, Lambda, and scripts live in the public repo linked at the end._", "The gap nobody fills: *your* identity in front of a shared-key API", "By 2026 most big SaaS shipped an official MCP server, and many even support OAuth. So \"my tool has no MCP\" and \"my tool has no OAuth\" are both fading problems — chasing vendors that lack one is a losing game; the list shrinks every week.", "Tavily — the search API I use here — is actually a good citizen: its remote MCP supports both a shared key in the URL (`https://mcp.tavily.com/mcp/?tavilyApiKey= `) *and* an OAuth flow. So why wrap it at all? Because even a vendor's OAuth answers a different question than the one a security team is asking:", "- **Shared keys are the default, and cost is why.** Plenty of these tools price per seat. So the cheapest integration — the one teams actually ship — is one account's API key wired into a single shared MCP for the whole group. The instant that key is shared, per-user identity is gone: every call is the same caller, and \"who ran this?\" has no answer. Vendor OAuth only helps if everyone pays for their own seat, which is exactly the bill teams are dodging. - **Claimed ≠ enforced.** Even when a tool *does* attribute per user, it often rides on a value the client supplies — Tavily's `X-Human-Id` header is exactly this. The client asserts it, so nothing stops a caller from sending someone else's; the attribution is a courtesy, not a control. Trustworthy attribution has to come from a token *your own* gateway cryptographically verified, not a string the client typed.", "And here's the part that doesn't shrink at all: **the upstreams that matter most to you will never ship OAuth.** Your internal billing API, the legacy claims system nobody wants to touch, the compliance-mandated third-party tool with a single team API key. For every one of those, the only auth is a shared key — shared by nature. And it's not only internal systems: plenty of public vendors are the same shape — Brave Search's own official MCP server, for instance, authenticates with a single `BRAVE_API_KEY` and no OAuth at all. One key, the whole team behind it.", "That's the real, durable gap: **no front door that authenticates callers as *your* identity, scopes them, and audits them — before handing off to a shared-key upstream.** That's what this post builds. Tavily is just a free, public stand-in you can actually run; mentally swap it for your own shared-key API.", "> 💡 **The one idea:** put a real OAuth 2.0 / OIDC authorization server bound to *your* identity (Amazon Cognito) — running the PKCE and client-credentials flows the MCP spec expects — in front of a shared-key upstream, in a single Lambda. Each caller gets their own scoped, cryptographically-verified, audited identity. The shared key never leaves the server.", "*Want the 30-second version first? [Walk through the interactive demo](https://gabrielkoo.github.io/tavily-oauth-mcp-wrapper/) — click through the whole OAuth flow, the scoped tool call, and the `403` when a caller reaches past its scope.*", "Two pictures tell the whole story. The shared-key path — one key for everyone, whether it sits in a URL or a config file:", "![Shared-key access — three callers (two engineers and an agent) all funnel through one shared key, which then hits the upstream API. No per-user scope under your control, no audit you own, the key leaks into logs, and rotating it breaks the whole team.](/assets/img/3c5b6ff475fb.png)", "With the wrapper, each caller arrives as themselves — verified against your own identity provider — and the key is locked away server-side:", "![The OAuth wrapper — each caller logs in to its own session, hits an MCP server fronted by Amazon Cognito OAuth that enforces per-user scope and audit, and the shared key sits server-side (Secrets Manager / Parameter Store), reached only by the server before it calls the upstream. Scoped per identity, fully audited, key never leaves the server, and one session can be revoked without touching the team.](/assets/img/14d02ac3c62f.png)", "Three managed AWS pieces do the work, each with one clear job:", "**Amazon Cognito — the bouncer who issues the wristbands.** It's the OAuth 2.0 / OIDC authorization server (with PKCE — the piece OAuth 2.1 and the MCP spec lean on). A caller proves who they are *to Amazon Cognito*, which hands back a short-lived signed token stamped with a scope (here, `tavily-mcp/search`). It handles both kinds of caller from the same pool: a backend agent authenticates machine-to-machine, a human logs in through a hosted login page with PKCE. Crucially, Amazon Cognito *is* your identity layer — swapping the demo's username/password for real corporate SSO (SAML, OIDC, or social logins like Google, Microsoft, Apple) is a configuration change, not a rebuild. The tokens, scopes, and everything downstream stay identical.", "**API Gateway HTTP API + its JWT authorizer — the wristband scanner at the door.** Every request to the MCP endpoint — a plain JSON-RPC call over HTTP POST (the Streamable HTTP transport; this wrapper uses request/response POSTs only, not the transport's optional SSE streaming channel) — hits API Gateway first. Its [built-in JWT authorizer](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-jwt-authorizer.html) — a native feature of the HTTP API (v2) flavour; the older REST API would need a Cognito or custom Lambda authorizer instead — checks the token's signature, issuer, and expiry *at the edge*, before a single line of your code runs. No token, expired token, forged token: rejected with a 401 right there. This is pure authentication: *are you who the token says you are?* Nothing reaches your logic until that passes. (One footgun worth flagging: Cognito access tokens carry no `aud` claim, so the authorizer's audience must be set to the Cognito app client ID — a mismatch here is the most common source of silent 401s in this stack.)", "**Lambda — the room you're actually allowed into.** Once the token is valid, the Lambda does the *authorization*: it re-reads the scope to confirm this identity may call this specific tool, writes an audit line tying the action to that caller, then — and only then — reaches into Secrets Manager for the shared upstream key and calls Tavily. The key lives only inside this function's execution role; it is never sent back to the client.", "![The deployed request path — MCP client sends an Amazon Cognito Bearer token; API Gateway's JWT authorizer validates it at the edge; the Lambda re-checks scope, logs the caller, and calls Tavily with a key it reads from Secrets Manager and never returns to the client.](/assets/img/c700c0918921.png)", "The split is the whole point: **API Gateway answers \"is this a real, valid token?\" and Lambda answers \"is this identity allowed to do this, and let's record that they did.\"** Authentication at the edge, authorization in your code, the secret sealed behind both.", "Why `type: http`, not `type: stdio`", "There's a reason this wrapper is a *remote* MCP server and not a local one — and it's the same reason the talk put OAuth-fronted remote servers at the centre of the architecture. An MCP client config gives you three transport choices, and the choice quietly decides your entire security story:", "- **`stdio`** — the client spawns a local process (`npx some-mcp`, a Python script) and talks to it over stdin/stdout. The catch: that process needs the upstream credential *on the developer's machine*. So the key lands in `mcp.json`, in an `env` block, in shell history, in a dotfile that gets synced to who-knows-where. Every laptop is now a copy of the shared key. This is the exact shape of the BYOAI / shared-key problem — one credential, sprayed everywhere, impossible to attribute or revoke cleanly. - **`sse`** — the older remote transport. Remote is the right instinct, but plain SSE was only ever a transport; it never carried an auth story of its own, so in practice people bolted a static bearer token onto it and landed right back at \"one shared secret.\" - **`http`** (Streamable HTTP) — a remote endpoint the client reaches over plain HTTPS, and the transport the 2025 MCP spec standardised on. Crucially, the [spec defines the MCP server as an **OAuth resource server**](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) — so authentication is a first-class part of the transport, not an afterthought. The client config holds *no secret at all*; it just poi", "That last line is the whole pitch. Compare the two configs a developer actually writes:", "// http — no secret, OAuth per user { \"tavily\": { \"type\": \"http\", \"url\": \"https://…/mcp\" } } ```", "The `http` version has nothing to leak. The credential never reaches the client; identity is established per-user through Cognito; and the server — not the laptop — is the only thing that ever touches the upstream key. Whenever an upstream *can* be reached as a remote OAuth-fronted `http` server, it should be. This wrapper exists precisely to turn a `stdio`-shaped shared-key tool into an `http`-shaped one.", "One endpoint, two kinds of caller", "A backend agent and a human engineer authenticate through completely different OAuth flows — `client_credentials` for the machine, `authorization_code` + PKCE for the person. But they end up carrying the *same* scope and hitting the *same* endpoint. The server treats them identically; the only difference is what lands in the audit log — a client ID for the machine, an email for the human.", "![The two token origins converge on one MCP endpoint — a machine client via client_credentials and a human via authorization_code + PKCE both arrive carrying the same tavily-mcp/search scope, and the server treats them identically while logging a different caller identity.](/assets/img/f42275ee79db.png)", "The last mile most \"MCP + OAuth\" posts skip", "One piece of the human flow is easy to gloss over, and it's exactly where most \"remote MCP + OAuth\" walkthroughs quietly wave their hands: a static MCP client config (the `claude_desktop_config.json` kind) can't pop a browser for a Cognito login on its own. The fix is a thin local helper — registered as the client's `stdio` command — that checks for a cached token, triggers the PKCE browser login when it's missing or expired, stashes the result, and forwards each request with the `Authorization: Bearer …` header. This is exactly what tools like [`mcp-remote`](https://github.com/geelen/mcp-remote) exist to do — and the [rough edges around that login flow](https://github.com/geelen/mcp-remote/issues/251) are a recurring [source of confusion](https://www.reddit.com/r/mcp/comments/1mw09b5/how_are_you_handling_oauth_when_running_mcp/) when people first wire up a remote MCP server. It's a small shim, but without it the \"human logs in with PKCE\" line is doing a lot of unspoken work. The repo includes a runnable PKCE script you can wire in as that helper.", "**The Lambda is the *final* enforcement point — but the security of the whole scheme rests on the entire chain: the JWT authorizer validating token integrity at the edge, and the Lambda enforcing scope behind it.** The upstream still sees one shared key; it has no idea which human is behind a call. All the per-user identity, scoping, and audit live in *your* layer. So the scheme reduces to a few disciplines:", "- **Lock the secret to only the Lambda's execution role.** Anyone who can read it gets full upstream access. - **Make sure every route to the function goes through the JWT authorizer.** A second unprotected trigger would bypass the whole thing. - **Machine callers blur the human behind them.** When a backend agent authenticates with `client_credentials`, the audit log names the *machine* client, not the person who prompted it. For a human-triggered agent you've narrowed \"who\" to one service identity, not one person. Closing that last gap means propagating the user's own token down to the agent (OAuth 2.0 token exchange, RFC 8693) instead of falling back to a shared machine credential — worth knowing before you lean on M2M audit lines as proof of *who* did something. - **Mind the 30-second ceiling.** API Gateway HTTP APIs cap each integration at 30 seconds. A search call returns in well under a second, so it's a non-issue here — but if you wrap a slower upstream (a heavy query, a multi-step scrape, a long-running agent tool), a call that overruns hits a `504` at the gateway. That's the point where you'd reach for the transport's SSE streaming channel, or an async submit-then-poll pa", "Get those right and you've genuinely converted a shared god-token into per-user delegation. Get them wrong and you've just added a proxy in front of the same shared key. This is also why the *audit log* matters more than it looks: it's the only place \"who ran what\" is recoverable at all — Tavily's logs will forever show one key.", "Deployed to a real AWS account, the end-to-end path is exactly what you'd hope. A caller fetches an Amazon Cognito token, calls the MCP tool, and a real Tavily answer comes back *through the wrapper* — the client never touches the upstream key:", "1. What is the Model Context Protocol (MCP)? https://www.databricks.com/blog/what-is-model-context-protocol ```", "And the boundary holds: a request with **no token, or a forged one, gets a `401` at the edge** — rejected by API Gateway before the Lambda ever runs. Meanwhile every successful call writes an audit line naming the caller — `client 59psk…` or `demo@example.com` — something the upstream's shared-key logs physically cannot produce. (Commands and full output are in the repo.)", "Swap Tavily for a legacy internal API, a SaaS whose own OAuth logs into *its* identity instead of yours, or a machine credential you don't want sprayed across developer laptops — the Lambda is the only thing that changes, and the payoff is the same. A leaked Amazon Cognito access token is short-lived and expires on its own, and you cut a compromised identity off at the source by revoking its refresh token (and disabling the user/app client) so it can't mint new ones — versus a leaked shared key, which means rotate-and-redeploy for the whole team. (Worth knowing: with the edge JWT authorizer, an already-issued access token stays valid until it expires; for instant kill-switch revocation you'd add a server-side check in the Lambda.) It ties straight back to the Golden Rule from the talk — *if a human can't do it in the UI, the agent can't do it via MCP* — because the agent inherits exactly the caller's scope, nothing more. That's what makes team-wide AI-assisted coding safe to roll out: developers plug a powerful shared-key tool straight into their own IDE, and security doesn't have to say no, because there's no shared credential on the laptop to leak — just the developer's own scope", "What does all this cost? Almost nothing.", "Here's the part that makes this an easy yes: the entire control plane is serverless and pay-per-use, so for a team's real workload it rounds to a rounding error. Amazon Cognito is free up to 50,000 monthly active users. Lambda's free tier covers a million requests a month. An idle HTTP API costs nothing and is about $1 per million requests after that.", "The one real decision is where the shared upstream key lives. An encrypted Lambda environment variable is free but readable by anyone with `lambda:GetFunctionConfiguration` and baked into the deployment — fine for a demo, not for production. SSM Parameter Store `SecureString` is also free and gives the same *at-rest* protection for a static key (KMS-encrypted, IAM-scoped reads) — the sweet spot for most single-key wrappers. AWS Secrets Manager costs ~$0.40/month but adds the operational layer you may actually need: automatic rotation, resource policies, cross-account sharing. I used Secrets Manager in the reference because it makes the lockdown story explicit, but Parameter Store swaps in with a few lines. Either way the boundary is identical: IAM grants read to *only* the Lambda's execution role, and the key never leaves the server.", "You also get *measurement* nearly for free: because every call carries a real identity, your Lambda can log structured per-user, per-tool fields, and CloudWatch Logs Insights (or an EMF metric) turns them into per-identity rate limits, anomaly alerts, or usage you bill back to a team. The shared-key world gives you one undifferentiated blob of traffic and none of that.", "Full source — `template.yaml`, the Lambda, both flow scripts, and a one-command teardown — is on GitHub. Clone it, point it at your own shared-key API, and you've got per-user OAuth in an afternoon.", "- **The talk this builds on** — [\"Empower Team-Wide Vibe Coding with LLM Gateway and Security-First MCPs\"](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/) (Gabriel Koo & Rakshit Jain, AgentCon HK 2026). - **Interactive demo** — [click through the full OAuth flow](https://gabrielkoo.github.io/tavily-oauth-mcp-wrapper/) (login, scoped call, 403 on out-of-scope tool, and the `stdio` vs `http` config contrast). - **Runnable demo & full source**: [github.com/gabrielkoo/tavily-oauth-mcp-wrapper](https://github.com/gabrielkoo/tavily-oauth-mcp-wrapper)" ] }, { "id": "article:resurface-claude-code-usage-across-your-team-with-cloudwatch-otel-no-lambda-4p0i", "source_type": "article", "title": "Resurface Claude Code Usage Across Your Team with CloudWatch OTEL (No Lambda)", "url": "https://gabrielkoo.com/blog/resurface-claude-code-usage-across-your-team-with-cloudwatch-otel-no-lambda-4p0i/", "canonical_url": "https://dev.to/aws-builders/resurface-claude-code-usage-across-your-team-with-cloudwatch-otel-no-lambda-4p0i", "published_at": "2026-04-18", "last_verified_at": "2026-08-23", "tags": [ "aws", "opentelemetry", "cloudwatch", "serverless" ], "description": "I've been building AI tooling infrastructure to empower a team of 50+ software engineers to do vibe...", "content": "I've been building AI tooling infrastructure to empower a team of 50+ software engineers to do vibe coding safely. We went from 3 engineers using AI full-time to 50+ in 6 months — including non-engineers. (I co-presented on this journey at [AgentCon Hong Kong 2026](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/).)\n\nOne thing we learned: **you can't improve what you can't measure.** Once you give a team AI coding tools, you want visibility into how they're being used — not to evaluate individual engineers, but to understand adoption patterns. Which tools are people reaching for? How large are the prompts? What tool calls are being made? Making these metrics visible to everyone helps the team learn from each other and helps champions pull others forward.\n\nThis post is about the plumbing: how to get that telemetry data from coding agents into CloudWatch with minimal infrastructure.\n\n**\"But we already have an LLM gateway.\"** If your team routes AI traffic through a gateway like [LiteLLM](https://github.com/BerriAI/litellm) or [AWS Bedrock](https://aws.amazon.com/bedrock/), you already have token-level usage data. But if your engineers are on coding plans — Claude Team/Max, OpenCode Go, GitHub Copilot seats, ChatGPT Codex — the LLM calls bypass your gateway entirely. You lose visibility into the interesting stuff: how many tool calls per session, prompt sizes, which tools are being invoked, who's active at what times. That's where OTEL telemetry fills the gap.\n\nAI coding tools are shipping with built-in OpenTelemetry support. [Claude Code](https://docs.anthropic.com/en/docs/claude-code/monitoring-usage), [Claude CoWork](https://support.claude.com/en/articles/14477985-monitor-claude-cowork-activity-with-opentelemetry), [GitHub Copilot](https://docs.github.com/copilot/how-tos/copilot-sdk/observability/opentelemetry), [Gemini CLI](https://geminicli.com/docs/cli/telemetry/), and [Cursor](https://github.com/LangGuard-AI/cursor-otel-hook) (via hooks) all export metrics, traces, and log events over OTLP/HTTP — token counts, tool durations, model latency, the works. [Kiro has an open feature request](https://github.com/kirodotdev/Kiro/issues/6319) for native OTEL support too.\n\nThere's one catch: **CloudWatch's OTLP endpoints require SigV4 signing.** These tools' OTEL SDKs can't do that. Neither can most OTEL SDKs without an AWS-specific exporter or a collector sidecar.\n\nThe usual fix is a Lambda function that receives OTLP, signs it, and forwards it. That means cold starts, packaging, and another thing to maintain.\n\nHere's a simpler way: **API Gateway REST API with AWS Service Integration.** APIGW signs the request with SigV4 using an execution role. No Lambda. No collector. No code.\n\n![Expanding Brain Meme](/assets/img/51d37c7f020b.png)\n\n> **Timeline:** CloudWatch has supported OTLP ingestion for [traces and logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) for some time (availability varies by region — check the [OTLP endpoints doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)). [Native OTLP metrics support launched April 2, 2026](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) in public preview, completing all three pillars of observability via OTLP.\n\n## Architecture\n\n![Architecture](/assets/img/91657a8ac5b6.png)\n\n```plaintext\nAI Coding Tool (OTEL SDK)\n ↓ OTLP/HTTP + x-api-key\nAPI Gateway REST API\n ├→ POST /v1/metrics → AWS Integration → monitoring (SigV4) → CloudWatch Metrics\n ├→ POST /v1/traces → AWS Integration → xray (SigV4) → X-Ray / CloudWatch Logs\n └→ POST /v1/logs → AWS Integration → logs (SigV4) → CloudWatch Logs\n```\n\nThe client sends standard OTLP/HTTP requests with an API key. APIGW validates the key, assumes an IAM role, signs the request with SigV4, and forwards it to the CloudWatch OTLP endpoint. That's it.\n\n## Why This Works\n\nAPI Gateway REST API has an integration type called **[AWS Service Integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html)**. It can call any AWS service API and sign the request with SigV4 using an execution role. The [CloudWatch OTLP endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) are standard AWS service endpoints:\n\n| Signal | Endpoint | Service |\n|--------|----------|---------|\n| Metrics | `monitoring.{region}.amazonaws.com/v1/metrics` | `monitoring` |\n| Traces | `xray.{region}.amazonaws.com/v1/traces` | `xray` |\n| Logs | `logs.{region}.amazonaws.com/v1/logs` | `logs` |\n\nAPIGW's integration URI format maps directly:\n\n```plaintext\narn:aws:apigateway:{region}:monitoring:path/v1/metrics\narn:aws:apigateway:{region}:xray:path/v1/traces\narn:aws:apigateway:{region}:logs:path/v1/logs\n```\n\n## Setup\n\nThe full infrastructure is defined in a CloudFormation template (link at the bottom). Here's what it creates:\n\n### IAM Execution Role\n\nAPIGW needs an IAM role to sign requests to CloudWatch. The policy is scoped to only the actions and resources needed for OTLP ingestion:\n\n```yaml\nParameters:\n OtlpLogGroupName:\n Type: String\n Default: \"otlp-logs\"\n Description: CloudWatch Logs log group for OTLP log ingestion\n\nResources:\n OtlpExecutionRole:\n Type: AWS::IAM::Role\n Properties:\n AssumeRolePolicyDocument:\n Statement:\n - Effect: Allow\n Principal:\n Service: apigateway.amazonaws.com\n Action: sts:AssumeRole\n Policies:\n - PolicyName: otlp-metrics\n PolicyDocument:\n Statement:\n - Effect: Allow\n Action:\n - cloudwatch:PutMetricData\n Resource: \"*\"\n - PolicyName: otlp-traces\n PolicyDocument:\n Statement:\n - Effect: Allow\n Action:\n - xray:PutTraceSegments\n - xray:PutTelemetryRecords\n Resource: \"*\"\n - PolicyName: otlp-logs\n PolicyDocument:\n Statement:\n - Effect: Allow\n Action:\n - logs:PutLogEvents\n - logs:CreateLogStream\n - logs:DescribeLogStreams\n Resource:\n - !Sub \"arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${OtlpLogGroupName}:*\"\n```\n\nNote: `cloudwatch:PutMetricData` doesn't support resource-level ARNs. The `cloudwatch:namespace` condition key exists but does not apply to the OTLP ingestion path — metrics are accepted regardless of namespace. X-Ray `PutTraceSegments` also doesn't support resource-level restrictions. Logs permissions are scoped to a specific log group via the `OtlpLogGroupName` parameter.\n\n### API Gateway with AWS Service Integration\n\nEach OTLP signal gets its own resource with an AWS integration:\n\n```yaml\nMetricsMethod:\n Type: AWS::ApiGateway::Method\n Properties:\n HttpMethod: POST\n AuthorizationType: NONE\n ApiKeyRequired: true\n Integration:\n Type: AWS\n IntegrationHttpMethod: POST\n Uri: !Sub \"arn:aws:apigateway:${AWS::Region}:monitoring:path/v1/metrics\"\n Credentials: !GetAtt OtlpExecutionRole.Arn\n PassthroughBehavior: WHEN_NO_MATCH\n ContentHandling: CONVERT_TO_TEXT\n```\n\nSame pattern for `/v1/traces` (service: `xray`) and `/v1/logs` (service: `logs`).\n\n### API Key Authentication\n\nProtect the endpoint with an API key so only your tools can send telemetry:\n\n```yaml\nApiKey:\n Type: AWS::ApiGateway::ApiKey\n Properties:\n Enabled: true\n\nUsagePlan:\n Type: AWS::ApiGateway::UsagePlan\n Properties:\n ApiStages:\n - ApiId: !Ref Api\n Stage: !Ref Stage\n```\n\n## Configure Your Tools\n\nThe proxy works with any tool that supports standard OTEL environment variables. Here's how to configure each:\n\n> **Disclaimer:** I personally use Claude Code routed through a custom LLM gateway (not a coding plan), since some coding plans aren't available in the region I live in. The configurations below are based on each tool's official documentation — your mileage may vary.\n\n### Claude Code\n\n[Official monitoring docs](https://docs.anthropic.com/en/docs/claude-code/monitoring-usage)\n\n```bash\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=otlp\nexport OTEL_LOGS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=http/json\nexport OTEL_EXPORTER_OTLP_ENDPOINT=https://xxx.execute-api.us-west-2.amazonaws.com/prod\nexport OTEL_EXPORTER_OTLP_HEADERS=x-api-key=your-api-key\nexport OTEL_SERVICE_NAME=claude-code\n```\n\nFor short-lived tasks, lower the export interval so data flushes before the process exits:\n\n```bash\nexport OTEL_METRIC_EXPORT_INTERVAL=1000\nexport OTEL_LOGS_EXPORT_INTERVAL=1000\n```\n\nFor traces (beta):\n\n```bash\nexport CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1\nexport OTEL_TRACES_EXPORTER=otlp\nexport OTEL_TRACES_EXPORT_INTERVAL=1000\n```\n\n**Enforcing OTEL across your team:** Claude Code supports [managed settings](https://code.claude.com/docs/en/settings#settings-files) via `managed-settings.json`, deployable through MDM (Jamf, Intune, etc.). This lets you enforce OTEL configuration org-wide — engineers don't need to set environment variables manually, and they can't opt out.\n\n### Claude CoWork (Team & Enterprise)\n\n[CoWork monitoring docs](https://support.claude.com/en/articles/14477985-monitor-claude-cowork-activity-with-opentelemetry) — configure via Admin Settings → Cowork → Monitoring:\n\n- OTLP endpoint: your APIGW URL\n- OTLP protocol: `http/json`\n- OTLP headers: `x-api-key=your-api-key`\n\nCoWork streams user prompts, tool/MCP invocations, file access, human approval decisions, and API request details. It shares the same OTel event schema as Claude Code via the Claude Agent SDK — you can distinguish them by `terminal.type` (`cowork` vs `cli`).\n\n### GitHub Copilot CLI\n\n[Copilot CLI OTel reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-command-reference#opentelemetry-monitoring) — available since Copilot CLI 1.0.4:\n\n```bash\nexport COPILOT_OTEL_ENDPOINT=https://xxx.execute-api.us-west-2.amazonaws.com/prod\nexport COPILOT_OTEL_HEADERS=x-api-key=your-api-key\n```\n\n### Gemini CLI\n\n[Gemini CLI telemetry docs](https://geminicli.com/docs/cli/telemetry/)\n\n```bash\nexport GEMINI_CLI_OTEL_EXPORT_ENDPOINT=https://xxx.execute-api.us-west-2.amazonaws.com/prod\n```\n\n### Cursor (via Hooks)\n\nCursor doesn't have native OTEL export yet, but the community [cursor-otel-hook](https://github.com/LangGuard-AI/cursor-otel-hook) project captures agent activity via Cursor's hook system and exports traces to any OTLP endpoint. Configure via `otel_config.json`:\n\n```json\n{\n \"OTEL_EXPORTER_OTLP_ENDPOINT\": \"https://xxx.execute-api.us-west-2.amazonaws.com/prod/v1/traces\",\n \"OTEL_EXPORTER_OTLP_PROTOCOL\": \"http/json\",\n \"OTEL_EXPORTER_OTLP_HEADERS\": { \"x-api-key\": \"your-api-key\" }\n}\n```\n\n## What You Get\n\nCloudWatch receives standard OTLP data. For Claude Code specifically:\n\n- **Metrics**: `claude_code.token.usage` (by `token.type`: input/output/cache_read/cache_creation), `claude_code.cost.usage` (USD), `claude_code.session.count`, `claude_code.lines_of_code.count`\n- **Traces** (beta): Spans linking each user prompt → API requests → tool executions\n- **Log events**: `claude_code.user_prompt`, `claude_code.tool_decision`, `claude_code.tool_result`, `claude_code.api_request` — all tagged with `session.id` and `service.name=claude-code`\n\nHere's what real Claude Code log events look like after flowing through the proxy into CloudWatch Logs. This is actual data from an E2E test — a single prompt that triggered a Bash tool call:\n\n**`claude_code.user_prompt`** — emitted when the user sends a prompt:\n\n```json\n{\n \"resource\": {\n \"attributes\": {\n \"host.arch\": \"arm64\",\n \"os.type\": \"linux\",\n \"service.name\": \"claude-code\",\n \"service.version\": \"2.1.114\",\n \"os.version\": \"6.17.0-1010-aws\"\n }\n },\n \"scope\": {\n \"name\": \"com.anthropic.claude_code.events\",\n \"version\": \"2.1.114\"\n },\n \"body\": \"claude_code.user_prompt\",\n \"attributes\": {\n \"event.sequence\": 0,\n \"user.id\": \"1c257d04...\",\n \"prompt_length\": \"40\",\n \"terminal.type\": \"non-interactive\",\n \"event.name\": \"user_prompt\",\n \"event.timestamp\": \"2026-04-18T11:23:13.187Z\",\n \"prompt\": \"\",\n \"session.id\": \"846ab649-8bba-471e-8ec5-8756116d0840\",\n \"prompt.id\": \"88475ee2-59c2-4137-9201-5540c6a6cad1\"\n }\n}\n```\n\n**`claude_code.tool_result`** — emitted after each tool execution:\n\n```json\n{\n \"body\": \"claude_code.tool_result\",\n \"attributes\": {\n \"tool_name\": \"Bash\",\n \"tool_result_size_bytes\": \"899\",\n \"tool_input\": \"{\\\"command\\\":\\\"ls\\\",\\\"description\\\":\\\"List files in current directory\\\"}\",\n \"duration_ms\": \"95\",\n \"success\": \"true\",\n \"session.id\": \"846ab649-8bba-471e-8ec5-8756116d0840\",\n \"prompt.id\": \"88475ee2-59c2-4137-9201-5540c6a6cad1\"\n }\n}\n```\n\n**`claude_code.api_request`** — emitted after each API call with token counts and cost:\n\n```json\n{\n \"body\": \"claude_code.api_request\",\n \"attributes\": {\n \"model\": \"claude-sonnet-4-5-20250929\",\n \"input_tokens\": \"142\",\n \"output_tokens\": \"61\",\n \"cache_read_tokens\": \"0\",\n \"cache_creation_tokens\": \"25848\",\n \"cost_usd\": \"0.098271\",\n \"duration_ms\": \"4950\",\n \"speed\": \"normal\",\n \"session.id\": \"846ab649-8bba-471e-8ec5-8756116d0840\",\n \"prompt.id\": \"88475ee2-59c2-4137-9201-5540c6a6cad1\"\n }\n}\n```\n\nAll events share the same `prompt.id`, linking them into a single interaction. The `event.sequence` field orders events within a prompt. Every record carries `service.name=claude-code` in resource attributes, so isolating Claude Code telemetry in a mixed pipeline is trivial — just filter on that in CloudWatch Logs Insights:\n\n```sql\nfields @timestamp, body, attributes.model, attributes.cost_usd, attributes.duration_ms\n| filter resource.attributes.`service.name` = 'claude-code'\n| filter body = 'claude_code.api_request'\n| sort @timestamp desc\n```\n\n## Region Availability\n\nCloudWatch OTLP endpoints are available in most regions but **not all**. The [OTLP metrics preview](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) launched in 5 regions:\n\n| Signal | Regions | Docs |\n|--------|---------|------|\n| Metrics (preview) | us-east-1, us-west-2, ap-southeast-1, ap-southeast-2, eu-west-1 | [Announcement](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) |\n| Traces | Most commercial regions | [OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) |\n| Logs | Most commercial regions | [OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) |\n\nTested and confirmed:\n\n| Region | Metrics | Traces | Logs |\n|--------|---------|--------|------|\n| us-east-1 | ✅ | ✅ | ✅ |\n| us-west-2 | ✅ | ✅ | ✅ |\n| ap-southeast-1 | ✅ | ✅ | ✅ |\n| ap-east-1 (Hong Kong) | ❌ | ❌ | ❌ |\n\nIf your primary region doesn't support it, deploy the proxy in a supported region. The APIGW endpoint is accessible from anywhere.\n\nFor the full list of CloudWatch service endpoints by region, see the [AWS General Reference](https://docs.aws.amazon.com/general/latest/gr/cw_region.html).\n\n## Gotchas\n\n**XRay traces require manual setup.** The CloudFormation template creates the proxy endpoints, but X-Ray traces need two additional steps that aren't in the template:\n\n1. Set CloudWatch Logs as the trace segment destination:\n\n```bash\naws xray update-trace-segment-destination --destination CloudWatchLogs\n```\n\n2. Create a CloudWatch Logs resource policy allowing X-Ray to write to the `aws/spans` log group:\n\n```bash\naws logs put-resource-policy \\\n --policy-name XRayAccessPolicy \\\n --policy-document '{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"xray.amazonaws.com\"},\"Action\":[\"logs:PutLogEvents\",\"logs:CreateLogGroup\",\"logs:CreateLogStream\"],\"Resource\":\"*\"}]}'\n```\n\nWithout these, traces will return `AccessDeniedException`.\n\n**CloudWatch Logs supports bearer token auth.** The `/v1/logs` endpoint supports [bearer token authentication](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) without SigV4 — but only for logs. Metrics and traces still require SigV4, which is why the APIGW proxy is needed for a unified endpoint.\n\n**Use `http/json`, not `http/protobuf`.** CloudWatch accepts both formats, but API Gateway's `CONVERT_TO_TEXT` content handling can corrupt binary protobuf payloads in transit. Set `OTEL_EXPORTER_OTLP_PROTOCOL=http/json` to avoid this. JSON is also easier to debug in APIGW execution logs. Most coding tools default to protobuf, so you'll need to override this explicitly.\n\n**API Gateway payload limit.** REST API has a 10MB payload limit. OTLP batches from coding tools are well under this, but keep it in mind if you're aggregating from multiple sources. CloudWatch's own limits are 1MB for metrics and logs, 5MB for traces ([full limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)).\n\n**REST API, not HTTP API.** Only REST API supports the `AWS` integration type needed for SigV4 service proxying. HTTP API does not.\n\n## Cost\n\nThis is about as cheap as it gets for a telemetry pipeline:\n\n| Component | Cost |\n|-----------|------|\n| API Gateway | ~$3.50 / million requests |\n| CloudWatch Metrics | [Standard CW pricing](https://aws.amazon.com/cloudwatch/pricing/) (free during OTel metrics preview) |\n| CloudWatch Logs | [Standard CW pricing](https://aws.amazon.com/cloudwatch/pricing/) |\n| Lambda | $0 (there is none) |\n\nNo idle cost. No provisioned capacity. Pure pay-per-request.\n\nFor comparison: a Lambda-based OTLP forwarder would add ~$0.20/million invocations plus compute time, but gives you retry logic and transformation capabilities. At typical coding agent volumes (a few hundred requests/day per developer), the cost difference is negligible — the real win is operational simplicity.\n\n## When NOT to Use This\n\nThis proxy is optimized for simplicity. It's the right choice for low-to-moderate telemetry volumes from coding agents and developer tools. But it has tradeoffs:\n\n| Approach | Complexity | Cost | Retries | Multi-destination | Transformation |\n|----------|-----------|------|---------|-------------------|----------------|\n| This proxy (APIGW) | Minimal | ~$3.50/M req | ❌ | ❌ | ❌ |\n| OTel Collector | Medium | Compute cost | ✅ | ✅ | ✅ |\n| Lambda forwarder | Medium | ~$0.20/M + compute | ✅ | ✅ | ✅ |\n| ADOT SDK (in-app) | Low | Free (SigV4 native) | ✅ | ❌ | ❌ |\n| SaaS (Datadog, etc.) | Low | $$$ | ✅ | N/A | ✅ |\n\nConsider an OTel Collector or Lambda forwarder instead if you need:\n\n- **High throughput** — thousands of requests/second from many sources\n- **Retry and buffering** — this proxy is fire-and-forget; if CloudWatch returns an error, the data is lost. OTEL SDKs have built-in retry, but only for transient failures\n- **Multi-destination routing** — fan out to CloudWatch + Datadog + S3 simultaneously\n- **Payload transformation** — filter, enrich, or redact telemetry before ingestion\n- **Compliance requirements** — audit trails, guaranteed delivery, or data residency controls\n\nFor most coding agent monitoring use cases (a team of 5-50 developers), this proxy handles the volume comfortably.\n\n## Security Considerations\n\nThe proxy uses API key authentication — simple but not the strongest option. Here's how to harden it:\n\n**Attach AWS WAF to the REST API.** Add rate limiting, IP allowlisting, or geo-blocking to prevent abuse. A single WAF WebACL with a rate-based rule (e.g., 1000 req/5min per IP) costs ~$6/month and stops most abuse patterns.\n\n**Rotate API keys.** APIGW supports multiple API keys per usage plan. Create a new key, distribute it, then disable the old one — zero downtime rotation.\n\n**Consider IAM auth for internal use.** If your tools run inside AWS (EC2, ECS, Lambda), switch `AuthorizationType` from `NONE` to `AWS_IAM` and drop the API key entirely. The caller signs requests with SigV4 using their IAM role — no shared secrets. This doesn't work for external tools like Claude Code on developer laptops, but it's ideal for CI/CD pipelines or server-side agents.\n\n**Egress control.** If you're running coding agents in a controlled environment, restrict outbound traffic to only your APIGW endpoint. This prevents telemetry from leaking to unauthorized collectors.\n\n## Beyond Coding Agents\n\nThis proxy works with **any OTEL SDK** that supports OTLP/HTTP. If your tool can set `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS`, it can ship telemetry to CloudWatch through this proxy.\n\nPotential use cases:\n- **AI coding agents** (Claude Code, CoWork, Copilot, Cursor, Gemini CLI) — track token usage, costs, and tool calls across your org\n- **Internal tools** — ship metrics without embedding AWS credentials in client apps\n- **CI/CD pipelines** — export build/test telemetry to CloudWatch\n- **On-premises services** — send OTLP from outside AWS without running ADOT Collector\n\nFor apps running inside AWS with IAM roles available, consider the [ADOT SDK](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLP-UsingADOT.html) for collector-less telemetry with native SigV4 signing — no proxy needed.\n\n## Source Code & One-Click Deploy\n\nThe CloudFormation template and full documentation are on GitHub:\n\n👉 [gabrielkoo/otlp-cloudwatch-proxy](https://github.com/gabrielkoo/otlp-cloudwatch-proxy)\n\nOne-click deploy to supported regions:\n\n| Region | Deploy |\n|--------|--------|\n| US East (N. Virginia) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) |\n| US West (Oregon) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) |\n| Asia Pacific (Singapore) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) |\n| Asia Pacific (Sydney) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) |\n| Europe (Ireland) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) |\n\n---\n\n*Built and validated on a Saturday morning with Claude Code + OpenClaw. Zero Lambda functions were harmed in the making of this article.*\n\n## Further Reading\n\n- **[AWS Guidance for Claude Code with Amazon Bedrock — Monitoring](https://github.com/aws-solutions-library-samples/guidance-for-claude-code-with-amazon-bedrock/blob/main/assets/docs/MONITORING.md)** — A comprehensive (and admittedly overkill) reference implementation using ECS Fargate + ALB + ADOT Collector + Lambda + DynamoDB + Kinesis + Athena. Great if you want to see the full spectrum of what can be measured: per-user token tracking, quota monitoring, cost dashboards, and an analytics data lake. If you need all of that, use it. If you just need telemetry flowing to CloudWatch, the one-template proxy in this post will do.\n- **[Claude Code Monitoring Docs](https://code.claude.com/docs/en/monitoring-usage)** — Official OTEL configuration reference, including all metrics, events, and traces.\n- **[Claude Code Managed Settings](https://code.claude.com/docs/en/settings#settings-files)** — How to deploy `managed-settings.json` via MDM for org-wide OTEL enforcement.\n- **[CloudWatch OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)** — AWS docs on native OTLP ingestion for metrics, traces, and logs.", "excerpts": [ "I've been building AI tooling infrastructure to empower a team of 50+ software engineers to do vibe coding safely. We went from 3 engineers using AI full-time to 50+ in 6 months — including non-engineers. (I co-presented on this journey at [AgentCon Hong Kong 2026](https://the-quantum-nargle.github.io/agentcon-2026-hk-slides/).)", "One thing we learned: **you can't improve what you can't measure.** Once you give a team AI coding tools, you want visibility into how they're being used — not to evaluate individual engineers, but to understand adoption patterns. Which tools are people reaching for? How large are the prompts? What tool calls are being made? Making these metrics visible to everyone helps the team learn from each other and helps champions pull others forward.", "This post is about the plumbing: how to get that telemetry data from coding agents into CloudWatch with minimal infrastructure.", "**\"But we already have an LLM gateway.\"** If your team routes AI traffic through a gateway like [LiteLLM](https://github.com/BerriAI/litellm) or [AWS Bedrock](https://aws.amazon.com/bedrock/), you already have token-level usage data. But if your engineers are on coding plans — Claude Team/Max, OpenCode Go, GitHub Copilot seats, ChatGPT Codex — the LLM calls bypass your gateway entirely. You lose visibility into the interesting stuff: how many tool calls per session, prompt sizes, which tools are being invoked, who's active at what times. That's where OTEL telemetry fills the gap.", "AI coding tools are shipping with built-in OpenTelemetry support. [Claude Code](https://docs.anthropic.com/en/docs/claude-code/monitoring-usage), [Claude CoWork](https://support.claude.com/en/articles/14477985-monitor-claude-cowork-activity-with-opentelemetry), [GitHub Copilot](https://docs.github.com/copilot/how-tos/copilot-sdk/observability/opentelemetry), [Gemini CLI](https://geminicli.com/docs/cli/telemetry/), and [Cursor](https://github.com/LangGuard-AI/cursor-otel-hook) (via hooks) all export metrics, traces, and log events over OTLP/HTTP — token counts, tool durations, model latency, the works. [Kiro has an open feature request](https://github.com/kirodotdev/Kiro/issues/6319) for native OTEL support too.", "There's one catch: **CloudWatch's OTLP endpoints require SigV4 signing.** These tools' OTEL SDKs can't do that. Neither can most OTEL SDKs without an AWS-specific exporter or a collector sidecar.", "The usual fix is a Lambda function that receives OTLP, signs it, and forwards it. That means cold starts, packaging, and another thing to maintain.", "Here's a simpler way: **API Gateway REST API with AWS Service Integration.** APIGW signs the request with SigV4 using an execution role. No Lambda. No collector. No code.", "![Expanding Brain Meme](/assets/img/51d37c7f020b.png)", "> **Timeline:** CloudWatch has supported OTLP ingestion for [traces and logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) for some time (availability varies by region — check the [OTLP endpoints doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)). [Native OTLP metrics support launched April 2, 2026](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) in public preview, completing all three pillars of observability via OTLP.", "![Architecture](/assets/img/91657a8ac5b6.png)", "The client sends standard OTLP/HTTP requests with an API key. APIGW validates the key, assumes an IAM role, signs the request with SigV4, and forwards it to the CloudWatch OTLP endpoint. That's it.", "API Gateway REST API has an integration type called **[AWS Service Integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html)**. It can call any AWS service API and sign the request with SigV4 using an execution role. The [CloudWatch OTLP endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) are standard AWS service endpoints:", "| Signal | Endpoint | Service | |--------|----------|---------| | Metrics | `monitoring.{region}.amazonaws.com/v1/metrics` | `monitoring` | | Traces | `xray.{region}.amazonaws.com/v1/traces` | `xray` | | Logs | `logs.{region}.amazonaws.com/v1/logs` | `logs` |", "APIGW's integration URI format maps directly:", "The full infrastructure is defined in a CloudFormation template (link at the bottom). Here's what it creates:", "APIGW needs an IAM role to sign requests to CloudWatch. The policy is scoped to only the actions and resources needed for OTLP ingestion:", "Resources: OtlpExecutionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: apigateway.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: otlp-metrics PolicyDocument: Statement: - Effect: Allow Action: - cloudwatch:PutMetricData Resource: \"*\" - PolicyName: otlp-traces PolicyDocument: Statement: - Effect: Allow Action: - xray:PutTraceSegments - xray:PutTelemetryRecords Resource: \"*\" - PolicyName: otlp-logs PolicyDocument: Statement: - Effect: Allow Action: - logs:PutLogEvents - logs:CreateLogStream - logs:DescribeLogStreams Resource: - !Sub \"arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${OtlpLogGroupName}:*\" ```", "Note: `cloudwatch:PutMetricData` doesn't support resource-level ARNs. The `cloudwatch:namespace` condition key exists but does not apply to the OTLP ingestion path — metrics are accepted regardless of namespace. X-Ray `PutTraceSegments` also doesn't support resource-level restrictions. Logs permissions are scoped to a specific log group via the `OtlpLogGroupName` parameter.", "API Gateway with AWS Service Integration", "Each OTLP signal gets its own resource with an AWS integration:", "Same pattern for `/v1/traces` (service: `xray`) and `/v1/logs` (service: `logs`).", "Protect the endpoint with an API key so only your tools can send telemetry:", "UsagePlan: Type: AWS::ApiGateway::UsagePlan Properties: ApiStages: - ApiId: !Ref Api Stage: !Ref Stage ```", "The proxy works with any tool that supports standard OTEL environment variables. Here's how to configure each:", "> **Disclaimer:** I personally use Claude Code routed through a custom LLM gateway (not a coding plan), since some coding plans aren't available in the region I live in. The configurations below are based on each tool's official documentation — your mileage may vary.", "[Official monitoring docs](https://docs.anthropic.com/en/docs/claude-code/monitoring-usage)", "For short-lived tasks, lower the export interval so data flushes before the process exits:", "**Enforcing OTEL across your team:** Claude Code supports [managed settings](https://code.claude.com/docs/en/settings#settings-files) via `managed-settings.json`, deployable through MDM (Jamf, Intune, etc.). This lets you enforce OTEL configuration org-wide — engineers don't need to set environment variables manually, and they can't opt out.", "Claude CoWork (Team & Enterprise)", "[CoWork monitoring docs](https://support.claude.com/en/articles/14477985-monitor-claude-cowork-activity-with-opentelemetry) — configure via Admin Settings → Cowork → Monitoring:", "- OTLP endpoint: your APIGW URL - OTLP protocol: `http/json` - OTLP headers: `x-api-key=your-api-key`", "CoWork streams user prompts, tool/MCP invocations, file access, human approval decisions, and API request details. It shares the same OTel event schema as Claude Code via the Claude Agent SDK — you can distinguish them by `terminal.type` (`cowork` vs `cli`).", "[Copilot CLI OTel reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-command-reference#opentelemetry-monitoring) — available since Copilot CLI 1.0.4:", "[Gemini CLI telemetry docs](https://geminicli.com/docs/cli/telemetry/)", "Cursor doesn't have native OTEL export yet, but the community [cursor-otel-hook](https://github.com/LangGuard-AI/cursor-otel-hook) project captures agent activity via Cursor's hook system and exports traces to any OTLP endpoint. Configure via `otel_config.json`:", "CloudWatch receives standard OTLP data. For Claude Code specifically:", "- **Metrics**: `claude_code.token.usage` (by `token.type`: input/output/cache_read/cache_creation), `claude_code.cost.usage` (USD), `claude_code.session.count`, `claude_code.lines_of_code.count` - **Traces** (beta): Spans linking each user prompt → API requests → tool executions - **Log events**: `claude_code.user_prompt`, `claude_code.tool_decision`, `claude_code.tool_result`, `claude_code.api_request` — all tagged with `session.id` and `service.name=claude-code`", "Here's what real Claude Code log events look like after flowing through the proxy into CloudWatch Logs. This is actual data from an E2E test — a single prompt that triggered a Bash tool call:", "**`claude_code.user_prompt`** — emitted when the user sends a prompt:", "**`claude_code.tool_result`** — emitted after each tool execution:", "**`claude_code.api_request`** — emitted after each API call with token counts and cost:", "All events share the same `prompt.id`, linking them into a single interaction. The `event.sequence` field orders events within a prompt. Every record carries `service.name=claude-code` in resource attributes, so isolating Claude Code telemetry in a mixed pipeline is trivial — just filter on that in CloudWatch Logs Insights:", "CloudWatch OTLP endpoints are available in most regions but **not all**. The [OTLP metrics preview](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) launched in 5 regions:", "| Signal | Regions | Docs | |--------|---------|------| | Metrics (preview) | us-east-1, us-west-2, ap-southeast-1, ap-southeast-2, eu-west-1 | [Announcement](https://aws.amazon.com/about-aws/whats-new/2026/04/amazon-cloudwatch-opentelemetry-metrics/) | | Traces | Most commercial regions | [OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) | | Logs | Most commercial regions | [OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) |", "| Region | Metrics | Traces | Logs | |--------|---------|--------|------| | us-east-1 | ✅ | ✅ | ✅ | | us-west-2 | ✅ | ✅ | ✅ | | ap-southeast-1 | ✅ | ✅ | ✅ | | ap-east-1 (Hong Kong) | ❌ | ❌ | ❌ |", "If your primary region doesn't support it, deploy the proxy in a supported region. The APIGW endpoint is accessible from anywhere.", "For the full list of CloudWatch service endpoints by region, see the [AWS General Reference](https://docs.aws.amazon.com/general/latest/gr/cw_region.html).", "**XRay traces require manual setup.** The CloudFormation template creates the proxy endpoints, but X-Ray traces need two additional steps that aren't in the template:", "1. Set CloudWatch Logs as the trace segment destination:", "2. Create a CloudWatch Logs resource policy allowing X-Ray to write to the `aws/spans` log group:", "Without these, traces will return `AccessDeniedException`.", "**CloudWatch Logs supports bearer token auth.** The `/v1/logs` endpoint supports [bearer token authentication](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html) without SigV4 — but only for logs. Metrics and traces still require SigV4, which is why the APIGW proxy is needed for a unified endpoint.", "**Use `http/json`, not `http/protobuf`.** CloudWatch accepts both formats, but API Gateway's `CONVERT_TO_TEXT` content handling can corrupt binary protobuf payloads in transit. Set `OTEL_EXPORTER_OTLP_PROTOCOL=http/json` to avoid this. JSON is also easier to debug in APIGW execution logs. Most coding tools default to protobuf, so you'll need to override this explicitly.", "**API Gateway payload limit.** REST API has a 10MB payload limit. OTLP batches from coding tools are well under this, but keep it in mind if you're aggregating from multiple sources. CloudWatch's own limits are 1MB for metrics and logs, 5MB for traces ([full limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)).", "**REST API, not HTTP API.** Only REST API supports the `AWS` integration type needed for SigV4 service proxying. HTTP API does not.", "This is about as cheap as it gets for a telemetry pipeline:", "| Component | Cost | |-----------|------| | API Gateway | ~$3.50 / million requests | | CloudWatch Metrics | [Standard CW pricing](https://aws.amazon.com/cloudwatch/pricing/) (free during OTel metrics preview) | | CloudWatch Logs | [Standard CW pricing](https://aws.amazon.com/cloudwatch/pricing/) | | Lambda | $0 (there is none) |", "No idle cost. No provisioned capacity. Pure pay-per-request.", "For comparison: a Lambda-based OTLP forwarder would add ~$0.20/million invocations plus compute time, but gives you retry logic and transformation capabilities. At typical coding agent volumes (a few hundred requests/day per developer), the cost difference is negligible — the real win is operational simplicity.", "This proxy is optimized for simplicity. It's the right choice for low-to-moderate telemetry volumes from coding agents and developer tools. But it has tradeoffs:", "| Approach | Complexity | Cost | Retries | Multi-destination | Transformation | |----------|-----------|------|---------|-------------------|----------------| | This proxy (APIGW) | Minimal | ~$3.50/M req | ❌ | ❌ | ❌ | | OTel Collector | Medium | Compute cost | ✅ | ✅ | ✅ | | Lambda forwarder | Medium | ~$0.20/M + compute | ✅ | ✅ | ✅ | | ADOT SDK (in-app) | Low | Free (SigV4 native) | ✅ | ❌ | ❌ | | SaaS (Datadog, etc.) | Low | $$$ | ✅ | N/A | ✅ |", "Consider an OTel Collector or Lambda forwarder instead if you need:", "- **High throughput** — thousands of requests/second from many sources - **Retry and buffering** — this proxy is fire-and-forget; if CloudWatch returns an error, the data is lost. OTEL SDKs have built-in retry, but only for transient failures - **Multi-destination routing** — fan out to CloudWatch + Datadog + S3 simultaneously - **Payload transformation** — filter, enrich, or redact telemetry before ingestion - **Compliance requirements** — audit trails, guaranteed delivery, or data residency controls", "For most coding agent monitoring use cases (a team of 5-50 developers), this proxy handles the volume comfortably.", "The proxy uses API key authentication — simple but not the strongest option. Here's how to harden it:", "**Attach AWS WAF to the REST API.** Add rate limiting, IP allowlisting, or geo-blocking to prevent abuse. A single WAF WebACL with a rate-based rule (e.g., 1000 req/5min per IP) costs ~$6/month and stops most abuse patterns.", "**Rotate API keys.** APIGW supports multiple API keys per usage plan. Create a new key, distribute it, then disable the old one — zero downtime rotation.", "**Consider IAM auth for internal use.** If your tools run inside AWS (EC2, ECS, Lambda), switch `AuthorizationType` from `NONE` to `AWS_IAM` and drop the API key entirely. The caller signs requests with SigV4 using their IAM role — no shared secrets. This doesn't work for external tools like Claude Code on developer laptops, but it's ideal for CI/CD pipelines or server-side agents.", "**Egress control.** If you're running coding agents in a controlled environment, restrict outbound traffic to only your APIGW endpoint. This prevents telemetry from leaking to unauthorized collectors.", "This proxy works with **any OTEL SDK** that supports OTLP/HTTP. If your tool can set `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS`, it can ship telemetry to CloudWatch through this proxy.", "Potential use cases: - **AI coding agents** (Claude Code, CoWork, Copilot, Cursor, Gemini CLI) — track token usage, costs, and tool calls across your org - **Internal tools** — ship metrics without embedding AWS credentials in client apps - **CI/CD pipelines** — export build/test telemetry to CloudWatch - **On-premises services** — send OTLP from outside AWS without running ADOT Collector", "For apps running inside AWS with IAM roles available, consider the [ADOT SDK](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLP-UsingADOT.html) for collector-less telemetry with native SigV4 signing — no proxy needed.", "Source Code & One-Click Deploy", "The CloudFormation template and full documentation are on GitHub:", "👉 [gabrielkoo/otlp-cloudwatch-proxy](https://github.com/gabrielkoo/otlp-cloudwatch-proxy)", "One-click deploy to supported regions:", "| Region | Deploy | |--------|--------| | US East (N. Virginia) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) | | US West (Oregon) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) | | Asia Pacific (Singapore) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml&stackName=otlp-cloudwatch-proxy) | | Asia Pacific (Sydney) | [![Launch Stack](/assets/img/4bf452529163.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://raw.githubusercontent.com/gabrielkoo/otlp-cloudwatch-proxy/main/template.yaml", "*Built and validated on a Saturday morning with Claude Code + OpenClaw. Zero Lambda functions were harmed in the making of this article.*", "- **[AWS Guidance for Claude Code with Amazon Bedrock — Monitoring](https://github.com/aws-solutions-library-samples/guidance-for-claude-code-with-amazon-bedrock/blob/main/assets/docs/MONITORING.md)** — A comprehensive (and admittedly overkill) reference implementation using ECS Fargate + ALB + ADOT Collector + Lambda + DynamoDB + Kinesis + Athena. Great if you want to see the full spectrum of what can be measured: per-user token tracking, quota monitoring, cost dashboards, and an analytics data lake. If you need all of that, use it. If you just need telemetry flowing to CloudWatch, the one-template proxy in this post will do. - **[Claude Code Monitoring Docs](https://code.claude.com/docs/en/monitoring-usage)** — Official OTEL configuration reference, including all metrics, events, and traces. - **[Claude Code Managed Settings](https://code.claude.com/docs/en/settings#settings-files)** — How to deploy `managed-settings.json` via MDM for org-wide OTEL enforcement. - **[CloudWatch OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html)** — AWS docs on native OTLP ingestion for metrics, traces, and logs." ] }, { "id": "article:bedrock-for-ai-coding-tools-mantle-vs-gateway-vs-litellm-a-decision-guide-for-aws-credit-burners-1h01", "source_type": "article", "title": "Bedrock for AI Coding Tools: Mantle vs Gateway vs LiteLLM — A Decision Guide for AWS Credit Burners", "url": "https://gabrielkoo.com/blog/bedrock-for-ai-coding-tools-mantle-vs-gateway-vs-litellm-a-decision-guide-for-aws-credit-burners-1h01/", "canonical_url": "https://dev.to/aws-builders/bedrock-for-ai-coding-tools-mantle-vs-gateway-vs-litellm-a-decision-guide-for-aws-credit-burners-1h01", "published_at": "2026-03-22", "last_verified_at": "2026-08-23", "tags": [ "aws", "bedrock", "ai", "openai" ], "description": "You have AWS credits. You want to use them on AI coding tools — OpenCode, Codex CLI, Claude Code,...", "content": "You have AWS credits. You want to use them on AI coding tools — OpenCode, Codex CLI, Claude Code, whatever. Amazon Bedrock has the models. But how do you actually connect them?\n\nThere are three approaches, and picking the wrong one wastes time. Here's the decision guide I wish I had.\n\n> All data in this post is as of March 2026. Model counts and API support may change — check [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io) for the latest.\n\n## TL;DR\n\n- **Just want it to work?** Mantle + OpenCode. Five minutes, zero infra.\n- **Need Claude models via OpenAI API?** bedrock-access-gateway on Lambda.\n- **Need Claude Code specifically?** LiteLLM. It's the only path.\n- **Codex CLI?** Broken with all three. Wait for LiteLLM to fix a tool translation bug.\n\n## The three paths\n\n![Decision flowchart: Mantle vs bedrock-access-gateway vs LiteLLM](/assets/img/5ec03a820bf7.png)\n\n**One thing all three have in common: your API keys and code context stay within your AWS account or your own infrastructure.** Third-party AI gateways exist (Bifrost, Portkey, etc.), but they require routing your Bedrock API keys and code context through someone else's servers. Self-hosted or AWS-native — that's the baseline.\n\n### 1. Bedrock Mantle — no self-hosted infra required\n\n[Mantle](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) is AWS's native OpenAI-compatible endpoint. No Lambda, no container, no proxy — just set your base URL and API key:\n\n```bash\nexport OPENAI_BASE_URL=\"https://bedrock-mantle.us-east-1.api.aws/v1\"\nexport OPENAI_API_KEY=\"your-bedrock-api-key\"\n```\n\n**What's on Mantle:** 38 open-weight models — DeepSeek, Mistral, Qwen, GLM, NVIDIA Nemotron, MiniMax, Moonshot Kimi, Google Gemma, OpenAI gpt-oss, and Writer Palmyra.\n\n**What's NOT on Mantle:** Anthropic Claude, Amazon Nova, Meta Llama, AI21, Cohere — the proprietary/first-party models are absent.\n\n**API coverage:** Mantle exposes Chat Completions (`/v1/chat/completions`) and Responses API (`/v1/responses`). No Anthropic Messages API (`/v1/messages`).\n\nThe Responses API is limited — only 4 models support it: `openai.gpt-oss-120b-1:0`, `openai.gpt-oss-20b-1:0`, `openai.gpt-oss-120b`, and `openai.gpt-oss-20b`. Every other model is Chat Completions only. I verified this by scraping all 102 model card pages in the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).\n\n**Cost:** Standard Bedrock on-demand pricing. No gateway markup, no infra costs.\n\n**Best for:** OpenCode or any tool that speaks OpenAI Chat Completions.\n\n### 2. bedrock-access-gateway — self-hosted, all models\n\n[bedrock-access-gateway](https://github.com/aws-samples/bedrock-access-gateway) (or my fork, [bedrock-access-gateway-function-url](https://github.com/gabrielkoo/bedrock-access-gateway-function-url)) gives you an OpenAI-compatible proxy backed by all Bedrock models — including Claude, Nova, and Llama.\n\nDeploy it as a Lambda Function URL or on ECS, and you get:\n\n```bash\nexport OPENAI_BASE_URL=\"https://your-lambda-url.lambda-url.us-west-2.on.aws/api/v1\"\nexport OPENAI_API_KEY=\"your-gateway-api-key\"\n```\n\nThe tradeoff: you maintain infrastructure. But you get access to every Bedrock model through a single OpenAI-compatible endpoint.\n\n**Cost:** Bedrock on-demand pricing + Lambda/ECS compute costs (minimal for Lambda Function URLs — you pay per invocation).\n\n**Best for:** When you need Claude or Nova through OpenAI-compatible tools, or want full control over routing, caching, and logging.\n\n### 3. LiteLLM — the universal translator\n\n[LiteLLM](https://github.com/BerriAI/litellm) is the Swiss Army knife. It translates between API schemas — OpenAI, Anthropic, Bedrock native, and more. It's the only option that gives you Anthropic Messages API (`/v1/messages`) compatibility with Bedrock models.\n\nThis matters because **Claude Code uses the Anthropic API schema**, not OpenAI's. If you want to run Claude Code against Bedrock, LiteLLM is your best (and arguably only) option. I tested this end-to-end: Claude Code CLI → LiteLLM → Bedrock Converse API — it works, including streaming responses.\n\nIs it perfect? No. Setup is more complex (Python process or Docker container, optional PostgreSQL for analytics), and you're adding another layer of abstraction. But it's the most flexible gateway available.\n\n**Cost:** Bedrock on-demand pricing + your compute costs for hosting LiteLLM. No per-call markup from LiteLLM itself (open source).\n\n**Best for:** Claude Code, or when you need both OpenAI and Anthropic API compatibility from a single proxy.\n\n## Tool compatibility matrix\n\n| Tool | API Schema | Mantle | bedrock-access-gateway | LiteLLM |\n|------|-----------|--------|----------------------|---------|\n| OpenCode | OpenAI Chat | ✅ | ✅ | ✅ |\n| Codex CLI | OpenAI Responses | ❌ Auth issues | ❌ No Responses API | ⚠️ Tool bug |\n| Claude Code | Anthropic Messages | ❌ No support | ❌ Wrong schema | ✅ |\n\nNote: Anthropic-native tools like Kiro CLI also work through LiteLLM's Anthropic Messages API translation.\n\n### A note on Codex CLI\n\nCodex CLI requires the Responses API (`/v1/responses`), which limits your options:\n\n- **Mantle:** Only the 4 OpenAI gpt-oss models support Responses API. Even with those, I hit 401 auth errors (Bearer token not passed correctly through Codex's HTTPS transport) and tool type rejections (`web_search` type not supported — only `function` and `mcp`).\n- **bedrock-access-gateway:** No Responses API at all — `/v1/responses` returns 404. The gateway only implements Chat Completions.\n- **LiteLLM:** Supports Responses API ([v1.66.3+](https://github.com/BerriAI/litellm/releases)) and has an [official Codex CLI tutorial](https://docs.litellm.ai/docs/tutorials/openai_codex). However, as of v1.82.5, there's a tool translation bug: Codex CLI sends built-in tool types that LiteLLM converts to Bedrock Converse format with empty `toolSpec.name` fields, causing Bedrock validation errors. The Responses API itself works fine when tested with standard function tools. This should be fixable on the LiteLLM side.\n\n## Quick setup: OpenCode + Mantle\n\nIf you just want to burn AWS credits on a coding CLI today, here's the fastest path. [OpenCode](https://opencode.ai) (v1.2.27+) works with Mantle out of the box:\n\n```json\n{\n \"$schema\": \"https://opencode.ai/config.json\",\n \"provider\": {\n \"bedrock-mantle\": {\n \"npm\": \"@ai-sdk/openai-compatible\",\n \"name\": \"Bedrock Mantle\",\n \"options\": {\n \"baseURL\": \"https://bedrock-mantle.us-east-1.api.aws/v1\",\n \"apiKey\": \"{env:BEDROCK_API_KEY}\"\n },\n \"models\": {\n \"openai.gpt-oss-120b\": { \"name\": \"GPT OSS 120B\" },\n \"zai.glm-5\": { \"name\": \"GLM 5 (744B/40B MoE)\" },\n \"qwen.qwen3-coder-480b-a35b-instruct\": { \"name\": \"Qwen3 Coder 480B\" },\n \"deepseek.v3.2\": { \"name\": \"DeepSeek V3.2\" },\n \"mistral.mistral-large-3-675b-instruct\": { \"name\": \"Mistral Large 3\" }\n }\n }\n },\n \"model\": \"bedrock-mantle/openai.gpt-oss-120b\"\n}\n```\n\nSave to `~/.config/opencode/opencode.json`, set `BEDROCK_API_KEY`, and you're coding.\n\n## The bottom line\n\n| | Mantle | bedrock-access-gateway | LiteLLM |\n|---|--------|----------------------|---------|\n| Infra to maintain | None | Lambda/ECS | Container/process |\n| Models available | 38 (open-weight) | All Bedrock | All Bedrock |\n| OpenAI Chat API | ✅ | ✅ | ✅ |\n| OpenAI Responses API | ⚠️ gpt-oss only | ❌ | ✅ |\n| Anthropic Messages API | ❌ | ❌ | ✅ |\n| OpenCode | ✅ | ✅ | ✅ |\n| Codex CLI | ❌ | ❌ | ⚠️ Tool bug |\n| Claude Code | ❌ | ❌ | ✅ |\n| Extra cost | None | ~$0 (Lambda) | Your compute |\n| Setup time | 5 min | 30 min | 1 hr |\n\n## Track available Mantle models\n\nI maintain [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io) — a catalog of every Bedrock model with API support badges and endpoint support (Mantle vs Runtime), scraped from the AWS documentation.\n\n---\n\n*Burning AWS credits on something interesting? I'd love to hear what tools and models you're using — drop a comment.*\n comment.*", "excerpts": [ "You have AWS credits. You want to use them on AI coding tools — OpenCode, Codex CLI, Claude Code, whatever. Amazon Bedrock has the models. But how do you actually connect them?", "There are three approaches, and picking the wrong one wastes time. Here's the decision guide I wish I had.", "> All data in this post is as of March 2026. Model counts and API support may change — check [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io) for the latest.", "- **Just want it to work?** Mantle + OpenCode. Five minutes, zero infra. - **Need Claude models via OpenAI API?** bedrock-access-gateway on Lambda. - **Need Claude Code specifically?** LiteLLM. It's the only path. - **Codex CLI?** Broken with all three. Wait for LiteLLM to fix a tool translation bug.", "![Decision flowchart: Mantle vs bedrock-access-gateway vs LiteLLM](/assets/img/5ec03a820bf7.png)", "**One thing all three have in common: your API keys and code context stay within your AWS account or your own infrastructure.** Third-party AI gateways exist (Bifrost, Portkey, etc.), but they require routing your Bedrock API keys and code context through someone else's servers. Self-hosted or AWS-native — that's the baseline.", "1. Bedrock Mantle — no self-hosted infra required", "[Mantle](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) is AWS's native OpenAI-compatible endpoint. No Lambda, no container, no proxy — just set your base URL and API key:", "**What's on Mantle:** 38 open-weight models — DeepSeek, Mistral, Qwen, GLM, NVIDIA Nemotron, MiniMax, Moonshot Kimi, Google Gemma, OpenAI gpt-oss, and Writer Palmyra.", "**What's NOT on Mantle:** Anthropic Claude, Amazon Nova, Meta Llama, AI21, Cohere — the proprietary/first-party models are absent.", "**API coverage:** Mantle exposes Chat Completions (`/v1/chat/completions`) and Responses API (`/v1/responses`). No Anthropic Messages API (`/v1/messages`).", "The Responses API is limited — only 4 models support it: `openai.gpt-oss-120b-1:0`, `openai.gpt-oss-20b-1:0`, `openai.gpt-oss-120b`, and `openai.gpt-oss-20b`. Every other model is Chat Completions only. I verified this by scraping all 102 model card pages in the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).", "**Cost:** Standard Bedrock on-demand pricing. No gateway markup, no infra costs.", "**Best for:** OpenCode or any tool that speaks OpenAI Chat Completions.", "2. bedrock-access-gateway — self-hosted, all models", "[bedrock-access-gateway](https://github.com/aws-samples/bedrock-access-gateway) (or my fork, [bedrock-access-gateway-function-url](https://github.com/gabrielkoo/bedrock-access-gateway-function-url)) gives you an OpenAI-compatible proxy backed by all Bedrock models — including Claude, Nova, and Llama.", "Deploy it as a Lambda Function URL or on ECS, and you get:", "The tradeoff: you maintain infrastructure. But you get access to every Bedrock model through a single OpenAI-compatible endpoint.", "**Cost:** Bedrock on-demand pricing + Lambda/ECS compute costs (minimal for Lambda Function URLs — you pay per invocation).", "**Best for:** When you need Claude or Nova through OpenAI-compatible tools, or want full control over routing, caching, and logging.", "3. LiteLLM — the universal translator", "[LiteLLM](https://github.com/BerriAI/litellm) is the Swiss Army knife. It translates between API schemas — OpenAI, Anthropic, Bedrock native, and more. It's the only option that gives you Anthropic Messages API (`/v1/messages`) compatibility with Bedrock models.", "This matters because **Claude Code uses the Anthropic API schema**, not OpenAI's. If you want to run Claude Code against Bedrock, LiteLLM is your best (and arguably only) option. I tested this end-to-end: Claude Code CLI → LiteLLM → Bedrock Converse API — it works, including streaming responses.", "Is it perfect? No. Setup is more complex (Python process or Docker container, optional PostgreSQL for analytics), and you're adding another layer of abstraction. But it's the most flexible gateway available.", "**Cost:** Bedrock on-demand pricing + your compute costs for hosting LiteLLM. No per-call markup from LiteLLM itself (open source).", "**Best for:** Claude Code, or when you need both OpenAI and Anthropic API compatibility from a single proxy.", "| Tool | API Schema | Mantle | bedrock-access-gateway | LiteLLM | |------|-----------|--------|----------------------|---------| | OpenCode | OpenAI Chat | ✅ | ✅ | ✅ | | Codex CLI | OpenAI Responses | ❌ Auth issues | ❌ No Responses API | ⚠️ Tool bug | | Claude Code | Anthropic Messages | ❌ No support | ❌ Wrong schema | ✅ |", "Note: Anthropic-native tools like Kiro CLI also work through LiteLLM's Anthropic Messages API translation.", "Codex CLI requires the Responses API (`/v1/responses`), which limits your options:", "- **Mantle:** Only the 4 OpenAI gpt-oss models support Responses API. Even with those, I hit 401 auth errors (Bearer token not passed correctly through Codex's HTTPS transport) and tool type rejections (`web_search` type not supported — only `function` and `mcp`). - **bedrock-access-gateway:** No Responses API at all — `/v1/responses` returns 404. The gateway only implements Chat Completions. - **LiteLLM:** Supports Responses API ([v1.66.3+](https://github.com/BerriAI/litellm/releases)) and has an [official Codex CLI tutorial](https://docs.litellm.ai/docs/tutorials/openai_codex). However, as of v1.82.5, there's a tool translation bug: Codex CLI sends built-in tool types that LiteLLM converts to Bedrock Converse format with empty `toolSpec.name` fields, causing Bedrock validation errors. The Responses API itself works fine when tested with standard function tools. This should be fixable on the LiteLLM side.", "Quick setup: OpenCode + Mantle", "If you just want to burn AWS credits on a coding CLI today, here's the fastest path. [OpenCode](https://opencode.ai) (v1.2.27+) works with Mantle out of the box:", "Save to `~/.config/opencode/opencode.json`, set `BEDROCK_API_KEY`, and you're coding.", "| | Mantle | bedrock-access-gateway | LiteLLM | |---|--------|----------------------|---------| | Infra to maintain | None | Lambda/ECS | Container/process | | Models available | 38 (open-weight) | All Bedrock | All Bedrock | | OpenAI Chat API | ✅ | ✅ | ✅ | | OpenAI Responses API | ⚠️ gpt-oss only | ❌ | ✅ | | Anthropic Messages API | ❌ | ❌ | ✅ | | OpenCode | ✅ | ✅ | ✅ | | Codex CLI | ❌ | ❌ | ⚠️ Tool bug | | Claude Code | ❌ | ❌ | ✅ | | Extra cost | None | ~$0 (Lambda) | Your compute | | Setup time | 5 min | 30 min | 1 hr |", "I maintain [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io) — a catalog of every Bedrock model with API support badges and endpoint support (Mantle vs Runtime), scraped from the AWS documentation.", "*Burning AWS credits on something interesting? I'd love to hear what tools and models you're using — drop a comment.* comment.*" ] }, { "id": "article:from-3-minute-cold-starts-to-20-seconds-whisper-on-aws-lambda-efs-for-openclaw-9c5", "source_type": "article", "title": "From 3-Minute Cold Starts to ~20 Seconds: Whisper on AWS Lambda + EFS for OpenClaw", "url": "https://gabrielkoo.com/blog/from-3-minute-cold-starts-to-20-seconds-whisper-on-aws-lambda-efs-for-openclaw-9c5/", "canonical_url": "https://dev.to/aws-builders/from-3-minute-cold-starts-to-20-seconds-whisper-on-aws-lambda-efs-for-openclaw-9c5", "published_at": "2026-03-13", "last_verified_at": "2026-08-23", "tags": [ "whisper", "openclaw", "aws", "efs" ], "description": "Part 3 of my series on building a low-cost personal AI stack on AWS. Part 1 — Squeezing my $1k/month...", "content": "*Part 3 of my series on building a low-cost personal AI stack on AWS.* \n*[Part 1 — Squeezing my $1k/month API bill to $20/month with AWS Credits](https://dev.to/aws-builders/i-squeezed-my-1k-monthly-openclaw-api-bill-with-20month-in-aws-credits-heres-the-exact-setup-3gj4)*\n*[Part 2 — Drop-in Perplexity Sonar replacement with AWS Bedrock Nova Grounding](https://dev.to/aws-builders/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9)*\n\n---\n\n## TL;DR\n\nI built a self-hosted speech-to-text API on AWS Lambda using [faster-whisper](https://github.com/SYSTRAN/faster-whisper). After trying Amazon Transcribe, SageMaker Serverless, and Lambda with a bundled model, I landed on a **Lambda + EFS + S3** architecture that achieves ~20-30 second cold starts (once the model is cached on EFS) for ~$0.21/month in storage costs. Once warm, specifying the language drops response time to ~10s.\n\nOpen source: [gabrielkoo/aws-lambda-whisper-adaptor](https://github.com/gabrielkoo/aws-lambda-whisper-adaptor)\n\n---\n\n## The Problem\n\nI wanted to automatically transcribe Telegram voice messages. The requirements were simple:\n\n- **Accuracy**: Good enough for Cantonese\n- **Cost**: Pay-per-use, scales to zero when idle\n- **Latency**: Cold start under 60 seconds\n\nThere's a fourth constraint that's easy to overlook outside Hong Kong: **most managed STT APIs simply aren't available here**. OpenAI's Whisper API falls under their notorious China's regional restriction. Google's Gemini models are available and actually competitive on both accuracy and price — Gemini 3 Flash achieves 3.1% WER at ~$1.92/1000 minutes ([Artificial Analysis STT leaderboard](https://artificialanalysis.ai/speech-to-text)), cheaper than OpenAI's Whisper API and competitive with Lambda at low volume. The real reason I went with Lambda: AWS Credits from the Community Builder program (same theme as the rest of this series) make it effectively free.\n\nSimple enough. Except it took four attempts to get there.\n\n---\n\n## What I Tried (and Why It Didn't Work)\n\n### Option 1: Amazon Transcribe\n\nThe obvious first choice — fully managed, pay-per-use, native AWS integration.\n\n**Why I rejected it before even trying:**\n\nAmazon Transcribe [supports `zh-CN` and `zh-TW`, but not `yue` (Cantonese)](https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html). Whisper large-v3-turbo handles Cantonese significantly better, and accuracy matters more than convenience here.\n\n---\n\n### Option 2: SageMaker Serverless Inference\n\nSageMaker Serverless scales to zero and handles model serving — sounds perfect.\n\n**What happened:**\n\nI deployed a SageMaker Serverless endpoint with faster-whisper. The first invocation after idle:\n\n- Container provisioning: ~30s\n- Model loading: ~45-60s\n- **Total cold start: 60-90 seconds**\n\nFor a voice message that's 5-10 seconds long, waiting 90 seconds is a terrible experience.\n\n**The 6GB memory wall:**\n\nSageMaker Serverless [maxes out at 6144 MB (6 GB) RAM](https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html). Here's why that's a problem for Whisper:\n\n- [`whisper-large-v3-turbo` (INT8)](https://huggingface.co/Zoont/faster-whisper-large-v3-turbo-int8-ct2): ~780MB model + ~2GB Python/runtime overhead ≈ 2.8GB minimum\n- [`whisper-large-v3` (FP16)](https://huggingface.co/Systran/faster-whisper-large-v3): ~3GB model alone — barely fits, zero headroom for audio processing\n- Any concurrent requests? You're OOM.\n\n[Lambda goes up to 10,240 MB](https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html). That headroom matters.\n\n**Cost comparison:**\n\n[SageMaker Serverless bills per GB-second](https://aws.amazon.com/sagemaker/pricing/) of inference time. For sporadic voice message transcription (~10s per request, a few times a day), Lambda's per-invocation pricing is significantly cheaper. My Lambda setup costs ~$0.21/month in storage — the compute is essentially free at this volume.\n\nI deleted the endpoint after testing.\n\n---\n\n### Option 2b: Bedrock Marketplace\n\nAWS Bedrock Marketplace [does list Whisper Large V3 Turbo](https://aws.amazon.com/blogs/machine-learning/build-a-serverless-audio-summarization-solution-with-amazon-bedrock-and-whisper/) — but it deploys on a **dedicated endpoint instance**. Auto-scaling is available (including scale-to-zero), but that creates a different problem:\n\n- **Keep minimum 1 instance**: always paying for idle time, even at 3am\n- **Scale to zero**: cold starts when traffic resumes — SageMaker cold starts are measured in **minutes**, not seconds\n- Not token/usage-based pricing either way\n\nFor a Telegram bot that gets a few voice messages a day, you're either burning money on idle instances or waiting minutes for the first message to transcribe. Lambda's ~20-30s cold start looks great by comparison.\n\n---\n\n### Option 3: Lambda with Bundled Model\n\nNext idea: bundle the model directly into the Docker image. No external dependencies, simple architecture.\n\n**What happened:**\n\n```dockerfile\n# Download model during build\n# Note: using openai/whisper-large-v3-turbo converted to int8 via sync-model workflow\nRUN python -c \"from faster_whisper import WhisperModel; WhisperModel('openai/whisper-large-v3-turbo')\"\n```\n\n- Docker image size: **~10GB**\n- ECR push time: **5+ minutes**\n- Lambda cold start: **2 minutes 51 seconds**\n\nThe cold start is dominated by Lambda pulling the 10GB image from ECR. AWS Lambda caches images, but any cold start after the cache expires hits this wall.\n\n**Why it didn't work:**\n\n- 3-minute cold start is unusable for interactive transcription\n- Every code change requires rebuilding and pushing a 10GB image\n- ECR storage: ~$1/month just for the image\n\n---\n\n### Option 4: Lambda + S3 (No EFS)\n\nWhat if Lambda downloads the model from S3 on cold start, storing it in `/tmp`?\n\n**The problem:**\n\nLambda's `/tmp` is ephemeral. Every cold start re-downloads the model from S3:\n\n- S3 download for 1.6GB FP16 model: **30-60 seconds**\n- S3 download for 780MB INT8 model: **15-30 seconds**\n\nThis is better than the bundled model approach, but there's a bigger issue: **no caching between Lambda instances**. If you have 3 concurrent invocations, all 3 download the model independently. You're paying for S3 transfer on every cold start.\n\n**What about Lambda SnapStart or Durable Functions?**\n\nAWS added two relevant features since this was written:\n\n- **SnapStart for Python** (Nov 2024): snapshots the initialized execution environment — sounds perfect for caching a loaded model. The catch: [SnapStart doesn't support container images](https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html). This adaptor is container-based, so it's off the table.\n\n- **[Lambda Durable Functions](https://aws.amazon.com/about-aws/whats-new/2025/12/lambda-durable-multi-step-applications-ai-workflows/)** (re:Invent 2025): enables multi-step workflows with automatic checkpointing, pause/resume for up to one year, and failure recovery. This is workflow orchestration (think Azure Durable Functions) — useful for multi-step AI pipelines, but not for persisting a 780MB model binary between cold starts.\n\nEFS remains the right solution for model caching.\n\n---\n\n## What Actually Worked: Lambda + EFS + S3\n\nThe solution: use **EFS as a persistent model cache**, bootstrapped from S3. I've used EFS for [persistent Streamlit state on ECS](https://dev.to/aws-builders/scale-a-stateful-streamlit-chatbot-with-aws-ecs-and-efs-48gm) before — same pattern, different compute layer.\n\n```plaintext\nRequest → Lambda Function URL\n ↓\n Lambda (VPC)\n ↓ first cold start only: S3 → EFS\n EFS (model cached here permanently)\n```\n\n![Logic Flow](/assets/img/57f530f3907a.png)\n\n\n\n\n**How it works:**\n\n1. **First cold start** (once per model): Lambda checks for a marker file on EFS. If missing, downloads model from S3 to EFS (~55s for INT8). Writes marker file. Then loads model into RAM.\n2. **Subsequent cold starts** (new container, model already on EFS): Marker file exists → load model from EFS into RAM (~20-30s for INT8).\n3. **Warm invocations** (same container reused): Model already in memory → transcription-only time (~10-22s depending on audio length and whether language is specified).\n\n```python\nHF_MODEL_REPO = os.environ.get('HF_MODEL_REPO', 'openai/whisper-large-v3-turbo')\nMODEL_SLUG = HF_MODEL_REPO.replace('/', '--')\nEFS_MODEL_DIR = f'/mnt/whisper-models/{MODEL_SLUG}'\nMODEL_MARKER = f'/mnt/whisper-models/.ready-{MODEL_SLUG}'\n\ndef bootstrap_model():\n if os.path.exists(MODEL_MARKER):\n return WhisperModel(EFS_MODEL_DIR, device='cpu', compute_type='int8')\n \n # First run: sync model from S3 to EFS\n s3 = boto3.client('s3')\n prefix = f'models/{MODEL_SLUG}/'\n os.makedirs(EFS_MODEL_DIR, exist_ok=True)\n \n paginator = s3.get_paginator('list_objects_v2')\n for page in paginator.paginate(Bucket=os.environ['MODEL_S3_BUCKET'], Prefix=prefix):\n for obj in page.get('Contents', []):\n key = obj['Key']\n local_path = os.path.join(EFS_MODEL_DIR, key[len(prefix):])\n os.makedirs(os.path.dirname(local_path), exist_ok=True)\n s3.download_file(os.environ['MODEL_S3_BUCKET'], key, local_path)\n \n open(MODEL_MARKER, 'w').close() # Mark as ready\n return WhisperModel(EFS_MODEL_DIR, device='cpu', compute_type='int8')\n\nMODEL = bootstrap_model() # Runs at Lambda init time, cached for warm invocations\n```\n\n**Why EFS works:**\n\n- EFS persists across Lambda instances — model is downloaded **once**, reused forever\n- EFS is mounted at `/mnt/whisper-models` — Lambda reads it like a local filesystem\n- **S3 VPC Gateway Endpoint is free** — no NAT Gateway needed (saves ~$32/month)\n- **Zero internet egress** — Lambda → S3 via VPC Gateway Endpoint, Lambda → EFS within VPC. The Lambda function never reaches the internet. This is a meaningful security benefit when using third-party models from HuggingFace — model weights never leave the AWS network once synced to S3.\n- EFS storage: ~$0.19/month for the 780MB INT8 model\n\n> 🔒 **Security note:** The Lambda runs in a VPC with **no internet access** — no NAT Gateway, no public subnet. It can only reach EFS (VPC-internal) and S3 (via the free VPC Gateway Endpoint). This means even if you're using a third-party HuggingFace model, the model weights and your audio data never leave the AWS network. No data exfiltration risk, no outbound calls to unknown endpoints.\n\n---\n\n## INT8 vs FP16: The Model Size Trade-off\n\nThe `openai/whisper-large-v3-turbo` model on HuggingFace needs conversion to CTranslate2 format. The `sync-model` workflow handles this, converting to INT8 and fixing the `num_mel_bins` config. Alternatively, use [`Zoont/faster-whisper-large-v3-turbo-int8-ct2`](https://huggingface.co/Zoont/faster-whisper-large-v3-turbo-int8-ct2) — a pre-converted CTranslate2 INT8 model that works out of the box with `quantization=none`:\n\n| Model | Size (EFS) | First Bootstrap | EFS Cold Start | Warm (2.5s audio) | Memory |\n|-------|-----------|-----------------|----------------|-------------------|--------|\n| `Zoont/faster-whisper-large-v3-turbo-int8-ct2` | ~780MB | ~55s | **~22s** ✅ | **~10s** ✅ | ~2.8GB |\n| `openai/whisper-large-v3-turbo` (INT8, via sync-model) | ~780MB | ~55s | ~22s | ~10s | ~2.8GB |\n| `openai/whisper-large-v3-turbo` (FP16) | ~1.5GB | ~126s | ~40s | ~15s | ~4GB |\n| `Systran/faster-whisper-large-v3` (FP16, loaded as int8) | ~1.6GB | ~54s | ~30s | ~13s | 6GB |\n\n**Recommended:** `Zoont/faster-whisper-large-v3-turbo-int8-ct2` — no conversion step needed, identical performance to the openai model converted to INT8. Use `quantization=none` in the sync-model workflow since it's already in CTranslate2 format.\n\n---\n\n## Cost Breakdown\n\n| Resource | Monthly Cost |\n|----------|-------------|\n| EFS storage (780MB INT8) | ~$0.19 |\n| S3 storage (780MB) | ~$0.02 |\n| Lambda compute | ~$0.00167/warm invocation* |\n| S3 VPC Gateway Endpoint | **Free** |\n| NAT Gateway | **Not needed ($0)** |\n| **Total (storage only)** | **~$0.21/month** |\n\n*10GB × 10s = 100 GB-seconds per warm invocation. The [Lambda free tier](https://aws.amazon.com/lambda/pricing/) covers **400,000 GB-seconds/month** — roughly 4,000 warm invocations. For a personal bot, compute cost is effectively **$0**. Storage dominates.\n\nCompare to SageMaker Serverless: minimum ~$5-10/month for similar workloads, plus the 60-90s cold start penalty.\n\n> **Why not Provisioned Concurrency?** PC keeps Lambda permanently warm (no cold starts), but costs ~$0.0000097222/GB-second. For a 10GB function running 24/7: ~$252/month. Even a minimal 4GB setup runs ~$100/month — roughly 500x more than the $0.21 storage approach. For a personal bot with a few voice messages a day, the occasional ~60s cold start is a fine trade-off.\n\n### vs. OpenAI Whisper API\n\nOpenAI's Whisper API costs [**$0.006/minute**](https://openai.com/api/pricing/). Here's how it compares for a bot averaging 15s voice messages:\n\n| Volume | OpenAI Whisper API | Self-hosted Lambda |\n|--------|-------------------|-------------------|\n| 50 msgs/month | $0.08 | $0.21 (storage only) |\n| 140 msgs/month | $0.21 | **$0.21** ← break-even |\n| 500 msgs/month | $0.75 | $0.21 (storage only) |\n| 1,000 msgs/month | $1.50 | $0.21 (storage only) |\n| 4,000 msgs/month | $6.00 | $0.21 (storage only) |\n\nLambda compute is free within the free tier (~4,000 warm invocations/month). Beyond that, it's $0.00167/invocation — but that's a high volume for a personal bot.\n\nBreak-even: **~140 messages/month**. Above that, Lambda wins on cost.\n\nBut cost isn't the only reason to self-host:\n- **Geographic availability**: OpenAI's API is not available in Hong Kong — HK falls under China's regional restriction. Azure OpenAI does offer Whisper, but [only `whisper-1` (large-v2 based)](https://learn.microsoft.com/en-us/answers/questions/2237575/new-announced-speech-to-text-models-for-realtime) — large-v3 and large-v3-turbo are not available. If you're in HK (or other restricted regions), this approach isn't just cheaper — it's the only option for v3-quality transcription.\n- **Cantonese accuracy**: `language=yue` with Whisper large-v3-turbo is noticeably better than the managed API for Cantonese\n- **Privacy**: audio never leaves your infrastructure\n- **No rate limits**: Lambda scales independently\n\n---\n\n## Architecture\n\n```plaintext\nTelegram voice message\n ↓\n OpenClaw (gateway)\n ↓\nLambda Function URL (auth via token)\n ↓\nLambda (VPC, 10GB RAM, 900s timeout)\n ↓\nEFS /mnt/whisper-models/{model-slug}\n ↓\nfaster-whisper (CTranslate2, INT8)\n ↓\n Transcript\n```\n\n**Lambda configuration:**\n- Memory: 10,240 MB — actual usage is **~2.2GB** (INT8 model), but [Lambda allocates CPU proportional to memory](https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html). 10GB gives ~6 vCPUs vs ~2.3 vCPUs at 4GB, cutting warm transcription from ~16s to ~10s. You're paying for CPU, not RAM.\n- Timeout: 900s (handles long audio files)\n- VPC: Default VPC (no NAT Gateway)\n- EFS: Mounted at `/mnt/whisper-models`\n\n**Memory vs. cost trade-off (tested, 3 runs each):**\n\n| Config | Cold Start | Warm (2.5s audio) | GB-seconds/invocation |\n|--------|-----------|-------------------|----------------------|\n| 4,096 MB | ~30s | ~21s | 84 (~$0.00140) |\n| 6,144 MB | ~25s | ~16s | 96 (~$0.00160) |\n| 8,192 MB | ~24s | ~18s | 144 (~$0.00240) |\n| 10,240 MB | ~22s | ~15s | 150 (~$0.00250) |\n\nCold start is ~20-30s across all configs — it's EFS I/O bound, not CPU bound, so more memory doesn't help much here. Warm inference time does scale with memory (more vCPUs = faster CTranslate2 decoding). Interestingly, 4GB is the cheapest per invocation — the warm time savings at higher memory don't offset the extra GB-seconds. Within the free tier, cost differences are negligible regardless.\n\n---\n\n## API Compatibility\n\nThe adaptor exposes two endpoints so it works as a drop-in replacement for existing integrations:\n\n**OpenAI compatible** (`/v1/audio/transcriptions`):\n```bash\ncurl -X POST https:///v1/audio/transcriptions \\\n -H \"Authorization: Token \" \\\n -F \"file=@audio.ogg\" \\\n -F \"language=yue\"\n```\n```json\n{\"text\": \"transcript here\"}\n```\n\n**Deepgram compatible** (`/v1/listen`):\n```bash\ncurl -X POST https:///v1/listen?language=yue \\\n -H \"Authorization: Token \" \\\n -H \"Content-Type: audio/ogg\" \\\n --data-binary @audio.ogg\n```\n\n---\n\n## Model Management API\n\nOnce you've synced multiple models to EFS, there's no SSH access to see what's there or clean up. I added two non-standard endpoints:\n\n**List models on EFS:**\n```bash\ncurl https:///v1/models -H \"Authorization: Token \"\n```\n```json\n{\n \"object\": \"list\",\n \"data\": [\n {\"id\": \"openai/whisper-large-v3-turbo\", \"object\": \"model\", \"owned_by\": \"openai\"},\n {\"id\": \"Systran/faster-distil-whisper-large-v3\", \"object\": \"model\", \"owned_by\": \"Systran\"}\n ]\n}\n```\n\n**Delete a model from EFS** (the currently loaded model returns 409):\n```bash\ncurl -X DELETE https:///v1/models/Systran/faster-distil-whisper-large-v3 \\\n -H \"Authorization: Token \"\n```\n```json\n{\"id\": \"Systran/faster-distil-whisper-large-v3\", \"object\": \"model\", \"deleted\": true}\n```\n\nSlashes in model IDs work naturally — `rawPath` preserves the full path, so `DELETE /v1/models/openai/whisper-large-v3-turbo` correctly maps to model ID `openai/whisper-large-v3-turbo`.\n\n---\n\n## Performance Tip: Always Specify Language\n\nWhen no language is specified, Whisper runs language detection on the first audio chunk — adding noticeable overhead. For a 2.5s voice message:\n\n| Request | Response Time |\n|---------|--------------|\n| No language (auto-detect) | ~22s |\n| `language=yue` (Cantonese) | ~10s |\n\nThat's a **2x speedup** just from passing a language hint. Two ways to do it:\n\n**Option A — per-request query param** (recommended, keeps Lambda language-agnostic):\n```bash\n# Deepgram endpoint\ncurl -X POST https:///v1/listen?language=yue \\\n -H \"Authorization: Token \" \\\n -H \"Content-Type: audio/ogg\" \\\n --data-binary @audio.ogg\n\n# OpenAI endpoint\ncurl -X POST https:///v1/audio/transcriptions \\\n -H \"Authorization: Token \" \\\n -F \"file=@audio.ogg\" \\\n -F \"language=yue\"\n```\n\n**Option B — Lambda env var** (simpler if you only ever transcribe one language):\n```bash\nWHISPER_LANGUAGE=yue\n```\n\nI use Option A — the language is set in my OpenClaw config (`language: \"yue\"` in the audio model), which passes it as `?language=yue` to the Lambda on every request.\n\n### Real-time Factor\n\nOnce warm, the Lambda transcribes faster than real-time for typical voice messages:\n\n| Audio Duration | Warm Response Time | Real-time Factor |\n|---------------|-------------------|-----------------|\n| 2.5s | ~10s | 4x |\n| 33s | ~23s | **0.68x** ✅ faster than real-time |\n\nThe 2.5s result looks slow (4x), but Whisper processes audio in 30-second chunks — the overhead is fixed regardless of audio length. For longer messages, the real-time factor drops well below 1x.\n\n---\n\n## Open Source\n\nThe project is open source at [gabrielkoo/aws-lambda-whisper-adaptor](https://github.com/gabrielkoo/aws-lambda-whisper-adaptor).\n\nKey features:\n- Any [faster-whisper](https://huggingface.co/models?search=faster-whisper) model via `HF_MODEL_REPO` env var\n- GitHub Actions workflow to sync models from HuggingFace → S3 (`quantization=int8` for HF-format models, `quantization=none` for pre-converted CTranslate2 models)\n- `GET /v1/models` — list all models currently on EFS\n- `DELETE /v1/models/{owner}/{model}` — remove a model from EFS on demand\n- Pre-built Docker image: `ghcr.io/gabrielkoo/aws-lambda-whisper-adaptor:latest`\n- Configurable language detection via `WHISPER_LANGUAGE` env var or per-request parameter\n\n---\n\n## Pre-warming\n\n> **For OpenClaw voice prompts:** the ~20-30s cold start is often negligible in practice — if you're asking the agent to run a multi-step job, it'll take a few minutes anyway. Pre-warming only matters if you need the very first transcription to be fast.\n\nCold starts happen when Lambda hasn't been invoked recently. For predictable usage patterns (e.g. a morning standup bot), pre-warm the Lambda before you need it:\n\n```bash\n#!/bin/bash\n# prewarm.sh — trigger Lambda init before expected usage\ncurl -s -o /dev/null \\\n -X POST \"$WHISPER_LAMBDA_URL/v1/listen?language=yue\" \\\n -H \"Authorization: Token $WHISPER_API_SECRET\" \\\n -H \"Content-Type: audio/ogg\" \\\n --data-binary @sample.ogg\necho \"Lambda pre-warmed\"\n```\n\nSchedule with cron: `0 8 * * * /path/to/prewarm.sh` (runs at 8am daily).\n\nAlternatively, use an EventBridge rule to ping the Lambda every few minutes — though at that frequency, Provisioned Concurrency starts making more sense cost-wise.\n\n---\n\n## Conclusion\n\nThe Lambda + EFS + S3 architecture achieves:\n- **~20-30s cold start** (INT8 model on EFS); first-ever bootstrap from S3 takes ~55s (one-time only)\n- **~10s warm invocations** with `language=yue`\n- **~$0.21/month** storage cost\n- **Zero idle cost** (scales to zero)\n- **Deepgram and OpenAI compatible** APIs\n\nThe key insight: **EFS is the missing piece**. It provides persistent, fast storage that Lambda can access without a NAT Gateway (using the free S3 VPC Gateway Endpoint for bootstrapping).\n\nI couldn't find any existing write-up of Whisper on Lambda using EFS for persistent model caching — most approaches either bundle the model in Docker (3-minute cold starts) or re-download from S3 on every cold start (no caching between instances). If you've seen this done before, I'd love to know.\n\nTwo things worth knowing before you deploy:\n1. Use `Zoont/faster-whisper-large-v3-turbo-int8-ct2` with `quantization=none` in the sync-model workflow — it's pre-converted to CTranslate2 INT8 and works out of the box (the `openai/whisper-large-v3-turbo` model requires conversion and can hit `num_mel_bins` config issues)\n2. Always pass a `language` parameter if you know it — cuts response time roughly in half\n\nIf you're building voice transcription on AWS and want Whisper-quality accuracy without the SageMaker complexity, give it a try.\n\n---\n\n*Using EFS as a persistent model cache follows the same pattern I used earlier for [scaling a stateful Streamlit chatbot with ECS + EFS](https://dev.to/aws-builders/scale-a-stateful-streamlit-chatbot-with-aws-ecs-and-efs-48gm) — if you're building other stateful workloads on AWS, that one's worth a look too.*", "excerpts": [ "*Part 3 of my series on building a low-cost personal AI stack on AWS.* *[Part 1 — Squeezing my $1k/month API bill to $20/month with AWS Credits](https://dev.to/aws-builders/i-squeezed-my-1k-monthly-openclaw-api-bill-with-20month-in-aws-credits-heres-the-exact-setup-3gj4)* *[Part 2 — Drop-in Perplexity Sonar replacement with AWS Bedrock Nova Grounding](https://dev.to/aws-builders/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9)*", "I built a self-hosted speech-to-text API on AWS Lambda using [faster-whisper](https://github.com/SYSTRAN/faster-whisper). After trying Amazon Transcribe, SageMaker Serverless, and Lambda with a bundled model, I landed on a **Lambda + EFS + S3** architecture that achieves ~20-30 second cold starts (once the model is cached on EFS) for ~$0.21/month in storage costs. Once warm, specifying the language drops response time to ~10s.", "Open source: [gabrielkoo/aws-lambda-whisper-adaptor](https://github.com/gabrielkoo/aws-lambda-whisper-adaptor)", "I wanted to automatically transcribe Telegram voice messages. The requirements were simple:", "- **Accuracy**: Good enough for Cantonese - **Cost**: Pay-per-use, scales to zero when idle - **Latency**: Cold start under 60 seconds", "There's a fourth constraint that's easy to overlook outside Hong Kong: **most managed STT APIs simply aren't available here**. OpenAI's Whisper API falls under their notorious China's regional restriction. Google's Gemini models are available and actually competitive on both accuracy and price — Gemini 3 Flash achieves 3.1% WER at ~$1.92/1000 minutes ([Artificial Analysis STT leaderboard](https://artificialanalysis.ai/speech-to-text)), cheaper than OpenAI's Whisper API and competitive with Lambda at low volume. The real reason I went with Lambda: AWS Credits from the Community Builder program (same theme as the rest of this series) make it effectively free.", "Simple enough. Except it took four attempts to get there.", "What I Tried (and Why It Didn't Work)", "The obvious first choice — fully managed, pay-per-use, native AWS integration.", "**Why I rejected it before even trying:**", "Amazon Transcribe [supports `zh-CN` and `zh-TW`, but not `yue` (Cantonese)](https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html). Whisper large-v3-turbo handles Cantonese significantly better, and accuracy matters more than convenience here.", "Option 2: SageMaker Serverless Inference", "SageMaker Serverless scales to zero and handles model serving — sounds perfect.", "I deployed a SageMaker Serverless endpoint with faster-whisper. The first invocation after idle:", "- Container provisioning: ~30s - Model loading: ~45-60s - **Total cold start: 60-90 seconds**", "For a voice message that's 5-10 seconds long, waiting 90 seconds is a terrible experience.", "SageMaker Serverless [maxes out at 6144 MB (6 GB) RAM](https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html). Here's why that's a problem for Whisper:", "- [`whisper-large-v3-turbo` (INT8)](https://huggingface.co/Zoont/faster-whisper-large-v3-turbo-int8-ct2): ~780MB model + ~2GB Python/runtime overhead ≈ 2.8GB minimum - [`whisper-large-v3` (FP16)](https://huggingface.co/Systran/faster-whisper-large-v3): ~3GB model alone — barely fits, zero headroom for audio processing - Any concurrent requests? You're OOM.", "[Lambda goes up to 10,240 MB](https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html). That headroom matters.", "[SageMaker Serverless bills per GB-second](https://aws.amazon.com/sagemaker/pricing/) of inference time. For sporadic voice message transcription (~10s per request, a few times a day), Lambda's per-invocation pricing is significantly cheaper. My Lambda setup costs ~$0.21/month in storage — the compute is essentially free at this volume.", "I deleted the endpoint after testing.", "Option 2b: Bedrock Marketplace", "AWS Bedrock Marketplace [does list Whisper Large V3 Turbo](https://aws.amazon.com/blogs/machine-learning/build-a-serverless-audio-summarization-solution-with-amazon-bedrock-and-whisper/) — but it deploys on a **dedicated endpoint instance**. Auto-scaling is available (including scale-to-zero), but that creates a different problem:", "- **Keep minimum 1 instance**: always paying for idle time, even at 3am - **Scale to zero**: cold starts when traffic resumes — SageMaker cold starts are measured in **minutes**, not seconds - Not token/usage-based pricing either way", "For a Telegram bot that gets a few voice messages a day, you're either burning money on idle instances or waiting minutes for the first message to transcribe. Lambda's ~20-30s cold start looks great by comparison.", "Option 3: Lambda with Bundled Model", "Next idea: bundle the model directly into the Docker image. No external dependencies, simple architecture.", "- Docker image size: **~10GB** - ECR push time: **5+ minutes** - Lambda cold start: **2 minutes 51 seconds**", "The cold start is dominated by Lambda pulling the 10GB image from ECR. AWS Lambda caches images, but any cold start after the cache expires hits this wall.", "- 3-minute cold start is unusable for interactive transcription - Every code change requires rebuilding and pushing a 10GB image - ECR storage: ~$1/month just for the image", "Option 4: Lambda + S3 (No EFS)", "What if Lambda downloads the model from S3 on cold start, storing it in `/tmp`?", "Lambda's `/tmp` is ephemeral. Every cold start re-downloads the model from S3:", "- S3 download for 1.6GB FP16 model: **30-60 seconds** - S3 download for 780MB INT8 model: **15-30 seconds**", "This is better than the bundled model approach, but there's a bigger issue: **no caching between Lambda instances**. If you have 3 concurrent invocations, all 3 download the model independently. You're paying for S3 transfer on every cold start.", "**What about Lambda SnapStart or Durable Functions?**", "AWS added two relevant features since this was written:", "- **SnapStart for Python** (Nov 2024): snapshots the initialized execution environment — sounds perfect for caching a loaded model. The catch: [SnapStart doesn't support container images](https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html). This adaptor is container-based, so it's off the table.", "- **[Lambda Durable Functions](https://aws.amazon.com/about-aws/whats-new/2025/12/lambda-durable-multi-step-applications-ai-workflows/)** (re:Invent 2025): enables multi-step workflows with automatic checkpointing, pause/resume for up to one year, and failure recovery. This is workflow orchestration (think Azure Durable Functions) — useful for multi-step AI pipelines, but not for persisting a 780MB model binary between cold starts.", "EFS remains the right solution for model caching.", "What Actually Worked: Lambda + EFS + S3", "The solution: use **EFS as a persistent model cache**, bootstrapped from S3. I've used EFS for [persistent Streamlit state on ECS](https://dev.to/aws-builders/scale-a-stateful-streamlit-chatbot-with-aws-ecs-and-efs-48gm) before — same pattern, different compute layer.", "![Logic Flow](/assets/img/57f530f3907a.png)", "1. **First cold start** (once per model): Lambda checks for a marker file on EFS. If missing, downloads model from S3 to EFS (~55s for INT8). Writes marker file. Then loads model into RAM. 2. **Subsequent cold starts** (new container, model already on EFS): Marker file exists → load model from EFS into RAM (~20-30s for INT8). 3. **Warm invocations** (same container reused): Model already in memory → transcription-only time (~10-22s depending on audio length and whether language is specified).", "def bootstrap_model(): if os.path.exists(MODEL_MARKER): return WhisperModel(EFS_MODEL_DIR, device='cpu', compute_type='int8') # First run: sync model from S3 to EFS s3 = boto3.client('s3') prefix = f'models/{MODEL_SLUG}/' os.makedirs(EFS_MODEL_DIR, exist_ok=True) paginator = s3.get_paginator('list_objects_v2') for page in paginator.paginate(Bucket=os.environ['MODEL_S3_BUCKET'], Prefix=prefix): for obj in page.get('Contents', []): key = obj['Key'] local_path = os.path.join(EFS_MODEL_DIR, key[len(prefix):]) os.makedirs(os.path.dirname(local_path), exist_ok=True) s3.download_file(os.environ['MODEL_S3_BUCKET'], key, local_path) open(MODEL_MARKER, 'w').close() # Mark as ready return WhisperModel(EFS_MODEL_DIR, device='cpu', compute_type='int8')", "MODEL = bootstrap_model() # Runs at Lambda init time, cached for warm invocations ```", "- EFS persists across Lambda instances — model is downloaded **once**, reused forever - EFS is mounted at `/mnt/whisper-models` — Lambda reads it like a local filesystem - **S3 VPC Gateway Endpoint is free** — no NAT Gateway needed (saves ~$32/month) - **Zero internet egress** — Lambda → S3 via VPC Gateway Endpoint, Lambda → EFS within VPC. The Lambda function never reaches the internet. This is a meaningful security benefit when using third-party models from HuggingFace — model weights never leave the AWS network once synced to S3. - EFS storage: ~$0.19/month for the 780MB INT8 model", "> 🔒 **Security note:** The Lambda runs in a VPC with **no internet access** — no NAT Gateway, no public subnet. It can only reach EFS (VPC-internal) and S3 (via the free VPC Gateway Endpoint). This means even if you're using a third-party HuggingFace model, the model weights and your audio data never leave the AWS network. No data exfiltration risk, no outbound calls to unknown endpoints.", "INT8 vs FP16: The Model Size Trade-off", "The `openai/whisper-large-v3-turbo` model on HuggingFace needs conversion to CTranslate2 format. The `sync-model` workflow handles this, converting to INT8 and fixing the `num_mel_bins` config. Alternatively, use [`Zoont/faster-whisper-large-v3-turbo-int8-ct2`](https://huggingface.co/Zoont/faster-whisper-large-v3-turbo-int8-ct2) — a pre-converted CTranslate2 INT8 model that works out of the box with `quantization=none`:", "| Model | Size (EFS) | First Bootstrap | EFS Cold Start | Warm (2.5s audio) | Memory | |-------|-----------|-----------------|----------------|-------------------|--------| | `Zoont/faster-whisper-large-v3-turbo-int8-ct2` | ~780MB | ~55s | **~22s** ✅ | **~10s** ✅ | ~2.8GB | | `openai/whisper-large-v3-turbo` (INT8, via sync-model) | ~780MB | ~55s | ~22s | ~10s | ~2.8GB | | `openai/whisper-large-v3-turbo` (FP16) | ~1.5GB | ~126s | ~40s | ~15s | ~4GB | | `Systran/faster-whisper-large-v3` (FP16, loaded as int8) | ~1.6GB | ~54s | ~30s | ~13s | 6GB |", "**Recommended:** `Zoont/faster-whisper-large-v3-turbo-int8-ct2` — no conversion step needed, identical performance to the openai model converted to INT8. Use `quantization=none` in the sync-model workflow since it's already in CTranslate2 format.", "| Resource | Monthly Cost | |----------|-------------| | EFS storage (780MB INT8) | ~$0.19 | | S3 storage (780MB) | ~$0.02 | | Lambda compute | ~$0.00167/warm invocation* | | S3 VPC Gateway Endpoint | **Free** | | NAT Gateway | **Not needed ($0)** | | **Total (storage only)** | **~$0.21/month** |", "*10GB × 10s = 100 GB-seconds per warm invocation. The [Lambda free tier](https://aws.amazon.com/lambda/pricing/) covers **400,000 GB-seconds/month** — roughly 4,000 warm invocations. For a personal bot, compute cost is effectively **$0**. Storage dominates.", "Compare to SageMaker Serverless: minimum ~$5-10/month for similar workloads, plus the 60-90s cold start penalty.", "> **Why not Provisioned Concurrency?** PC keeps Lambda permanently warm (no cold starts), but costs ~$0.0000097222/GB-second. For a 10GB function running 24/7: ~$252/month. Even a minimal 4GB setup runs ~$100/month — roughly 500x more than the $0.21 storage approach. For a personal bot with a few voice messages a day, the occasional ~60s cold start is a fine trade-off.", "OpenAI's Whisper API costs [**$0.006/minute**](https://openai.com/api/pricing/). Here's how it compares for a bot averaging 15s voice messages:", "| Volume | OpenAI Whisper API | Self-hosted Lambda | |--------|-------------------|-------------------| | 50 msgs/month | $0.08 | $0.21 (storage only) | | 140 msgs/month | $0.21 | **$0.21** ← break-even | | 500 msgs/month | $0.75 | $0.21 (storage only) | | 1,000 msgs/month | $1.50 | $0.21 (storage only) | | 4,000 msgs/month | $6.00 | $0.21 (storage only) |", "Lambda compute is free within the free tier (~4,000 warm invocations/month). Beyond that, it's $0.00167/invocation — but that's a high volume for a personal bot.", "Break-even: **~140 messages/month**. Above that, Lambda wins on cost.", "But cost isn't the only reason to self-host: - **Geographic availability**: OpenAI's API is not available in Hong Kong — HK falls under China's regional restriction. Azure OpenAI does offer Whisper, but [only `whisper-1` (large-v2 based)](https://learn.microsoft.com/en-us/answers/questions/2237575/new-announced-speech-to-text-models-for-realtime) — large-v3 and large-v3-turbo are not available. If you're in HK (or other restricted regions), this approach isn't just cheaper — it's the only option for v3-quality transcription. - **Cantonese accuracy**: `language=yue` with Whisper large-v3-turbo is noticeably better than the managed API for Cantonese - **Privacy**: audio never leaves your infrastructure - **No rate limits**: Lambda scales independently", "**Lambda configuration:** - Memory: 10,240 MB — actual usage is **~2.2GB** (INT8 model), but [Lambda allocates CPU proportional to memory](https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html). 10GB gives ~6 vCPUs vs ~2.3 vCPUs at 4GB, cutting warm transcription from ~16s to ~10s. You're paying for CPU, not RAM. - Timeout: 900s (handles long audio files) - VPC: Default VPC (no NAT Gateway) - EFS: Mounted at `/mnt/whisper-models`", "**Memory vs. cost trade-off (tested, 3 runs each):**", "| Config | Cold Start | Warm (2.5s audio) | GB-seconds/invocation | |--------|-----------|-------------------|----------------------| | 4,096 MB | ~30s | ~21s | 84 (~$0.00140) | | 6,144 MB | ~25s | ~16s | 96 (~$0.00160) | | 8,192 MB | ~24s | ~18s | 144 (~$0.00240) | | 10,240 MB | ~22s | ~15s | 150 (~$0.00250) |", "Cold start is ~20-30s across all configs — it's EFS I/O bound, not CPU bound, so more memory doesn't help much here. Warm inference time does scale with memory (more vCPUs = faster CTranslate2 decoding). Interestingly, 4GB is the cheapest per invocation — the warm time savings at higher memory don't offset the extra GB-seconds. Within the free tier, cost differences are negligible regardless.", "The adaptor exposes two endpoints so it works as a drop-in replacement for existing integrations:", "**OpenAI compatible** (`/v1/audio/transcriptions`): ```bash curl -X POST https:// /v1/audio/transcriptions \\ -H \"Authorization: Token \" \\ -F \"file=@audio.ogg\" \\ -F \"language=yue\" ``` ```json {\"text\": \"transcript here\"} ```", "**Deepgram compatible** (`/v1/listen`): ```bash curl -X POST https:// /v1/listen?language=yue \\ -H \"Authorization: Token \" \\ -H \"Content-Type: audio/ogg\" \\ --data-binary @audio.ogg ```", "Once you've synced multiple models to EFS, there's no SSH access to see what's there or clean up. I added two non-standard endpoints:", "**List models on EFS:** ```bash curl https:// /v1/models -H \"Authorization: Token \" ``` ```json { \"object\": \"list\", \"data\": [ {\"id\": \"openai/whisper-large-v3-turbo\", \"object\": \"model\", \"owned_by\": \"openai\"}, {\"id\": \"Systran/faster-distil-whisper-large-v3\", \"object\": \"model\", \"owned_by\": \"Systran\"} ] } ```", "**Delete a model from EFS** (the currently loaded model returns 409): ```bash curl -X DELETE https:// /v1/models/Systran/faster-distil-whisper-large-v3 \\ -H \"Authorization: Token \" ``` ```json {\"id\": \"Systran/faster-distil-whisper-large-v3\", \"object\": \"model\", \"deleted\": true} ```", "Slashes in model IDs work naturally — `rawPath` preserves the full path, so `DELETE /v1/models/openai/whisper-large-v3-turbo` correctly maps to model ID `openai/whisper-large-v3-turbo`.", "Performance Tip: Always Specify Language", "When no language is specified, Whisper runs language detection on the first audio chunk — adding noticeable overhead. For a 2.5s voice message:", "| Request | Response Time | |---------|--------------| | No language (auto-detect) | ~22s | | `language=yue` (Cantonese) | ~10s |", "That's a **2x speedup** just from passing a language hint. Two ways to do it:", "**Option A — per-request query param** (recommended, keeps Lambda language-agnostic): ```bash Deepgram endpoint curl -X POST https:// /v1/listen?language=yue \\ -H \"Authorization: Token \" \\ -H \"Content-Type: audio/ogg\" \\ --data-binary @audio.ogg", "OpenAI endpoint curl -X POST https:// /v1/audio/transcriptions \\ -H \"Authorization: Token \" \\ -F \"file=@audio.ogg\" \\ -F \"language=yue\" ```", "**Option B — Lambda env var** (simpler if you only ever transcribe one language): ```bash WHISPER_LANGUAGE=yue ```", "I use Option A — the language is set in my OpenClaw config (`language: \"yue\"` in the audio model), which passes it as `?language=yue` to the Lambda on every request." ] }, { "id": "article:i-squeezed-my-1k-monthly-openclaw-api-bill-with-20month-in-aws-credits-heres-the-exact-setup-3gj4", "source_type": "article", "title": "I Squeezed My $1k Monthly OpenClaw API Bill with ~$20/Month in AWS Credits — Here's the Exact Setup", "url": "https://gabrielkoo.com/blog/i-squeezed-my-1k-monthly-openclaw-api-bill-with-20month-in-aws-credits-heres-the-exact-setup-3gj4/", "canonical_url": "https://dev.to/aws-builders/i-squeezed-my-1k-monthly-openclaw-api-bill-with-20month-in-aws-credits-heres-the-exact-setup-3gj4", "published_at": "2026-02-21", "last_verified_at": "2026-08-23", "tags": [ "aws", "bedrock", "openclaw", "kiro" ], "description": "I've got OpenClaw running locally on a Raspberry Pi — where computation power is scarce, and it's...", "content": "I've got OpenClaw(MoltBot(ClawdBot)) running locally on a Raspberry Pi — where computation power is scarce, and it's gone unresponsive on me more than a few times. But even on constrained hardware, every chat turn, every memory search, every web lookup is hitting paid APIs. The bill is small at first, then it isn't. I had been using `qwen3-coder-480b` for a week or two, and the daily cost skyrocketed to as much as $50.\n\n> **Assumption:** OpenClaw is running on hardware you already own or pay for separately — a Raspberry Pi, home server, or existing cloud instance. The compute cost of the host itself isn't counted in the ~$20/month figure here.\n\nIf you've picked up AWS Credits from events, the [AWS Community Builder program](https://builder.aws.com/content/32g2lQ7kc3Py8kKIYGS15Pe8VSS/aws-community-builders-program) ($500/year), or AWS Activate — or if your company prefers to keep spend within AWS rather than onboarding yet another SaaS API provider — there's a way to run the whole OpenClaw stack on credits.\n\nThis is how I did it.\n\n> **Disclaimer**: The crux of this hack relies heavily on Amazon Q Developer Pro's undocumented while generously high usage ceiling while it lasts. If it's eventually deprecated, we will still need to switch to Kiro plans with overage pricings - still covered by AWS Credits with lower cost/token ratio. \n\n---\n\n## Who This Is For\n\nTwo very different reasons to care about this setup.\n\n**If you have AWS Credits to burn:**\nCredits from re:Invent, AWS Community Builder, AWS Activate, or customer programs come with expiry dates. Running your AI assistant stack on them is one of the most practical ways to put idle credits to work — at ~$20/month, $100 in credits covers 5 months of the full stack. If you're sitting on a few hundred dollars with an end-of-year deadline, this is a productive use before they lapse.\n\n**If you're in a company with procurement or compliance requirements:**\nEvery new SaaS vendor is a TPRM exercise. OpenAI for embeddings, Perplexity for web search, Anthropic for Claude — each one is a separate vendor assessment, a separate DPA, and a separate conversation with your security team. For FSI and regulated industries, that's not just overhead — it can be a blocker.\n\nAWS is likely already in your vendor register. Consolidating on Bedrock means single billing, fewer third-party relationships to manage, and data residency you control. For anything touching customer data in banking, insurance, or healthcare, that's the difference between a quick internal approval and a 3-month procurement cycle.\n\n---\n\n## Prerequisites\n\n- **AWS account** with Bedrock access enabled in `us-east-1` (or another US region)\n- **AWS credentials** — a [Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) is the simplest option if your account supports it. Otherwise, a long-term IAM access key/secret key pair works fine and is easier to manage than SSO. IAM Identity Center is only required for the Q Developer Pro layer.\n- **Python 3.10+** — used by kiro-gateway, LiteLLM, and the Nova grounding proxy\n- **Amazon Q Developer Pro subscription** ($19/user/month, credit-eligible) — required for Layer 1 (kiro-gateway). Kiro Pro, Pro+, or Power plans also work but are credit-based with overage charges — Q Developer Pro is the better deal.\n\n---\n\n## What Actually Costs Money in OpenClaw?\n\nBefore reaching for solutions, it helps to know exactly where the spend goes. OpenClaw has four distinct cost centers:\n\n**1. Main model (LLM)**\nEvery chat turn, every agent action, every tool call — all routed through your primary LLM. This is the biggest variable cost. On a busy day it adds up fast.\n\n**2. Memory search (embeddings)**\nOpenClaw's `memory_search` tool converts your memory files into vector embeddings and queries them semantically. Every search = an embedding API call. Low cost per call, but it runs constantly in the background.\n\n**3. Web search**\nThe `web_search` tool hits Perplexity or Brave APIs. Perplexity charges per query on paid plans; Brave gives you $5/month free then charges beyond that.\n\n**4. Browser automation**\nThe `browser` tool spins up a Chromium instance for web scraping, form filling, and screenshots. Running a full browser on a low-compute machine (Raspberry Pi, t4g.small) is heavy — and cloud browser options cost per session.\n\nThat's it. Four layers. The goal: drive variable cost to zero.\n\n---\n\n## My Config: All 4 Layers on AWS Credits\n\nHere's the full picture before we go deep:\n\n| Layer | Solution | Credit |\n|-------|----------|--------|\n| Main model | [kiro-gateway](https://github.com/jwadow/kiro-gateway) → Amazon Q Developer Pro | [@Jwadow](https://github.com/Jwadow) |\n| Memory search | Native Bedrock embeddings via [PR #20191](https://github.com/openclaw/openclaw/pull/20191) | [@gabrielkoo](https://github.com/gabrielkoo) |\n| Web search | [bedrock-web-search-proxy](https://github.com/gabrielkoo/bedrock-web-search-proxy) — Nova Grounding as Perplexity drop-in | [@gabrielkoo](https://github.com/gabrielkoo) |\n| Browser | [agent-browser + AgentCore provider](https://github.com/vercel-labs/agent-browser/pull/397) | [@pahudnet](https://x.com/pahudnet) |\n\nTwo of these I built myself. Two were built by other community members. All four are open source.\n\n---\n\n## Layer 1: Main Model + Image Analysis — Kiro CLI — Covered by AWS Credits\n\n### Amazon Q Developer Pro: flat-rate access to Claude\n\nThe key difference between Amazon Q Developer Pro and Kiro Pro is the billing model. Kiro Pro is credit-based — 1,000 credits/month, pay more if you exceed them. Amazon Q Developer Pro is a flat monthly subscription: **$19/user/month, no per-token billing, no surprise overages.**\n\n| Plan | Cost | Usage |\n|------|------|-------|\n| Kiro Free | $0/mo | 50 credits/month |\n| Kiro Pro | $20/mo | 1,000 credits + $0.04/credit overage |\n| Kiro Pro+ | $40/mo | 2,000 credits + $0.04/credit overage |\n| Kiro Power | $200/mo | 10,000 credits + $0.04/credit overage |\n| Amazon Q Developer Pro (legacy) | $19/user/mo | Flat-rate, not credit-capped |\n\n> **Note:** Amazon Q Developer Pro is now a legacy plan in the Kiro ecosystem. AWS has stopped allowing new Builder ID subscriptions to Q Developer Pro — new users can only subscribe through Kiro plans. The undocumented usage limits on Q Pro are likely part of why AWS made this transition. If you're already on Q Developer Pro, you retain access and it remains the better deal for OpenClaw.\n\nYour Q Developer Pro subscription grants access to `kiro-cli`. The documented quota is [10,000 inference calls/month](https://docs.aws.amazon.com/general/latest/gr/amazonqdev.html) — for a personal AI assistant, that's more than enough.\n\n> **Real-world cost check:** In 4 days of active OpenClaw usage after switching to kiro-gateway, I consumed ~40M input tokens and ~865K output tokens with Claude Sonnet. OpenClaw loads memory files, system prompts, and tool results into every turn — the context window fills up fast. At standard Bedrock pricing ($3/1M input, $15/1M output), that's ~$135 for 4 days, or roughly **$1,000/month**. Q Developer Pro covers all of it for $19/month flat.\n\nIn practice, I've been running Kiro CLI with OpenClaw daily and haven't hit any rate limits in active use. Note: the `/usage` command isn't available under the Q Developer Pro plan — monitor your usage via the AWS console instead. That said, after running OpenClaw with kiro-gateway for several days, I checked the Q Developer usage metrics in the AWS console and the figures hadn't moved at all. It's unclear whether Kiro CLI usage is counted against the same quota as Q Developer's agentic requests, or tracked separately. The [Amazon Q Developer pricing page](https://aws.amazon.com/q/developer/pricing/) only states \"Included (with limits)\" for the Pro tier — no specifics on what those limits are or how Kiro CLI calls are metered.\n\n> **Note:** Q Developer Pro requires [AWS IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html) (SSO) — you can't use it with a free Builder ID. If you're already set up with Identity Center (common in enterprise teams and AWS Community Builders with corporate accounts), you're good to go.\n\n> **Important:** Standard AWS Credits don't cover per-token Claude usage via Anthropic's marketplace agreement. But the Q Developer Pro subscription fee itself **is** credit-eligible — making the whole stack fundable with AWS credits. Kiro's flat-rate subscription is currently the only practical way to run Claude in OpenClaw without per-token billing.\n\n> **New AWS accounts:** Even if you'd prefer to pay per-token via direct Bedrock API, new accounts often come with [ultra-low default rate limits](https://dev.to/aws-builders/ultra-low-bedrock-llm-rate-limits-for-new-aws-accounts-time-to-wake-up-your-inactive-aws-accounts-3no0) that can't reliably serve OpenClaw — even when you're willing to pay. The flat-rate Q Developer Pro route sidesteps this entirely.\n\n### kiro-gateway: the bridge\n\n[kiro-gateway](https://github.com/jwadow/kiro-gateway) — built by [@Jwadow](https://github.com/Jwadow) — wraps Kiro CLI and exposes OpenAI-compatible and Anthropic-compatible API endpoints. OpenClaw talks to it like any other provider.\n\n```bash\ngit clone https://github.com/jwadow/kiro-gateway\ncd kiro-gateway\npip install -r requirements.txt\ncp .env.example .env\n```\n\nEdit `.env`:\n\n```env\nPROXY_API_KEY=\"your-secret-key\"\nKIRO_CREDS_FILE=\"~/.aws/sso/cache/kiro-auth-token.json\"\n```\n\nRun `kiro-cli login` once to authenticate — this populates `KIRO_CREDS_FILE` automatically. (`kiro-cli` is only needed for this initial login; `kiro-gateway` reads the token it generates. Re-run if your token expires.) Then:\n\n```bash\npython main.py --port 9000\n```\n\n> **Heads up:** kiro-gateway's hardcoded fallback model list may lag behind new Claude releases. If a model isn't showing up at `/v1/models`, add it manually to `FALLBACK_MODELS` in `kiro/config.py`.\n\nAvailable models via Q Developer Pro:\n\n| Model | Best for |\n|-------|---------|\n| `claude-sonnet-4.6` | General tasks, coding, writing |\n| `claude-haiku-4.5` | Fast, lightweight responses |\n| `claude-opus-4.6` | Complex reasoning, long context |\n\nOpenClaw config:\n\n```json\n{\n \"models\": {\n \"providers\": {\n \"kiro\": {\n \"baseUrl\": \"http://localhost:9000\",\n \"apiKey\": \"your-secret-key\",\n \"api\": \"anthropic-messages\"\n }\n }\n },\n \"agents\": {\n \"defaults\": {\n \"model\": {\n \"primary\": \"kiro/claude-sonnet-4.6\"\n },\n \"imageModel\": {\n \"primary\": \"kiro/claude-sonnet-4.6\"\n }\n }\n }\n}\n```\n\n> **Bonus:** kiro-gateway works with any tool that supports OpenAI or Anthropic APIs — not just OpenClaw. To use it with Claude Code: `ANTHROPIC_BASE_URL=http://localhost:9000` and `ANTHROPIC_API_KEY=your-secret-key`.\n\n---\n\n## Layer 2: Memory Search — Bedrock Embeddings — Covered by AWS Credits\n\nOpenClaw's `memory_search` needs an embedding model. [Amazon Nova Multimodal Embeddings](https://docs.aws.amazon.com/nova/latest/userguide/nova-embeddings.html) costs ~$0.00014 per 1K tokens — fractions of a cent per query, and covered by AWS Credits.\n\nOpenClaw's native Bedrock provider doesn't wire up embeddings cleanly yet — [PR #24892](https://github.com/openclaw/openclaw/pull/24892) - (I made a novice mistake with [PR #20191](https://github.com/openclaw/openclaw/pull/20191)) is pending merge. Until then, you'll need a local OpenAI-compatible proxy in front of Bedrock. Two options:\n\n### Option A: LiteLLM\n\n```yaml\n# litellm_config.yaml\nmodel_list:\n - model_name: nova-2-multimodal-embeddings-v1.0\n litellm_params:\n model: bedrock/amazon.nova-2-multimodal-embeddings-v1:0\n aws_region_name: us-east-1\n\nlitellm_settings:\n drop_params: true\n master_key: \"local-only\"\n```\n\n```bash\npip install 'litellm[proxy]'\nlitellm --config litellm_config.yaml --port 4000\n```\n\n```json\n\"memorySearch\": {\n \"enabled\": true,\n \"provider\": \"openai\",\n \"remote\": { \"baseUrl\": \"http://localhost:4000\", \"apiKey\": \"local-only\" },\n \"model\": \"nova-2-multimodal-embeddings-v1.0\"\n}\n```\n\n### Option B: bedrock-access-gateway-function-url (serverless, no fixed cost)\n\nMy own fork of the original `bedrock-access-gateway` — deployed as a Lambda Function URL instead of ALB+Fargate, so there's no $16+/month fixed cost. Full writeup: [Use Amazon Bedrock Models with OpenAI SDKs with a Serverless Proxy Endpoint](https://dev.to/aws-builders/use-amazon-bedrock-models-via-an-openai-api-compatible-serverless-endpoint-now-without-fixed-cost-5hf5).\n\n> **Note:** My [PR #222](https://github.com/aws-samples/bedrock-access-gateway/pull/222) for Nova 2 embedding support against the original `bedrock-access-gateway` project has been merged — so my fork pulls from this upstream automatically via `prepare_source.sh`.\n\n```bash\ngit clone --depth=1 https://github.com/gabrielkoo/bedrock-access-gateway-function-url\ncd bedrock-access-gateway-function-url\n./prepare_source.sh\nsam build\nsam deploy --guided\n```\n\nGrab the `FunctionUrl` output after deploy, then:\n\n```json\n\"memorySearch\": {\n \"enabled\": true,\n \"provider\": \"openai\",\n \"remote\": { \"baseUrl\": \"https://.lambda-url.us-east-1.on.aws\", \"apiKey\": \"your-api-key\" },\n \"model\": \"amazon.nova-2-multimodal-embeddings-v1:0\"\n}\n```\n\n> **Region note:** `amazon.nova-2-multimodal-embeddings-v1:0` availability varies — check the [Bedrock model availability page](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html). Make sure your IAM credentials have `bedrock:InvokeModel` in your target region.\n\nOnce [PR #24892](https://github.com/openclaw/openclaw/pull/24892) merges, no proxy needed — the config simplifies to:\n\n```json\n\"memorySearch\": {\n \"enabled\": true,\n \"provider\": \"bedrock\",\n \"model\": \"amazon.nova-2-multimodal-embeddings-v1:0\",\n \"region\": \"us-east-1\"\n}\n```\n\n---\n\n## Layer 3: Web Search — Nova Grounding Proxy — Covered by AWS Credits\n\nI built [`bedrock-web-search-proxy`](https://github.com/gabrielkoo/bedrock-web-search-proxy) — a FastAPI wrapper that makes Bedrock Nova Grounding look like the Perplexity Sonar API. No Perplexity or Brave API key needed. Runs entirely on AWS Credits.\n\nFull writeup: [Drop-in Perplexity Sonar Replacement with AWS Bedrock Nova Grounding](https://dev.to/aws-builders/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9).\n\n### Option A: Run locally\n\n```bash\ngit clone https://github.com/gabrielkoo/bedrock-web-search-proxy\ncd bedrock-web-search-proxy\npip install fastapi uvicorn boto3\nuvicorn main:app --port 7000\n```\n\n### Option B: Lambda Function URL (zero idle cost)\n\nSee the [deployment guide in the repo](https://github.com/gabrielkoo/bedrock-web-search-proxy) — SAM-based, arm64, python3.13. Once deployed, you get a persistent HTTPS endpoint with no local process to manage.\n\nOpenClaw config:\n\n```json\n{\n \"tools\": {\n \"web\": {\n \"search\": {\n \"provider\": \"perplexity\",\n \"perplexity\": {\n \"apiKey\": \"your-proxy-key\",\n \"baseUrl\": \"http://localhost:7000/v1\",\n \"model\": \"sonar-pro\"\n }\n }\n }\n }\n}\n```\n\n> All US Nova CRIS (Cross-Region Inference Services) profiles support web grounding (`us.amazon.nova-premier-v1:0`, `us.amazon.nova-pro-v1:0`, etc.). Native model IDs without the `us.` prefix do NOT work — must use CRIS profiles. Web grounding is US regions only (us-east-1, us-east-2, us-west-2).\n\n---\n\n## Layer 4: Cloud Browser — Bedrock AgentCore — Covered by AWS Credits\n\n[`agent-browser`](https://github.com/vercel-labs/agent-browser) by Vercel Labs, with the AgentCore provider contributed by [Pahud Hsieh](https://github.com/pahud) ([@pahudnet](https://x.com/pahudnet)) — [PR #397](https://github.com/vercel-labs/agent-browser/pull/397).\n\nThe browser runs in AWS — no local Chromium needed. Particularly useful on low-compute instances (Pi, t4g.small) where running a local browser would be too heavy. Covered by AWS Credits.\n\nNode.js and pnpm required. Since [PR #397](https://github.com/vercel-labs/agent-browser/pull/397) isn't merged yet, check out the branch directly:\n\n```bash\ngit clone https://github.com/vercel-labs/agent-browser\ncd agent-browser\ngit fetch origin pull/397/head:agentcore\ngit checkout agentcore\npnpm install && pnpm build\n```\n\nThen use it:\n\n```bash\nagent-browser -p agentcore open https://example.com\nagent-browser close\n```\n\nYour AWS identity needs these IAM permissions:\n\n- `bedrock-agentcore:StartBrowserSession`\n- `bedrock-agentcore:ConnectBrowserAutomationStream`\n- `bedrock-agentcore:StopBrowserSession`\n\n> On a desktop machine with enough RAM, local CDP (OpenClaw's built-in browser) is free and works fine. AgentCore is the play for headless/low-compute setups.\n\n---\n\n## The Cost Math\n\nWithout this setup, Claude Sonnet alone runs ~**$1,000/month** at standard Bedrock pricing — based on real token usage from my own sessions. OpenClaw's large context window (memory files, system prompts, tool results loaded every turn) means the token bill compounds fast.\n\nThe full stack with this setup runs at **~$20/month**:\n\n- **$19/mo** — Amazon Q Developer Pro (flat-rate, covers all LLM calls)\n- **≤$1/mo** — Bedrock embeddings for memory search (Nova 2 at $0.00014/1K tokens)\n\nWeb search and browser automation are covered by AWS Credits — no separate line item.\n\nWith **$100 in AWS Credits**, you cover roughly **5 months** of the full stack. Both the Q Developer Pro subscription and Bedrock embeddings are credit-eligible — if you're an AWS Community Builder, that $500/year allocation more than covers it.\n\n### Where AWS Credits Come From\n\n- **AWS event participant/speaker** — re:Invent, Summit, local user groups\n- **AWS Community Builder** — $500/year for active builders ([builder.aws.com](https://builder.aws.com/content/32g2lQ7kc3Py8kKIYGS15Pe8VSS/aws-community-builders-program)). The application opens a few rounds per year — I'm one of the builders in the program.\n- **AWS Customer Council** — participation typically includes credits\n- **AWS Activate** (startups) — up to $100K\n- **AWS Educate / Academy** — educators and students\n\nCheck your balance: [console.aws.amazon.com/billing/home#/credits](https://console.aws.amazon.com/billing/home#/credits)\n\n---\n\n## Closing\n\nFour layers. Two built by community members, two I built myself. All open source, all running on AWS Credits.\n\nTo be clear: **kiro-gateway is the most crucial piece here.** [@Jwadow](https://github.com/Jwadow) built the bridge that makes Claude accessible without per-token billing — I built the embedding proxy and web search proxy to fill the remaining gaps. [Web search](https://github.com/gabrielkoo/bedrock-web-search-proxy) and [cloud browser](https://github.com/vercel-labs/agent-browser) (Layers 3 and 4) are purely AWS Credits — no subscription, per-token billing well covered by AWS Credits.\n\nIf you're already an AWS Community Builder or have credits sitting in your account, there's no reason to be paying per-token for a personal AI assistant. Wire it up once, and the stack runs itself.\n\nPut those credits to work.", "excerpts": [ "I've got OpenClaw(MoltBot(ClawdBot)) running locally on a Raspberry Pi — where computation power is scarce, and it's gone unresponsive on me more than a few times. But even on constrained hardware, every chat turn, every memory search, every web lookup is hitting paid APIs. The bill is small at first, then it isn't. I had been using `qwen3-coder-480b` for a week or two, and the daily cost skyrocketed to as much as $50.", "> **Assumption:** OpenClaw is running on hardware you already own or pay for separately — a Raspberry Pi, home server, or existing cloud instance. The compute cost of the host itself isn't counted in the ~$20/month figure here.", "If you've picked up AWS Credits from events, the [AWS Community Builder program](https://builder.aws.com/content/32g2lQ7kc3Py8kKIYGS15Pe8VSS/aws-community-builders-program) ($500/year), or AWS Activate — or if your company prefers to keep spend within AWS rather than onboarding yet another SaaS API provider — there's a way to run the whole OpenClaw stack on credits.", "> **Disclaimer**: The crux of this hack relies heavily on Amazon Q Developer Pro's undocumented while generously high usage ceiling while it lasts. If it's eventually deprecated, we will still need to switch to Kiro plans with overage pricings - still covered by AWS Credits with lower cost/token ratio.", "Two very different reasons to care about this setup.", "**If you have AWS Credits to burn:** Credits from re:Invent, AWS Community Builder, AWS Activate, or customer programs come with expiry dates. Running your AI assistant stack on them is one of the most practical ways to put idle credits to work — at ~$20/month, $100 in credits covers 5 months of the full stack. If you're sitting on a few hundred dollars with an end-of-year deadline, this is a productive use before they lapse.", "**If you're in a company with procurement or compliance requirements:** Every new SaaS vendor is a TPRM exercise. OpenAI for embeddings, Perplexity for web search, Anthropic for Claude — each one is a separate vendor assessment, a separate DPA, and a separate conversation with your security team. For FSI and regulated industries, that's not just overhead — it can be a blocker.", "AWS is likely already in your vendor register. Consolidating on Bedrock means single billing, fewer third-party relationships to manage, and data residency you control. For anything touching customer data in banking, insurance, or healthcare, that's the difference between a quick internal approval and a 3-month procurement cycle.", "- **AWS account** with Bedrock access enabled in `us-east-1` (or another US region) - **AWS credentials** — a [Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) is the simplest option if your account supports it. Otherwise, a long-term IAM access key/secret key pair works fine and is easier to manage than SSO. IAM Identity Center is only required for the Q Developer Pro layer. - **Python 3.10+** — used by kiro-gateway, LiteLLM, and the Nova grounding proxy - **Amazon Q Developer Pro subscription** ($19/user/month, credit-eligible) — required for Layer 1 (kiro-gateway). Kiro Pro, Pro+, or Power plans also work but are credit-based with overage charges — Q Developer Pro is the better deal.", "What Actually Costs Money in OpenClaw?", "Before reaching for solutions, it helps to know exactly where the spend goes. OpenClaw has four distinct cost centers:", "**1. Main model (LLM)** Every chat turn, every agent action, every tool call — all routed through your primary LLM. This is the biggest variable cost. On a busy day it adds up fast.", "**2. Memory search (embeddings)** OpenClaw's `memory_search` tool converts your memory files into vector embeddings and queries them semantically. Every search = an embedding API call. Low cost per call, but it runs constantly in the background.", "**3. Web search** The `web_search` tool hits Perplexity or Brave APIs. Perplexity charges per query on paid plans; Brave gives you $5/month free then charges beyond that.", "**4. Browser automation** The `browser` tool spins up a Chromium instance for web scraping, form filling, and screenshots. Running a full browser on a low-compute machine (Raspberry Pi, t4g.small) is heavy — and cloud browser options cost per session.", "That's it. Four layers. The goal: drive variable cost to zero.", "My Config: All 4 Layers on AWS Credits", "Here's the full picture before we go deep:", "| Layer | Solution | Credit | |-------|----------|--------| | Main model | [kiro-gateway](https://github.com/jwadow/kiro-gateway) → Amazon Q Developer Pro | [@Jwadow](https://github.com/Jwadow) | | Memory search | Native Bedrock embeddings via [PR #20191](https://github.com/openclaw/openclaw/pull/20191) | [@gabrielkoo](https://github.com/gabrielkoo) | | Web search | [bedrock-web-search-proxy](https://github.com/gabrielkoo/bedrock-web-search-proxy) — Nova Grounding as Perplexity drop-in | [@gabrielkoo](https://github.com/gabrielkoo) | | Browser | [agent-browser + AgentCore provider](https://github.com/vercel-labs/agent-browser/pull/397) | [@pahudnet](https://x.com/pahudnet) |", "Two of these I built myself. Two were built by other community members. All four are open source.", "Layer 1: Main Model + Image Analysis — Kiro CLI — Covered by AWS Credits", "Amazon Q Developer Pro: flat-rate access to Claude", "The key difference between Amazon Q Developer Pro and Kiro Pro is the billing model. Kiro Pro is credit-based — 1,000 credits/month, pay more if you exceed them. Amazon Q Developer Pro is a flat monthly subscription: **$19/user/month, no per-token billing, no surprise overages.**", "| Plan | Cost | Usage | |------|------|-------| | Kiro Free | $0/mo | 50 credits/month | | Kiro Pro | $20/mo | 1,000 credits + $0.04/credit overage | | Kiro Pro+ | $40/mo | 2,000 credits + $0.04/credit overage | | Kiro Power | $200/mo | 10,000 credits + $0.04/credit overage | | Amazon Q Developer Pro (legacy) | $19/user/mo | Flat-rate, not credit-capped |", "> **Note:** Amazon Q Developer Pro is now a legacy plan in the Kiro ecosystem. AWS has stopped allowing new Builder ID subscriptions to Q Developer Pro — new users can only subscribe through Kiro plans. The undocumented usage limits on Q Pro are likely part of why AWS made this transition. If you're already on Q Developer Pro, you retain access and it remains the better deal for OpenClaw.", "Your Q Developer Pro subscription grants access to `kiro-cli`. The documented quota is [10,000 inference calls/month](https://docs.aws.amazon.com/general/latest/gr/amazonqdev.html) — for a personal AI assistant, that's more than enough.", "> **Real-world cost check:** In 4 days of active OpenClaw usage after switching to kiro-gateway, I consumed ~40M input tokens and ~865K output tokens with Claude Sonnet. OpenClaw loads memory files, system prompts, and tool results into every turn — the context window fills up fast. At standard Bedrock pricing ($3/1M input, $15/1M output), that's ~$135 for 4 days, or roughly **$1,000/month**. Q Developer Pro covers all of it for $19/month flat.", "In practice, I've been running Kiro CLI with OpenClaw daily and haven't hit any rate limits in active use. Note: the `/usage` command isn't available under the Q Developer Pro plan — monitor your usage via the AWS console instead. That said, after running OpenClaw with kiro-gateway for several days, I checked the Q Developer usage metrics in the AWS console and the figures hadn't moved at all. It's unclear whether Kiro CLI usage is counted against the same quota as Q Developer's agentic requests, or tracked separately. The [Amazon Q Developer pricing page](https://aws.amazon.com/q/developer/pricing/) only states \"Included (with limits)\" for the Pro tier — no specifics on what those limits are or how Kiro CLI calls are metered.", "> **Note:** Q Developer Pro requires [AWS IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html) (SSO) — you can't use it with a free Builder ID. If you're already set up with Identity Center (common in enterprise teams and AWS Community Builders with corporate accounts), you're good to go.", "> **Important:** Standard AWS Credits don't cover per-token Claude usage via Anthropic's marketplace agreement. But the Q Developer Pro subscription fee itself **is** credit-eligible — making the whole stack fundable with AWS credits. Kiro's flat-rate subscription is currently the only practical way to run Claude in OpenClaw without per-token billing.", "> **New AWS accounts:** Even if you'd prefer to pay per-token via direct Bedrock API, new accounts often come with [ultra-low default rate limits](https://dev.to/aws-builders/ultra-low-bedrock-llm-rate-limits-for-new-aws-accounts-time-to-wake-up-your-inactive-aws-accounts-3no0) that can't reliably serve OpenClaw — even when you're willing to pay. The flat-rate Q Developer Pro route sidesteps this entirely.", "[kiro-gateway](https://github.com/jwadow/kiro-gateway) — built by [@Jwadow](https://github.com/Jwadow) — wraps Kiro CLI and exposes OpenAI-compatible and Anthropic-compatible API endpoints. OpenClaw talks to it like any other provider.", "Run `kiro-cli login` once to authenticate — this populates `KIRO_CREDS_FILE` automatically. (`kiro-cli` is only needed for this initial login; `kiro-gateway` reads the token it generates. Re-run if your token expires.) Then:", "> **Heads up:** kiro-gateway's hardcoded fallback model list may lag behind new Claude releases. If a model isn't showing up at `/v1/models`, add it manually to `FALLBACK_MODELS` in `kiro/config.py`.", "Available models via Q Developer Pro:", "| Model | Best for | |-------|---------| | `claude-sonnet-4.6` | General tasks, coding, writing | | `claude-haiku-4.5` | Fast, lightweight responses | | `claude-opus-4.6` | Complex reasoning, long context |", "> **Bonus:** kiro-gateway works with any tool that supports OpenAI or Anthropic APIs — not just OpenClaw. To use it with Claude Code: `ANTHROPIC_BASE_URL=http://localhost:9000` and `ANTHROPIC_API_KEY=your-secret-key`.", "Layer 2: Memory Search — Bedrock Embeddings — Covered by AWS Credits", "OpenClaw's `memory_search` needs an embedding model. [Amazon Nova Multimodal Embeddings](https://docs.aws.amazon.com/nova/latest/userguide/nova-embeddings.html) costs ~$0.00014 per 1K tokens — fractions of a cent per query, and covered by AWS Credits.", "OpenClaw's native Bedrock provider doesn't wire up embeddings cleanly yet — [PR #24892](https://github.com/openclaw/openclaw/pull/24892) - (I made a novice mistake with [PR #20191](https://github.com/openclaw/openclaw/pull/20191)) is pending merge. Until then, you'll need a local OpenAI-compatible proxy in front of Bedrock. Two options:", "litellm_settings: drop_params: true master_key: \"local-only\" ```", "Option B: bedrock-access-gateway-function-url (serverless, no fixed cost)", "My own fork of the original `bedrock-access-gateway` — deployed as a Lambda Function URL instead of ALB+Fargate, so there's no $16+/month fixed cost. Full writeup: [Use Amazon Bedrock Models with OpenAI SDKs with a Serverless Proxy Endpoint](https://dev.to/aws-builders/use-amazon-bedrock-models-via-an-openai-api-compatible-serverless-endpoint-now-without-fixed-cost-5hf5).", "> **Note:** My [PR #222](https://github.com/aws-samples/bedrock-access-gateway/pull/222) for Nova 2 embedding support against the original `bedrock-access-gateway` project has been merged — so my fork pulls from this upstream automatically via `prepare_source.sh`.", "Grab the `FunctionUrl` output after deploy, then:", "> **Region note:** `amazon.nova-2-multimodal-embeddings-v1:0` availability varies — check the [Bedrock model availability page](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html). Make sure your IAM credentials have `bedrock:InvokeModel` in your target region.", "Once [PR #24892](https://github.com/openclaw/openclaw/pull/24892) merges, no proxy needed — the config simplifies to:", "Layer 3: Web Search — Nova Grounding Proxy — Covered by AWS Credits", "I built [`bedrock-web-search-proxy`](https://github.com/gabrielkoo/bedrock-web-search-proxy) — a FastAPI wrapper that makes Bedrock Nova Grounding look like the Perplexity Sonar API. No Perplexity or Brave API key needed. Runs entirely on AWS Credits.", "Full writeup: [Drop-in Perplexity Sonar Replacement with AWS Bedrock Nova Grounding](https://dev.to/aws-builders/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9).", "Option B: Lambda Function URL (zero idle cost)", "See the [deployment guide in the repo](https://github.com/gabrielkoo/bedrock-web-search-proxy) — SAM-based, arm64, python3.13. Once deployed, you get a persistent HTTPS endpoint with no local process to manage.", "> All US Nova CRIS (Cross-Region Inference Services) profiles support web grounding (`us.amazon.nova-premier-v1:0`, `us.amazon.nova-pro-v1:0`, etc.). Native model IDs without the `us.` prefix do NOT work — must use CRIS profiles. Web grounding is US regions only (us-east-1, us-east-2, us-west-2).", "Layer 4: Cloud Browser — Bedrock AgentCore — Covered by AWS Credits", "[`agent-browser`](https://github.com/vercel-labs/agent-browser) by Vercel Labs, with the AgentCore provider contributed by [Pahud Hsieh](https://github.com/pahud) ([@pahudnet](https://x.com/pahudnet)) — [PR #397](https://github.com/vercel-labs/agent-browser/pull/397).", "The browser runs in AWS — no local Chromium needed. Particularly useful on low-compute instances (Pi, t4g.small) where running a local browser would be too heavy. Covered by AWS Credits.", "Node.js and pnpm required. Since [PR #397](https://github.com/vercel-labs/agent-browser/pull/397) isn't merged yet, check out the branch directly:", "Your AWS identity needs these IAM permissions:", "- `bedrock-agentcore:StartBrowserSession` - `bedrock-agentcore:ConnectBrowserAutomationStream` - `bedrock-agentcore:StopBrowserSession`", "> On a desktop machine with enough RAM, local CDP (OpenClaw's built-in browser) is free and works fine. AgentCore is the play for headless/low-compute setups.", "Without this setup, Claude Sonnet alone runs ~**$1,000/month** at standard Bedrock pricing — based on real token usage from my own sessions. OpenClaw's large context window (memory files, system prompts, tool results loaded every turn) means the token bill compounds fast.", "The full stack with this setup runs at **~$20/month**:", "- **$19/mo** — Amazon Q Developer Pro (flat-rate, covers all LLM calls) - **≤$1/mo** — Bedrock embeddings for memory search (Nova 2 at $0.00014/1K tokens)", "Web search and browser automation are covered by AWS Credits — no separate line item.", "With **$100 in AWS Credits**, you cover roughly **5 months** of the full stack. Both the Q Developer Pro subscription and Bedrock embeddings are credit-eligible — if you're an AWS Community Builder, that $500/year allocation more than covers it.", "- **AWS event participant/speaker** — re:Invent, Summit, local user groups - **AWS Community Builder** — $500/year for active builders ([builder.aws.com](https://builder.aws.com/content/32g2lQ7kc3Py8kKIYGS15Pe8VSS/aws-community-builders-program)). The application opens a few rounds per year — I'm one of the builders in the program. - **AWS Customer Council** — participation typically includes credits - **AWS Activate** (startups) — up to $100K - **AWS Educate / Academy** — educators and students", "Check your balance: [console.aws.amazon.com/billing/home#/credits](https://console.aws.amazon.com/billing/home#/credits)", "Four layers. Two built by community members, two I built myself. All open source, all running on AWS Credits.", "To be clear: **kiro-gateway is the most crucial piece here.** [@Jwadow](https://github.com/Jwadow) built the bridge that makes Claude accessible without per-token billing — I built the embedding proxy and web search proxy to fill the remaining gaps. [Web search](https://github.com/gabrielkoo/bedrock-web-search-proxy) and [cloud browser](https://github.com/vercel-labs/agent-browser) (Layers 3 and 4) are purely AWS Credits — no subscription, per-token billing well covered by AWS Credits.", "If you're already an AWS Community Builder or have credits sitting in your account, there's no reason to be paying per-token for a personal AI assistant. Wire it up once, and the stack runs itself." ] }, { "id": "article:drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9", "source_type": "article", "title": "Drop-in Perplexity Sonar Replacement with AWS Bedrock Nova Grounding", "url": "https://gabrielkoo.com/blog/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9/", "canonical_url": "https://dev.to/aws-builders/drop-in-perplexity-sonar-replacement-with-aws-bedrock-nova-grounding-35o9", "published_at": "2026-02-20", "last_verified_at": "2026-08-23", "tags": [ "aws", "bedrock", "python", "webdev" ], "description": "If you're running an AI assistant or agent framework that uses Perplexity's Sonar API for web search,...", "content": "If you're running an AI assistant or agent framework that uses Perplexity's Sonar API for web search, you're paying per query — or burning through your monthly credit allocation faster than you'd like.\n\nI'm on Perplexity Pro, which comes with $5/month in API credits. Sounds fine until you hit mid-month and realize OpenClaw has quietly burned through all of it. I wanted something uncapped that didn't add another bill. If you're an AWS user with any credits sitting around — that $25 from a workshop, an event promo, or re:Invent swag — there's a better option: route those queries through Amazon Bedrock's Nova Premier grounding instead.\n\nI built [`bedrock-web-search-proxy`](https://github.com/gabrielkoo/bedrock-web-search-proxy), a FastAPI proxy that makes Bedrock Nova Premier look exactly like the Perplexity Sonar API. Change one URL, keep everything else the same.\n\n## What is Nova Grounding?\n\nAmazon Nova Premier supports a `nova_grounding` system tool that lets the model search the web in real-time and return answers with citations — similar to Perplexity Sonar. The difference: it runs on Bedrock, so it counts against your AWS credits rather than a separate Perplexity subscription.\n\n## Why Not Just Use Brave Search's Free Tier?\n\nBrave does have an AI Answers API that returns synthesized answers with citations — similar to Perplexity. Two catches though:\n\n1. **Credit card required** — even the $5/month free tier needs a card on file as an anti-fraud measure\n2. **Undocumented model** — Brave doesn't clearly disclose which LLM powers the answers, so you're trusting a black box\n\nWith Nova grounding, you know exactly what's running (Nova Premier on Bedrock), and it counts against AWS credits you likely already have. No new billing relationship, no mystery model.\n\n## Apps That Use Perplexity API\n\nThe wrapper is a drop-in for any app that supports Perplexity as a provider:\n\n- **OpenClaw** — `tools.web.search.perplexity.baseUrl` config\n- **Open WebUI** — web search integration\n- **LibreChat** — via Perplexity MCP server\n- **Cursor** — Perplexity MCP for web research\n- **Continue.dev** — Sonar models for codebase context\n- **AnythingLLM** — Perplexity as cloud LLM provider\n- **LiteLLM** — web search interception\n\n## Proof It's Actually Grounded (Not Hallucinated)\n\nHere's a direct API call asking for the current Bitcoin price:\n\n```bash\ncurl -s http://localhost:7000/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"nova-premier-web-grounding\",\n \"messages\": [{\"role\": \"user\", \"content\": \"What is the Bitcoin price right now?\"}],\n \"max_tokens\": 200\n }'\n```\n\nResponse:\n\n```json\n{\n \"choices\": [{\n \"message\": {\n \"content\": \"The current price of Bitcoin (BTC) is $67,254.57 USD, reflecting a 0.54% increase in the last 24 hours. Last updated February 20, 2026 at 04:34 UTC.\"\n }\n }],\n \"citations\": [\n \"https://www.latestly.com/technology/bitcoin-price-today-february-20-2026-btc-price-at-usd-67243-up-compared-to-yesterdays-usd-66941-mark-7321498.html\",\n \"https://www.binance.com/en/price/bitcoin\"\n ]\n}\n```\n\nThe citation URL contains today's date in the slug. Not hallucinated — Nova Premier actually fetched and synthesized live web content.\n\n## Setup\n\n### 1. Install and run (one line, no cloning needed)\n\n```bash\nuvx --from git+https://github.com/gabrielkoo/bedrock-web-search-proxy bedrock-web-search-proxy\n```\n\nOr run directly from the raw script:\n\n```bash\nuv run https://raw.githubusercontent.com/gabrielkoo/bedrock-web-search-proxy/main/main.py\n```\n\nBoth require [uv](https://docs.astral.sh/uv/) and AWS credentials with `bedrock:InvokeModel` on `us.amazon.nova-premier-v1:0`. Region defaults to `us-east-1` — override with `AWS_DEFAULT_REGION` if needed.\n\n### 2. Configure your app\n\nFor **OpenClaw**, update `~/.openclaw/openclaw.json`:\n\n```json\n{\n \"tools\": {\n \"web\": {\n \"search\": {\n \"provider\": \"perplexity\",\n \"perplexity\": {\n \"baseUrl\": \"http://localhost:7000/v1\",\n \"apiKey\": \"nova-grounding\",\n \"model\": \"nova-premier-web-grounding\"\n }\n }\n }\n }\n}\n```\n\n> ⚠️ The `apiKey` must **not** be a real `pplx-` key — OpenClaw detects that prefix and overrides `baseUrl` back to Perplexity's servers.\n\nFor other apps, just point the Perplexity base URL to `http://your-host:7000/v1` and use any model name — the wrapper routes everything to Nova Premier.\n\n## Model Aliases\n\nAll standard Perplexity model names are accepted and routed to Nova Premier (the only Nova model that currently supports the grounding tool):\n\n| Request model | Bedrock model |\n|---|---|\n| `nova-premier-web-grounding` | `us.amazon.nova-premier-v1:0` |\n| `sonar-pro`, `sonar-pro-online` | `us.amazon.nova-premier-v1:0` |\n| `sonar`, `sonar-mini`, `sonar-turbo` | `us.amazon.nova-premier-v1:0` |\n\n## Cost\n\nNova Premier usage counts against your AWS credits — so if you have any sitting around (a $25 promo from an AWS event, workshop, or re:Invent swag bag), this is effectively free. Check your [Billing console](https://console.aws.amazon.com/billing/home#/credits) — you might have more than you think.\n\n- **AWS Community Builders**: covered by $500/year credits\n- **Others with AWS credits**: same deal — credits apply\n- **No credits**: check the [Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/) for current Nova Premier rates\n\n## Caveats\n\n- **Streaming doesn't return `citations[]`** — Nova limitation. Non-streaming works fine, and OpenClaw's `web_search` tool uses non-streaming.\n- **`MAX_CONCURRENT` semaphore** defaults to 5 — tune via env var if needed.\n- **Region**: Nova Premier grounding requires `us-east-1`.\n\n## Wrapping Up\n\nIf you're already on AWS Bedrock for your LLM workloads, there's no reason to pay Perplexity separately for web-grounded search. The wrapper is ~350 lines of Python, has 44 tests, and is OpenAI SDK-compatible — so it works with anything that speaks the Perplexity or OpenAI chat completions API.\n\nRepo: [github.com/gabrielkoo/bedrock-web-search-proxy](https://github.com/gabrielkoo/bedrock-web-search-proxy)", "excerpts": [ "If you're running an AI assistant or agent framework that uses Perplexity's Sonar API for web search, you're paying per query — or burning through your monthly credit allocation faster than you'd like.", "I'm on Perplexity Pro, which comes with $5/month in API credits. Sounds fine until you hit mid-month and realize OpenClaw has quietly burned through all of it. I wanted something uncapped that didn't add another bill. If you're an AWS user with any credits sitting around — that $25 from a workshop, an event promo, or re:Invent swag — there's a better option: route those queries through Amazon Bedrock's Nova Premier grounding instead.", "I built [`bedrock-web-search-proxy`](https://github.com/gabrielkoo/bedrock-web-search-proxy), a FastAPI proxy that makes Bedrock Nova Premier look exactly like the Perplexity Sonar API. Change one URL, keep everything else the same.", "Amazon Nova Premier supports a `nova_grounding` system tool that lets the model search the web in real-time and return answers with citations — similar to Perplexity Sonar. The difference: it runs on Bedrock, so it counts against your AWS credits rather than a separate Perplexity subscription.", "Why Not Just Use Brave Search's Free Tier?", "Brave does have an AI Answers API that returns synthesized answers with citations — similar to Perplexity. Two catches though:", "1. **Credit card required** — even the $5/month free tier needs a card on file as an anti-fraud measure 2. **Undocumented model** — Brave doesn't clearly disclose which LLM powers the answers, so you're trusting a black box", "With Nova grounding, you know exactly what's running (Nova Premier on Bedrock), and it counts against AWS credits you likely already have. No new billing relationship, no mystery model.", "The wrapper is a drop-in for any app that supports Perplexity as a provider:", "- **OpenClaw** — `tools.web.search.perplexity.baseUrl` config - **Open WebUI** — web search integration - **LibreChat** — via Perplexity MCP server - **Cursor** — Perplexity MCP for web research - **Continue.dev** — Sonar models for codebase context - **AnythingLLM** — Perplexity as cloud LLM provider - **LiteLLM** — web search interception", "Proof It's Actually Grounded (Not Hallucinated)", "Here's a direct API call asking for the current Bitcoin price:", "The citation URL contains today's date in the slug. Not hallucinated — Nova Premier actually fetched and synthesized live web content.", "1. Install and run (one line, no cloning needed)", "Or run directly from the raw script:", "Both require [uv](https://docs.astral.sh/uv/) and AWS credentials with `bedrock:InvokeModel` on `us.amazon.nova-premier-v1:0`. Region defaults to `us-east-1` — override with `AWS_DEFAULT_REGION` if needed.", "For **OpenClaw**, update `~/.openclaw/openclaw.json`:", "> ⚠️ The `apiKey` must **not** be a real `pplx-` key — OpenClaw detects that prefix and overrides `baseUrl` back to Perplexity's servers.", "For other apps, just point the Perplexity base URL to `http://your-host:7000/v1` and use any model name — the wrapper routes everything to Nova Premier.", "All standard Perplexity model names are accepted and routed to Nova Premier (the only Nova model that currently supports the grounding tool):", "| Request model | Bedrock model | |---|---| | `nova-premier-web-grounding` | `us.amazon.nova-premier-v1:0` | | `sonar-pro`, `sonar-pro-online` | `us.amazon.nova-premier-v1:0` | | `sonar`, `sonar-mini`, `sonar-turbo` | `us.amazon.nova-premier-v1:0` |", "Nova Premier usage counts against your AWS credits — so if you have any sitting around (a $25 promo from an AWS event, workshop, or re:Invent swag bag), this is effectively free. Check your [Billing console](https://console.aws.amazon.com/billing/home#/credits) — you might have more than you think.", "- **AWS Community Builders**: covered by $500/year credits - **Others with AWS credits**: same deal — credits apply - **No credits**: check the [Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/) for current Nova Premier rates", "- **Streaming doesn't return `citations[]`** — Nova limitation. Non-streaming works fine, and OpenClaw's `web_search` tool uses non-streaming. - **`MAX_CONCURRENT` semaphore** defaults to 5 — tune via env var if needed. - **Region**: Nova Premier grounding requires `us-east-1`.", "If you're already on AWS Bedrock for your LLM workloads, there's no reason to pay Perplexity separately for web-grounded search. The wrapper is ~350 lines of Python, has 44 tests, and is OpenAI SDK-compatible — so it works with anything that speaks the Perplexity or OpenAI chat completions API.", "Repo: [github.com/gabrielkoo/bedrock-web-search-proxy](https://github.com/gabrielkoo/bedrock-web-search-proxy)" ] }, { "id": "article:aws-silently-releases-kimi-k25-and-glm-47-models-to-bedrock-1514", "source_type": "article", "title": "AWS Silently Releases Kimi K2.5 and GLM 4.7 Models to Bedrock", "url": "https://gabrielkoo.com/blog/aws-silently-releases-kimi-k25-and-glm-47-models-to-bedrock-1514/", "canonical_url": "https://dev.to/aws-builders/aws-silently-releases-kimi-k25-and-glm-47-models-to-bedrock-1514", "published_at": "2026-02-08", "last_verified_at": "2026-08-23", "tags": [ "aws", "vibecoding", "ai" ], "description": "[UPDATE 10 Feb 2026] - It seems it was part of AWS’s rollout plan for rolling out open weight models...", "content": "**[UPDATE 10 Feb 2026]** - It seems it was part of AWS’s rollout plan for rolling out open weight models for Kiro and Kiro CLI! [Open weight models are here: more choice, more speed, less cost](https://kiro.dev/blog/open-weight-models/)\n\nBut interestingly among the models covered in this article, only DeepSeek v3.2, Minimax 2.1 and Qwen Coder next were covered. Moonshot K2.5 and GLM-4.7 were missing.\n————\nI was refreshing my [Bedrock model catalog](https://amazonbedrockmodels.github.io) script our of random curiosity when a few unfamiliar model IDs showed up in us-east-1. No AWS blog post. No tweet thread. Just a few new entries in the API response.\n\n**If you've been waiting for a Claude-adjacent model you could swap in seamlessly via AWS credits — this is it.**\n\nBut there’s a drawback for early adopters - Do read till the end to learn about the flaw!\n\n**Kimi K2.5** (by Moonshot AI), **GLM 4.7** (by Zhipu AI), and several other new models like DeepSeek 3.2 and Qwen3 Coder Next are now live on Bedrock, all with full support for the Converse API, tool calling, and — in Kimi K2.5's case — native image understanding.\n\n![Image description](/assets/img/44497e3244dd.png)\n\n![Image description](/assets/img/3b89e0bed041.png)\n\n**Quick note:** These models aren't listed in the [AWS Bedrock models-supported documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), yet they're fully functional via the Converse API. That's the whole \"silent release\" thing — available in production, just not reflected in the canonical docs yet. Worth bookmarking your region's actual model list from the Bedrock console instead of relying solely on the written guides.\n\n## The models: what just landed\n\n![Image description](/assets/img/e1fb65a68813.png)\n\n**Kimi K2.5** ([Moonshot AI blog](https://www.kimi.com/blog/kimi-k2.5.html)) is the eye-catcher here:\n\n- **Tool calling (function calling):** ✓ Fully supported via Bedrock Converse API\n- **Image understanding:** ✓ Native image inputs (base64 or URL)\n- **Code generation:** In my testing, it held its own against Claude 4.5 Sonnet on typical coding prompts — it handled a multi-file refactor of a FastAPI router cleanly on the first try\n- **Bedrock Model ID:** `moonshotai.kimi-k2.5`\n- **Availability:** us-east-1, us-west-2 (and expanding)\n- **Use case fit:** Drop-in replacement for Claude if you're already on AWS credits\n\n**GLM 4.7** ([Zhipu AI blog](https://z.ai/blog/glm-4.7)) fills a quieter but useful role:\n\n- **Tool calling:** ✓ Supported, though less aggressively tested in my flows\n- **Code generation:** Strong; competitive with Deepseek for certain workloads\n- **Bedrock Model ID:** `zai.glm-4.7(-flash)`\n- **Availability:** us-east-1, us-west-2\n- **Use case fit:** Solid all-arounder; good for prompts that don't strictly require image handling\n\n**The real unlock:** Both are live on `converse` API, which means they work seamlessly with Bedrock's function-calling infrastructure. \n\n### When to pick which\n\n| Need | Pick | Why |\n|------|------|-----|\n| Image understanding + tool calling | **Kimi K2.5** | Only Bedrock open-weight flagship model with both |\n| Text-only tasks, cost-conscious | **GLM 4.7** | Solid all-arounder, no vision overhead |\n| Maximum reliability & ecosystem | **Claude** | Battle-tested, widest documentation |\n\n## Why this matters for vibe coding\n\n*\"Vibe coding\"* — the practice of rapidly iterating on code with LLM assistance, swapping models mid-session, and optimizing for flow over perfection — lives or dies on how frictionless your model-switching is.\n\nIf you're sitting on expiring AWS credits (I've got ~$700 by July 2026), the bottleneck isn't usually \"which model is smartest?\" — it's \"how fast can I swap without rewriting everything?\"\n\nKimi K2.5 solves a real pain point: until now, if you wanted image understanding + tool calling + AWS-native billing, you were stuck with Claude. And the only way you could have done so is via purchasing Kiro CLI subscription ($20/$40/$200 per month) - Since Anthropic Claude models are not covered by the typical AWS Credits.\n\nI initially considered subscribing to the $200 Kiro Power plan, but then I am not confident that I could utilize the plan fully every month, but I'm worried that sticking to the $20/$40 plans will result to paying for Kiro credit overages (which is double of the average plan price per credit). Therefore any PAYG option would perfectly fit my usage pattern.\n\nSo now you have an option that:\n\n1. **Bills directly to your AWS account** — no vendor intermediary, no separate API key, just your existing credits burning down\n2. Runs on the same Bedrock Converse API \n3. Calls tools reliably\n4. Natively understands images\n\nFor experimentation loops (refactors, code generation, visual analysis), that's a genuinely useful escape hatch.\n\n## The lightweight setup: local LiteLLM gateway\n\nYou don't need a complex setup. My entire gateway is:\n\n- A Python venv with `litellm` installed\n- A single YAML config file (shown in the next section)\n- A systemd unit to keep it running on port 4000\n\nNo containers, no Kubernetes. One command to install, one service file to manage. Once running, any client on that machine calls `http://localhost:4000/chat/completions` with the standard OpenAI format, and LiteLLM translates it to Bedrock Converse API automatically.\n\n**Performance note:** In my testing, the LiteLLM translation layer adds negligible latency (~20–50ms overhead). Streaming responses from Kimi K2.5 feel comparable to calling Claude directly — first tokens arrive within 1–2 seconds for typical prompts.\n\n## Bonus: Claude Code & OpenCode integration\n\nHere's the slightly cheeky part: you can point [Claude Code](https://docs.anthropic.com/en/docs/claude-code) or [OpenCode](https://opencode.ai/) at your local LiteLLM gateway and route requests through to Kimi K2.5 or GLM 4.7 on Bedrock — all while staying on your AWS credits.\n\nLiteLLM supports the Anthropic `/v1/messages` API endpoint, so it's a two-liner to set up:\n\n```bash\nexport ANTHROPIC_BASE_URL=http://localhost:4000\nexport ANTHROPIC_AUTH_TOKEN=sk-your-litellm-key\nexport ANTHROPIC_MODEL=kimi-k2.5\nexport DISABLE_PROMPT_CACHING=true\n```\n\nThe `DISABLE_PROMPT_CACHING=true` is essential here (Special thanks my colleague _**[at]Marty**_ for troubleshooting and fixing that) - since by default Claude Code tries to apply prompt caching for speed, but not every model on Amazon Bedrock supports that:\n\n![Image description](/assets/img/b70d22146480.png)\n\n\n\nThen launch Claude Code or OpenCode as usual. LiteLLM intercepts the Anthropic-format requests and translates them to Bedrock Converse calls. It's not officially blessed by Anthropic, but it works cleanly for local experimentation — and your AWS credits take the hit instead of your Anthropic billing.\n\n![Image description](/assets/img/00e6c23d9623.png)\n\n\n\n## Config: explicit about capabilities\n\nHere's how I route Kimi K2.5 and GLM 4.7:\n\n```yaml\nmodel_list:\n - model_name: kimi-k2.5\n litellm_params:\n model: bedrock/converse/moonshotai.kimi-k2.5\n aws_region_name: us-east-1\n allowed_openai_params: ['reasoning_effort', 'tools', 'tool_choice']\n model_info:\n mode: completion\n\n - model_name: glm-4.7\n litellm_params:\n model: bedrock/converse/zai.glm-4.7\n aws_region_name: us-east-1\n allowed_openai_params: ['reasoning_effort', 'tools', 'tool_choice'] \n model_info:\n mode: completion\n\nlitellm_settings:\n modify_params: true\n log_responses: true\n```\n\nKey patterns here:\n\n- **Friendly names** (`kimi-k2.5`, `glm-4.7`) instead of long model IDs\n- **Explicit capability flags** (`supports_function_calling`, `supports_vision`)\n- **`modify_params: true`** for Bedrock edge-case smoothing\n- **Single region** (us-east-1) since both models are there\n\nThe capability flags matter. They let your orchestration layer (or agent framework) gracefully degrade if a model can't do tools or images. No more \"half-attempt to call a function and fail mysteriously.\"\n\n## Testing: quick verification\n\nI tested both models with the Bedrock Converse API in us-east-1. Here's what actually happened:\n\n**Kimi K2.5:** I threw a \"get current weather in Tokyo\" tool spec at it — standard JSON Schema function definition, nothing fancy. It correctly structured the function call on the first attempt, including proper argument types in the response. For code generation, I asked it to refactor a Python CLI script into async; the output was clean and ran without edits. Image support is declared in the model schema but I haven't validated it hands-on yet — that's next on my list.\n\n**GLM 4.7:** Solid on text queries and code generation. Tool calling works, though it was slightly less eager to invoke tools unprompted compared to Kimi — it sometimes answered directly when I expected a function call. No image support, as expected; Zhipu hasn't added vision capabilities to GLM 4.7.\n\n## Why the quiet release?\n\nThese aren't show-stopping announcements. They're bread-and-butter additions to Bedrock's model portfolio. AWS likely brought them in as part of an ongoing expansion to reduce vendor lock-in on \"you have to use Claude for everything.\" That's healthy — more options, better pricing pressure, cleaner credit utilization.\n\nThis is a pattern, not an anomaly. Anthropic Claude, AI21 Jamba, and several Mistral variants all appeared on Bedrock before official blog posts or documentation updates. If you're only checking AWS launch announcements, you're always behind.\n\n> **📌 That's exactly why I built [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io)** — a living catalog of what's *actually* available on Bedrock, in which regions, and what each model can do. Bookmark it. It updates faster than the docs.\n\n## The model-swapping checklist\n\nIf you want to swap models without rewriting your code:\n\n1. **Use a gateway** (LiteLLM, LLMProxy, or similar) to normalize requests\n2. **Pin the Bedrock route explicitly** (`bedrock/converse/modelid`) in your config\n3. **Mark capability per model** (tool calling, vision, etc.) — don't assume\n4. **Test the tool spec** — even \"supported\" models sometimes have quirky implementations\n5. **Keep a catalog** so you don't rediscover the same model twice\n\nKimi K2.5 fits this playbook cleanly. It's a genuine Claude replacement for Bedrock users, not a \"wait and see if it works\" experiment.\n\n## Next steps\n\n- **If you're on AWS credits:** Spin up a local LiteLLM instance and try both models\n- **If you find more quietly-available models:** Open a PR against the catalog or message me\n- **If you're in an AWS org:** Check your Bedrock region — availability is still expanding\n\nThe AWS credits will expire whether you use them or not. Might as well pick models that fit your workflow instead of forcing your workflow around Kiro CLI only.\n\n---\n\nP.S. During my testing, indeed I observed occasional longer LLM inference times - but I guess AWS is working on providing more compute capability on these newer beta models.\n\n---\n\n## References\n\n[1] [Moonshot AI — Kimi K2.5 Announcement](https://www.kimi.com/blog/kimi-k2.5.html)\n[2] [Zhipu AI — GLM 4.7 Announcement](https://z.ai/blog/glm-4.7)\n[3] [AWS Bedrock — Converse API Reference](https://docs.aws.amazon.com/cli/latest/reference/bedrock-runtime/converse.html)\n[4] [LiteLLM — AWS Bedrock Provider Documentation](https://docs.litellm.ai/docs/providers/bedrock)\n[5] [Unofficial - Amazon Bedrock Model Catalog I Created](https://amazonbedrockmodels.github.io)", "excerpts": [ "**[UPDATE 10 Feb 2026]** - It seems it was part of AWS’s rollout plan for rolling out open weight models for Kiro and Kiro CLI! [Open weight models are here: more choice, more speed, less cost](https://kiro.dev/blog/open-weight-models/)", "But interestingly among the models covered in this article, only DeepSeek v3.2, Minimax 2.1 and Qwen Coder next were covered. Moonshot K2.5 and GLM-4.7 were missing. ———— I was refreshing my [Bedrock model catalog](https://amazonbedrockmodels.github.io) script our of random curiosity when a few unfamiliar model IDs showed up in us-east-1. No AWS blog post. No tweet thread. Just a few new entries in the API response.", "**If you've been waiting for a Claude-adjacent model you could swap in seamlessly via AWS credits — this is it.**", "But there’s a drawback for early adopters - Do read till the end to learn about the flaw!", "**Kimi K2.5** (by Moonshot AI), **GLM 4.7** (by Zhipu AI), and several other new models like DeepSeek 3.2 and Qwen3 Coder Next are now live on Bedrock, all with full support for the Converse API, tool calling, and — in Kimi K2.5's case — native image understanding.", "![Image description](/assets/img/44497e3244dd.png)", "![Image description](/assets/img/3b89e0bed041.png)", "**Quick note:** These models aren't listed in the [AWS Bedrock models-supported documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), yet they're fully functional via the Converse API. That's the whole \"silent release\" thing — available in production, just not reflected in the canonical docs yet. Worth bookmarking your region's actual model list from the Bedrock console instead of relying solely on the written guides.", "![Image description](/assets/img/e1fb65a68813.png)", "**Kimi K2.5** ([Moonshot AI blog](https://www.kimi.com/blog/kimi-k2.5.html)) is the eye-catcher here:", "- **Tool calling (function calling):** ✓ Fully supported via Bedrock Converse API - **Image understanding:** ✓ Native image inputs (base64 or URL) - **Code generation:** In my testing, it held its own against Claude 4.5 Sonnet on typical coding prompts — it handled a multi-file refactor of a FastAPI router cleanly on the first try - **Bedrock Model ID:** `moonshotai.kimi-k2.5` - **Availability:** us-east-1, us-west-2 (and expanding) - **Use case fit:** Drop-in replacement for Claude if you're already on AWS credits", "**GLM 4.7** ([Zhipu AI blog](https://z.ai/blog/glm-4.7)) fills a quieter but useful role:", "- **Tool calling:** ✓ Supported, though less aggressively tested in my flows - **Code generation:** Strong; competitive with Deepseek for certain workloads - **Bedrock Model ID:** `zai.glm-4.7(-flash)` - **Availability:** us-east-1, us-west-2 - **Use case fit:** Solid all-arounder; good for prompts that don't strictly require image handling", "**The real unlock:** Both are live on `converse` API, which means they work seamlessly with Bedrock's function-calling infrastructure.", "| Need | Pick | Why | |------|------|-----| | Image understanding + tool calling | **Kimi K2.5** | Only Bedrock open-weight flagship model with both | | Text-only tasks, cost-conscious | **GLM 4.7** | Solid all-arounder, no vision overhead | | Maximum reliability & ecosystem | **Claude** | Battle-tested, widest documentation |", "Why this matters for vibe coding", "*\"Vibe coding\"* — the practice of rapidly iterating on code with LLM assistance, swapping models mid-session, and optimizing for flow over perfection — lives or dies on how frictionless your model-switching is.", "If you're sitting on expiring AWS credits (I've got ~$700 by July 2026), the bottleneck isn't usually \"which model is smartest?\" — it's \"how fast can I swap without rewriting everything?\"", "Kimi K2.5 solves a real pain point: until now, if you wanted image understanding + tool calling + AWS-native billing, you were stuck with Claude. And the only way you could have done so is via purchasing Kiro CLI subscription ($20/$40/$200 per month) - Since Anthropic Claude models are not covered by the typical AWS Credits.", "I initially considered subscribing to the $200 Kiro Power plan, but then I am not confident that I could utilize the plan fully every month, but I'm worried that sticking to the $20/$40 plans will result to paying for Kiro credit overages (which is double of the average plan price per credit). Therefore any PAYG option would perfectly fit my usage pattern.", "So now you have an option that:", "1. **Bills directly to your AWS account** — no vendor intermediary, no separate API key, just your existing credits burning down 2. Runs on the same Bedrock Converse API 3. Calls tools reliably 4. Natively understands images", "For experimentation loops (refactors, code generation, visual analysis), that's a genuinely useful escape hatch.", "The lightweight setup: local LiteLLM gateway", "You don't need a complex setup. My entire gateway is:", "- A Python venv with `litellm` installed - A single YAML config file (shown in the next section) - A systemd unit to keep it running on port 4000", "No containers, no Kubernetes. One command to install, one service file to manage. Once running, any client on that machine calls `http://localhost:4000/chat/completions` with the standard OpenAI format, and LiteLLM translates it to Bedrock Converse API automatically.", "**Performance note:** In my testing, the LiteLLM translation layer adds negligible latency (~20–50ms overhead). Streaming responses from Kimi K2.5 feel comparable to calling Claude directly — first tokens arrive within 1–2 seconds for typical prompts.", "Bonus: Claude Code & OpenCode integration", "Here's the slightly cheeky part: you can point [Claude Code](https://docs.anthropic.com/en/docs/claude-code) or [OpenCode](https://opencode.ai/) at your local LiteLLM gateway and route requests through to Kimi K2.5 or GLM 4.7 on Bedrock — all while staying on your AWS credits.", "LiteLLM supports the Anthropic `/v1/messages` API endpoint, so it's a two-liner to set up:", "The `DISABLE_PROMPT_CACHING=true` is essential here (Special thanks my colleague _**[at]Marty**_ for troubleshooting and fixing that) - since by default Claude Code tries to apply prompt caching for speed, but not every model on Amazon Bedrock supports that:", "![Image description](/assets/img/b70d22146480.png)", "Then launch Claude Code or OpenCode as usual. LiteLLM intercepts the Anthropic-format requests and translates them to Bedrock Converse calls. It's not officially blessed by Anthropic, but it works cleanly for local experimentation — and your AWS credits take the hit instead of your Anthropic billing.", "![Image description](/assets/img/00e6c23d9623.png)", "Config: explicit about capabilities", "Here's how I route Kimi K2.5 and GLM 4.7:", "- model_name: glm-4.7 litellm_params: model: bedrock/converse/zai.glm-4.7 aws_region_name: us-east-1 allowed_openai_params: ['reasoning_effort', 'tools', 'tool_choice'] model_info: mode: completion", "litellm_settings: modify_params: true log_responses: true ```", "- **Friendly names** (`kimi-k2.5`, `glm-4.7`) instead of long model IDs - **Explicit capability flags** (`supports_function_calling`, `supports_vision`) - **`modify_params: true`** for Bedrock edge-case smoothing - **Single region** (us-east-1) since both models are there", "The capability flags matter. They let your orchestration layer (or agent framework) gracefully degrade if a model can't do tools or images. No more \"half-attempt to call a function and fail mysteriously.\"", "I tested both models with the Bedrock Converse API in us-east-1. Here's what actually happened:", "**Kimi K2.5:** I threw a \"get current weather in Tokyo\" tool spec at it — standard JSON Schema function definition, nothing fancy. It correctly structured the function call on the first attempt, including proper argument types in the response. For code generation, I asked it to refactor a Python CLI script into async; the output was clean and ran without edits. Image support is declared in the model schema but I haven't validated it hands-on yet — that's next on my list.", "**GLM 4.7:** Solid on text queries and code generation. Tool calling works, though it was slightly less eager to invoke tools unprompted compared to Kimi — it sometimes answered directly when I expected a function call. No image support, as expected; Zhipu hasn't added vision capabilities to GLM 4.7.", "These aren't show-stopping announcements. They're bread-and-butter additions to Bedrock's model portfolio. AWS likely brought them in as part of an ongoing expansion to reduce vendor lock-in on \"you have to use Claude for everything.\" That's healthy — more options, better pricing pressure, cleaner credit utilization.", "This is a pattern, not an anomaly. Anthropic Claude, AI21 Jamba, and several Mistral variants all appeared on Bedrock before official blog posts or documentation updates. If you're only checking AWS launch announcements, you're always behind.", "> **📌 That's exactly why I built [amazonbedrockmodels.github.io](https://amazonbedrockmodels.github.io)** — a living catalog of what's *actually* available on Bedrock, in which regions, and what each model can do. Bookmark it. It updates faster than the docs.", "If you want to swap models without rewriting your code:", "1. **Use a gateway** (LiteLLM, LLMProxy, or similar) to normalize requests 2. **Pin the Bedrock route explicitly** (`bedrock/converse/modelid`) in your config 3. **Mark capability per model** (tool calling, vision, etc.) — don't assume 4. **Test the tool spec** — even \"supported\" models sometimes have quirky implementations 5. **Keep a catalog** so you don't rediscover the same model twice", "Kimi K2.5 fits this playbook cleanly. It's a genuine Claude replacement for Bedrock users, not a \"wait and see if it works\" experiment.", "- **If you're on AWS credits:** Spin up a local LiteLLM instance and try both models - **If you find more quietly-available models:** Open a PR against the catalog or message me - **If you're in an AWS org:** Check your Bedrock region — availability is still expanding", "The AWS credits will expire whether you use them or not. Might as well pick models that fit your workflow instead of forcing your workflow around Kiro CLI only.", "P.S. During my testing, indeed I observed occasional longer LLM inference times - but I guess AWS is working on providing more compute capability on these newer beta models.", "[1] [Moonshot AI — Kimi K2.5 Announcement](https://www.kimi.com/blog/kimi-k2.5.html) [2] [Zhipu AI — GLM 4.7 Announcement](https://z.ai/blog/glm-4.7) [3] [AWS Bedrock — Converse API Reference](https://docs.aws.amazon.com/cli/latest/reference/bedrock-runtime/converse.html) [4] [LiteLLM — AWS Bedrock Provider Documentation](https://docs.litellm.ai/docs/providers/bedrock) [5] [Unofficial - Amazon Bedrock Model Catalog I Created](https://amazonbedrockmodels.github.io)" ] }, { "id": "article:ultra-low-bedrock-llm-rate-limits-for-new-aws-accounts-time-to-wake-up-your-inactive-aws-accounts-3no0", "source_type": "article", "title": "Ultra Low Bedrock LLM Rate Limits for New AWS Accounts? Time to Wake Up Your Inactive AWS Accounts!", "url": "https://gabrielkoo.com/blog/ultra-low-bedrock-llm-rate-limits-for-new-aws-accounts-time-to-wake-up-your-inactive-aws-accounts-3no0/", "canonical_url": "https://dev.to/aws-builders/ultra-low-bedrock-llm-rate-limits-for-new-aws-accounts-time-to-wake-up-your-inactive-aws-accounts-3no0", "published_at": "2025-11-26", "last_verified_at": "2026-08-23", "tags": [ "bedrock", "ai", "aws" ], "description": "Are You Struggling With Amazon Bedrock’s Ultra-Low Quotas on New AWS Accounts? 🤯 Are you...", "content": "## Are You Struggling With Amazon Bedrock’s Ultra-Low Quotas on New AWS Accounts? 🤯\n\nAre you hitting painfully low rate limits when running LLMs on Amazon Bedrock from a newly created AWS account? You’re definitely not alone — many developers are discovering that new accounts often start with **extremely restrictive quotas**, sometimes as low as **2 requests per minute**.\n\nOfficial guidance usually suggests contacting an account manager to escalate your limits, but for startups, hobby projects, or personal experimentation, that path is far from simple.\n\n## New Account? Big Ambitions, Tiny Quota 🤏🚧\n\nSince 2024 or maybe 2025, AWS quietly adjusted Bedrock’s default model access for newly created accounts - together with other lower account defaults such as a maximum concurreny of 10 for AWS Lambda. Even when using global endpoints, many fresh accounts get **just a few requests per minute** (e.g., 2 rpm for Claude 4.5 Sonnet). This severely slows down prototyping or early-stage AI development.\n\nMeanwhile, **older AWS accounts** — even ones that never touched Bedrock before — often start with dramatically higher limits, approaching **200+ rpm** for the exact same models.\n\n![Sonnet 4.5 Rates](/assets/img/21649e128927.png)\n\nThis creates a real operational advantage for teams with access to aged accounts.\n\n## The Elder Account Advantage 🕰️✨\n\nAWS appears to apply significantly stricter defaults to newer accounts while preserving far more permissive limits for older ones. This aligns with AWS’s long-standing pattern of maintaining stable experiences for long-time customers.\n\nIn practice, a dormant but years-old AWS account can immediately receive much **higher Bedrock limits** purely due to its age.\n\nBelow is a striking example: **3 rpm** for new accounts vs **250 rpm** for older accounts on Claude 4.5 Opus.\n\n![Opus 4.5 Rates](/assets/img/ad1a563932cb.png)\n\n## Why AWS Is Unlikely to Reduce Older Accounts’ Limits 🔒🏢\n\nReducing quotas for older accounts would create major risk and break expectations for long-standing customers — especially enterprises.\n\n* Many organizations have stable, long-lived workloads.\n* Retroactively lowering quotas could break pipelines and violate performance assumptions.\n* AWS historically avoids backward-incompatible changes unless absolutely necessary.\n\nBecause of this, it’s unlikely AWS will apply newer, stricter defaults to older accounts. Teams spinning up new AWS accounts face a much steeper ramp for experimenting with Bedrock.\n\n## Finding and Reusing Older AWS Accounts 🔎📦\n\nIf your organization has older AWS accounts lying around, they may offer instant scaling advantages. With the new **[AWS Organizations Direct Account Transfer](https://aws.amazon.com/about-aws/whats-new/2025/11/aws-organizations-direct-account-transfers/)** feature, accounts can move between Organizations without removing payment methods or performing the old, painful detachment workflow.\n\nWhen moving such accounts, remember to update:\n\n* **Legal entity name**\n* **Root user email**\n* **Addresses and billing contacts**\n* **Tax information**\n\nIf your organization uses [Trusted Access for AWS Account Management](https://docs.aws.amazon.com/accounts/latest/reference/using-orgs-trusted-access.html), these updates are straightforward. Also make sure to audit the account for any leftover resources before dedicating it to GenAI workloads.\n\n\n## Why New Accounts? Why Not Run Everything in One AWS Account? 🧩💼\n\nPutting Bedrock experiments, different production services, and dev workloads into a single account sounds convenient — until it isn’t. Combining everything into one single AWS Account creates unnecessary risk and operational noise.\n\n1. Isolation Protects You 🔥🧱\nAccount boundaries are AWS’s strongest safety net. One bad experiment or IAM mistake shouldn’t touch production. Isolation limits accidental data access, cost spikes, and incident blast radius.\n\n2. Governance Stays Clean 📜✨\nDifferent workloads need different controls. Separate accounts keep audits simpler, give clear ownership, and let you apply SCPs and guardrails without compromise.\n\n3. Costs Stay Transparent 💸📊\nMixing Bedrock prototyping with core services muddies cost reporting. Individual accounts let you track usage, set budgets, and avoid team-to-team disputes.\n\n4. Experiments Move Faster ⚡🔬\nA sandbox (ideally an older account with higher limits) lets you test models, tweak IAM, and push boundaries freely — without risking production stability.\n\n5. It Matches AWS Best Practices 🏗️📚\nAWS recommends multi-account setups for lifecycle separation and blast-radius control. Using older accounts for Bedrock while keeping core workloads isolated follows this playbook perfectly.\n\n\n## How to Break Free From Bedrock’s Slow Lane 🚀💡\n\n* Identify older AWS accounts that haven't been used recently.\n* Transfer them into your AWS Organization using the streamlined Direct Account Transfer workflow.\n* Update all account metadata for compliance.\n* Deploy your Bedrock workloads — and unlock higher default limits instantly.\n\nThis approach helps teams accelerate their AI development journey despite the strict constraints placed on newly created AWS accounts.\n\nHave you seen similar quota differences in your environment? Share your experience — more data points help the community understand the pattern! 🙌", "excerpts": [ "Are You Struggling With Amazon Bedrock’s Ultra-Low Quotas on New AWS Accounts? 🤯", "Are you hitting painfully low rate limits when running LLMs on Amazon Bedrock from a newly created AWS account? You’re definitely not alone — many developers are discovering that new accounts often start with **extremely restrictive quotas**, sometimes as low as **2 requests per minute**.", "Official guidance usually suggests contacting an account manager to escalate your limits, but for startups, hobby projects, or personal experimentation, that path is far from simple.", "New Account? Big Ambitions, Tiny Quota 🤏🚧", "Since 2024 or maybe 2025, AWS quietly adjusted Bedrock’s default model access for newly created accounts - together with other lower account defaults such as a maximum concurreny of 10 for AWS Lambda. Even when using global endpoints, many fresh accounts get **just a few requests per minute** (e.g., 2 rpm for Claude 4.5 Sonnet). This severely slows down prototyping or early-stage AI development.", "Meanwhile, **older AWS accounts** — even ones that never touched Bedrock before — often start with dramatically higher limits, approaching **200+ rpm** for the exact same models.", "![Sonnet 4.5 Rates](/assets/img/21649e128927.png)", "This creates a real operational advantage for teams with access to aged accounts.", "The Elder Account Advantage 🕰️✨", "AWS appears to apply significantly stricter defaults to newer accounts while preserving far more permissive limits for older ones. This aligns with AWS’s long-standing pattern of maintaining stable experiences for long-time customers.", "In practice, a dormant but years-old AWS account can immediately receive much **higher Bedrock limits** purely due to its age.", "Below is a striking example: **3 rpm** for new accounts vs **250 rpm** for older accounts on Claude 4.5 Opus.", "![Opus 4.5 Rates](/assets/img/ad1a563932cb.png)", "Why AWS Is Unlikely to Reduce Older Accounts’ Limits 🔒🏢", "Reducing quotas for older accounts would create major risk and break expectations for long-standing customers — especially enterprises.", "* Many organizations have stable, long-lived workloads. * Retroactively lowering quotas could break pipelines and violate performance assumptions. * AWS historically avoids backward-incompatible changes unless absolutely necessary.", "Because of this, it’s unlikely AWS will apply newer, stricter defaults to older accounts. Teams spinning up new AWS accounts face a much steeper ramp for experimenting with Bedrock.", "Finding and Reusing Older AWS Accounts 🔎📦", "If your organization has older AWS accounts lying around, they may offer instant scaling advantages. With the new **[AWS Organizations Direct Account Transfer](https://aws.amazon.com/about-aws/whats-new/2025/11/aws-organizations-direct-account-transfers/)** feature, accounts can move between Organizations without removing payment methods or performing the old, painful detachment workflow.", "When moving such accounts, remember to update:", "* **Legal entity name** * **Root user email** * **Addresses and billing contacts** * **Tax information**", "If your organization uses [Trusted Access for AWS Account Management](https://docs.aws.amazon.com/accounts/latest/reference/using-orgs-trusted-access.html), these updates are straightforward. Also make sure to audit the account for any leftover resources before dedicating it to GenAI workloads.", "Why New Accounts? Why Not Run Everything in One AWS Account? 🧩💼", "Putting Bedrock experiments, different production services, and dev workloads into a single account sounds convenient — until it isn’t. Combining everything into one single AWS Account creates unnecessary risk and operational noise.", "1. Isolation Protects You 🔥🧱 Account boundaries are AWS’s strongest safety net. One bad experiment or IAM mistake shouldn’t touch production. Isolation limits accidental data access, cost spikes, and incident blast radius.", "2. Governance Stays Clean 📜✨ Different workloads need different controls. Separate accounts keep audits simpler, give clear ownership, and let you apply SCPs and guardrails without compromise.", "3. Costs Stay Transparent 💸📊 Mixing Bedrock prototyping with core services muddies cost reporting. Individual accounts let you track usage, set budgets, and avoid team-to-team disputes.", "4. Experiments Move Faster ⚡🔬 A sandbox (ideally an older account with higher limits) lets you test models, tweak IAM, and push boundaries freely — without risking production stability.", "5. It Matches AWS Best Practices 🏗️📚 AWS recommends multi-account setups for lifecycle separation and blast-radius control. Using older accounts for Bedrock while keeping core workloads isolated follows this playbook perfectly.", "How to Break Free From Bedrock’s Slow Lane 🚀💡", "* Identify older AWS accounts that haven't been used recently. * Transfer them into your AWS Organization using the streamlined Direct Account Transfer workflow. * Update all account metadata for compliance. * Deploy your Bedrock workloads — and unlock higher default limits instantly.", "This approach helps teams accelerate their AI development journey despite the strict constraints placed on newly created AWS accounts.", "Have you seen similar quota differences in your environment? Share your experience — more data points help the community understand the pattern! 🙌" ] }, { "id": "article:use-openai-codex-cli-with-amazon-bedrock-models-pay-as-you-go-48eb", "source_type": "article", "title": "Use OpenAI Codex CLI with Amazon Bedrock Models - Pay As You Go", "url": "https://gabrielkoo.com/blog/use-openai-codex-cli-with-amazon-bedrock-models-pay-as-you-go-48eb/", "canonical_url": "https://dev.to/aws-builders/use-openai-codex-cli-with-amazon-bedrock-models-pay-as-you-go-48eb", "published_at": "2025-08-27", "last_verified_at": "2026-08-23", "tags": [ "vibecoding", "genai", "aws" ], "description": "OpenAI Codex CLI on Amazon Bedrock Models: Why Bother? Here’s why Codex plus the Amazon...", "content": "**NEW** (2026 Jan) Newer versions of Codex uses `/v1/responses` API by default and dropped support for chat completions endpoint. You'll need to add `wire_api = \"responses\"` to your existing config to use the new endpoint instead: .\n\n## OpenAI Codex CLI on Amazon Bedrock Models: Why Bother?\n\nHere’s why Codex plus the Amazon Bedrock models make sense under some cases:\n\n1. **Pay as you go**: No fixed cost—just pay for Bedrock tokens and Lambda invocations. No monthly minimum while Amazon Q Developer CLI has a free-tier quota, you must upgrade to the $19 USD/month paid plan if you have breached it, which *still* enforces usage caps.\n2. **Use your own fine-tuned models**: Swap model endpoints easily; the gateway can even route to your own Amazon Bedrock fine-tunes (e.g. Nova) without friction.\n3. **Transparent logging**: Codex’s request/response logs give you full visibility — a plus for debugging and cost tracking.\n4. **No AWS IAM/Identity required - Perfect for Headless Workloads**: You only need your Bedrock Access Gateway API key; no need to log into your AWS identities with an inconvenient console authentication with your AWS Identity Center user/Builder ID (great for CI/CD and ephemeral cloud instances).\n5. **Regional flexibility**: Yes, you could use [Claude Code with Amazon Bedrock](https://docs.anthropic.com/en/docs/claude-code/amazon-bedrock), but then I live in Hong Kong where Claude model usage is not allowed. \n6. **Amazon Nova Micro: Price King**: For pure simple text LLM tasks, swapping Sonnet 4 for Nova Micro cuts costs by a factor of 85 — Comparing between Nova Micro and Sonnet 4.\n7. If you have a bunch of AWS Credits from AWS events - you're cover with your usages with `gpt-oss` / Nova family of models!\n\n## Setup: Codex CLI + Bedrock Gateway\n\n(UPDATE: Deprecated after Codex v0.80.0 https://github.com/openai/codex/discussions/7782)\n\nGet your Lambda Gateway Function URL and API Key after deployment. (Check my earlier article for a step-by-step guide to get it running on Lambda via AWS SAM: )\n\nHere's a no-brainer if you want to skip my article and deploy it right away: \n\n```bash\n(\n cd /tmp && \\\n git clone --depth=1 https://github.com/gabrielkoo/bedrock-access- gateway-function-url && \\\n cd bedrock-access-gateway-function-url && \\\n ./prepare_source.sh && \\\n sam build && \\\n sam deploy --guided\n)\n```\n\nNow [install Codex](https://github.com/openai/codex?tab=readme-ov-file#installing-and-running-codex-cli):\n\n```bash\nnpm i -g @openai/codex\n```\n\nConfigure Codex like so:\n\n```toml\n# ~/.codex/config.toml\nprofile = 'bedrock'\n\n[profiles.bedrock]\nmodel = 'openai.gpt-oss-120b-1:0'\n# OR\n# model = 'us.amazon.nova-premier-v1:0'\nmodel_provider = 'bedrock'\nmodel_reasoning_effort = \"low\"\n# NEW! Newer versions of Codex uses /v1/responses API by default.\nwire_api = \"chat\"\n\n[model_providers.bedrock]\nname = 'bedrock'\nbase_url = 'https://RANDOM_HASH_HERE.lambda-url.AWS_REGION.on.aws/api/v1'\nenv_key = 'CODEX_OPENAI_API_KEY'\n```\n\nAlternatively, if you want to stick to only `gpt-oss` models but not e.g. Claude/Nova families of models, you can use the latest official OpenAI compatible endpoint with an Amazon Bedrock API Key instead - there will be no need to host the Bedrock Access Gateway:\n\n```toml\n...\nweb_search = \"disabled\"\n\n[model_providers.bedrock]\nname = \"AmazonBedrock\"\nbase_url = \"https://bedrock-mantle.us-west-1.api.aws/v1\"\nenv_key = \"ENV_KEY_FOR_YOUR_BEDROCK_API_KEY\"\n\n...\n\n[profiles.gpt-oss]\n# NOTE: The model ID is truncated if you use the responses API.\nmodel = \"openai.gpt-oss-120b\"\n\n```\n\nQuery the LLM:\n\n```bash\ncodex --profile bedrock \"What is my public IP address?\"\n```\n\n![Codex with Bedrock model in action](/assets/img/f87b08972878.png)\n\n\n## Model Support\n\nNote that not all Bedrock models work over the gateway. Models must support **tool calls**.\n\n**GPT OSS (20b/120b)**: Optimzied with Codex\n**Nova family (Premier, Pro, Lite, Micro):** All tested and working.\n**Claude, Llama, Mistral, Command R:** Working, subject to regional restrictions (e.g. Hong Kong).\n\n## Amazon Q Developer CLI vs Codex CLI on Bedrock\n\nAmazon Q Developer CLI is indeed **officially supported in Hong Kong** — but after your free usage ([50 agentic chats/month](https://aws.amazon.com/q/developer/pricing/)), you'll need the $19/month paid plan, and may hit quotas even then.\n\nCodex CLI via Amazon Bedrock gives *unmetered usage* (subject to whatever quotas you have on Amazon Bedrock itself and Lambda), no AWS login required - you just need to prepare the API key that you defined your self when you deployed the Bedrock Access Gateway.\n\n## Why Don't I Just Use the new OpenAI Compatible Endpoint?\n\nRefer to my other blog article [AWS Launches OpenAI-Compatible API for Bedrock (and I Did Some Tests!)](https://dev.to/aws-builders/aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd), the new OpenAI compatible Amazon Bedrock API endpoint supports `gpt-oss` 20b as well as 120b out of the box, other models like Nova or Claude are not supported.\n\nSo with my solution of wrapping the calls via a Bedrock Access Gateway, you can switch to other models whenever you want according your choice.\n\n## Summary\n\nCodex CLI + Amazon Bedrock (via OpenAI-compatible gateway) gives developers a way to use pay-as-you-go agentic CLI Agents, swap fine-tuned models easily, and avoid region/pricing issues present in other AWS or Anthropic toolings. For minimal cost, Nova Micro is unbeatable for text workloads. And yes, my serverless gateway solution is the backbone — but more about that in [my previous blog](https://dev.to/aws-builders/use-amazon-bedrock-models-via-an-openai-api-compatible-serverless-endpoint-now-without-fixed-cost-5hf5)!", "excerpts": [ "**NEW** (2026 Jan) Newer versions of Codex uses `/v1/responses` API by default and dropped support for chat completions endpoint. You'll need to add `wire_api = \"responses\"` to your existing config to use the new endpoint instead: .", "OpenAI Codex CLI on Amazon Bedrock Models: Why Bother?", "Here’s why Codex plus the Amazon Bedrock models make sense under some cases:", "1. **Pay as you go**: No fixed cost—just pay for Bedrock tokens and Lambda invocations. No monthly minimum while Amazon Q Developer CLI has a free-tier quota, you must upgrade to the $19 USD/month paid plan if you have breached it, which *still* enforces usage caps. 2. **Use your own fine-tuned models**: Swap model endpoints easily; the gateway can even route to your own Amazon Bedrock fine-tunes (e.g. Nova) without friction. 3. **Transparent logging**: Codex’s request/response logs give you full visibility — a plus for debugging and cost tracking. 4. **No AWS IAM/Identity required - Perfect for Headless Workloads**: You only need your Bedrock Access Gateway API key; no need to log into your AWS identities with an inconvenient console authentication with your AWS Identity Center user/Builder ID (great for CI/CD and ephemeral cloud instances). 5. **Regional flexibility**: Yes, you could use [Claude Code with Amazon Bedrock](https://docs.anthropic.com/en/docs/claude-code/amazon-bedrock), but then I live in Hong Kong where Claude model usage is not allowed. 6. **Amazon Nova Micro: Price King**: For pure simple text LLM tasks, swapping Sonnet 4 for Nova Micro cuts costs by a factor of ", "Setup: Codex CLI + Bedrock Gateway", "(UPDATE: Deprecated after Codex v0.80.0 https://github.com/openai/codex/discussions/7782)", "Get your Lambda Gateway Function URL and API Key after deployment. (Check my earlier article for a step-by-step guide to get it running on Lambda via AWS SAM: )", "Here's a no-brainer if you want to skip my article and deploy it right away:", "Now [install Codex](https://github.com/openai/codex?tab=readme-ov-file#installing-and-running-codex-cli):", "[profiles.bedrock] model = 'openai.gpt-oss-120b-1:0' OR model = 'us.amazon.nova-premier-v1:0' model_provider = 'bedrock' model_reasoning_effort = \"low\" NEW! Newer versions of Codex uses /v1/responses API by default. wire_api = \"chat\"", "[model_providers.bedrock] name = 'bedrock' base_url = 'https://RANDOM_HASH_HERE.lambda-url.AWS_REGION.on.aws/api/v1' env_key = 'CODEX_OPENAI_API_KEY' ```", "Alternatively, if you want to stick to only `gpt-oss` models but not e.g. Claude/Nova families of models, you can use the latest official OpenAI compatible endpoint with an Amazon Bedrock API Key instead - there will be no need to host the Bedrock Access Gateway:", "[model_providers.bedrock] name = \"AmazonBedrock\" base_url = \"https://bedrock-mantle.us-west-1.api.aws/v1\" env_key = \"ENV_KEY_FOR_YOUR_BEDROCK_API_KEY\"", "[profiles.gpt-oss] NOTE: The model ID is truncated if you use the responses API. model = \"openai.gpt-oss-120b\"", "![Codex with Bedrock model in action](/assets/img/f87b08972878.png)", "Note that not all Bedrock models work over the gateway. Models must support **tool calls**.", "**GPT OSS (20b/120b)**: Optimzied with Codex **Nova family (Premier, Pro, Lite, Micro):** All tested and working. **Claude, Llama, Mistral, Command R:** Working, subject to regional restrictions (e.g. Hong Kong).", "Amazon Q Developer CLI vs Codex CLI on Bedrock", "Amazon Q Developer CLI is indeed **officially supported in Hong Kong** — but after your free usage ([50 agentic chats/month](https://aws.amazon.com/q/developer/pricing/)), you'll need the $19/month paid plan, and may hit quotas even then.", "Codex CLI via Amazon Bedrock gives *unmetered usage* (subject to whatever quotas you have on Amazon Bedrock itself and Lambda), no AWS login required - you just need to prepare the API key that you defined your self when you deployed the Bedrock Access Gateway.", "Why Don't I Just Use the new OpenAI Compatible Endpoint?", "Refer to my other blog article [AWS Launches OpenAI-Compatible API for Bedrock (and I Did Some Tests!)](https://dev.to/aws-builders/aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd), the new OpenAI compatible Amazon Bedrock API endpoint supports `gpt-oss` 20b as well as 120b out of the box, other models like Nova or Claude are not supported.", "So with my solution of wrapping the calls via a Bedrock Access Gateway, you can switch to other models whenever you want according your choice.", "Codex CLI + Amazon Bedrock (via OpenAI-compatible gateway) gives developers a way to use pay-as-you-go agentic CLI Agents, swap fine-tuned models easily, and avoid region/pricing issues present in other AWS or Anthropic toolings. For minimal cost, Nova Micro is unbeatable for text workloads. And yes, my serverless gateway solution is the backbone — but more about that in [my previous blog](https://dev.to/aws-builders/use-amazon-bedrock-models-via-an-openai-api-compatible-serverless-endpoint-now-without-fixed-cost-5hf5)!" ] }, { "id": "article:aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd", "source_type": "article", "title": "🚀 AWS Launches OpenAI-Compatible API for Bedrock (and I Did Some Tests!)", "url": "https://gabrielkoo.com/blog/aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd/", "canonical_url": "https://dev.to/aws-builders/aws-launches-openai-compatible-api-for-bedrock-and-i-did-some-tests-49cd", "published_at": "2025-08-07", "last_verified_at": "2026-08-23", "tags": [ "bedrock", "aws", "openai" ], "description": "🚀 AWS OpenAI-Compatible API for Bedrock OSS GPT: Real-World Dev Tests & Insights AWS...", "content": "## 🚀 AWS OpenAI-Compatible API for Bedrock OSS GPT: Real-World Dev Tests & Insights\n\nAWS just dropped a [bombshell](https://aws.amazon.com/blogs/aws/openai-open-weight-models-now-available-on-aws/) by launching open-weight GPT models (`gpt-oss-120b`, `gpt-oss-20b`) on Amazon Bedrock as serverless, pay-as-you-go option to self hosting — this caught a lot of headlines 👀.\n\nBut as someone obsessed with **Developer Experience (DX)**, I was even more stoked that AWS **finally** launched [an official OpenAI-compatible endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html): \n\n`https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1` \n\nThis puts AWS right alongside Gemini/VertexAI and Anthropic as companies providing first-party OpenAI SDK compatibility, and finally brings native OpenAI \"plug-and-play\" infra to AWS 🤩\n\nBefore we dive in, **here’s my previous deep-dive blog for context on using Amazon Bedrock models with OpenAI compatibility (without fixed costs):**\n \n👉 \n\n---\n\n## 😎 Wait—Does This Make the Bedrock Proxy Gateway Projects Stale?\n\nNope! While simple OpenAI API calls still runs perfectly with these new official compatibile Amazon Bedrock Runtimeendpoints, **there are critical compatibility gaps** — so custom proxies/gateways like my [bedrock-access-gateway-function-url](https://github.com/gabrielkoo/bedrock-access-gateway-function-url) project and the official [bedrock-access-gateway](https://github.com/aws-samples/bedrock-access-gateway/) project are nowhere near obsolete yet.\n\n---\n\n## By the Way, AWS Credits Does Cover the `gpt-oss` models!\n\nIt has been quite a pity that typical AWS Credits do not cover other models like Claude or Mistral. It's great that this time with the official partnership between AWS and OpenAI, the usage for `gpt-oss` models seems covered finally:\n\n![Image description](/assets/img/23f1c54741f8.png)\n\n---\n\n## 🧑‍💻 I Vibe-Coded the Tests with Kiro AI IDE — Here’s What Happened\n\nYou can check my **actual repo and test scripts here:** \n📦 \n\n**AWS Official Docs:** [AWS Bedrock Docs: OpenAI Chat Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html) \n*(Note: The docs didn’t mention support is limited to latest OpenAI `gpt-oss` models only, and didn't mention directly support for `tool_call` or `response_format` features/parameters, so do read on for my real-world dev findings!)*\n\n## Summary Table:\n\nAs a control test, I also ran the same testing script with my local Ollama `gpt-oss:20b` model:\n\n| Test | Bedrock OpenAI Endpoint | Local Ollama + `gpt-oss` | Notes |\n|------------------------------------------------------|------------------------------------------|---------------------------------------|--------------------------------------------------------------------------------------------|\n| Basic Chat Completion + Reasoning | ✅ Works: Step-wise reasoning included | ✅ Works, chain-of-thought | Bedrock wraps reasoning in `` tags, both are great for math/logic explanations |\n| `reasoning_effort` parameter | ✅ Supported, adjusts explanation depth | ✅ Supported, works | Fine-tune output detail—rare outside OpenAI OSS models |\n| 🔧 Tool/Function Calling (`tools`/`tool_choice`) | ❌ Only intent as JSON, *not* native `tool_calls` | ✅ True OpenAI tool_call schema | Just like the initial state of `o1-mini`: AWS \"gets\" the tools, but you can’t delegate natively |\n| `response_format` (JSON schema) | ❌ Not supported, API error | ✅ Works, emits JSON | Bedrock doesn’t do response schema yet |\n| Token param (`max_tokens` vs `max_completion_tokens`) | ❌ Must use `max_completion_tokens` | ✅ Accepts either | Bedrock is stricter, adjust scripts accordingly |\n| Non-OSS Model (`Nova`, `Titan`, etc.) | ✅ \"Model not found\", as expected | ✅ Same | Endpoint only exposes OSS GPT models (`gpt-oss-*`) for now |\n\n---\n\n## 🌟 Key DX Takeaways\n\n- **API Parity, But Not Quite Full Compatibility:** \n Any OpenAI SDK code (and most OpenAI agent frameworks) can now point at AWS’s Bedrock Runtime `/openai/v1` endpoint with just env var tweaks — no code rewriting. This UX is 🔥 for engineers and innovation teams!\n- **Transparent, Adjustable Reasoning:** \n Get full chain-of-thought out-of-the-box, plus parametric reasoning level control. Perfect for math, science, coding, and explainability applications!\n- **Tool Calling Support—Almost There:** \n AWS Bedrock OSS GPT models *recognize* function call intent (return JSON), but don’t yet emit proper `tool_calls` blocks in the response. (While the same test case passed for my local Ollama test on the same `gpt-oss` model). This feels just like the early days of the `o1-mini` models, so: expect better in future, but for now, your OpenAI tool-calling apps will need adapters or proxies. 🙂\n- **Strict OpenAI Param Parsing:** \n If your code uses `max_tokens`, it’ll break here — swap to `max_completion_tokens` for as `gpt-oss` is a reasoning model.\n- **Model Registry/Open Weight Only:** \n The official endpoint supports only `gpt-oss` models so far (`gpt-oss-120b`, `gpt-oss-20b`). Other Amazon Bedrock serverless models like `Nova` will get you a fast fail as of now.\n- **Still Need Dev Proxy Power:** \n [bedrock-access-gateway-function-url](https://github.com/gabrielkoo/bedrock-access-gateway-function-url) is *not* obsolete! It remains highly useful for compatibility workarounds or for non-`gpt-oss` models.\n\nAs a disclaimer, if we refer to the model documentation of `gpt-oss` on AWS, , tool calls were not supported directly for `gpt-oss` on Amazon Bedrock hosted version as of now, so it’s totally possible when AWS release more model support for the new OpenAI endpoint, these could no longer become a concern!\n\n---\n\n## 💡 DX Tips for Devs\n\n- **Add backward/forward compatibility adapters** if you’re building serious agentic stacks.\n- **Watch the docs** ([Invoke a model with the OpenAI Chat Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html)), but always test yourself — AWS hasn't listed all the current limitations yet clearly in the documentation!\n- **Port your scripts with care:** \n - Reasoning, logic, coding, and OpenAI-style chat: 👍 \n - Tool use, output schemas: ⚠️ adapter or proxy needed today!\n- **OSS all the things!** Making the full open-weight LLM stack *API-portable* is a big win for multi-cloud, local/cloud hybrid, and plain old sovereignty nerds. It feels like a new era for \"bring your own infra, but keep your SDKs.\"\n\n---\n\n## Closing\n\nThanks to [Kiro AI IDE](https://kiro.dev/) for the vibe-code collab and to AWS's team on making the OpenAI compatible endpoint finally launched!\n\nAWS’s new official OpenAI compatible endpoint for `gpt-oss` models is a major leap for cloud/dev portability, even if a few “classic” OpenAI features like native function calling aren’t fully there (yet). If you need adapters, or want total control, proxies like mine remain essential. But for standard chat/reasoning patterns, Bedrock is now OpenAI SDK ready — from your own existing code based on OpenAI SDKs.\n\nHappy building! 🚀🤖\n\nP.S. This article was written on 2025-08-06, 1 day after the new endpoint it was launched, so it's totally possible that by the time you read this blog article, AWS could have already rolled out more support for other common OpenAI SDK features!", "excerpts": [ "🚀 AWS OpenAI-Compatible API for Bedrock OSS GPT: Real-World Dev Tests & Insights", "AWS just dropped a [bombshell](https://aws.amazon.com/blogs/aws/openai-open-weight-models-now-available-on-aws/) by launching open-weight GPT models (`gpt-oss-120b`, `gpt-oss-20b`) on Amazon Bedrock as serverless, pay-as-you-go option to self hosting — this caught a lot of headlines 👀.", "But as someone obsessed with **Developer Experience (DX)**, I was even more stoked that AWS **finally** launched [an official OpenAI-compatible endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html):", "`https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1`", "This puts AWS right alongside Gemini/VertexAI and Anthropic as companies providing first-party OpenAI SDK compatibility, and finally brings native OpenAI \"plug-and-play\" infra to AWS 🤩", "Before we dive in, **here’s my previous deep-dive blog for context on using Amazon Bedrock models with OpenAI compatibility (without fixed costs):** 👉", "😎 Wait—Does This Make the Bedrock Proxy Gateway Projects Stale?", "Nope! While simple OpenAI API calls still runs perfectly with these new official compatibile Amazon Bedrock Runtimeendpoints, **there are critical compatibility gaps** — so custom proxies/gateways like my [bedrock-access-gateway-function-url](https://github.com/gabrielkoo/bedrock-access-gateway-function-url) project and the official [bedrock-access-gateway](https://github.com/aws-samples/bedrock-access-gateway/) project are nowhere near obsolete yet.", "By the Way, AWS Credits Does Cover the `gpt-oss` models!", "It has been quite a pity that typical AWS Credits do not cover other models like Claude or Mistral. It's great that this time with the official partnership between AWS and OpenAI, the usage for `gpt-oss` models seems covered finally:", "![Image description](/assets/img/23f1c54741f8.png)", "🧑‍💻 I Vibe-Coded the Tests with Kiro AI IDE — Here’s What Happened", "You can check my **actual repo and test scripts here:** 📦", "**AWS Official Docs:** [AWS Bedrock Docs: OpenAI Chat Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html) *(Note: The docs didn’t mention support is limited to latest OpenAI `gpt-oss` models only, and didn't mention directly support for `tool_call` or `response_format` features/parameters, so do read on for my real-world dev findings!)*", "As a control test, I also ran the same testing script with my local Ollama `gpt-oss:20b` model:", "| Test | Bedrock OpenAI Endpoint | Local Ollama + `gpt-oss` | Notes | |------------------------------------------------------|------------------------------------------|---------------------------------------|--------------------------------------------------------------------------------------------| | Basic Chat Completion + Reasoning | ✅ Works: Step-wise reasoning included | ✅ Works, chain-of-thought | Bedrock wraps reasoning in ` ` tags, both are great for math/logic explanations | | `reasoning_effort` parameter | ✅ Supported, adjusts explanation depth | ✅ Supported, works | Fine-tune output detail—rare outside OpenAI OSS models | | 🔧 Tool/Function Calling (`tools`/`tool_choice`) | ❌ Only intent as JSON, *not* native `tool_calls` | ✅ True OpenAI tool_call schema | Just like the initial state of `o1-mini`: AWS \"gets\" the tools, but you can’t delegate natively | | `response_format` (JSON schema) | ❌ Not supported, API error | ✅ Works, emits JSON | Bedrock doesn’t do response schema yet | | Token param (`max_tokens` vs `max_completion_tokens`) | ❌ Must use `max_completion_tokens` | ✅ Accepts either | Bedrock is stricter, adjust scripts accordingly | | Non-OSS Model (`Nova`, `Titan", "- **API Parity, But Not Quite Full Compatibility:** Any OpenAI SDK code (and most OpenAI agent frameworks) can now point at AWS’s Bedrock Runtime `/openai/v1` endpoint with just env var tweaks — no code rewriting. This UX is 🔥 for engineers and innovation teams! - **Transparent, Adjustable Reasoning:** Get full chain-of-thought out-of-the-box, plus parametric reasoning level control. Perfect for math, science, coding, and explainability applications! - **Tool Calling Support—Almost There:** AWS Bedrock OSS GPT models *recognize* function call intent (return JSON), but don’t yet emit proper `tool_calls` blocks in the response. (While the same test case passed for my local Ollama test on the same `gpt-oss` model). This feels just like the early days of the `o1-mini` models, so: expect better in future, but for now, your OpenAI tool-calling apps will need adapters or proxies. 🙂 - **Strict OpenAI Param Parsing:** If your code uses `max_tokens`, it’ll break here — swap to `max_completion_tokens` for as `gpt-oss` is a reasoning model. - **Model Registry/Open Weight Only:** The official endpoint supports only `gpt-oss` models so far (`gpt-oss-120b`, `gpt-oss-20b`). Other Amazon Bedrock se", "As a disclaimer, if we refer to the model documentation of `gpt-oss` on AWS, , tool calls were not supported directly for `gpt-oss` on Amazon Bedrock hosted version as of now, so it’s totally possible when AWS release more model support for the new OpenAI endpoint, these could no longer become a concern!", "- **Add backward/forward compatibility adapters** if you’re building serious agentic stacks. - **Watch the docs** ([Invoke a model with the OpenAI Chat Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html)), but always test yourself — AWS hasn't listed all the current limitations yet clearly in the documentation! - **Port your scripts with care:** - Reasoning, logic, coding, and OpenAI-style chat: 👍 - Tool use, output schemas: ⚠️ adapter or proxy needed today! - **OSS all the things!** Making the full open-weight LLM stack *API-portable* is a big win for multi-cloud, local/cloud hybrid, and plain old sovereignty nerds. It feels like a new era for \"bring your own infra, but keep your SDKs.\"", "Thanks to [Kiro AI IDE](https://kiro.dev/) for the vibe-code collab and to AWS's team on making the OpenAI compatible endpoint finally launched!", "AWS’s new official OpenAI compatible endpoint for `gpt-oss` models is a major leap for cloud/dev portability, even if a few “classic” OpenAI features like native function calling aren’t fully there (yet). If you need adapters, or want total control, proxies like mine remain essential. But for standard chat/reasoning patterns, Bedrock is now OpenAI SDK ready — from your own existing code based on OpenAI SDKs.", "P.S. This article was written on 2025-08-06, 1 day after the new endpoint it was launched, so it's totally possible that by the time you read this blog article, AWS could have already rolled out more support for other common OpenAI SDK features!" ] }, { "id": "article:regain-access-to-amazon-q-cli-in-cloudshell-with-this-simple-trick-58m3", "source_type": "article", "title": "Regain access to Amazon Q CLI in CloudShell with This Simple Trick", "url": "https://gabrielkoo.com/blog/regain-access-to-amazon-q-cli-in-cloudshell-with-this-simple-trick-58m3/", "canonical_url": "https://dev.to/aws-builders/regain-access-to-amazon-q-cli-in-cloudshell-with-this-simple-trick-58m3", "published_at": "2025-08-03", "last_verified_at": "2026-08-23", "tags": [ "aws", "cloud", "amazonqcli" ], "description": "If you’ve tried to use the Amazon Q CLI a.k.a. q chat command in AWS CloudShell recently, you might...", "content": "If you’ve tried to use the Amazon Q CLI a.k.a. `q chat` command in **AWS CloudShell** recently, you might have seen the frustrating message 🤯:\n\n> _Q CLI integration is temporarily disabled. For continued access to Q Chat, please use the integrated version in your AWS Console._\n\nThis is annoying especially for me as a true lover of Amazon Q CLI which is so far the best GenAI CLI agent for AWS daily users.\n\nThe good news is here’s a guide on how to force reinstall Amazon Q CLI and log in with your own account. Plus, some perspective on why this happens, and why Amazon Q CLI is still a must-have for me 🛠️ — especially if you’re experimenting with GenAI in cloud environments.\n\n## What’s Happening?\n\nAs of time of writing (2025-08-02), AWS has temporarily disabled Amazon Q chat CLI features in CloudShell due to an internal issue (officially documented by AWS [here](https://docs.aws.amazon.com/cloudshell/latest/userguide/q-cli-features-in-cloudshell.html)). Q-based inline suggestions and the actual chat functionality are both disabled in CloudShell. ⛔️\n\n## Why Is This an Issue (My Own Speculation)?\n\nI saw an user reporting the issue ⚠️ as early as 2025-05-23: .\n\nHere’s why I think it’s happening 🧠:\n\n**CloudShell** authenticates you for AWS credentials natively using **AWS IAM users, roles, or Identity Center Access Role sessions** — so that you can do API calls like `aws sts get-caller-identity` within CloudShell without the hassle of configuring the credentials.\n\nIn contrast, **Amazon Q CLI** expects authentication via **AWS Builder ID** or **AWS IAM Identity Center** for Pro plan.\n\nThese authentication models may not be exactly compatible, especially for chat-based workflows requiring licensing or entitlements outside basic AWS IAM integration.\n\nOne last possibility is that it is actually quite hard to apply limits or attribute usage rates of Amazon Q CLI. Imagine if one’s Amazon Q CLI usage with a particular IAM user in CloudShell session exceeds a fair pre-defined limit, one can just create another IAM user and have rate limits reset as a separate user. One could even programmatically set this up for literally unlimited Amazon Q CLI usage quota (if there is not any AWS Account wide limit applied). 💸\n\nUntil AWS aligns these behind the scenes, expect more time from the Amazon Q CLI team to work on a fix.\n\n## But Amazon Q CLI is Still Cool!\n\nEven with this hiccup, Amazon Q CLI remains hugely valuable ⚡️:\n\n- AI-driven CLI suggestions right where you work\n- Natural language command generation\n- Instant code explanations and diagnostics\n- Makes security and automation work faster and less error-prone\n\nIf you use it outside CloudShell, it truly supercharges your workflow.\n\n## Force Reinstall & Login Steps\n\nHere’s comes to the main topic - how you can force a clean install and log in outside CloudShell.\n\nEasy. Since AWS CloudShell computing environment is just x64 Amazon Linux under the hood, just force uninstall it and then install it according to the [official documentation](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/command-line-installing-ssh-setup-autocomplete.html#command-line-download-install-file).\n\n```bash\n# This removes the existing installation\nsudo rm `which q`\n# Download the latest ZIP-based installer\ncurl --proto '=https' --tlsv1.2 -sSf \"https://desktop-release.q.us-east-1.amazonaws.com/latest/q-x86_64-linux-musl.zip\" -o \"q.zip\"\n\nunzip q.zip\n./q/install.sh\n```\n\nWhen prompted:\n\n> Do you want q to modify your shell config (you will have to manually do this otherwise)?\nAnswer: **Yes**\n\nThis helps set up the **Amazon Q CLI** and autocompletion back.\n\nNext, authenticate with your preferred credentials:\n\n```bash\nq login\n```\n\nYou’ll be guided through a browser-based process (powered by Builder ID, Identity Center, etc.), enter the code, and your CLI session will be authenticated. Once authorized, you get full access (outside limitations of CloudShell).\n\n![Image description](/assets/img/ea59f7ecf7b4.jpeg)\n\nNow, let’s test it out. Try:\n\n```bash\nq help\nq inline enable\n```\n\nWith proper authentication, you’ll see **Amazon Q CLI** in all its glory (in supported environments) 🚀.\n\n![Image description](/assets/img/5979c6e0bf80.jpeg)\n\n## Important Points to Note:\n\n- With my method, your Amazon Q CLI usages in CloudShell are bind to your own AWS Builder ID account / your Identity Center user. Be aware of usage rate limits. 🧾\n- AWS CloudShell environments would be automatically deleted after **120 days of inactivity**. You might need to re-do this hack if you haven’t used a particular CloudShell environment for more than 120 days. 🗑️\n\n## Key Takeaways\n\n- Don’t stress if `q chat` is broken in CloudShell—it’s a temporary AWS-side limitation.\n- Use my way to force reinstall and login outside **AWS CloudShell** for a working setup.\n- Integration hurdles are likely caused by the difference in authentication flows: IAM / Identity Center built into CloudShell vs. Builder ID or Pro Plan requirements for Amazon Q CLI.\n- Amazon Q CLI is a great tool—just leverage it where AWS supports full features, and watch for future CloudShell improvements.\n\nHappy hacking — enjoy more intelligent, secure, and productive time in your CloudShell terminal! ☁️", "excerpts": [ "If you’ve tried to use the Amazon Q CLI a.k.a. `q chat` command in **AWS CloudShell** recently, you might have seen the frustrating message 🤯:", "> _Q CLI integration is temporarily disabled. For continued access to Q Chat, please use the integrated version in your AWS Console._", "This is annoying especially for me as a true lover of Amazon Q CLI which is so far the best GenAI CLI agent for AWS daily users.", "The good news is here’s a guide on how to force reinstall Amazon Q CLI and log in with your own account. Plus, some perspective on why this happens, and why Amazon Q CLI is still a must-have for me 🛠️ — especially if you’re experimenting with GenAI in cloud environments.", "As of time of writing (2025-08-02), AWS has temporarily disabled Amazon Q chat CLI features in CloudShell due to an internal issue (officially documented by AWS [here](https://docs.aws.amazon.com/cloudshell/latest/userguide/q-cli-features-in-cloudshell.html)). Q-based inline suggestions and the actual chat functionality are both disabled in CloudShell. ⛔️", "Why Is This an Issue (My Own Speculation)?", "I saw an user reporting the issue ⚠️ as early as 2025-05-23: .", "Here’s why I think it’s happening 🧠:", "**CloudShell** authenticates you for AWS credentials natively using **AWS IAM users, roles, or Identity Center Access Role sessions** — so that you can do API calls like `aws sts get-caller-identity` within CloudShell without the hassle of configuring the credentials.", "In contrast, **Amazon Q CLI** expects authentication via **AWS Builder ID** or **AWS IAM Identity Center** for Pro plan.", "These authentication models may not be exactly compatible, especially for chat-based workflows requiring licensing or entitlements outside basic AWS IAM integration.", "One last possibility is that it is actually quite hard to apply limits or attribute usage rates of Amazon Q CLI. Imagine if one’s Amazon Q CLI usage with a particular IAM user in CloudShell session exceeds a fair pre-defined limit, one can just create another IAM user and have rate limits reset as a separate user. One could even programmatically set this up for literally unlimited Amazon Q CLI usage quota (if there is not any AWS Account wide limit applied). 💸", "Until AWS aligns these behind the scenes, expect more time from the Amazon Q CLI team to work on a fix.", "But Amazon Q CLI is Still Cool!", "Even with this hiccup, Amazon Q CLI remains hugely valuable ⚡️:", "- AI-driven CLI suggestions right where you work - Natural language command generation - Instant code explanations and diagnostics - Makes security and automation work faster and less error-prone", "If you use it outside CloudShell, it truly supercharges your workflow.", "Here’s comes to the main topic - how you can force a clean install and log in outside CloudShell.", "Easy. Since AWS CloudShell computing environment is just x64 Amazon Linux under the hood, just force uninstall it and then install it according to the [official documentation](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/command-line-installing-ssh-setup-autocomplete.html#command-line-download-install-file).", "unzip q.zip ./q/install.sh ```", "> Do you want q to modify your shell config (you will have to manually do this otherwise)? Answer: **Yes**", "This helps set up the **Amazon Q CLI** and autocompletion back.", "Next, authenticate with your preferred credentials:", "You’ll be guided through a browser-based process (powered by Builder ID, Identity Center, etc.), enter the code, and your CLI session will be authenticated. Once authorized, you get full access (outside limitations of CloudShell).", "![Image description](/assets/img/ea59f7ecf7b4.jpeg)", "With proper authentication, you’ll see **Amazon Q CLI** in all its glory (in supported environments) 🚀.", "![Image description](/assets/img/5979c6e0bf80.jpeg)", "- With my method, your Amazon Q CLI usages in CloudShell are bind to your own AWS Builder ID account / your Identity Center user. Be aware of usage rate limits. 🧾 - AWS CloudShell environments would be automatically deleted after **120 days of inactivity**. You might need to re-do this hack if you haven’t used a particular CloudShell environment for more than 120 days. 🗑️", "- Don’t stress if `q chat` is broken in CloudShell—it’s a temporary AWS-side limitation. - Use my way to force reinstall and login outside **AWS CloudShell** for a working setup. - Integration hurdles are likely caused by the difference in authentication flows: IAM / Identity Center built into CloudShell vs. Builder ID or Pro Plan requirements for Amazon Q CLI. - Amazon Q CLI is a great tool—just leverage it where AWS supports full features, and watch for future CloudShell improvements.", "Happy hacking — enjoy more intelligent, secure, and productive time in your CloudShell terminal! ☁️" ] }, { "id": "article:building-github-style-contribution-grids-for-devto-articles-with-ai-3fpn", "source_type": "article", "title": "Building GitHub-Style Contribution Grids for dev.to Articles with Kiro AI IDE", "url": "https://gabrielkoo.com/blog/building-github-style-contribution-grids-for-devto-articles-with-ai-3fpn/", "canonical_url": "https://dev.to/kirodotdev/building-github-style-contribution-grids-for-devto-articles-with-ai-3fpn", "published_at": "2025-08-02", "last_verified_at": "2026-08-23", "tags": [ "devto", "kirodev", "seo", "analytics" ], "description": "How I created beautiful GitHub-style contribution graphs and traffic analytics for my dev.to articles...", "content": "*How I created beautiful GitHub-style contribution graphs and traffic analytics for my dev.to articles using Kiro.dev - without writing much code myself*\n\n![Image description](/assets/img/5a9d039e9765.png)\n\n---\n\n## The Problem: Understanding Your Content's Impact\n\nAs a technical writer publishing on dev.to, I found myself constantly wondering about my articles' performance. Sure, dev.to provides basic stats, but I wanted something more comprehensive - something that could show me:\n\n- **GitHub-style contribution grids** for my writing activity\n- **Traffic source analysis** to understand where readers discover my content\n- **Historical trends** to see how articles perform over time\n- **Beautiful visualizations** I could embed anywhere\n\nWorking frequently with SEO teams, I was particularly interested in understanding traffic sources. Were my articles getting discovered through Google searches? LinkedIn shares? Twitter threads? Newsletter mentions? This data would be crucial for optimizing my content distribution strategy.\n\n## The Solution: GitHub-Style Analytics with Traffic Insights\n\nWhat started as a simple idea turned into a comprehensive analytics system that fetches data from dev.to's API and generates beautiful SVG visualizations. The system creates GitHub-style contribution grids and tracks:\n\n### 📊 Core Metrics\n- **Views**: Daily article view counts\n- **Reactions**: Likes, unicorns, and bookmarks\n- **Comments**: Reader engagement levels\n- **Combined Activity**: Weighted scoring system\n\n### 🌐 Traffic Source Analysis\nThe system tracks where your readers come from and visualizes it in a beautiful pie chart:\n- **Direct traffic** (users typing your URL directly)\n- **Search engines** (Google, Bing, DuckDuckGo)\n- **Social media** (LinkedIn, Twitter, Facebook)\n- **Developer platforms** (GitHub, dev.to internal traffic)\n- **Newsletters and blogs** (Substack, personal blogs)\n- **Professional tools** (Slack, Microsoft Office)\n\n![Traffic sources pie chart showing distribution of 19,096 total views](/assets/img/2bad2a13bc04.png)\n\n### 📈 GitHub-Style Contribution Grids\nThe system generates six types of beautiful SVG visualizations:\n\n1. **Views Activity Grid** - GitHub-style contribution graph with green color scheme\n2. **Reactions Activity Grid** - Purple-themed grid showing engagement patterns \n3. **Combined Activity Grid** - Orange-themed weighted activity visualization\n4. **Top Articles by Views** - Horizontal bar chart of most-viewed content\n5. **Top Articles by Reactions** - Bar chart highlighting most engaging articles\n6. **Traffic Sources Pie Chart** - Visual breakdown of where your readers come from\n\n## The Magic: Built with Kiro.dev\n\nHere's where the story gets interesting - **I didn't write most of this code myself**. The entire project was created using [Kiro.dev](https://kiro.dev/), an AI-powered development environment that acts as your coding partner.\n\n### How Kiro.dev Transformed My Workflow\n\nInstead of spending weeks researching APIs, designing data structures, and debugging visualization code, I simply described what I wanted:\n\n> \"I want to create GitHub-style contribution grids for my dev.to articles, track traffic sources, and generate beautiful SVG visualizations I can embed anywhere.\"\n\nKiro.dev understood the requirements and generated the complete system:\n\n1. **Data fetching** - [`fetch_stats.py`](fetch_stats.py) handles API integration with error handling and rate limiting\n2. **Contribution grids** - [`generate_advanced_graph.py`](generate_advanced_graph.py) creates GitHub-style activity visualizations\n3. **Top articles charts** - [`generate_top_articles.py`](generate_top_articles.py) generates ranking visualizations\n4. **Traffic analysis** - [`generate_traffic_pie_chart.py`](generate_traffic_pie_chart.py) creates traffic source breakdowns\n5. **Automation** - [`.github/workflows/`](.github/workflows/) sets up daily updates\n6. **Data management** - Smart incremental updates and referrer tracking\n\n### The Development Experience\n\nWorking with Kiro.dev felt like pair programming with an expert developer who:\n- **Understood context** - Grasped the full project scope from minimal descriptions\n- **Made smart decisions** - Chose appropriate data structures and algorithms\n- **Wrote clean code** - Generated well-documented, maintainable Python scripts\n- **Anticipated needs** - Added features I hadn't even thought of, like referrer tracking\n\nThe AI didn't just write code - it architected a complete solution.\n\n## Technical Deep Dive\n\n### Data Architecture\n\nThe system organizes data efficiently:\n\n```\ndata/\n├── articles/ # Individual article analytics\n│ ├── {id}-{slug}.json\n├── account.json # Aggregated account stats\n└── top_articles.json # Rankings by metrics\n```\n\nEach article file contains:\n- Basic metrics (views, comments, reactions)\n- Daily breakdown of activity\n- **Referrer data** showing traffic sources\n- Metadata for URL construction\n\n### API Integration\n\nThe system uses three dev.to API endpoints:\n- `/analytics/historical` - Historical analytics data\n- `/analytics/referrers` - Traffic source information\n- `/articles/me/published` - Article listings\n\n### Smart Updates\n\nThe fetcher implements intelligent incremental updates:\n- Only fetches new data since last update\n- Refreshes the second-to-last day to catch delayed analytics\n- Handles API rate limits gracefully\n- Maintains data integrity with backup strategies\n\n### Visualization Engine\n\nThe SVG generation system creates publication-ready graphics:\n- **Responsive design** that works at any size\n- **Interactive tooltips** with detailed information\n- **Clickable elements** linking to original articles\n- **Multiple color schemes** for different contexts\n\n## Key Insights from Traffic Analysis\n\nAfter implementing this system, I discovered fascinating patterns in my content's reach:\n\n### Traffic Source Breakdown\nFrom my 19,096 total views, the top 10 referrers account for 14,974 views (78.4%):\n\n1. **Direct traffic (41.6%)** - 7,939 views from users directly accessing articles\n2. **Google.com (24.3%)** - 4,639 views from organic search\n3. **t.co (3.2%)** - 619 views from Twitter links\n4. **LinkedIn.com (3.0%)** - 582 views from professional network shares\n5. **dev.to (2.4%)** - 459 views from platform internal discovery\n6. **office.net (0.9%)** - 172 views from Microsoft Office applications\n7. **duckduckgo.com (0.9%)** - 172 views from privacy-focused search\n8. **bing.com (0.8%)** - 158 views from Microsoft search\n9. **facebook.com (0.6%)** - 121 views from social media shares\n10. **mrugalski.pl (0.6%)** - 113 views from a personal blog referral\n\n### SEO Team Goldmine\n\nThis data proved invaluable for SEO strategy discussions:\n- **Search dominance** - Google alone drives 24.3% of traffic, with additional search engines contributing more\n- **Social amplification** - Twitter (3.2%) and LinkedIn (3.0%) show strong professional sharing\n- **Platform dynamics** - Only 2.4% comes from dev.to's internal discovery, showing external reach\n- **Direct engagement** - 41.6% direct traffic indicates strong brand recognition and bookmarking\n\n### Content Performance Patterns\n\nThe top-performing article \"I bought us-east-1.com\" generated 9,021 views with fascinating traffic patterns:\n- **71% direct traffic** - Indicating strong word-of-mouth sharing\n- **6.6% Twitter traffic** - Viral social media spread\n- **5.7% LinkedIn traffic** - Professional network engagement\n\n## Showcasing Results: Raw GitHub Content Integration\n\nOne of the most powerful features is the ability to embed these visualizations anywhere using GitHub's raw content URLs. Here's how:\n\n### Step 1: Generate Your Visualizations\n```bash\n# GitHub-style contribution grids\npython3 generate_advanced_graph.py --metric views --color github\npython3 generate_advanced_graph.py --metric reactions --color purple\n\n# Top articles charts \npython3 generate_top_articles.py --metric reactions --count 3\n\n# Traffic sources analysis\npython3 generate_traffic_pie_chart.py\n```\n\n### Step 2: Embed in README Files\n```markdown\n![Views Activity](/assets/img/1b55399b9f84.svg)\n![Top Articles](/assets/img/81a2e8aa0cee.svg)\n![Traffic Sources](/assets/img/e7a039bc0214.svg)\n```\n\n### Step 3: Use in Documentation\nThe SVGs work perfectly in:\n- **GitHub README files** - Showcase your writing activity\n- **Portfolio websites** - Demonstrate content creation consistency\n- **Blog posts** - Visual proof of engagement metrics\n- **Social media** - Eye-catching performance summaries\n\n![README file showing embedded SVG graphs](/assets/img/4724e2869be9.png)\n\n### Pro Tips for Display\n\n1. **Repository structure** - Keep graphs in a dedicated `/graphs` folder\n2. **Naming convention** - Use descriptive filenames like `devto_views_graph.svg`\n3. **Update automation** - GitHub Actions ensures graphs stay current\n4. **Multiple formats** - Generate different metrics and color schemes for various contexts\n\n## The Automation Layer\n\nGitHub Actions keeps everything current with daily updates:\n\n```yaml\n# Runs daily at midnight UTC\n# Fetches latest analytics\n# Generates updated visualizations\n# Commits changes automatically\n```\n\nThis means your analytics dashboard stays fresh without any manual intervention.\n\n## Lessons Learned\n\n### 1. AI-Assisted Development is Transformative\nKiro.dev didn't just speed up development - it elevated the entire solution. The AI suggested features and optimizations I wouldn't have considered, like:\n- Logarithmic scaling for better visualization distribution\n- Referrer data aggregation for traffic source analysis\n- Incremental update strategies for API efficiency\n\n### 2. Data Visualization Drives Insights\nHaving visual representations of my writing activity revealed patterns invisible in raw numbers:\n- Consistency gaps in publishing schedule\n- Correlation between article topics and engagement\n- Traffic source diversity indicating content reach\n\n### 3. SEO Integration is Crucial\nThe referrer tracking proved invaluable for SEO discussions:\n- Identifying high-performing traffic sources\n- Understanding content discovery patterns\n- Optimizing distribution strategies\n\n## Getting Started\n\nWant to build your own analytics dashboard? The complete system is available as an open-source project. Here's how to get started:\n\n### Quick Setup\n1. Fork the repository - \n**[📊 devto-stats-github-action](https://github.com/gabrielkoo/devto-stats-github-action)**\n2. Add your dev.to API key to GitHub Actions Secrets\n3. Enable GitHub Actions for automatic updates\n4. Customize visualizations to match your brand\n\n### Customization Options\n- **Color schemes** - GitHub green, purple, blue, or orange themes\n- **Metrics focus** - Emphasize views, reactions, or combined activity\n- **Time ranges** - Adjust historical data collection periods\n- **Graph types** - Modify visualization styles and layouts\n\n## Conclusion\n\nBuilding this analytics dashboard taught me that the future of development isn't about replacing developers - it's about amplifying our capabilities. With Kiro.dev as my coding partner, I created a sophisticated system that would have taken weeks to build manually.\n\nThe insights gained from traffic source analysis have already improved my content strategy, and the beautiful visualizations provide compelling evidence of writing consistency and engagement.\n\nMost importantly, this project demonstrates how AI-assisted development can help creators focus on what matters most - creating great content - while still building the tools needed to understand and optimize their impact.\n\nWhether you're a technical writer, developer advocate, or content creator, having detailed analytics about your work's reach and impact is invaluable. And with tools like Kiro.dev, building these systems is more accessible than ever.\n\n---\n\n*Ready to build your own analytics dashboard? Check out the complete source code and start tracking your content's impact today.*\n\n\n**[📊 devto-stats-github-action](https://github.com/gabrielkoo/devto-stats-github-action)**\n\n![Sample](/assets/img/1151c14ece05.png)", "excerpts": [ "*How I created beautiful GitHub-style contribution graphs and traffic analytics for my dev.to articles using Kiro.dev - without writing much code myself*", "![Image description](/assets/img/5a9d039e9765.png)", "The Problem: Understanding Your Content's Impact", "As a technical writer publishing on dev.to, I found myself constantly wondering about my articles' performance. Sure, dev.to provides basic stats, but I wanted something more comprehensive - something that could show me:", "- **GitHub-style contribution grids** for my writing activity - **Traffic source analysis** to understand where readers discover my content - **Historical trends** to see how articles perform over time - **Beautiful visualizations** I could embed anywhere", "Working frequently with SEO teams, I was particularly interested in understanding traffic sources. Were my articles getting discovered through Google searches? LinkedIn shares? Twitter threads? Newsletter mentions? This data would be crucial for optimizing my content distribution strategy.", "The Solution: GitHub-Style Analytics with Traffic Insights", "What started as a simple idea turned into a comprehensive analytics system that fetches data from dev.to's API and generates beautiful SVG visualizations. The system creates GitHub-style contribution grids and tracks:", "📊 Core Metrics - **Views**: Daily article view counts - **Reactions**: Likes, unicorns, and bookmarks - **Comments**: Reader engagement levels - **Combined Activity**: Weighted scoring system", "🌐 Traffic Source Analysis The system tracks where your readers come from and visualizes it in a beautiful pie chart: - **Direct traffic** (users typing your URL directly) - **Search engines** (Google, Bing, DuckDuckGo) - **Social media** (LinkedIn, Twitter, Facebook) - **Developer platforms** (GitHub, dev.to internal traffic) - **Newsletters and blogs** (Substack, personal blogs) - **Professional tools** (Slack, Microsoft Office)", "![Traffic sources pie chart showing distribution of 19,096 total views](/assets/img/2bad2a13bc04.png)", "📈 GitHub-Style Contribution Grids The system generates six types of beautiful SVG visualizations:", "1. **Views Activity Grid** - GitHub-style contribution graph with green color scheme 2. **Reactions Activity Grid** - Purple-themed grid showing engagement patterns 3. **Combined Activity Grid** - Orange-themed weighted activity visualization 4. **Top Articles by Views** - Horizontal bar chart of most-viewed content 5. **Top Articles by Reactions** - Bar chart highlighting most engaging articles 6. **Traffic Sources Pie Chart** - Visual breakdown of where your readers come from", "The Magic: Built with Kiro.dev", "Here's where the story gets interesting - **I didn't write most of this code myself**. The entire project was created using [Kiro.dev](https://kiro.dev/), an AI-powered development environment that acts as your coding partner.", "How Kiro.dev Transformed My Workflow", "Instead of spending weeks researching APIs, designing data structures, and debugging visualization code, I simply described what I wanted:", "> \"I want to create GitHub-style contribution grids for my dev.to articles, track traffic sources, and generate beautiful SVG visualizations I can embed anywhere.\"", "Kiro.dev understood the requirements and generated the complete system:", "1. **Data fetching** - [`fetch_stats.py`](fetch_stats.py) handles API integration with error handling and rate limiting 2. **Contribution grids** - [`generate_advanced_graph.py`](generate_advanced_graph.py) creates GitHub-style activity visualizations 3. **Top articles charts** - [`generate_top_articles.py`](generate_top_articles.py) generates ranking visualizations 4. **Traffic analysis** - [`generate_traffic_pie_chart.py`](generate_traffic_pie_chart.py) creates traffic source breakdowns 5. **Automation** - [`.github/workflows/`](.github/workflows/) sets up daily updates 6. **Data management** - Smart incremental updates and referrer tracking", "Working with Kiro.dev felt like pair programming with an expert developer who: - **Understood context** - Grasped the full project scope from minimal descriptions - **Made smart decisions** - Chose appropriate data structures and algorithms - **Wrote clean code** - Generated well-documented, maintainable Python scripts - **Anticipated needs** - Added features I hadn't even thought of, like referrer tracking", "The AI didn't just write code - it architected a complete solution.", "The system organizes data efficiently:", "Each article file contains: - Basic metrics (views, comments, reactions) - Daily breakdown of activity - **Referrer data** showing traffic sources - Metadata for URL construction", "The system uses three dev.to API endpoints: - `/analytics/historical` - Historical analytics data - `/analytics/referrers` - Traffic source information - `/articles/me/published` - Article listings", "The fetcher implements intelligent incremental updates: - Only fetches new data since last update - Refreshes the second-to-last day to catch delayed analytics - Handles API rate limits gracefully - Maintains data integrity with backup strategies", "The SVG generation system creates publication-ready graphics: - **Responsive design** that works at any size - **Interactive tooltips** with detailed information - **Clickable elements** linking to original articles - **Multiple color schemes** for different contexts", "Key Insights from Traffic Analysis", "After implementing this system, I discovered fascinating patterns in my content's reach:", "Traffic Source Breakdown From my 19,096 total views, the top 10 referrers account for 14,974 views (78.4%):", "1. **Direct traffic (41.6%)** - 7,939 views from users directly accessing articles 2. **Google.com (24.3%)** - 4,639 views from organic search 3. **t.co (3.2%)** - 619 views from Twitter links 4. **LinkedIn.com (3.0%)** - 582 views from professional network shares 5. **dev.to (2.4%)** - 459 views from platform internal discovery 6. **office.net (0.9%)** - 172 views from Microsoft Office applications 7. **duckduckgo.com (0.9%)** - 172 views from privacy-focused search 8. **bing.com (0.8%)** - 158 views from Microsoft search 9. **facebook.com (0.6%)** - 121 views from social media shares 10. **mrugalski.pl (0.6%)** - 113 views from a personal blog referral", "This data proved invaluable for SEO strategy discussions: - **Search dominance** - Google alone drives 24.3% of traffic, with additional search engines contributing more - **Social amplification** - Twitter (3.2%) and LinkedIn (3.0%) show strong professional sharing - **Platform dynamics** - Only 2.4% comes from dev.to's internal discovery, showing external reach - **Direct engagement** - 41.6% direct traffic indicates strong brand recognition and bookmarking", "The top-performing article \"I bought us-east-1.com\" generated 9,021 views with fascinating traffic patterns: - **71% direct traffic** - Indicating strong word-of-mouth sharing - **6.6% Twitter traffic** - Viral social media spread - **5.7% LinkedIn traffic** - Professional network engagement", "Showcasing Results: Raw GitHub Content Integration", "One of the most powerful features is the ability to embed these visualizations anywhere using GitHub's raw content URLs. Here's how:", "Step 1: Generate Your Visualizations ```bash GitHub-style contribution grids python3 generate_advanced_graph.py --metric views --color github python3 generate_advanced_graph.py --metric reactions --color purple", "Top articles charts python3 generate_top_articles.py --metric reactions --count 3", "Traffic sources analysis python3 generate_traffic_pie_chart.py ```", "Step 2: Embed in README Files ```markdown ![Views Activity](/assets/img/1b55399b9f84.svg) ![Top Articles](/assets/img/81a2e8aa0cee.svg) ![Traffic Sources](/assets/img/e7a039bc0214.svg) ```", "Step 3: Use in Documentation The SVGs work perfectly in: - **GitHub README files** - Showcase your writing activity - **Portfolio websites** - Demonstrate content creation consistency - **Blog posts** - Visual proof of engagement metrics - **Social media** - Eye-catching performance summaries", "![README file showing embedded SVG graphs](/assets/img/4724e2869be9.png)", "1. **Repository structure** - Keep graphs in a dedicated `/graphs` folder 2. **Naming convention** - Use descriptive filenames like `devto_views_graph.svg` 3. **Update automation** - GitHub Actions ensures graphs stay current 4. **Multiple formats** - Generate different metrics and color schemes for various contexts", "GitHub Actions keeps everything current with daily updates:", "This means your analytics dashboard stays fresh without any manual intervention.", "1. AI-Assisted Development is Transformative Kiro.dev didn't just speed up development - it elevated the entire solution. The AI suggested features and optimizations I wouldn't have considered, like: - Logarithmic scaling for better visualization distribution - Referrer data aggregation for traffic source analysis - Incremental update strategies for API efficiency", "2. Data Visualization Drives Insights Having visual representations of my writing activity revealed patterns invisible in raw numbers: - Consistency gaps in publishing schedule - Correlation between article topics and engagement - Traffic source diversity indicating content reach", "3. SEO Integration is Crucial The referrer tracking proved invaluable for SEO discussions: - Identifying high-performing traffic sources - Understanding content discovery patterns - Optimizing distribution strategies", "Want to build your own analytics dashboard? The complete system is available as an open-source project. Here's how to get started:", "Quick Setup 1. Fork the repository - **[📊 devto-stats-github-action](https://github.com/gabrielkoo/devto-stats-github-action)** 2. Add your dev.to API key to GitHub Actions Secrets 3. Enable GitHub Actions for automatic updates 4. Customize visualizations to match your brand", "Customization Options - **Color schemes** - GitHub green, purple, blue, or orange themes - **Metrics focus** - Emphasize views, reactions, or combined activity - **Time ranges** - Adjust historical data collection periods - **Graph types** - Modify visualization styles and layouts", "Building this analytics dashboard taught me that the future of development isn't about replacing developers - it's about amplifying our capabilities. With Kiro.dev as my coding partner, I created a sophisticated system that would have taken weeks to build manually.", "The insights gained from traffic source analysis have already improved my content strategy, and the beautiful visualizations provide compelling evidence of writing consistency and engagement.", "Most importantly, this project demonstrates how AI-assisted development can help creators focus on what matters most - creating great content - while still building the tools needed to understand and optimize their impact.", "Whether you're a technical writer, developer advocate, or content creator, having detailed analytics about your work's reach and impact is invaluable. And with tools like Kiro.dev, building these systems is more accessible than ever.", "*Ready to build your own analytics dashboard? Check out the complete source code and start tracking your content's impact today.*", "**[📊 devto-stats-github-action](https://github.com/gabrielkoo/devto-stats-github-action)**", "![Sample](/assets/img/1151c14ece05.png)" ] }, { "id": "article:why-i-built-a-web-ui-for-amazon-q-developer-cli-and-how-i-vibe-coded-it-54d6", "source_type": "article", "title": "Why I Built a Web UI for Amazon Q Developer CLI (And How I Vibe-Coded It)", "url": "https://gabrielkoo.com/blog/why-i-built-a-web-ui-for-amazon-q-developer-cli-and-how-i-vibe-coded-it-54d6/", "canonical_url": "https://dev.to/aws-builders/why-i-built-a-web-ui-for-amazon-q-developer-cli-and-how-i-vibe-coded-it-54d6", "published_at": "2025-06-26", "last_verified_at": "2026-08-23", "tags": [ "aws", "ai", "vibecoding", "devex" ], "description": "🚀 Beyond the Console: Why I Built a Web UI for the Amazon Q Developer CLI Amazon Q is a...", "content": "## 🚀 **Beyond the Console: Why I Built a Web UI for the Amazon Q Developer CLI**\n\n[Amazon Q](https://aws.amazon.com/q/) is a **game-changer for developers**, but if you've only used it inside the **AWS Console**, you're only seeing part of the picture. Using Q in the Console is great for general AWS questions, but when it comes to **interacting with your own code and environment**, it hits fundamental limits.\n\nMy project started not from a corporate need, but from a personal one familiar to many developers: the **homelab**. I wanted a way to make quick fixes on my server from anywhere, but was constantly frustrated by the **clunky experience of mobile terminals**. This is the story of why I built the **Amazon Q Developer CLI WebUI**.\n\n## 📱 **The Real Problem: The Mobile Terminal Nightmare**\n\nPicture this: you're on the couch with your phone and have an idea for a quick change on your homelab server. You log in through a mobile web terminal and immediately hit a wall. Have you ever tried to **paste a multiline command** into one? Or **send a `Ctrl+C` to stop a runaway process**? It's a nightmare of awkward UIs, missed keystrokes, and immense frustration.\n\n### 🧱 **Example 1**: The Multi-Line `docker run` Command\n\nA common task in a homelab is running a new Docker container with specific ports and volumes:\n\n```bash\ndocker run -d --name=my-app \\\n -p 8080:80 \\\n -v $(pwd)/data:/app/data \\\n nginx\n```\n\n**Mobile Frustrations**:\n\n* **Line Breaks**: Typing a backslash (`\\`) then hitting Enter is clumsy and error-prone on mobile keyboards.\n* **Special Characters**: Juggling `-`, `:`, `=`, and `$` requires constant switching between keyboard symbol layouts.\n* **Editing**: Fixing a typo in the middle of this command on a touchscreen is **incredibly difficult**.\n\n### 🔍 **Example 2**: The Piped `find` and `grep` Command\n\nYou need to search for a specific configuration line within all `.conf` files:\n\n```bash\nfind . -name \"*.conf\" | xargs grep \"listen_port\"\n```\n\n**Mobile Frustrations**:\n\n* **Keyboard Gymnastics**: This short command is packed with hard-to-access symbols on mobile keyboards: `.`, `*`, `\"`, and especially `|`.\n* **The Pipe (`|`)**: Often deeply hidden, requiring multiple taps to find and type.\n* **No `Ctrl` Key**: If a command hangs, the lack of an easy `Ctrl+C` makes stopping it a nightmare.\n\nThis frustration becomes a **major barrier** when using powerful tools like the **Amazon Q Developer CLI**. The CLI is the perfect assistant for the job—but it's **shackled to a terminal** that mobile browsers just can't handle.\n\n## 🤯 Other Reasons for Build This Project \n\n- You cannot install the great Amazon Q Developer CLI on **Windows** 🪟 unless you setup Windows Subsystem for Linux (WSL).\n- For other AI Agent alternatives, they usually belong to one of the cases below:\n - Pure server side AI Agents - like Gemini/ChatGPT, they have access to tools/canvas and even run code for you - but they are not connected to your computer/servers\n - CLI AI Agents - like Codex CLI, requires you to use in a Bring-Your-Own-Key way. You have to be very careful with your context length as you need to worry about token usage. \n\n## 🤖 **Q Developer CLI: An Agent with Better Tools**\n\nThink of the [Amazon Q Developer CLI](https://github.com/aws/amazon-q-developer-cli) as a **specialized AI agent** with more powerful \"tools\" than the AWS Console version. Its biggest advantage? **File system access**, giving it the **context it needs to be a true coding partner**.\n\nAlso look at how I vibe coded a Star Wars inspired lightsaber duel game: [Building A Plasma Sword Fighter Game with Amazon Q CLI](https://dev.to/aws-builders/building-a-plasma-sword-fighter-game-with-amazon-q-cli-279g)\n\n### 🔍 **Access Comparison**\n\n| Feature | Q in AWS Console | Amazon Q Developer CLI | Amazon Q CLI via WebUI |\n| --------------------- | ------------------------ | --------------------------------- | ------------------------------------ |\n| **Primary Use Case** | General AWS service Q\\&A | Deep, contextual code development | Contextual development on any device |\n| **Code Awareness** | ❌ No File System Access | ✅ Full Project Context | ✅ Full Project Context |\n| **Interaction Model** | GUI Chat Window | Command-Line Interface | Browser-Based Terminal |\n| **Accessibility** | Requires AWS login | Requires desktop terminal | Access from any browser |\n| **Mobile Usability** | ✅ Serviceable UI | ❌ Very Difficult | ✅ Designed for Web/Mobile |\n\nThe table shows the gap: the **most powerful version** (CLI) is the **least accessible**—unless we bridge that gap.\n\n## 🛠️ **The Solution: A Self-Hosted Web Interface for the Q CLI**\n\n![Image description](/assets/img/b38418371754.jpg)\n\nSeeing it in action: \n\nMy vibe-coded solution: **Amazon Q Developer CLI WebUI**—a bridge to the full CLI that runs in **any modern browser**. It wraps around the native `q` command, exposing its full feature set without needing a traditional terminal.\n\nI started with telling Amazon Q Developer CLI _\"I want to re-create the Amazon Q Developer CLI experience in a web UI\"_. Q came up with the plan and the initial tech stack, I iterated a few rounds with testings - in less than an hour, my final solution came:\n\n**Tech Stack:**\n\n* Backend: `Node.js`, `Express`\n* Real-Time Communication: `socket.io`\n* Terminal Emulation: `node-pty` for spawning a real `pty`\n\nThe result? A **fluid, interactive CLI experience** from a browser—even on mobile.\n\n## 🚀 **Getting Started**\n\nMake sure you have [`Amazon Q Developer CLI` installed](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/command-line-installing.html).\n\n```bash\ngit clone --depth 1 https://github.com/gabrielkoo/amazon-q-developer-cli-webui\ncd amazon-q-developer-cli-webui\nnpm install\nq login && npm start # —-host 0.0.0.0\n#\n# > amazon-q-cli-webui@0.0.1 start\n# > node server.js\n#\n# Server running on http://localhost:3000\n```\n\n## 🌐 **Making Web UI Available On Your Mobile**\n\nUse any of the typical homelab methods to expose `localhost:3000` online:\n\n* 🌍 **TailScale**\n* ☁️ **CloudFlare Tunnel**\n* 🚪 **Ngrok**\n* 🔐 **Open port 3000 + IP Whitelisting**\n\nThe first two options are preferred if you care very much about security.\n\n## 📲 **Unlocking a Truly Mobile Workflow**\n\nLet’s revisit the homelab scenario. You’re reviewing a Python script that needs a fix.\n\n* **Before**: You’d struggle with a mobile SSH app, fighting to type commands.\n* **After**: You just open a browser tab to your WebUI. Type: *\"Review `fix_script.py`, identify bugs, and suggest improvements.\"* Copy, paste, and interact—**touch-friendly and frustration-free**.\n\n![Image description](/assets/img/92f5982c1743.png)\n\n![Image description](/assets/img/6bba35601296.png)\n\n![Image description](/assets/img/f1eeb4214f14.png)\n\nLive demo: \n\n## ✨ **More Than a Wrapper: An Enhanced User Experience**\n\nI focused on **authenticity and comfort**, solving the mobile pain points:\n\n* 📝 **Proper Input**: A real multiline `