AI Gateway with Self-Validating Cache

Cut your LLM API costs.
Without breaking responses.

SemanticGuard caches your LLM responses with multi-layer verification and continuous AI-judged sampling. One line of code. OpenAI, Anthropic, Google.

50% median savings, 100% cache correctness on our public benchmark. See the methodology on the benchmark page.

Start free, no credit card View docs

import { createOpenAI } from "@ai-sdk/openai";
import { withSemanticGuard } from "@semanticguard/ai-sdk";

const openai = createOpenAI({
  apiKey: "sk-...",
  fetch: withSemanticGuard({
    apiKey: "sg-your-key-here",
  }),
});

// All calls now cached + tracked automatically
const result = await generateText({
  model: openai("gpt-4o"),
  prompt: "Summarize this document...",
});

See your savings in real time

Start with Shadow Mode to measure what you'd save. Enable caching when you're ready.

semanticguard.dev/dashboard/cost

Shadow Mode

Illustrative

Spend

$12.9K

Would Save

$8.7K

Shadow Hits

142K

Where your requests went

Total Requests

100%

Direct Match

34%

Smart Match

11%

Verified Match

15%

Cache Miss

33%

You would save by end of month

$12.8K

Enable caching to start saving. Zero risk, instant rollback.

semanticguard.dev/dashboard/cost

Caching Enabled

Illustrative

Cost

$4.2K

Saved

$8.7K

Hit Rate

67%

Cost vs Savings Over Time

Savings

Cost

Correctness

100%

Validated

847

Projected Monthly Savings

$12.8K

at $427/day from cache hits

67% saved vs without SG

How It Works

Add one line of code

Add fetch: withSemanticGuard() to your AI SDK config. Works with any provider.

Measure with Shadow Mode

See cost per request, per model, and exactly how much caching would save you. No cached responses served until you're ready.

Save with confidence

Enable caching. Cache hits return in under 50ms. Multi-layer verification on every hit; sampled hits also AI-judged for correctness with failures flagged.

Built for Vercel

Hosted today, self-host soon

Start on our hosted gateway in one minute. Native Vercel Marketplace listing is in submission; we'll migrate your account into your own Vercel project when it goes live, no rework needed.

Get started free Learn more

✓Encrypted in transit and at rest

✓Prompts stored only if you opt in

✓One-line SDK integration

✓Self-host in your Vercel soon

Built for production

The only LLM cache that proves its own correctness.

Self-validating cache

Multi-layer verification on every hit; sampled cache hits also judged by your own AI for correctness with failures flagged. 100% measured on our public benchmark.

Continuous learning

Your cheapest model learns what varies in your prompts. Names, IDs, dates, and more. Anything regex misses, your AI catches.

Fail-open design

If cache is down, requests go straight to your provider. Zero downtime risk.

Your keys, your data

Upstream API keys pass through at request time, never stored. Prompts logged only if you opt in. Full security posture.

Already using built-in prompt caching?

Stack on top. Catch everything they miss.

Provider prompt caching helps when the same exact prefix shows up again within minutes. That covers a small slice of real production traffic. SemanticGuard catches the rest.

Match type

Provider built-in

Byte-identical prefix only

SemanticGuard

Same meaning, even with different names, dates, or IDs

Across providers

Provider built-in

Locked to one vendor's cache

SemanticGuard

One cache across OpenAI, Anthropic, Google, Azure, Bedrock, Mistral

Across users in your org

Provider built-in

Each session on its own

SemanticGuard

One person's question can serve another

How long it lasts

Provider built-in

Minutes, then gone

SemanticGuard

As long as the answer stays fresh. Seconds for live data, hours for daily content, days for static reference

Setup

Provider built-in

Mark breakpoints or manage cache objects

SemanticGuard

One line of code. No prompt changes

What you save

Provider built-in

Discount on input tokens only

SemanticGuard

Full request eliminated. Input and output both

	Provider built-in caching	SemanticGuard
Match type	Byte-identical prefix only	Same meaning, even with different names, dates, or IDs
Across providers	Locked to one vendor's cache	One cache across OpenAI, Anthropic, Google, Azure, Bedrock, Mistral
Across users in your org	Each session on its own	One person's question can serve another
How long it lasts	Minutes, then gone	As long as the answer stays fresh. Seconds for live data, hours for daily content, days for static reference
Setup	Mark breakpoints or manage cache objects	One line of code. No prompt changes
What you save	Discount on input tokens only	Full request eliminated. Input and output both

Use provider caching for your static system prompts. Use SemanticGuard for everything else, especially any product where multiple users ask overlapping questions.

Built for the AI-native stack

Your AI agents already know how to use us

SemanticGuard speaks the protocols AI agents and dev tools use natively. No adapters, no glue code, no manual setup.

API

OpenAI-compatible endpoint

Same wire format as OpenAI. Any tool or agent that calls OpenAI works with SemanticGuard by changing one URL. Zero code migration.

MCP

Model Context Protocol

Built-in MCP server lets Claude, Cursor, and other AI tools query costs, cache performance, and request traces directly from your IDE.

SDK

One-line integration

TypeScript and Python SDKs with fail-open by default. If the gateway is unreachable, requests go directly to your provider. Zero downtime risk.

Machine-readable responses

Every response includes headers for cache status, latency, cost, and confidence score. Agents inspect caching behavior without parsing the body.

/v1

Health and metrics

Health check and Prometheus metrics endpoints out of the box. Plug into Grafana, Datadog, or any monitoring your agents already use.

Multi-provider, one gateway

OpenAI, Anthropic, Google, Azure, AWS Bedrock, Mistral. Route all providers through one gateway. One API key, one dashboard, one cache.

Pricing

Start free with Shadow Mode. See your savings before you commit.

Free

10K requests/mo

Shadow Mode shows potential savings
Identical-match cache
Cost analytics dashboard
Request tracing and logging

Get started

Popular

Pro

$49/mo

50K included, then $0.50/1K

Full multi-layer caching
Advanced pattern matching
Advanced analytics + projections
Up to 500K requests/mo

Start free, upgrade later

Enterprise

15%

of documented savings

$500/mo minimum commitment
Unlimited requests
We win when you save
AWS/GCP marketplace billing

Billed monthly on the dollars cached, with a full audit log on every invoice. If the cache doesn't deliver, you pay only the $500 floor.

Talk to sales

FAQ

10,000 requests per month, the full cost analytics dashboard, exact-match caching, request tracing, and shadow mode for measuring potential savings on your real traffic before turning caching on. No credit card required to sign up.

SemanticGuard uses multiple caching strategies that go far beyond simple key-value matching. It understands prompt structure, detects reusable patterns across requests, and verifies every cache match before serving. Median 50% savings across our public benchmark, with 100% measured cache correctness. See /benchmark for the per-vertical breakdown. Your actual savings depend on workload mix; RAG with many distinct documents will save less than customer support with overlapping intents.

Provider caching only fires when the same exact prompt prefix shows up again within minutes, on the same provider. SemanticGuard catches the rest. Same question worded differently, same intent with different names or IDs, same user returning the next day, or a different user in your org asking the same thing. Use both together. Provider caching handles your static system prompts, SemanticGuard handles everything else.

Validation runs in two layers. (1) Every cache match goes through multi-layer verification before serving to reject obvious mismatches. (2) A configurable sample of served hits (~0.5% by default, up to 5%) is judged by your cheapest model after the fact; failures are flagged to admins so you can see if anything snuck through. Plus, your AI learns what varies in your prompts (names, IDs, dates) so the cache never confuses one person's data with another's. Our public benchmark shows 100% correctness on wording-tolerant cache returns. See /benchmark for the methodology.

Your upstream API keys are passed through to the provider at request time and never stored in plaintext. We store only a one-way hash for identification. Your data stays with your chosen vendor.

Start with Shadow Mode (free tier default). It logs every request and shows what you would save if caching were enabled. No cached responses are served until you explicitly turn on caching.

One line of code. Add fetch: withSemanticGuard() to your AI SDK provider config. No API format changes, no vendor lock-in. Works with any provider that accepts a custom fetch function.

Yes. SemanticGuard exposes an OpenAI-compatible API, so any agent framework that accepts a custom base URL works by changing one URL. It also includes a built-in MCP server for Claude, Cursor, and other AI tools to query cost and cache analytics directly.

Start saving on LLM costs today

50% median savings and 100% cache correctness on our public benchmark. Multi-layer verification on every hit, AI-judged sampling on top. Free tier: 10K requests per month, no credit card required.

Get started free Read the docs

Cut your LLM API costs.Without breaking responses.

See your savings in real time

How It Works

Add one line of code

Measure with Shadow Mode

Save with confidence

Hosted today, self-host soon

Built for production

Stack on top. Catch everything they miss.

Your AI agents already know how to use us

Pricing

Free

Pro

Enterprise

FAQ

Start saving on LLM costs today

Cut your LLM API costs.
Without breaking responses.