Skip to content
Cloudflare Docs

Features

AI Gateway provides a comprehensive set of features to help you build, deploy, and manage AI applications with confidence. From performance optimization to security and observability, these features work together to create a robust AI infrastructure.

Core Features

Performance & Cost Optimization

Caching

Serve identical requests directly from Cloudflare's global cache, reducing latency by up to 90% and significantly cutting costs by avoiding repeated API calls to AI providers.

Key benefits:

  • Reduced response times for repeated queries
  • Lower API costs through cache hits
  • Configurable TTL and per-request cache control
  • Works across all supported AI providers
Use Caching

Rate Limiting

Control application scaling and protect against abuse with flexible rate limiting options. Set limits based on requests per time window with sliding or fixed window techniques.

Key benefits:

  • Prevent API quota exhaustion
  • Control costs and usage patterns
  • Configurable per gateway or per request
  • Multiple rate limiting techniques available
Use Rate Limiting

Dynamic Routing

Create sophisticated request routing flows without code changes. Route requests based on user segments, geography, content analysis, or A/B testing requirements through a visual interface.

Key benefits:

  • Visual flow-based configuration
  • User-based and geographic routing
  • A/B testing and fractional traffic splitting
  • Context-aware routing based on request content
  • Dynamic rate limiting with automatic fallbacks
Use Dynamic Routing

Security & Safety

Guardrails

Deploy AI applications safely with real-time content moderation. Automatically detect and block harmful content in both user prompts and model responses across all providers.

Key benefits:

  • Consistent moderation across all AI providers
  • Real-time prompt and response evaluation
  • Configurable content categories and actions
  • Compliance and audit capabilities
  • Enhanced user safety and trust
Use Guardrails

Authentication

Secure your AI Gateway with token-based authentication. Control access to your gateways and protect against unauthorized usage.

Key benefits:

  • Token-based access control
  • Configurable per gateway
  • Integration with Cloudflare's security infrastructure
  • Audit trail for access attempts
Use Authentication

Bring Your Own Keys (BYOK)

Securely store and manage AI provider API keys in Cloudflare's encrypted infrastructure. Remove hardcoded keys from your applications while maintaining full control.

Key benefits:

  • Encrypted key storage at rest and in transit
  • Centralized key management across providers
  • Easy key rotation without code changes
  • Support for 20+ AI providers
  • Enhanced security and compliance
Use Bring Your Own Keys (BYOK)

Observability & Analytics

Analytics

Gain deep insights into your AI application usage with comprehensive analytics. Track requests, tokens, costs, errors, and performance across all providers.

Key benefits:

  • Real-time usage metrics and trends
  • Cost tracking and estimation across providers
  • Error monitoring and troubleshooting
  • Cache hit rates and performance insights
  • GraphQL API for custom dashboards
Use Analytics

Logging

Capture detailed logs of all AI requests and responses for debugging, compliance, and analysis. Configure log retention and export options.

Key benefits:

  • Complete request/response logging
  • Configurable log retention policies
  • Export capabilities via Logpush
  • Custom metadata support
  • Compliance and audit support
Use Logging

Custom Metadata

Enrich your logs and analytics with custom metadata. Tag requests with user IDs, team information, or any custom data for enhanced filtering and analysis.

Key benefits:

  • Enhanced request tracking and filtering
  • User and team-based analytics
  • Custom business logic integration
  • Improved debugging and troubleshooting
Use Custom Metadata

Advanced Configuration

Custom Costs

Override default pricing with your negotiated rates or custom cost models. Apply custom costs at the request level for accurate cost tracking.

Key benefits:

  • Accurate cost tracking with negotiated rates
  • Per-request cost customization
  • Better budget planning and forecasting
  • Support for enterprise pricing agreements
Use Custom Costs

Feature Comparison by Use Case

Use CaseRecommended Features
Cost OptimizationCaching, Rate Limiting, Custom Costs
High AvailabilityFallbacks using Dynamic Routing
Security & ComplianceGuardrails, Authentication, BYOK, Logging
Performance MonitoringAnalytics, Logging, Custom Metadata
A/B TestingDynamic Routing, Custom Metadata, Analytics

Getting Started with Features

  1. Start with the basics: Enable Caching and Analytics for immediate benefits
  2. Add reliability: Configure Fallbacks and Rate Limiting using Dynamic routing
  3. Enhance security: Implement Guardrails and Authentication

All features work seamlessly together and across all 20+ supported AI providers. Get started with AI Gateway to begin using these features in your applications.