Vercel AI Gateway Token Utilities

Helper functions for managing tokens with the AI-Inferoxy proxy server for Vercel AI Gateway.

Overview

This module provides essential functions for:

• Getting managed tokens from the proxy server for Vercel AI Gateway (requires authentication)
• Reporting token usage status (success/error)
• Automatic error classification and handling
• Environment variable management
• Vercel AI Gateway specific features (key:email format, OpenAI-compatible API)

⚠️ Important: Authentication Required

All client operations now require authentication with the AI-Inferoxy server. This is part of the Role-Based Access Control (RBAC) system that provides secure access to the proxy services.

Getting Your API Key

1. Default Admin User: The system creates a default admin user on first run. Check your server logs or the users.json file for the default admin credentials.

2. Create a User Account: Use the admin account to create a regular user account:

   curl -X POST "http://localhost:8000/admin/users" \
     -H "Authorization: Bearer ADMIN_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"username": "youruser", "email": "user@example.com", "full_name": "Your Name", "role": "user"}'
   

3. Use the Generated API Key: The response will include an API key that you’ll use in all client operations.

For detailed RBAC setup and user management, see RBAC_README.md.

Code Example

import os
import requests
import json
from typing import Dict, Optional, Any, Tuple

def get_proxy_token(provider: str = "vc", proxy_url: str = "http://localhost:8000", api_key: str = None) -> Tuple[str, str]:
    """
    Get a valid token from the proxy server for Vercel AI Gateway.
    
    Args:
        provider: Provider type ("vc" for Vercel AI Gateway)
        proxy_url: URL of the AI-Inferoxy server
        api_key: Your API key for authenticating with the proxy server (REQUIRED)
        
    Returns:
        Tuple of (token, token_id)
        
    Raises:
        Exception: If token provisioning fails
    """
    headers = {}
    if api_key:
        headers["Authorization"] = f"Bearer {api_key}"
    
    response = requests.get(f"{proxy_url}/keys/provision/{provider}", headers=headers)
    if response.status_code != 200:
        raise Exception(f"Failed to provision token: {response.text}")
    
    data = response.json()
    token = data["token"]
    token_id = data["token_id"]
    
    # For convenience, also set environment variable
    os.environ["VC_TOKEN"] = token
    
    return token, token_id

def report_token_status(
    token_id: str, 
    status: str = "success", 
    error: Optional[str] = None,
    provider: str = "vc",
    proxy_url: str = "http://localhost:8000",
    api_key: str = None,
    client_name: Optional[str] = None,
) -> bool:
    """
    Report token usage status back to the proxy server for Vercel AI Gateway.
    
    Args:
        token_id: ID of the token to report (from get_proxy_token)
        status: Status to report ('success' or 'error')
        error: Error message if status is 'error'
        provider: Provider type ("vc" for Vercel AI Gateway)
        proxy_url: URL of the AI-Inferoxy server
        api_key: Your API key for authenticating with the proxy server (REQUIRED)
        client_name: Optional end-user identifier for attribution; defaults to username on server
        
    Returns:
        True if report was accepted, False otherwise
    """
    payload = {"token_id": token_id, "status": status}
    
    if error:
        payload["error"] = error
        
        # Extract error classification based on actual VC error patterns
        error_type = None
        if "401 Client Error" in error or "authentication_error" in error:
            error_type = "invalid_credentials"
        elif "402 Client Error" in error or "insufficient_funds" in error:
            error_type = "credits_exceeded"
        elif "404 Client Error" in error or "model_not_found" in error:
            error_type = "model_not_found"
        elif "429 Client Error" in error or "rate_limit_exceeded" in error:
            error_type = "rate_limited"
            
        if error_type:
            payload["error_type"] = error_type
    
    if client_name:
        payload["client_name"] = client_name

    headers = {"Content-Type": "application/json"}
    if api_key:
        headers["Authorization"] = f"Bearer {api_key}"
        
    try:
        response = requests.post(f"{proxy_url}/keys/report/{provider}", json=payload, headers=headers)
        return response.status_code == 200
    except Exception as e:
        # Silently fail to avoid breaking the client application
        # In production, consider logging this error
        return False

Functions

get_proxy_token(provider: str = "vc", proxy_url: str = "http://localhost:8000", api_key: str = None) -> Tuple[str, str]

Retrieves a valid token from the AI-Inferoxy proxy server for Vercel AI Gateway.

Parameters:

• provider: Provider type (“vc” for Vercel AI Gateway)
• proxy_url: URL of the AI-Inferoxy server (defaults to localhost:8000)
• api_key: REQUIRED - Your API key for authenticating with the proxy server

Returns:

• token: The actual API token to use with Vercel AI Gateway
• token_id: Unique identifier for tracking this token’s usage

Features:

• Automatically sets VC_TOKEN environment variable for convenience
• Handles HTTP errors gracefully
• Returns both token and tracking ID
• Requires authentication with your proxy API key

report_token_status(token_id: str, status: str = "success", error: Optional[str] = None, provider: str = "vc", proxy_url: str = "http://localhost:8000", api_key: str = None) -> bool

Reports the status of a token back to the proxy server for health monitoring.

Parameters:

• token_id: ID returned from get_proxy_token()
• status: Either “success” or “error”
• error: Error message if status is “error”
• provider: Provider type (“vc” for Vercel AI Gateway)
• proxy_url: URL of the AI-Inferoxy server
• api_key: REQUIRED - Your API key for authenticating with the proxy server

Returns:

• True if report was accepted, False otherwise

Features:

• Automatic error classification (invalid credentials, credits exceeded, model not found, rate limited)
• Silent failure to avoid breaking client applications
• Supports custom proxy server URLs
• Requires authentication with your proxy API key

Usage Examples

Basic Token Usage with OpenAI Client

from openai import OpenAI
from vc_token_utils import get_proxy_token, report_token_status

# You need to get your API key from the admin or create a user account
# See RBAC_README.md for details on user management
proxy_api_key = "your_proxy_api_key_here"  # Get this from admin

# Get a token (requires authentication)
token, token_id = get_proxy_token(provider="vc", api_key=proxy_api_key)

# Create OpenAI client with Vercel AI Gateway
client = OpenAI(
    api_key=token,
    base_url="https://ai-gateway.vercel.sh"
)

try:
    # Use the token with OpenAI-compatible API
    response = client.chat.completions.create(
        model="openai/gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    
    # Report success
    report_token_status(token_id, "success", provider="vc", api_key=proxy_api_key)
    
    print(response.choices[0].message.content)
    
except Exception as e:
    # Report error
    report_token_status(token_id, "error", str(e), provider="vc", api_key=proxy_api_key)
    raise

Using with Hugging Face Hub InferenceClient

from huggingface_hub import InferenceClient
from vc_token_utils import get_proxy_token, report_token_status

# Get a token (requires authentication)
proxy_api_key = "your_proxy_api_key_here"  # Get this from admin
token, token_id = get_proxy_token(provider="vc", api_key=proxy_api_key)

# Create client with Vercel AI Gateway
client = InferenceClient(
    provider="openai",
    api_key=token,
    base_url="https://ai-gateway.vercel.sh"
)

try:
    # Make chat completion request
    completion = client.chat.completions.create(
        model="openai/gpt-4o-mini",
        messages=[
            {"role": "user", "content": "What is the capital of France?"}
        ]
    )
    
    # Report success
    report_token_status(token_id, "success", provider="vc", api_key=proxy_api_key)
    
    print(completion.choices[0].message.content)
    
except Exception as e:
    # Report error
    report_token_status(token_id, "error", str(e), provider="vc", api_key=proxy_api_key)
    raise

Custom Proxy Server

# Use a different proxy server
token, token_id = get_proxy_token("vc", "https://my-proxy.example.com", api_key=proxy_api_key)

# Report to the same server
report_token_status(token_id, "success", provider="vc", proxy_url="https://my-proxy.example.com", api_key=proxy_api_key)

Error Handling

try:
    # Your Vercel AI Gateway API call
    response = client.chat.completions.create(
        model="openai/gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    report_token_status(token_id, "success", provider="vc", api_key=proxy_api_key)
    
except Exception as e:
    error_msg = str(e)
    
    # The function automatically classifies common errors
    report_token_status(token_id, "error", error_msg, provider="vc", api_key=proxy_api_key)
    
    # You can also check the error type manually
    if "401 Client Error" in error_msg or "authentication_error" in error_msg:
        print("Authentication failed - token may be invalid")
    elif "402 Client Error" in error_msg or "insufficient_funds" in error_msg:
        print("Credits exceeded - token may be out of funds")
    elif "404 Client Error" in error_msg or "model_not_found" in error_msg:
        print("Model not found - check model name")
    elif "429 Client Error" in error_msg or "rate_limit_exceeded" in error_msg:
        print("Rate limited - too many requests")

Vercel AI Gateway Specific Features

Key:Email Format Support

When adding VC keys, you can use the key:email format:

# Example of adding VC keys with email information
# This is handled by the server, not the client utilities
# But it's good to know the format exists

# Server-side key addition example:
# curl -X POST "http://localhost:8000/add-key/vc" \
#   -H "Authorization: Bearer $ADMIN_API_KEY" \
#   -H "Content-Type: application/json" \
#   -d '{
#     "keys": [
#       "vck_ABC123:user@example.com",
#       "vck_DEF456:admin@example.com"
#     ]
#   }'

OpenAI-Compatible API

Vercel AI Gateway provides an OpenAI-compatible API, making it easy to use with existing OpenAI client libraries:

from openai import OpenAI

# Standard OpenAI client works with Vercel AI Gateway
client = OpenAI(
    api_key=token,  # From get_proxy_token()
    base_url="https://ai-gateway.vercel.sh"
)

# All OpenAI API methods work
response = client.chat.completions.create(...)
response = client.embeddings.create(...)
response = client.images.generate(...)

Installation

1. Copy this file to your project directory
2. Install required dependencies: uv add requests openai
3. Import and use the functions in your code
4. Get your API key from the AI-Inferoxy admin or create a user account

Error Classification

The utility automatically classifies common Vercel AI Gateway errors:

• invalid_credentials: 401 errors indicating token authentication failure
• credits_exceeded: 402 errors indicating insufficient funds
• model_not_found: 404 errors indicating the requested model doesn’t exist
• rate_limited: 429 errors indicating rate limit exceeded

This helps the proxy server make intelligent decisions about token health and rotation.

Related Examples

• HF Token Utilities - Helper functions for Hugging Face token management
• Simple Chat Completion - Basic usage example
• Streaming Chat Completion - Streaming usage example
• RBAC Setup - User management and authentication setup

Provider Comparison

Feature	Hugging Face	Vercel AI Gateway
API Compatibility	Hugging Face Hub	OpenAI-compatible
Model Ecosystem	Extensive HF models + providers	OpenAI models + others
Key Format	hf_...	vck_...
Email Support	Via whoami endpoint	Via key:email format
Error Handling	HF-specific errors	OpenAI-compatible errors
Base URL	https://router.huggingface.co	https://ai-gateway.vercel.sh
Client Libraries	huggingface_hub	openai, huggingface_hub