Skip to main content

Quick Start

Get started with Tokligence Gateway in 5 minutes.

Prerequisites​

  • Tokligence Gateway installed (Installation Guide)
  • At least one LLM provider API key (OpenAI, Anthropic, or Google) OR a local LLM

Step 1: Start the Gateway​

# Using the npm package
tgw start

# Or using the binary
./bin/gatewayd

# Default address: http://localhost:8081

Step 2: Configure a Provider​

Using OpenAI​

export TOKLIGENCE_OPENAI_API_KEY=sk-...

Using Anthropic​

export TOKLIGENCE_ANTHROPIC_API_KEY=sk-ant-...

Using Local LLM (Ollama)​

ollama serve
ollama run llama3

Step 3: Test the Gateway​

Using the CLI​

tgw chat "Hello, how are you?"

Using curl​

curl -X POST http://localhost:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer test" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Using with Coding Agents​

With OpenAI Codex CLI​

Point Codex to the gateway:

# Configure Codex to use the gateway
export OPENAI_BASE_URL=http://localhost:8081/v1
export OPENAI_API_KEY=your-gateway-key

# Run Codex with Claude via the gateway
codex --full-auto --config 'model="claude-3-5-sonnet-20241022"'

With Claude Code​

Point Claude Code to the gateway:

# Set the base URL in Claude Code settings
# API URL: http://localhost:8081/anthropic

# Configure gateway to translate to OpenAI
export TOKLIGENCE_OPENAI_API_KEY=sk-...

API Endpoints​

OpenAI-Compatible​

EndpointDescription
POST /v1/chat/completionsChat with tool calling support
POST /v1/responsesResponses API with session management
GET /v1/modelsList available models
POST /v1/embeddingsText embeddings

Anthropic Native​

EndpointDescription
POST /anthropic/v1/messagesNative Anthropic chat
POST /anthropic/v1/messages/count_tokensToken estimation

Gemini Native​

EndpointDescription
POST /v1beta/models/{model}:generateContentGemini chat completion
POST /v1beta/models/{model}:streamGenerateContentGemini streaming
GET /v1beta/modelsList Gemini models

Model Routing​

The gateway routes requests based on model name patterns:

# Configure via environment variable
export TOKLIGENCE_ROUTES="claude*=>anthropic,gpt-*=>openai,gemini-*=>google"

Examples:

  • model: "claude-3-haiku" → Anthropic API
  • model: "gpt-4" → OpenAI API
  • model: "gemini-pro" → Google API

Work Modes​

The gateway supports three operation modes:

ModeBehaviorUse Case
auto (default)Smart routing based on model and endpointBest for mixed workloads
passthroughDirect passthrough to upstream providersForce native provider APIs
translationOnly allows translation between formatsForce protocol conversion
export TOKLIGENCE_WORK_MODE=auto  # or passthrough, translation

Next Steps​