Platform ConfigLLM Model & PTU Configuration

LLM Model & PTU Configuration

14.4 · ID103 · Platform Admin · AI Inference Layer

1 critical, 1 warning

5

Available Models

3

PTU Deployments

1

Critical PTU Alerts

$2,520

Month-to-Date Spend

GPT-4o (Azure)RecommendedAzure OpenAI

gpt-4o

Available

Context Window

128K tokens

Input Cost

$0.005/1K

Output Cost

$0.015/1K

Latency P50

1200ms

Capabilities

textvisionfunction_calling
GPT-4o Mini (Azure)Azure OpenAI

gpt-4o-mini

Available

Context Window

128K tokens

Input Cost

$0.00015/1K

Output Cost

$0.0006/1K

Latency P50

450ms

Capabilities

textfunction_callingjson_mode
Claude 3.5 SonnetAnthropic

claude-3-5-sonnet-20241022

Available

Context Window

200K tokens

Input Cost

$0.003/1K

Output Cost

$0.015/1K

Latency P50

900ms

Capabilities

textvisionfunction_calling
Gemini 1.5 ProGoogle Gemini

gemini-1.5-pro

Available

Context Window

1000K tokens

Input Cost

$0.00125/1K

Output Cost

$0.005/1K

Latency P50

1100ms

Capabilities

textvisionaudio
Perplexity Sonar LargePerplexity

llama-3.1-sonar-large-128k-online

Available

Context Window

127K tokens

Input Cost

$0.001/1K

Output Cost

$0.001/1K

Latency P50

2000ms

Capabilities

textweb_search