AIGatewayTargetModelDashscopeConfig

object

Alibaba DashScope-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:dashscope

internationalboolean

Whether to use the international DashScope endpoint.

Default:true

Example

AIGatewayTargetModelDatabricksConfig

object

Databricks-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:databricks

workspace_instance_idstring

The Databricks workspace instance ID.

Example

AIGatewayTargetModelGeminiConfig

object

Google Gemini-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:gemini

location_idstring

The Google Cloud location ID for the model endpoint.

api_endpointstring

The custom API endpoint for the Gemini model.

Example

AIGatewayTargetModelHuggingfaceConfig

object

Hugging Face-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:huggingface

use_cacheboolean

Whether to use the Hugging Face inference cache.

Default:false

wait_for_modelboolean

Whether to wait for the model to load if it is not ready.

Default:false

Example

AIGatewayTargetModelLlama2Config

object

Llama2-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:llama2

formatstring

The request format to use when communicating with the Llama2 model.

Allowed values:ollamaopenairaw

Example