AIGatewayTargetModelCohereConfig

object

Cohere-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:cohere

embedding_input_typestring

The intended downstream use of the embeddings to improve model quality.

Allowed values:classificationclusteringimagesearch_documentsearch_query

Default:classification

wait_for_modelboolean

Whether to wait for the model to be ready before sending the request.

Default:false

Example

AIGatewayTargetModelDashscopeConfig

object

Alibaba DashScope-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:dashscope

internationalboolean

Whether to use the international DashScope endpoint.

Default:true

Example

AIGatewayTargetModelDatabricksConfig

object

Databricks-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:databricks

workspace_instance_idstring

The Databricks workspace instance ID.

Example

AIGatewayTargetModelGeminiConfig

object

Google Gemini-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:gemini

location_idstring

The Google Cloud location ID for the model endpoint.

api_endpointstring

The custom API endpoint for the Gemini model.

Example

AIGatewayTargetModelHuggingfaceConfig

object

Hugging Face-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:huggingface

use_cacheboolean

Whether to use the Hugging Face inference cache.

Default:false

wait_for_modelboolean

Whether to wait for the model to load if it is not ready.

Default:false

Example