AIGatewayTargetModelAzureConfig

object

Azure-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:azure

deployment_idstring

The Azure deployment ID for the model.

api_versionstring

The Azure OpenAI API version to use.

Default:2023-05-15

Example

AIGatewayTargetModelBedrockConfig

object

AWS Bedrock-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:bedrock

batch_bucket_prefixstring

S3 bucket prefix for batch inference jobs.

embeddings_normalizeboolean

Whether to normalize embedding vectors in the response.

Default:false

performance_config_latencystring

Latency performance configuration for the model invocation.

video_output_s3_uristring

S3 URI for storing video generation outputs.

Example

AIGatewayTargetModelCohereConfig

object

Cohere-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:cohere

embedding_input_typestring

The intended downstream use of the embeddings to improve model quality.

Allowed values:classificationclusteringimagesearch_documentsearch_query

Default:classification

wait_for_modelboolean

Whether to wait for the model to be ready before sending the request.

Default:false

Example

AIGatewayTargetModelDashscopeConfig

object

Alibaba DashScope-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:dashscope

internationalboolean

Whether to use the international DashScope endpoint.

Default:true

Example

AIGatewayTargetModelDatabricksConfig

object

Databricks-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:databricks

workspace_instance_idstring

The Databricks workspace instance ID.

Example