AIGatewayTargetModelAzureConfig
objectAzure-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:azure
The Azure deployment ID for the model.
The Azure OpenAI API version to use.
Default:2023-05-15
AIGatewayTargetModelBedrockConfig
objectAWS Bedrock-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:bedrock
S3 bucket prefix for batch inference jobs.
Whether to normalize embedding vectors in the response.
Default:false
Latency performance configuration for the model invocation.
S3 URI for storing video generation outputs.
AIGatewayTargetModelCohereConfig
objectCohere-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:cohere
The intended downstream use of the embeddings to improve model quality.
Allowed values:classificationclusteringimagesearch_documentsearch_query
Default:classification
Whether to wait for the model to be ready before sending the request.
Default:false
AIGatewayTargetModelDashscopeConfig
objectAlibaba DashScope-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:dashscope
Whether to use the international DashScope endpoint.
Default:true
AIGatewayTargetModelDatabricksConfig
objectDatabricks-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:databricks
The Databricks workspace instance ID.