AIGatewayTargetModelCohereConfig
objectCohere-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:cohere
The intended downstream use of the embeddings to improve model quality.
Allowed values:classificationclusteringimagesearch_documentsearch_query
Default:classification
Whether to wait for the model to be ready before sending the request.
Default:false
AIGatewayTargetModelDashscopeConfig
objectAlibaba DashScope-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:dashscope
Whether to use the international DashScope endpoint.
Default:true
AIGatewayTargetModelDatabricksConfig
objectDatabricks-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:databricks
The Databricks workspace instance ID.
AIGatewayTargetModelGeminiConfig
objectGoogle Gemini-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:gemini
The Google Cloud location ID for the model endpoint.
The custom API endpoint for the Gemini model.
AIGatewayTargetModelHuggingfaceConfig
objectHugging Face-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:huggingface
Whether to use the Hugging Face inference cache.
Default:false
Whether to wait for the model to load if it is not ready.
Default:false