AIGatewayTargetModelHuggingfaceConfig
objectHugging Face-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:huggingface
Whether to use the Hugging Face inference cache.
Default:false
Whether to wait for the model to load if it is not ready.
Default:false
AIGatewayTargetModelLlama2Config
objectLlama2-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:llama2
The request format to use when communicating with the Llama2 model.
Allowed values:ollamaopenairaw
AIGatewayTargetModelMistralConfig
objectMistral-specific configuration for a model.
The number of dimensions for embedding outputs.
The maximum number of tokens to generate in the response.
Cost per input token for billing and cost tracking.
Cost per output token for billing and cost tracking.
Controls randomness in the model output. Higher values produce more varied responses.
Limits the number of highest-probability tokens considered during generation.
Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.
The upstream URL for the model endpoint.
Allowed values:mistral
The request format to use when communicating with the Mistral model.
Allowed values:ollamaopenai
AIGatewayManagedBy
objectIndicates the managing entities for a resource. A maximum of 5 keys is allowed.
Keys must be 63 characters or less, starting and ending with an alphanumeric character,
and may include dashes (-), underscores (_), and dots (.).
Values must be non-empty strings of 127 characters or less.
Keys and values must not contain spaces or control characters, and are case-insensitive.
AIGatewayACLS
objectList of consumer group names permitted access.
List of consumer group names denied access.