AIGatewayTargetModelHuggingfaceConfig

object

Hugging Face-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:huggingface

use_cacheboolean

Whether to use the Hugging Face inference cache.

Default:false

wait_for_modelboolean

Whether to wait for the model to load if it is not ready.

Default:false

Example

AIGatewayTargetModelLlama2Config

object

Llama2-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:llama2

formatstring

The request format to use when communicating with the Llama2 model.

Allowed values:ollamaopenairaw

Example

AIGatewayTargetModelMistralConfig

object

Mistral-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:mistral

formatstring

The request format to use when communicating with the Mistral model.

Allowed values:ollamaopenai

Example

AIGatewayManagedBy

object

Indicates the managing entities for a resource. A maximum of 5 keys is allowed.

Keys must be 63 characters or less, starting and ending with an alphanumeric character,
and may include dashes (-), underscores (_), and dots (.).

Values must be non-empty strings of 127 characters or less.

Keys and values must not contain spaces or control characters, and are case-insensitive.

Example

AIGatewayACLS

object
* Additional properties are NOT allowed.
allowarray[string]required

List of consumer group names permitted access.

denyarray[string]required

List of consumer group names denied access.

Example