AIGatewayTargetModelLlama2Config

object

Llama2-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:llama2

formatstring

The request format to use when communicating with the Llama2 model.

Allowed values:ollamaopenairaw

Example

AIGatewayTargetModelMistralConfig

object

Mistral-specific configuration for a model.

* Additional properties are NOT allowed.
embeddings_dimensionsinteger

The number of dimensions for embedding outputs.

max_tokensinteger

The maximum number of tokens to generate in the response.

input_costnumber

Cost per input token for billing and cost tracking.

output_costnumber

Cost per output token for billing and cost tracking.

temperaturenumber

Controls randomness in the model output. Higher values produce more varied responses.

top_kinteger

Limits the number of highest-probability tokens considered during generation.

top_pnumber

Nucleus sampling probability mass. Tokens with cumulative probability up to top_p are considered.

upstream_urlstring(uri)

The upstream URL for the model endpoint.

typestringrequired

Allowed values:mistral

formatstring

The request format to use when communicating with the Mistral model.

Allowed values:ollamaopenai

Example

AIGatewayManagedBy

object

Indicates the managing entities for a resource. A maximum of 5 keys is allowed.

Keys must be 63 characters or less, starting and ending with an alphanumeric character,
and may include dashes (-), underscores (_), and dots (.).

Values must be non-empty strings of 127 characters or less.

Keys and values must not contain spaces or control characters, and are case-insensitive.

Example

AIGatewayACLS

object
* Additional properties are NOT allowed.
allowarray[string]required

List of consumer group names permitted access.

denyarray[string]required

List of consumer group names denied access.

Example

AIGatewayPolicyReference

object

Lightweight object used to reference a policy instance, identified by its ref.

* Additional properties are NOT allowed.
refstringrequired

The user-defined unique identifier for a policy instance.

Example:ai-pii-sanitizer-1234

Example