Skip to main content
Version: Next

Azure OpenAI Models

To use a language model hosted on Azure OpenAI, specify the azure path in the from field and the following parameters from the Azure OpenAI Model Deployment page:

ParamDescriptionDefault
azure_api_keyThe Azure OpenAI API key from the models deployment page.-
azure_api_versionThe API version used for the Azure OpenAI service.-
azure_deployment_nameThe name of the model deployment.Model name
endpointThe Azure OpenAI resource endpoint, e.g., https://resource-name.openai.azure.com.-
azure_entra_tokenA Microsoft Entra access token for authentication. Spice sends the token as given and does not renew it, so requests fail once it expires. For a token that Spice renews, use workload identity federation.-
azure_tenant_idThe Microsoft Entra tenant of the identity the model authenticates as: a directory (tenant) ID, which is a GUID, or a domain name such as contoso.onmicrosoft.com. With azure_client_id, the model authenticates with workload identity federation instead of azure_api_key or azure_entra_token.AZURE_TENANT_ID
azure_client_idThe client ID of the user-assigned managed identity or app registration whose federated credential trusts the workload's identity token. Required for workload identity federation.AZURE_CLIENT_ID
azure_federated_token_fileA file that holds the workload's identity token, such as a Kubernetes projected service account token. Spice reads it again for every exchange. When neither this parameter nor AZURE_FEDERATED_TOKEN_FILE is set, Spice requests the token from GitHub Actions.AZURE_FEDERATED_TOKEN_FILE
azure_federated_token_audienceThe audience Spice requests a GitHub Actions identity token for. It must match the audience of the federated credential. Applies only to a token from GitHub Actions.api://AzureADTokenExchange
responses_apienabled or disabled. Controls the Chat Completions backend: enabled proxies /v1/chat/completions through the backend Responses API; disabled uses the standard Chat Completions backend. The /v1/responses endpoint is available for all models regardless of this setting.disabled
openai_responses_toolsComma-separated list of OpenAI-hosted tools exposed via the Responses API for this model. The key has no azure_ prefix, and takes effect only when the model also sets tools (for example, tools: auto). These hosted tools are not available from the /v1/chat/completions HTTP endpoint. Supported tools: code_interpreter, web_search, web_search_preview (the legacy tool, for older models that do not support web_search).-

A model authenticates with exactly one of azure_api_key, azure_entra_token, or workload identity federation. A model that sets none of them uses an identity from its environment: a workload identity, then a managed identity, then the Azure CLI login.

Example:

models:
- from: azure:gpt-4o-mini
name: gpt-4o-mini
params:
endpoint: ${ secrets:SPICE_AZURE_AI_ENDPOINT }
azure_api_version: 2024-08-01-preview
azure_deployment_name: gpt-4o-mini
azure_api_key: ${ secrets:SPICE_AZURE_API_KEY }

# Responses API configuration
responses_api: enabled
tools: auto
openai_responses_tools: web_search

Workload Identity Federation​

An Azure OpenAI model can authenticate with Microsoft Entra workload identity federation instead of an API key or an Entra token. Spice exchanges the identity token that the platform gives the workload, such as a GitHub Actions OIDC token or a Kubernetes service account token, for a Microsoft Entra access token, and exchanges again before each access token expires, so no Azure OpenAI API key or Entra token is stored with Spice. The same parameters apply to /v1/chat/completions, /v1/responses, and Azure OpenAI embedding models.

Workload identity federation for Azure OpenAI models is new in v2.5.0. v2.4.1 and earlier do not have these parameters.

models:
- from: azure:gpt-4o-mini
name: gpt
params:
endpoint: https://my-resource.openai.azure.com
azure_api_version: 2024-10-21
azure_deployment_name: gpt-4o-mini
azure_tenant_id: ${ secrets:AZURE_TENANT_ID }
azure_client_id: ${ secrets:AZURE_CLIENT_ID }
azure_federated_token_file: /var/run/secrets/tokens/azure-identity-token

azure_tenant_id and azure_client_id are both required. When the Spicepod does not set one of them, Spice uses the AZURE_TENANT_ID or AZURE_CLIENT_ID environment variable. The identity token comes from the first of these sources that is available:

  1. The file in azure_federated_token_file.
  2. The file in the AZURE_FEDERATED_TOKEN_FILE environment variable.
  3. The GitHub Actions job that Spice runs in, which needs permissions: id-token: write (see OpenID Connect in GitHub Actions). Spice requests a new token from GitHub for every exchange, for the audience in azure_federated_token_audience, which defaults to api://AzureADTokenExchange, the audience Microsoft Entra ID recommends for a federated credential.

Spice reads a token file again for every exchange, so the platform can rotate the token in place. The audience of a token in a file is set where the token is issued, such as the audience of a Kubernetes projected volume, so a model that sets azure_federated_token_audience while its token comes from azure_federated_token_file or AZURE_FEDERATED_TOKEN_FILE fails to load with a configuration error that names both.

Spice exchanges the token with the Microsoft Entra ID authority of the cloud that endpoint belongs to, for a token scoped to Azure OpenAI in that cloud:

endpoint hostAuthorityScope
Ends in .azure.ushttps://login.microsoftonline.ushttps://cognitiveservices.azure.us/.default
Ends in .azure.cnhttps://login.chinacloudapi.cnhttps://cognitiveservices.azure.cn/.default
Any other hosthttps://login.microsoftonline.comhttps://cognitiveservices.azure.com/.default

When the AZURE_AUTHORITY_HOST environment variable is set, it replaces the authority, and the scope still follows endpoint. Because Spice sends the identity token to the authority, AZURE_AUTHORITY_HOST must be an https:// URL: a value without a scheme, such as login.microsoftonline.us, gets https://, and any other scheme fails the model with:

`AZURE_AUTHORITY_HOST` is not usable: The Microsoft Entra authority must be an https:// URL, such as https://login.microsoftonline.us, because the workload identity token is sent to it. See: https://spiceai.org/docs/components/models/azure

After the first exchange succeeds, Spice logs one info line that names where each part of the model's identity came from, because part of it can come from the environment, which the Spicepod does not show. Each source is a parameter (azure_tenant_id, azure_client_id, azure_federated_token_file), an environment variable (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_FEDERATED_TOKEN_FILE), or, for the token, this GitHub Actions job. The line names the sources, not the IDs:

Authenticated model 'gpt' (azure) as a Microsoft Entra workload identity, with the tenant from `AZURE_TENANT_ID`, the client from `AZURE_CLIENT_ID` and the identity token from `AZURE_FEDERATED_TOKEN_FILE`.

Spice exchanges again one minute before the access token expires, or, for a token that lasts less than four minutes, after three quarters of its lifetime. When an exchange after the first one fails, Spice keeps using the current access token and retries, waiting at most 15 seconds between attempts, and logs a warning that names the model.

A model that sets azure_api_key or azure_entra_token together with azure_tenant_id, azure_client_id, azure_federated_token_file, or azure_federated_token_audience fails to load with an error that names both. A model that has only one of the tenant and the client ID fails with an error that names the missing parameter, and a tenant that is neither a GUID nor a domain name fails with an error that names azure_tenant_id. A model that sets a federation parameter but has no identity token source fails with an error that names azure_federated_token_file. The errors for a model that sets no federation parameter are under Troubleshooting. When Microsoft Entra ID refuses the first exchange, the model fails to load with Entra's reason, such as an AADSTS700213 error for a token whose subject matches no federated credential.

AKS Workload Identity​

A model that sets none of azure_api_key, azure_entra_token, and the federation parameters uses the workload identity of its environment, as the Azure SDKs do: AZURE_TENANT_ID, AZURE_CLIENT_ID, and the token file in AZURE_FEDERATED_TOKEN_FILE. AKS workload identity sets these variables in a pod that has the azure.workload.identity/use: "true" label and a service account with the azure.workload.identity/client-id annotation, so on AKS the model needs only its endpoint:

models:
- from: azure:gpt-4o-mini
name: gpt
params:
endpoint: https://my-resource.openai.azure.com

Without the label, the workload identity webhook does not set the variables or mount the token, and the model fails to load (see Troubleshooting). Configure workload identity for Azure access shows the Helm values that add the label and the annotation.

The environment supplies a workload identity when it has a tenant, a client ID, and an identity token, from AZURE_FEDERATED_TOKEN_FILE or the GitHub Actions job. A model that sets azure_api_key or azure_entra_token never uses the environment's workload identity.

When the environment has no workload identity, the model uses a managed identity, such as on an Azure VM, App Service, Container Apps, or Functions, user-assigned when AZURE_CLIENT_ID names one, and then the Azure CLI login from az login, for local development. A configured workload identity whose token exchange fails stops the model from loading; Spice does not fall back to the other identities.

Kubernetes Outside AKS​

On another Kubernetes cluster, project a service account token with the audience api://AzureADTokenExchange into the pod, and name its file in azure_federated_token_file, as in the Spicepod at the top of this section:

# Pod spec
spec:
serviceAccountName: spiceai
containers:
- name: spiceai
volumeMounts:
- name: azure-identity-token
mountPath: /var/run/secrets/tokens
readOnly: true
volumes:
- name: azure-identity-token
projected:
sources:
- serviceAccountToken:
path: azure-identity-token
audience: api://AzureADTokenExchange
expirationSeconds: 3600

Kubernetes rotates a projected token before it expires (service account token volume projection). The federated credential's issuer is the cluster's service account issuer, and Microsoft Entra ID must be able to read that issuer's OIDC discovery document and signing keys.

GitHub Actions​

In a GitHub Actions job, Spice requests the job's OIDC token, so the model needs no token file. With AZURE_TENANT_ID and AZURE_CLIENT_ID in the job's environment, a model that sets only endpoint uses the job's identity, as on AKS. The job needs permissions: id-token: write:

jobs:
integration:
runs-on: ubuntu-latest
environment: prod # The federated credential's subject is repo:my-org/my-repo:environment:prod
permissions:
contents: read
id-token: write # Spice requests the job's OIDC token
env:
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
steps:
- uses: actions/checkout@v5
- run: curl https://install.spiceai.org | /bin/bash
- run: ~/.spice/bin/spice run &

install.spiceai.org installs the latest Spice release, which must be v2.5.0 or later for this job.

Spice requests a new OIDC token from the job for every exchange, so the model keeps its access for as long as the job runs. The load log line ends with the identity token from this GitHub Actions job.

Create the Identity in Azure​

The identity is a user-assigned managed identity or an app registration. It needs a federated credential that trusts the workload's identity token, and the Cognitive Services OpenAI User role on the Azure OpenAI resource. With the Azure CLI:

RG=my-rg

# 1. Create a user-assigned managed identity, and read the IDs Spice needs.
az identity create --resource-group $RG --name spice-models
AZURE_CLIENT_ID=$(az identity show --resource-group $RG --name spice-models --query clientId -o tsv)
AZURE_TENANT_ID=$(az identity show --resource-group $RG --name spice-models --query tenantId -o tsv)
PRINCIPAL_ID=$(az identity show --resource-group $RG --name spice-models --query principalId -o tsv)

# 2. Trust the workload's identity token.
az identity federated-credential create \
--resource-group $RG \
--identity-name spice-models \
--name github-prod \
--issuer https://token.actions.githubusercontent.com \
--subject repo:my-org/my-repo:environment:prod \
--audiences api://AzureADTokenExchange

# 3. Grant the identity access to the Azure OpenAI resource.
OPENAI_ID=$(az cognitiveservices account show --resource-group $RG --name my-resource --query id -o tsv)
az role assignment create \
--assignee-object-id $PRINCIPAL_ID \
--assignee-principal-type ServicePrincipal \
--role "Cognitive Services OpenAI User" \
--scope $OPENAI_ID

The issuer and subject of the federated credential depend on where Spice runs:

Workload--issuer--subject
GitHub Actions job in the environment prodhttps://token.actions.githubusercontent.comrepo:my-org/my-repo:environment:prod
AKS pod with the service account spiceai in the namespace spiceaiThe cluster's OIDC issuer, from az aks show --resource-group $RG --name my-cluster --query oidcIssuerProfile.issuerUrl -o tsvsystem:serviceaccount:spiceai:spiceai
Pod on another Kubernetes clusterThe cluster's service account issuersystem:serviceaccount:<namespace>:<service-account>

The subject must match the token's sub claim exactly. GitHub lists the subject of each kind of job in OpenID Connect in Azure. For an app registration, add the federated credential with az ad app federated-credential create, and use the application (client) ID as azure_client_id. A new federated credential can take a few seconds to take effect.

Troubleshooting​

When the model loads, the info line above shows which parameter or environment variable supplied each part of the identity. When loading fails, the error names the cause:

  • The environment has an identity token but no tenant or client ID. A model with no key and no federation parameter, in an environment with AZURE_FEDERATED_TOKEN_FILE but without AZURE_TENANT_ID or AZURE_CLIENT_ID, fails to load. This happens, for example, on AKS when the webhook mounts the token but the service account lacks the azure.workload.identity/client-id annotation:

    Found `AZURE_FEDERATED_TOKEN_FILE` for model 'gpt' (azure), but `AZURE_CLIENT_ID` is not set. On AKS, label the pod `azure.workload.identity/use: "true"` and annotate its service account with `azure.workload.identity/client-id`, so the workload identity webhook sets all three. See: https://spiceai.org/docs/components/models/azure
  • No identity answered. A model with no key and no federation parameter, in an environment with no workload identity, no managed identity, and no Azure CLI login, fails to load with an error that lists what it tried:

    Found no credential for model 'gpt' (azure): it sets no `azure_api_key` or `azure_entra_token`, and no Microsoft Entra identity answered. Tried …. Set `azure_api_key`, or give the workload an identity: on AKS, label the pod `azure.workload.identity/use: "true"`; on an Azure VM, App Service or Container Apps, assign it a managed identity; locally, run `az login`. See: https://spiceai.org/docs/components/models/azure
  • Microsoft Entra ID refuses the exchange. The error quotes Entra's reason. AADSTS700213 means that no federated credential of the identity matches the token: compare the credential's issuer, subject, and audience with the token's (see Create the Identity in Azure).

Refer to the Azure OpenAI Service models for more details on available models and configurations.

Follow the Azure OpenAI Models Cookbook to try Azure OpenAI models for vector-based search and chat functionalities with structured (taxi trips) and unstructured (GitHub files) data.