Skip to main content
Version: Next

LLM Tools (Function Calling)

A tool is a function or operation that can be called directly or by a language model (LLMs). The Spice runtime has several tools available by default, giving LLMs access to various parts of the runtime. Tools can also be added or configured by the user by declaring them in the tools section of spicepod.yaml.

For details about providing LLMs tool access, see Language Model Tools.

For details on tool specifications, see the Tools Spicepod Reference.

Available Tools​

NameDescriptionDefault Group
list_datasetsList the datasets the runtime can query, 50 at a time, optionally searched by name, description, and columns.auto
sqlExecute SQL queries on the runtime. Write statements (INSERT/UPDATE/DELETE/DDL) are accepted only when the request is authenticated with a ReadWrite API key and the target dataset is configured access: read_write; otherwise the tool runs as read-only.auto
table_schemaList a table's columns with their SQL types, descriptions, and keys, optionally only the columns matching a search.auto
searchSearches a configured dataset based on an input query.auto
get_readinessReport the readiness state of every runtime component (datasets, accelerators, models, embeddings, catalogs).auto
get_current_datetimeReturn the current UTC date and time as an ISO 8601 timestamp.auto
sample_distinct_columnsGenerate a synthetic sample of data with distinct values.all
random_sampleSample random rows from a table.all
top_n_sampleSample the top N rows from a table based on a specified ordering.all
memory:loadRetrieve all stored memories from the last time period.memory
memory:storeStore information from LLM interaction(s) for future reference.memory
websearchSearch the web for information. (Deprecated)-

The auto group is a subset of all: the sampling tools (sample_distinct_columns, random_sample, top_n_sample) are provided only by the all (and nsql) groups, or by naming them individually.

Tool Groups​

Tool groups are predefined sets of tools that can be provided to LLMs in a single tool name. For example, the auto tool group provides all default tools to the LLM (see above table).

models:
- name: full-runtime
from: openai:gpt-4o
params:
tools: auto # Automatically choose direct or registry-based discovery

Available tool groups​

NameDescription
autoAutomatically choose between direct tools and searchable registry discovery based on the number of tools and embedding model availability.
allAll built-in and Spicepod-configured tools, provided directly to the LLM.
search_registryUse searchable tool registry discovery via tool_search and tool_invoke meta-tools. Requires an embedding model (see tool_embedding_model).
memoryMemory tools for storing and retrieving information across conversations.
nsqlBuilt-in tools relevant to text-to-SQL: table_schema, sql, list_datasets, get_current_datetime, and the sampling tools.
disabledProvide no tools to the LLM.
MCPTools provided from an MCP server. Can be run within Spice, or connected to over HTTP(s) SSE

Results​

The sql tool and the sampling tools (sample_distinct_columns, random_sample, top_n_sample) return a result sized for a model's context window:

{"columns": ["d_year", "net_paid"], "rows": [[1998, 1234567.89], [1999, 1301122.07]]}

Each row is an array of values in columns order. Numbers are JSON numbers, NULL is null, a decimal with more than 15 significant digits is a string so that no digit is lost, and NaN and ±Infinity are the strings "NaN", "Infinity" and "-Infinity".

A result holds at most 100 rows and at most 10% of the calling model's context window, taken as 4 bytes per token, between 4 KiB and 32 KiB. A call without a model, such as from an MCP client or POST /v1/tools/{name}, gets 32 KiB. A result with more rows shows the rows that fit, and says so. This example shows 2 of its 100 rows:

{
"columns": ["ss_item_sk", "ss_net_paid"],
"rows": [[1, 12.5], [2, 8.0]],
"truncated": true,
"message": "Showing the first 100 rows; the result has more. Do not compute totals, counts, or averages from these rows. Run an aggregate query instead, or narrow the query with a WHERE clause."
}

A text value longer than 1,024 characters keeps its first 1,024 followed by …[+N chars], and a list or map longer than 16 elements keeps 16 followed by "…[+N items]" or "…[+N entries]". The message then says to read the rest with substr(column, 1025, 1024) or array_slice(column, 17, 32). A binary value is shown as "<binary N bytes>", and the message says to read it with encode(column, 'hex'). A map is a list of [key, value] pairs. A result never exceeds its budget, column names included: a result whose column names alone do not fit lists the names that fit, then …[+N columns], and no rows.

Over MCP, these tools declare an outputSchema and return the result as structuredContent, with the same JSON as text. Programs that need every row use /v1/sql instead.

In v2.4.x and earlier, the sql tool returned a JSON array of row objects, and the sampling tools a text table, with no size limit.

Datasets and Schemas​

A model that has the list_datasets tool is told which datasets it can query in a system message, after the Spicepod system_prompt and the client's own system messages. A model without list_datasets, such as one with tools: memory, sql, gets no list:

Datasets (use these names in sql, search, and table_schema):
- spice.public.orders: Customer orders
- spice.docs.pages: Product docs [searchable]

On /v1/responses, the list goes at the end of the request's instructions instead, after the client's. A response that continues a conversation with previous_response_id does not carry instructions over, so the conversation holds one copy of the list, not one per turn.

When the list would take more than about 2,000 tokens (8 KiB), the message instead says how many datasets there are and to find them with list_datasets:

992 datasets are available. Call list_datasets with search to find the ones relevant to the question.

list_datasets takes optional search, limit (default 50, at most 200), and offset. search matches dataset names, descriptions, column names, and column descriptions, best match first. A catalog table matches on its columns only when the runtime has its schema at hand:

{"datasets": [{"table": "spice.docs.pages", "description": "Product docs", "can_search_documents": true}], "total": 992, "has_more": true}

can_search_documents appears only when search can query the dataset, and description and metadata only when the dataset has them.

table_schema lists each table's columns, one per line, with its SQL type and NOT NULL. With output: full (the default), it adds column descriptions and the table's keys:

spice.tpcds.store_sales: TPC-DS store_sales table
ss_item_sk BIGINT NOT NULL
ss_net_paid DECIMAL(7,2) -- Net paid
primary key (ss_item_sk, ss_ticket_number)

A column name that SQL must quote, such as one with capitals or spaces, is listed in double quotes, as a query writes it: "Net Paid" DOUBLE.

For a wide table, search lists only the columns whose names or descriptions match, followed by a count of the others. output: minimal lists names, types, and NOT NULL only.

In v2.4.x and earlier, Spice added a list_datasets call and its result to every chat request, list_datasets returned every dataset at once, and table_schema returned a Markdown table.

Loading and Reloading Tools​

Spice registers the built-in tools first, then loads the tools in the Spicepod concurrently. A Spicepod tool that cannot load is logged and left out of the tool list. The built-in tools, such as sql, and the other Spicepod tools still register, and /v1/tools and the MCP tools/list method list them. A Spicepod tool with the same name as a built-in tool is not used: Spice logs a warning and keeps the built-in tool.

Spice waits up to 2 minutes for each tool, then loads models and workers while a tool that has not loaded keeps retrying in the background. A tool whose configuration is invalid is not retried: a from that no tool source handles, an invalid mcp: directive, or an mcp_args value that cannot be parsed. It stays unavailable until its configuration is fixed. A tool that has not loaded keeps /v1/ready reporting not ready, like any component that failed to load. When such a tool loads later, Spice reloads the models that use it, so they offer the tool from then on.

With the pods watcher enabled (spice run, or spiced --pods-watcher-enabled), changes to tools: in the Spicepod take effect without a restart. Later requests can no longer see or call a removed tool, and an MCP server that Spice started as a subprocess for it is stopped. A request already in progress keeps the tools it started with and can still call a removed tool, so removing a tool is not an immediate revocation of its access, and a remote MCP server keeps running. A changed tool loads again with its new settings, and an added tool loads as it would at startup. Spice then reloads every model that can use a changed tool: a model whose tools parameter names the tool (or catalog:tool), or that uses auto, all, or search_registry, alone or in a list such as auto, my_tool. Changes to workers: also take effect without a restart: a removed worker stops running on its schedule, including a run in progress.

In v2.4.1 and earlier, changes to tools: and workers: took effect only after a restart, a model did not receive a tool that loaded after the model, an MCP server that never answered kept models, workers, and built-in tools from loading, and a Spicepod tool that failed to load kept retrying and kept the Spicepod tools listed after it, and the built-in tools such as sql, from loading.

For details on auto, all, and search_registry tool modes, see Language Model Tools.