Announcing the release of Spice v1.8.0! 🧊
Spice v1.8.0 delivers major advances in data writes, scalable vector search, and now in preview—managed acceleration snapshots for fast cold starts. This release introduces write support for Iceberg tables using standard SQL INSERT INTO, partitioned S3 Vector indexes for petabyte-scale vector search, and preview of the AI SQL function for direct LLM integration in SQL. Additional improvements include improved reliability, and the v3.0.3 release of the Spice.js Node.js SDK.
Append Data to Iceberg Tables with SQL INSERT INTO: Spice now supports writing to Iceberg tables and catalogs using standard SQL INSERT INTO statements. This enables data ingestion, transformation, and pipeline use cases—no Spark or external writer required.
- Append-only: Initial version targets appends; no overwrite or delete.
- Schema validation: Inserted data must match the target table schema.
- Secure by default: Writes are only enabled for datasets or catalogs explicitly marked with
access: read_write.
Example Spicepod configuration:
catalogs:
- from: iceberg:https://glue.ap-northeast-3.amazonaws.com/iceberg/v1/catalogs/111111/namespaces
name: ice
access: read_write
datasets:
- from: iceberg:https://iceberg-catalog-host.com/v1/namespaces/my_namespace/tables/my_table
name: iceberg_table
access: read_write
Example SQL usage:
INSERT INTO iceberg_table
SELECT * FROM existing_table;
INSERT INTO iceberg_table (id, name, amount)
VALUES (1, 'John', 100.0), (2, 'Jane', 200.0);
INSERT INTO ice.sales.transactions
VALUES (1001, '2025-01-15', 299.99, 'completed');
Note: Only Iceberg datasets and catalogs with access: read_write support writes. Internal Spice tables and other connectors remain read-only.
Learn more in the Iceberg Data Connector documentation.
Bootstrap Managed Accelerations from Object Storage: Spice now supports managed acceleration snapshots in preview, enabling datasets accelerated with file-based engines (DuckDB or SQLite) to bootstrap from a snapshot stored in object storage (such as S3) if the local acceleration file does not exist on startup. This dramatically reduces cold start times and enables ephemeral storage for accelerations with persistent recovery.
Key features:
- Rapid readiness: Datasets can become ready in seconds by downloading a pre-built snapshot, skipping lengthy initial acceleration.
- Hive-style partitioning: Snapshots are organized by month, day, and dataset for easy retention and management.
- Flexible bootstrapping: Configurable fallback and retry behavior if a snapshot is missing or corrupted.
Example Spicepod configuration:
snapshots:
enabled: true
location: s3://some_bucket/some_folder/
bootstrap_on_failure_behavior: warn
params:
s3_auth: iam_role
datasets:
- from: s3://some_bucket/some_table/
name: some_table
params:
file_format: parquet
s3_auth: iam_role
acceleration:
enabled: true
snapshots: enabled
engine: duckdb
mode: file
params:
duckdb_file: /nvme/some_table.db
How it works:
- On startup, if the acceleration file does not exist, Spice checks the snapshot location for the latest snapshot and downloads it.
- Snapshots are stored as:
s3://some_bucket/some_folder/month=2025-09/day=2025-09-30/dataset=some_table/some_table_<timestamp>.db
- If no snapshot is found, a new acceleration file is created as usual.
- Snapshots are written after each refresh (unless configured otherwise).
Supported snapshot modes:
enabled: Download and write snapshots.bootstrap_only: Only download on startup, do not write new snapshots.create_only: Only write snapshots, do not download on startup.disabled: No snapshotting.
Note: This feature is only supported for file-based accelerations (DuckDB or SQLite) with dedicated files.
Why use acceleration snapshots?
- Faster cold starts: Skip waiting for full acceleration on startup.
- Ephemeral storage: Use fast local disks (e.g., NVMe) for acceleration, with persistent recovery from object storage.
- Disaster recovery: Recover from federated source outages by bootstrapping from the latest snapshot.
Efficient, Scalable Vector Search with Partitioning: Spice now supports partitioning Amazon S3 Vector indexes and scatter-gather queries using a partition_by expression in the dataset vector engine configuration. Partitioned indexes enable faster ingestion, lower query latency, and scale to billions of vectors.
Example Spicepod configuration:
datasets:
- name: reviews
vectors:
enabled: true
engine: s3_vectors
params:
s3_vectors_bucket: my-bucket
s3_vectors_index: base-embeddings
partition_by:
- 'bucket(50, PULocationID)'
columns:
- name: body
embeddings:
from: bedrock_titan
- name: title
embeddings:
from: bedrock_titan
See the Amazon S3 Vectors documentation for details.
LLMs Directly In SQL: A new asynchronous ai SQL function enables direct calls to LLMs from SQL queries for text generation, translation, classification, and more. This feature is released in preview and supports both default and model-specific invocation.
Example Spicepod model configuration:
models:
- name: gpt-4o
from: openai:gpt-4o
params:
openai_api_key: ${secrets:openai_key}
Example SQL usage:
SELECT ai('hi, this prompt is directly from SQL.');
SELECT ai('hi, this prompt is directly from SQL.', 'gpt-4o');
SELECT ai(concat_ws(' ', 'Categorize the zone', Zone, 'in a single word. Only return the word.')) AS category
FROM taxi_zones
LIMIT 10;
Learn more in the SQL Reference AI documentation.
Run CLI Commands Remotely: The Spice CLI now supports connecting to remote Spice instances, enabling you to run spice sql, spice search, and spice chat commands from your local machine against a remote spiced daemon or to Spice Cloud. Previously, these commands required running on the same machine as the runtime. Now, new flags allow remote execution:
--cloud: Connect to a Spice Cloud instance (requires --api-key).--endpoint <endpoint>: Connect to a remote Spice instance via HTTP or Arrow Flight SQL (gRPC). Supports http://, https://, grpc://, or grpc+tls:// schemes.
Examples:
spice sql --endpoint http://remote-host:8090
spice chat --cloud --api-key <your-api-key>
spice search --cloud --api-key <your-api-key>
Supported CLI Commands:
spice sql --cloud / spice sql --endpoint <endpoint>spice search --cloud / spice search --endpoint <endpoint>spice chat --cloud / spice chat --endpoint <endpoint>
Additional Flags:
--headers: Pass custom HTTP headers to the remote endpoint.--tls-root-certificate-file: Specify a root certificate for TLS verification.--user-agent: Set a custom user agent for requests.
For more details, see the Spice CLI Command Reference.
Spice.js v3.0.3 Released: The official Spice.ai Node.js/JavaScript SDK has been updated to v3.0.3, bringing cross-platform support, new APIs, and improved reliability for both Node.js and browser environments.
- Modern Query Methods: Use
sql(), sqlJson(), and nsql() for flexible querying, streaming, and natural language to SQL.
- Browser Support: SDK now works in browsers and web applications, automatically selecting the optimal transport (gRPC or HTTP).
- Health Checks & Dataset Refresh: Easily monitor Spice runtime health and trigger dataset refreshes on demand.
- Automatic HTTP Fallback: If gRPC/Flight is unavailable, the SDK falls back to HTTP automatically.
- Migration Guidance: v3 requires Node.js 20+, uses camelCase parameters, and introduces a new package structure.
Example usage:
import { SpiceClient } from '@spiceai/spice'
const client = new SpiceClient(apiKey)
const table = await client.sql('SELECT * FROM my_table LIMIT 10')
console.table(table.toArray())
See Spice.js SDK documentation for full details, migration tips, and advanced usage.
- Reliability: Improved logging, error handling, and network readiness checks across connectors (Iceberg, Databricks, etc.).
- Vector search durability and scale: Refined logging, stricter default limits, safeguards against index-only scans and duplicate results, and always-accessible metadata for robust queryability at scale.
- Cache behavior: Tightened cache logic for modification queries.
- Full-Text Search: FTS metadata columns now usable in projections; max search results increased to 1000.
- RRF Hybrid Search: Reciprocal Rank Fusion (RRF) UDTF enhancements for advanced hybrid search scenarios.
This release introduces two breaking changes associated with the search observability and tooling.
Firstly, the document_similarity tool has been renamed to search. This has the equivalent change to tracing of these tool calls:
>> spice trace tool_use::document_similarity
>> curl -XPOST http://localhost:8090/v1/tools/document_similarity \
-d '{
"datasets": ["my_tbl"],
"text": "Welcome to another Spice release"
}'
>> spice trace tool_use::search
>> curl -XPOST http://localhost:8090/v1/tools/search \
-d '{
"datasets": ["my_tbl"],
"text": "Welcome to another Spice release"
}'
Secondly, the vector_search task in runtime.task_history has been renamed to search.
The Spice Cookbook now includes 80 recipes to help you get started with Spice quickly and easily.
To upgrade to v1.8.0, use one of the following methods:
CLI:
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.8.0 image:
docker pull spiceai/spiceai:1.8.0
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai
AWS Marketplace:
🎉 Spice is now available in the AWS Marketplace!
- iceberg-rust: Upgraded to v0.7.0-rc.1
- mimalloc: Upgraded from 0.1.47 to 0.1.48
- azure_core: Upgraded from 0.27.0 to 0.28.0
- Jimver/cuda-toolkit: Upgraded from 0.2.27 to 0.2.28
Source: 
