Vector Embeddings

This guide covers practical decisions for working with vector embeddings in ArcadeDB: choosing dimensions, creating indexes, tuning parameters, and combining vector search with other query types.

Choosing an Embedding Model

Your embedding model determines the dimensions parameter for the index:

Model Dimensions Notes

Model	Dimensions	Notes
OpenAI `text-embedding-3-small`	1536	General purpose, high quality
OpenAI `text-embedding-3-large`	3072	Highest quality, largest memory footprint
Sentence Transformers `all-MiniLM-L6-v2`	384	Fast, open source, good quality
Sentence Transformers `all-mpnet-base-v2`	768	Better quality, slower
Cohere `embed-english-v3.0`	1024	Good balance of quality and size
CLIP (image + text)	512	Multi-modal image/text

OpenAI text-embedding-3-small

1536

General purpose, high quality

OpenAI text-embedding-3-large

3072

Highest quality, largest memory footprint

Sentence Transformers all-MiniLM-L6-v2

384

Fast, open source, good quality

Sentence Transformers all-mpnet-base-v2

768

Better quality, slower

Cohere embed-english-v3.0

1024

Good balance of quality and size

CLIP (image + text)

512

Multi-modal image/text

Start with 384 dimensions (MiniLM) for prototyping. Move to 768+ for production quality. Use quantization to manage memory at higher dimensions.

Creating a Vector Index

Recommended index creation with INT8 quantization:

CREATE VERTEX TYPE Document
CREATE PROPERTY Document.content STRING
CREATE PROPERTY Document.embedding LIST OF FLOAT

CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8'
}

INT8 quantization is recommended for all production workloads. It provides 2.5x faster search and 4x lower memory usage with negligible accuracy loss (see concepts/vector-search.adoc#quantization-performance). Only omit quantization for very small datasets (< 10K vectors) where maximum precision matters.

Production-ready index with additional tuning:

CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8',
  maxConnections: 16,
  beamWidth: 100
}

Choosing a Similarity Function

Function	Choose When	Avoid When
COSINE	Using text embedding models (most common). Vectors may have varying magnitudes.	Vectors represent absolute quantities (distances, counts).
DOT_PRODUCT	Vectors are already L2-normalized. You need maximum query speed.	Vectors are not normalized (results will be incorrect).
EUCLIDEAN	Working with spatial data, sensor readings, or continuous measurements.	Comparing text embeddings of different lengths.

Function

Choose When

Avoid When

COSINE

Using text embedding models (most common). Vectors may have varying magnitudes.

Vectors represent absolute quantities (distances, counts).

DOT_PRODUCT

Vectors are already L2-normalized. You need maximum query speed.

Vectors are not normalized (results will be incorrect).

EUCLIDEAN

Working with spatial data, sensor readings, or continuous measurements.

Comparing text embeddings of different lengths.

Quantization Trade-offs

Use INT8 quantization for most use cases. It provides 4x memory savings with minimal accuracy loss and significantly faster ingestion and search:

< 10K vectors: NONE is fine, but INT8 works well too
10K - 1M vectors: Use INT8 (4x memory savings, < 2% accuracy loss) — recommended
> 1M vectors: Use INT8 for general use, or PRODUCT for zero-disk-I/O graph construction on very large datasets
Extreme compression: Use BINARY for first-pass filtering, then rerank with full vectors

-- INT8: recommended for most workloads
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 768,
  similarity: 'COSINE',
  quantization: 'INT8'
}

-- PRODUCT: for very large datasets, enables in-memory graph build
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 1024,
  similarity: 'COSINE',
  quantization: 'PRODUCT'
}

INT8 Pre-Quantized Ingest

See Vector Encoding for the underlying concept.

When your embedding provider already emits signed-int8 vectors (Cohere int8 endpoints, OpenAI text-embedding-3-large reduced precision, Sentence Transformers with int8 quantization), use encoding: 'INT8' to keep the bytes byte-shaped end-to-end:

CREATE PROPERTY Doc.embedding BINARY;
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  "dimensions": 1024,
  "similarity": "COSINE",
  "encoding": "INT8"
};

What this saves:

HTTP payload: 4x smaller (1 byte/dim vs 4 bytes/dim) when sending vectors with the typed-marker convention — see HTTP wire convention.
Document bucket storage: 4x smaller, since the property column is BINARY instead of ARRAY_OF_FLOATS.
No client-side int8 → float32 → server round trip; the precision the provider already discarded does not get padded back out for the wire.

What this does not change today:

HNSW graph and search internally still run on float32 (JVector 4.0.0-rc.8 contract). The engine dequantizes once on the read path. Native int8 HNSW is tracked upstream at datastax/jvector#665.
encoding is independent of quantization. Combining encoding: 'INT8' with quantization: 'INT8' is rejected at index creation — pick one, not both.

When not to use INT8 encoding:

Your provider emits float32 (or float64) and you do not have an int8 quantizer client-side. The default FLOAT32 encoding skips the dequantize hop on every search and keeps full precision in the documents.
You want the index-internal compression benefit (search-time memory footprint). That is quantization: 'INT8', not encoding: 'INT8'. They are orthogonal.

Tuning for Recall vs Speed

Adjust maxConnections and beamWidth based on your priorities:

Profile	maxConnections	beamWidth	Trade-off
Default	32	100	Balanced for most workloads; matches hnswlib `M=16` density
High recall	48	200	Better accuracy, slower builds, more memory
Fast indexing	24	80	Faster builds, slightly lower recall
Memory constrained	16	60	Minimal memory footprint

maxConnections is the Vamana per-layer graph degree; unlike hnswlib’s M it is not doubled at the base layer, so to match an hnswlib M set maxConnections = 2 * M.

For datasets over 100K vectors or with 1024+ dimensions, enable hierarchical mode:

CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 1536,
  similarity: 'COSINE',
  quantization: 'INT8',
  addHierarchy: true,
  maxConnections: 32,
  beamWidth: 200
}

Tuning efSearch

The efSearch parameter controls how many candidates the search explores at query time. By default, ArcadeDB uses an adaptive strategy that works well for most workloads. You only need to tune efSearch if you have specific recall or latency requirements.

Profile efSearch Trade-off

Profile	efSearch	Trade-off
Adaptive (default)	auto	Two-pass: fast first pass (`2×k`), wider retry (`10×k`) if needed
High recall	200-500	Consistent high accuracy, higher latency
Low latency	20-50	Fast responses, lower recall on hard queries

Adaptive (default)

auto

Two-pass: fast first pass (2×k), wider retry (10×k) if needed

High recall

200-500

Consistent high accuracy, higher latency

Low latency

20-50

Fast responses, lower recall on hard queries

You can override efSearch per-query without changing the index:

-- High recall for a critical search
SELECT expand(vector.neighbors('Doc[embedding]', $queryVector, 10, 500))

-- Low latency for autocomplete/typeahead
SELECT expand(vector.neighbors('Doc[embedding]', $queryVector, 5, 30))

Or set a default on the index:

CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 768,
  similarity: 'COSINE',
  quantization: 'INT8',
  efSearch: 200
}

Store multiple embeddings per record for different search modalities:

CREATE VERTEX TYPE Product
CREATE PROPERTY Product.imageEmbedding ARRAY_OF_FLOATS
CREATE PROPERTY Product.textEmbedding  ARRAY_OF_FLOATS

CREATE INDEX ON Product (imageEmbedding) LSM_VECTOR METADATA {dimensions: 512, similarity: 'COSINE'}
CREATE INDEX ON Product (textEmbedding)  LSM_VECTOR METADATA {dimensions: 768, similarity: 'COSINE'}

Query each index independently:

-- Search by image similarity
SELECT name, distance FROM (
  SELECT expand(vector.neighbors('Product[imageEmbedding]', $imageVector, 10))
)

-- Search by text similarity
SELECT name, distance FROM (
  SELECT expand(vector.neighbors('Product[textEmbedding]', $textVector, 10))
)

Hybrid Search: Dense + Sparse + Full-Text

Combine dense vector similarity with sparse retrieval and/or keyword matching in a single server-side query. vector.fuse accepts any number of ranked sub-pipelines plus a fusion strategy (RRF, DBSF, LINEAR) and returns one ranked top-K:

-- Schema: dense + sparse properties + indexes (sparse needs LSM_SPARSE_VECTOR).
CREATE PROPERTY Document.dense   ARRAY_OF_FLOATS;
CREATE PROPERTY Document.tokens  ARRAY_OF_INTEGERS;
CREATE PROPERTY Document.weights ARRAY_OF_FLOATS;

CREATE INDEX ON Document (dense) LSM_VECTOR
  METADATA { dimensions: 384, similarity: 'COSINE' }

CREATE INDEX ON Document (tokens, weights) LSM_SPARSE_VECTOR
  METADATA { dimensions: 30000, modifier: 'IDF' }

-- Hybrid retrieval in one statement.
SELECT expand(`vector.fuse`(
    `vector.neighbors`('Document[dense]', :denseVec, 50),
    `vector.sparseNeighbors`('Document[tokens,weights]', :qIdx, :qVal, 50),
    { fusion: 'RRF', groupBy: 'source_file', groupSize: 1 }
)) LIMIT 10

vector.neighbors exposes distance (lower = better); vector.fuse auto-flips it so dense and sparse sources compose without manual rescaling.
groupBy + groupSize collapse same-source duplicates server-side. Drop the option to return chunk-level results.
Pre-fusion grouping is also possible by attaching { groupBy: 'source_file', groupSize: 1 } to each individual source.

To include full-text alongside dense/sparse, add a third source built from SEARCH_INDEX:

SELECT expand(`vector.fuse`(
    `vector.neighbors`('Document[dense]', :denseVec, 100),
    `vector.sparseNeighbors`('Document[tokens,weights]', :qIdx, :qVal, 100),
    (SELECT @rid, $score FROM Document
     WHERE SEARCH_INDEX('Document[content]', 'machine learning') = true),
    { fusion: 'RRF' }
)) LIMIT 10

Pick the strategy that matches your scoring shape:

RRF — rank-only, indifferent to score scales. Default, safest with mixed source types.
DBSF — mean +/- 3sigma normalisation per source then weighted sum. Use when scores are roughly Gaussian on each side.
LINEAR — per-source min-max normalisation then weighted sum. Use with already-tuned offline weights.

For the legacy two-query workaround (still supported via vector.rrfScore and vector.hybridScore on already-computed scores), see the SQL Vector Functions reference.

Batch Ingestion

For bulk loading vectors, batch your inserts within transactions:

BEGIN

CREATE VERTEX Document SET content = 'First document',  embedding = [0.1, 0.2, ...]
CREATE VERTEX Document SET content = 'Second document', embedding = [0.3, 0.4, ...]
-- ... more inserts ...

COMMIT

For large bulk loads, increase mutationsBeforeRebuild to delay index rebuilds until after the load completes, then trigger a rebuild.

Since v26.10.1 a bulk load through GraphBatch - which is what GraphImporter, the arcadedb-integration importers and the HTTP batch endpoint use - needs none of that tuning. ArcadeDB holds the automatic graph rebuild off for the whole load and runs it once at the end. Before that, the rebuild was triggered by the index going quiet, which during a load means the loader paused for an index compaction or a garbage collection rather than that it had finished: a 4.2M-vertex load rebuilt the graph four times, each over a partial dataset and each made obsolete by the rest of the load, and had not completed after six and a half hours. The same release also bounds the memory the waiting vectors occupy (see deltaCacheSize under Settings) - together these are what let a load with a vector index finish in the same time as one without.

When vectors are inserted below the rebuild threshold, an inactivity timer ensures the graph is still rebuilt after a period of no new mutations (default: 15 seconds). On a small graph (under 1,000 vectors) this rebuild is cheap and fires for any number of pending mutations, so buffered vectors never sit in the brute-force delta buffer for long during low-volume ingestion. On a larger graph a rebuild re-indexes the whole graph, so the timer only fires once pending mutations reach at least 10% of the effective rebuild threshold - a single stray insert into a large, otherwise-settled index no longer costs a full graph rebuild on the next quiet period. "How large" is measured against the vectors the index actually holds, not against the part of the graph the current session has loaded: the graph loads lazily on the first query, so a process that reopens a database, writes and then goes idle without ever querying it is gated exactly like one that queried first. Configure the timer via inactivityRebuildTimeoutMs (per-index metadata or arcadedb.vectorIndex.inactivityRebuildTimeoutMs globally); set to 0 to disable it.

Since v26.10.1 the same thresholds also decide what happens on the first search after reopening a database. Vectors written since the graph was last persisted are read back into the delta buffer and answered from there, and the rebuild that folds them into the graph is scheduled only once they reach the threshold, or once the delta scan they cause grows expensive (see maxDeltaScanRatio). Previously that first search always started a full rebuild however small the gap, so a process that opened a database, searched once and exited re-indexed the whole graph for a single buffered vector, waited for that rebuild on close(), and found the same vector waiting again on the next open - the rebuild having been cancelled before it could be persisted.

Searching Vectors Not Yet in the Graph

Between rebuilds, everything written since the last one lives in an in-memory delta buffer rather than in the HNSW graph. Those vectors are still searchable: every query scores the buffer and merges the result into the graph’s own, in distance order, so a search never returns a corpus older than the last write. The cost is linear in the size of the buffer, which is what mutationsBeforeRebuild, rebuildGraphRatio and the inactivity timer between them decide.

A graph build also occasionally leaves a vector with no incoming edges, which no search can reach at any efSearch however close it is to the query. Those are served from the delta buffer too, and since v26.10.1 the list of them is stored next to the graph, so they stay findable after a restart. unreachableGraphNodes reports how many the current graph has, readable straight after a reopen and before the first query:

-- Normally 0, or a handful in a large index. A high value means the corpus holds many near-identical vectors,
-- which a build cannot link into a navigable graph, and every query pays a scan for them
SELECT unreachableGraphNodes, deltaVectorsCount FROM (SELECT expand(stats) FROM schema:indexes WHERE name = 'Document[embedding]')

Before v26.10.1 that list lived only in the memory of the session that built the graph. After a restart those vectors were absent from every search while the index still counted them, and since a query still returned k results there was nothing to indicate it.

Before v26.9.1 this was true of every query except a grouped one. Adding groupBy routed the search down a path that read the graph only, so it answered from the corpus as of the last rebuild while the same query without groupBy, on the same index at the same instant, returned the newer rows - with no error, no warning and nothing in the statistics to say so. On default settings the invisible window was as wide as rebuildGraphRatio allows the buffer to grow, and stayed open for as long as writes kept arriving. If you worked around this by dropping groupBy and grouping in the application, that workaround is no longer needed.

The groupedSearchesMergingDelta counter reports how many grouped searches actually took rows out of the buffer:

-- Rising in step with your grouped query rate means the graph is persistently behind the write rate,
-- and every grouped query is paying a linear scan of the buffer; deltaVectorsCount is how long that scan is
SELECT groupedSearchesMergingDelta, deltaVectorsCount FROM (SELECT expand(stats) FROM schema:indexes WHERE name = 'Document[embedding]')

A high value is not itself a fault - it is the index doing what it should under ingestion. It is the signal to shorten the scan by lowering mutationsBeforeRebuild or rebuildGraphRatio, if query latency matters more than the CPU the extra rebuilds cost.

Rebuild Memory and Deferred Rebuilds

A graph rebuild is the most memory-hungry thing a vector index does, and an online rebuild - one triggered by the mutation threshold or the inactivity timer while the index keeps serving queries - is the most expensive kind. The graph being replaced stays resident so searches keep working, and the new build pays for its own working set on top of it (the build cache, the graph under construction, the ordinal map). Measured on a 50,000 x 128 corpus built and rebuilt inside one session, that is roughly 1.7x the peak of building the same index from nothing. It is much less after a reopen, where the graph being replaced is served from pages rather than from the heap: since v26.10.1 the gate measures what that graph actually costs instead of assuming a second copy of it in memory.

Two behaviours follow from that, both new in v26.9.1:

The auto-sized search cache budgets against available heap, not total heap. searchCacheMaxHeapPercent is a share of the heap actually free at the time, so a rebuild holding the old graph resident asks for a smaller cache instead of the same one. Previously it was a share of -Xmx, which meant a larger heap grew the cache proportionally and a rebuild that did not fit still did not fit after raising it.
A rebuild that will not fit is deferred instead of attempted. Before starting, the estimated peak is compared against arcadedb.vectorIndex.rebuildMaxHeapPercent (default 90) of the available heap. If it does not fit, the cycle is skipped rather than run into an OutOfMemoryError. Nothing is lost: the pending vectors remain fully searchable through the in-memory delta buffer, so the cost is a longer brute-force scan per query until a later rebuild succeeds.

Since v26.10.1 both sides of that comparison are more accurate, and a large index in a tight heap that used to log deferrals indefinitely may now rebuild:

What keeping the old graph resident costs is measured, not assumed. After a reopen the graph being replaced lives in pages, so it retains a fraction of what a freshly built one does. Charging it as a second in-memory graph could refuse a rebuild that fitted comfortably - and one no larger than the rebuild the close path runs without asking.
The page cache counts as reclaimable, because it is. Available heap is measured after the last collection, and cached pages count as used even though the engine can drop any of them and read them back from disk. When a rebuild fits only by giving some of that up, exactly the shortfall is evicted first - oldest pages, across every open database, since the cache is shared - and if the cache cannot actually hand those bytes over the rebuild is still deferred rather than run on heap it was not given. Expect a few slower queries afterwards while the evicted pages are read back.

Since v26.10.1 the graph-build cache is sized differently from the search cache, and the difference matters when you are planning a large ingest:

graphBuildCacheMaxHeapPercent is a share of -Xmx, capped at 90% of the heap currently available. Taking the percentage of free heap made the same corpus build very differently depending on how the data got in. A server that has just ingested the corpus over HTTP holds it in its own heap, so the free-heap reading is small and the default paid for a fraction of the corpus; an embedded build of the identical data, held outside the JVM until the build, saw almost the whole heap free and cached everything. On a 10M x 96-dim corpus with the same -Xmx that was a 3.4x difference in build time from the same default, and two identical runs could differ 4x between themselves depending on when the JVM had last collected. Taking the percentage of the ceiling makes the choice a property of your configuration rather than of the moment; the available-heap cap is what still shrinks the cache when an online rebuild is genuinely holding the old graph.
The percentage now applies to inline-quantized (INT8, BINARY) indexes too. It previously did nothing for them: their build cache was pinned at 100,000 vectors whatever the setting said, and only an explicit graphBuildCacheSize moved it. Note the consequence for a tight heap - an INT8 index that used to hold ~45 MB of build cache at 96 dimensions will now use up to its share of -Xmx, because the cache holds decoded float vectors regardless of how they are stored. Set graphBuildCacheMaxHeapPercent lower, or pin graphBuildCacheSize, if that share is not available to you.
The chosen capacity is logged next to the corpus size, as cache enabled: size=3674697 of 9990000, so a cache that is too small to cover the corpus is visible in one line at INFO.

A deferral is deliberately visible. It logs a warning naming what to change, and increments the rebuildsDeferredForMemory counter in the index statistics:

-- Non-zero and climbing means rebuilds keep being declined for lack of heap:
-- the graph is going stale and every query is paying a longer delta scan for it
SELECT rebuildsDeferredForMemory FROM (SELECT expand(stats) FROM schema:indexes WHERE name = 'Document[embedding]')

If it keeps climbing, give the JVM more heap, lower arcadedb.vectorIndex.graphBuildCacheMaxHeapPercent so the build asks for less, or split the index. Setting arcadedb.vectorIndex.rebuildMaxHeapPercent to 0 disables the check and restores the previous attempt-regardless behaviour.

Because a deferral does not consume the mutations that triggered it, the trigger would otherwise fire again on the very next query. arcadedb.vectorIndex.rebuildDeferralCooldownMs (default 30 seconds) is the minimum gap before another attempt; a rebuild that completes clears it immediately.

Only online rebuilds are subject to this check. A first build, the rebuild performed when a database closes, an explicit REBUILD INDEX and COMPACT INDEX are never declined - nothing would retry them, so declining one would mean it never happens at all.

If you create the index before inserting data (e.g., during schema setup), set buildGraphNow: false to skip the initial (empty) graph build. The graph will be built lazily on the first search:

-- Schema setup phase: defer graph build since no data exists yet
CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8',
  buildGraphNow: false
}

-- Bulk load data...
-- Graph is built automatically on first vector.neighbors() query

If you create the index after data is already loaded, leave buildGraphNow at its default (true) so the index is immediately ready to query.

Global Configuration

Set database-wide defaults for vector index parameters:

ALTER DATABASE `arcadedb.vectorIndex.locationCacheSize` 100000
ALTER DATABASE `arcadedb.vectorIndex.graphBuildCacheSize` 10000
ALTER DATABASE `arcadedb.vectorIndex.mutationsBeforeRebuild` 100
ALTER DATABASE `arcadedb.vectorIndex.inactivityRebuildTimeoutMs` 15000
ALTER DATABASE `arcadedb.vectorIndex.storeVectorsInGraph` false

Per-index metadata overrides these global settings.

Vector Embeddings

Choosing an Embedding Model

Creating a Vector Index

Choosing a Similarity Function

Quantization Trade-offs

INT8 Pre-Quantized Ingest

Tuning for Recall vs Speed

Tuning efSearch

Multi-Modal Embeddings

Hybrid Search: Dense + Sparse + Full-Text

Batch Ingestion

Searching Vectors Not Yet in the Graph

Rebuild Memory and Deferred Rebuilds

Global Configuration

Further Reading