Tag: ChromaDB

  • Troubleshooting ChromaDB InvalidDimensionException: Resolving the 1536 vs 384 Dimension Mismatch

    Troubleshooting ChromaDB InvalidDimensionException: Resolving the 1536 vs 384 Dimension Mismatch

    TL;DR

    • ChromaDB raises an InvalidDimensionException when a collection defaults to a 384-dimensional embedder at creation but receives 1536-dimensional query vectors from models like OpenAI.
    • ChromaDB collections enforce a strict, immutable coordinate space that cannot be modified in place once initialized.
    • To prevent dimension mismatches, developers must explicitly configure the desired embedding function when creating the collection.

    Diagnosing the InvalidDimensionException When Using OpenAI Embeddings

    This exception typically manifests when a collection is populated using ChromaDB’s default embedding pipeline and subsequently queried with an external OpenAI embedding model. In a report involving ChromaDB 0.4.24 and Python 3.11.8, a user encountered this failure during retrieval:

    InvalidDimensionException: Embedding dimension 1536 does not match collection dimensionality 384

    The underlying issue stems from a mismatch between the embedding function used at document ingestion and the embedding function used at query time. The following pattern illustrates how this mismatch occurs in code:

    persistent_client = chromadb.PersistentClient()
collection = persistent_client.create_collection("emp6")

# Adding documents without specifying an embedding function defaults to 384 dimensions
for doc in docs:
    collection.add(
        ids=[str(uuid.uuid1())], metadatas=doc.metadata, documents=doc.page_content
    )

# Querying via LangChain Chroma initialized with OpenAI embeddings (1536 dimensions)
langchain_chroma = Chroma(
    client=persistent_client,
    collection_name="emp6",
    embedding_function=OpenAIEmbeddings(),
)
docs1 = langchain_chroma.similarity_search("What is venture")

    This snippet demonstrates creating a collection and adding documents without providing an explicit embedding_function, followed by executing a similarity search through LangChain’s Chroma wrapper configured with OpenAIEmbeddings().

    When calling create_collection without an explicit embedding_function parameter, ChromaDB applies its default embedding function, producing vector embeddings with 384 dimensions when documents are added via collection.add(...). When querying that same collection through an interface configured with OpenAIEmbeddings()—such as LangChain’s similarity_search—the incoming query text is converted into a 1536-dimensional vector. Because the existing collection was already indexed at 384 dimensions, ChromaDB rejects the 1536-dimensional query vector with chromadb.errors.InvalidDimensionException.

    Sources: [Bug]: InvalidDimensionException: Embedding dimension 1536 does not match collection dimensionality 384

    Why ChromaDB Defaults to 384 Dimensions and Enforces Fixed Index Dimensionality

    To understand why this mismatch occurs, it helps to look at how ChromaDB handles vector spaces by default. ChromaDB sets a default dimensionality of 384 because its built-in embedder, DefaultEmbeddingFunction, uses the Sentence Transformers model all-MiniLM-L6-v2, which generates 384-dimensional embeddings. When a collection is initialized using create_collection or get_or_create_collection without specifying the embedding_function parameter, ChromaDB applies this default embedder and sets the collection dimensionality to 384.

    ChromaDB enforces strict dimensionality consistency across all records. By default, the underlying vector index is an HNSW graph index constructed for a single, fixed coordinate space, meaning a collection cannot store or query mixed-dimension vectors.

    Furthermore, collection dimensionality cannot be modified in place once initialized. If an operation—including add, upsert, update, or query—attempts to pass embeddings that do not match the collection’s expected dimensionality, ChromaDB rejects the operation and raises chromadb.errors.InvalidDimensionException.

    Sources: Embedding Functions – Chroma Docs, Adding Data to Chroma Collections – Chroma Docs, Configure Collections – Chroma Docs, chromadb.errors.InvalidDimensionException: Dimensionality of (512) does not match index dimensionality (768) ·, Sentence Transformer – Chroma Docs

    Configuring ChromaDB Collections with OpenAI's 1536-Dimension Embedding Function

    To prevent this conflict and ensure your collection accepts 1536-dimensional vectors, you must explicitly supply an embedding function during collection creation. In Python, the chromadb package includes built-in wrappers for providers, allowing you to import OpenAIEmbeddingFunction directly from chromadb.utils.embedding_functions. You must also set your OPENAI_API_KEY environment variable.

    When calling create_collection, link the embedding function to the collection using the embedding_function parameter:

    # Set your OPENAI_API_KEY environment variable
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

collection = client.create_collection(
    name="my_collection",
    embedding_function=OpenAIEmbeddingFunction(
        model_name="text-embedding-3-small"
    ),
)

# ChromaDB will use OpenAIEmbeddingFunction to embed your documents
collection.add(
    ids=["id1", "id2"],
    documents=["doc1", "doc2"],
)

    This configuration ensures ChromaDB automatically invokes OpenAIEmbeddingFunction to generate vector representations whenever you call add, update, upsert, or query on the collection using raw text documents.

    For TypeScript applications, install the provider package using a package manager (such as npm install @chroma-core/openai, pnpm add @chroma-core/openai, bun add @chroma-core/openai, or yarn add @chroma-core/openai). You can then pass the initialized instance to createCollection:

    // Set your OPENAI_API_KEY environment variable
import { OpenAIEmbeddingFunction } from "@chroma-core/openai";

collection = await client.createCollection({
  name: "my_collection",
  embedding_function: new OpenAIEmbeddingFunction({
    modelName: "text-embedding-3-small",
  }),
});

await collection.add({
  ids: ["id1", "id2"],
  documents: ["doc1", "doc2"],
});

    This TypeScript setup links OpenAIEmbeddingFunction to the collection so that document insertions and queries are embedded using the specified model rather than the default.

    In environments using the Rust client, embedding functions are not attached to the collection; instead, embeddings must be generated separately with your provider SDK and passed directly into calls such as add and query.

    Sources: Embedding Functions – Chroma Docs

    Deleting, Recreating, and Re-indexing Data in ChromaDB

    Once an appropriately configured collection is in place—or when recreating one from scratch—you can safely begin populating it. When inserting records into a collection, ChromaDB raises chromadb.errors.InvalidDimensionException if the supplied embeddings do not match the dimensionality of embeddings already in the collection. Additionally, attempting to re-add a record using an ID that already exists in the collection will be ignored without raising an error; overwriting existing data instead requires updating the data.

    To index new records into a collection, use .add. Each record requires a unique string ID, and you must supply either documents, embeddings, or both. Metadata is optional and can contain strings, integers, floats, booleans, or arrays of those types (empty arrays are not permitted, and all elements in an array must share the same type).

    If you pass only documents, ChromaDB automatically computes embeddings using the collection’s configured embedding_function:

    collection.add(
    ids=["id1", "id2", "id3"],
    documents=["lorem ipsum...", "doc2", "doc3"],
    metadatas=[{"chapter": 3, "verse": 16}, {"chapter": 3, "verse": 5}, {"chapter": 29, "verse": 11}],
)

    This call generates embeddings for each document via the configured collection embedding_function before inserting them.

    If embeddings are precomputed, you can provide them alongside documents. ChromaDB stores the provided embeddings as-is without re-embedding:

    collection.add(
    ids=["id1", "id2", "id3"],
    embeddings=[[1.1, 2.3, 3.2], [4.5, 6.9, 4.4], [1.1, 2.3, 3.2]],
    documents=["doc1", "doc2", "doc3"],
    metadatas=[{"chapter": 3, "verse": 16}, {"chapter": 3, "verse": 5}, {"chapter": 29, "verse": 11}],
)

    This method is used when you already have vector representations and want to avoid recalculating embeddings during ingestion. If your documents are stored externally (such as large images or videos), you can also pass only ids, embeddings, and metadatas, using the IDs to link the stored vectors back to external files.

    Sources: Adding Data to Chroma Collections – Chroma Docs

    Closing thoughts

    Ultimately, this error highlights the hidden hazard of ChromaDB’s default behavior: while falling back to a 384-dimensional model simplifies initial prototyping, it makes multi-tool pipelines—especially those bridging LangChain and OpenAI—deceptively fragile. Because Chroma’s underlying HNSW index enforces a strict, immutable coordinate space that cannot be modified in place, treating the embedding function as an optional setting is a risky architectural shortcut rather than a true convenience. Developers should view embedding function configuration as a mandatory schema definition from day one, explicitly establishing the expected dimensionality at collection creation to avoid the costly necessity of tearing down and re-indexing stored data.

    Frequently Asked Questions

    What causes the error ‘Embedding dimension 1536 does not match collection dimensionality 384’ in ChromaDB?

    This error occurs when a collection is created without an explicit embedding function, defaulting to 384 dimensions, and is subsequently queried with an external model like OpenAI that generates 1536-dimensional embeddings.

    Why does ChromaDB default to 384 dimensions?

    ChromaDB sets a default dimensionality of 384 because its built-in DefaultEmbeddingFunction relies on the Sentence Transformers model all-MiniLM-L6-v2, which outputs 384-dimensional vectors.

    Can you modify the dimensionality of an existing ChromaDB collection?

    No, collection dimensionality cannot be modified in place once initialized. ChromaDB’s underlying HNSW index enforces a fixed coordinate space, and any operation supplying mismatched vector dimensions will trigger an InvalidDimensionException.

    How can you configure a collection in Python to accept 1536-dimensional OpenAI embeddings?

    You can import OpenAIEmbeddingFunction from chromadb.utils.embedding_functions and pass it to the embedding_function parameter when calling create_collection, while ensuring the OPENAI_API_KEY environment variable is set.


    Keep reading: LLM Agents & Tooling

    All guides in this series: LLM Agent Guides: LangChain, LangGraph & AI Tooling

  • Resolving SQLite Database Lock Errors in ChromaDB During Concurrent Writes

    Resolving SQLite Database Lock Errors in ChromaDB During Concurrent Writes

    TL;DR

    • SQLite enforces a global write lock in Embedded ChromaDB, which causes SQLITE_BUSY database lock errors during concurrent write operations and read-to-write lock upgrades.
    • Standard write contention can be mitigated by keeping insert batch sizes between 50 and 250 records and raising SQLite’s own PRAGMA busy_timeout to at least 5 seconds — this is a general SQLite setting, not a parameter ChromaDB itself exposes.
    • Transitioning to ChromaDB’s Client-Server mode using HttpClient or AsyncHttpClient resolves embedded SQLite locking limits by offloading concurrency control to a standalone server process.

    Understanding Embedded ChromaDB and SQLite Persistence Engine

    Embedded ChromaDB handles local database persistence using PersistentClient. This client automatically saves database files to the local machine and reloads them on startup if previous data exists.

    The storage location on disk is configured through the path parameter, which determines where database files are written and loaded from. If no path is explicitly provided, the client defaults to using .chroma.

    The persistence client exposes programmatic methods for system management:

    • heartbeat(): Returns a nanosecond heartbeat value, useful for confirming the client is still connected.
    • reset(): Empties and completely resets the database. Executing this method is destructive and cannot be reversed.

    Sources: docs.trychroma.com — Persistent Client

    Root Causes of SQLite Locking Issues in Concurrent Write Workloads

    Underneath this local persistence mechanism, SQLite manages concurrent write operations by enforcing a global write lock that permits only one writer at a time. When a write transaction begins, it holds this global lock for its entire duration, blocking all other write attempts until the transaction completes.

    The SQLITE_BUSY error (“database is locked”) occurs whenever a transaction cannot acquire this global write lock. If another transaction holds the lock, even a simple insert operation fails immediately with this error.

    A distinct edge case occurs when an active read transaction attempts to upgrade to a write transaction. If another database connection has already modified the database or is in the process of modifying it, this upgrade attempt fails immediately with SQLITE_BUSY. Unlike general write lock contention, this specific read-to-write lock upgrade failure is not helped by setting a busy timeout.

    To address standard write lock contention, SQLite itself — independent of ChromaDB, which does not expose this as a documented client parameter — provides a general pragma, PRAGMA busy_timeout. This sets the duration that transactions will wait to acquire the write lock before returning “database is locked” instead of failing immediately. In production systems, a PRAGMA busy_timeout setting of 5 seconds or more is recommended.

    Sources: tenthousandmeters.com — SQLite concurrent writes and "database is locked" errors

    Mitigating Write Contention Through Batch Size and Understanding SQLite Storage Behavior

    Beyond PRAGMA busy_timeout, how ingestion parameters are configured also shapes how often write locks get contended in the first place. ChromaDB documentation recommends keeping insert batch sizes on the smaller side, specifically between 50 and 250 records. Choosing batch sizes in this range yields lower and more consistent latency while making writes less likely to encounter timeout errors. Although overall throughput plateaus and remains fairly flat across batch sizes between 100 and 500, smaller batches are preferred to prevent latency spikes and timeouts.

    For insert write concurrency, ChromaDB records writes to a log and flushes them every N operations. Because of this logging and flushing mechanism, mean latency does not fluctuate as the number of concurrent writers increases.

    Separately from locking behavior, ChromaDB’s on-disk footprint is worth understanding on its own: the database saves metadata and documents via SQLite, and disk usage is highly variable, depending entirely on whether full documents and metadata are being retained. For example, a sample collection containing approximately 40,000 documents (averaging 1,000 words each) and roughly 600,000 metadata entries requires about 1.7GB of storage. SQLite handles database disk paging effectively and supports database sizes scaling into the terabyte range.

    Sources: docs.trychroma.com — Performance guide (single node)

    Architectural Alternatives: Embedded SQLite vs Client-Server Mode for Multi-Threaded Writes

    When local tuning strategies and parameter adjustments prove insufficient for multi-threaded or multi-process write workloads, ChromaDB supports Client-Server ChromaDB where applications interact with a standalone server process instead of managing local storage. Synchronous access is established using HttpClient, whereas non-blocking access is provided via AsyncHttpClient. The two client implementations maintain identical method signatures and behaviors, but AsyncHttpClient makes all methods that would otherwise block run asynchronously.

    Configuring the standalone server instance is handled through environment variables:

    • CHROMA_PERSIST_PATH: Dictates the directory used for persisted data. This defaults to ./chroma in the frontend configuration and is commonly set to /data in container deployments.
    • CHROMA_SQLITE_FILENAME: Specifies the SQLite database filename created under the persist path, defaulting to chroma.sqlite3.
    • CHROMA_LISTEN_ADDRESS: Sets the bind address for the frontend server, defaulting to 0.0.0.0.
    • CHROMA_PORT: Defines the HTTP listening port for the frontend server, defaulting to 8000.

    Adopting Client-Server ChromaDB provides an architectural alternative to running Embedded ChromaDB directly inside client processes. In one third-party case, developers maintaining the MemPalace project reported encountering chromadb.errors.InternalError: Error in compaction: Failed to apply logs to the hnsw segment writer during concurrent multi-process operation. Their assessment was that Embedded ChromaDB’s SQLite database and HNSW segment writers are inherently not thread- or process-safe for concurrent writers. To address this within their application, they implemented a workaround by switching from the embedded PersistentClient to HttpClient, offloading concurrency and execution control entirely to the server process.

    It’s worth being precise about what this MemPalace failure actually is: an HNSW compaction error is a different failure mode from the SQLITE_BUSY lock errors described earlier in this article. It isn’t SQLite’s write-lock contention — it originates from concurrent access to the HNSW vector-index segment writer instead. PRAGMA busy_timeout has no bearing on it. The two problems just happen to share the same fix here (moving to Client-Server mode), not the same underlying mechanism.

    Sources: docs.trychroma.com — Client-Server mode (HttpClient / AsyncHttpClient), docs.trychroma.com — Server environment variables, MemPalace/mempalace issue #832

    Closing thoughts

    Ultimately, while Embedded ChromaDB provides a convenient local persistence mechanism, attempting to push it through heavy concurrent write workloads reveals clear architectural limitations inherent to embedded SQLite. Keeping batch sizes between 50 and 250 records, and raising SQLite’s own PRAGMA busy_timeout (a general SQLite setting, not something ChromaDB exposes as a client parameter), can alleviate standard SQLite write contention, but neither helps with unhandled read-to-write lock upgrades, and neither has any bearing on HNSW compaction failures — a separate concurrency issue in the HNSW segment writer, illustrated so far only by one third party’s reported case, rather than a SQLite locking problem. In my judgment, pushing heavy concurrent write workloads through Embedded ChromaDB carries stability risk on both fronts — SQLite lock contention and (per that one reported case) HNSW segment-writer safety alike. Transitioning to ChromaDB’s Client-Server mode using HttpClient or AsyncHttpClient remains the most definitive solution: it doesn’t make the underlying SQLite database disappear (the server itself still persists to chroma.sqlite3), but it stops multiple application processes from contending directly over the same local SQLite file, and in the MemPalace case, moving to HttpClient was also reported to resolve the HNSW segment-writer failure.

    Frequently Asked Questions

    What causes the SQLITE_BUSY database locked error in ChromaDB?

    The SQLITE_BUSY error occurs when a transaction cannot acquire SQLite’s global write lock because another transaction is holding it. It also happens when an active read transaction attempts to upgrade to a write transaction while another connection is modifying the database.

    What batch size is recommended for ChromaDB insert operations?

    ChromaDB documentation recommends keeping insert batch sizes between 50 and 250 records. This range yields lower, more consistent latency and makes writes less likely to hit timeout errors.

    How does PRAGMA busy_timeout help handle SQLite write lock contention?

    PRAGMA busy_timeout sets the duration that transactions will wait to acquire the write lock before failing with a database locked error. A setting of 5 seconds or more is recommended in production systems.

    How does switching to Client-Server mode solve embedded SQLite locking limitations?

    Transitioning to Client-Server mode offloads concurrency and execution control to a dedicated server process using HttpClient or AsyncHttpClient. This architecture bypasses the embedded SQLite locking constraints and thread-safety limitations.


    Keep reading: LLM Agents & Tooling

    All guides in this series: LLM Agent Guides: LangChain, LangGraph & AI Tooling