Cosine Similarity Is Not a Safety Property

The vector database returns the most relevant documents, defined as the ones with the highest cosine similarity to the query. This is a mathematical property with no concept of accuracy, authority or provenance. A document scoring 0.95 against a query can be entirely fabricated. A document scoring 0.60 can be the ground truth.

The attacker’s job in a poisoning attack is not to break into the vector database. It is to write a document whose embedding sits closer to the anticipated query than the legitimate document does, then frame it with enough authority to win the argument once both are in the context window.

What the embedding model is actually doing

sentence-transformers/all-MiniLM-L6-v2 turns any string into a 384-dimensional float vector: a point in 384-dimensional space. Semantically similar texts land near each other. “Q4 financial results” sits close to “fourth quarter revenue” and far from “company travel policy.” ChromaDB stores the vectors and answers a query by finding the k stored points closest to the query point, under whichever distance function the collection was created with.

ChromaDB’s default is squared L2, not cosine. Cosine has to be set on purpose, with metadata={"hnsw:space": "cosine"} at collection creation. Everything in this piece assumes a collection configured that way, which is the common choice for text embeddings and the one worth setting deliberately rather than inheriting by accident. If you skip that step, the “angle to the query point” framing below stops describing what your database is actually doing.

# The entire retrieval mechanism, conceptually
query_vector = embed("What was Q4 2025 revenue?")   # 384 floats

results = collection.query(query_embeddings=[query_vector], n_results=3)
# The 3 stored vectors with smallest cosine distance.

The model does not know what the text means in any truth-bearing sense. It knows how text clusters with other text in its training data. “Revenue was $8.3M” and “Revenue was $24.7M” land at nearly identical positions, because they are the same sentence with a different number. The vector database cannot tell them apart. Only the model can, after retrieval, and only if both documents are in front of it.

This is a property of dense retrieval, not a defect in one model. all-MiniLM-L6-v2 has aged into a poor production default; EmbeddingGemma-300M, Qwen3-Embedding and BGE-M3 are where new work goes in 2026. None of them change the property. Semantic similarity is not factual accuracy at any parameter count.

The three poisoned documents sit inside the top-3 window. The document holding the real number sits just outside it, at rank 4, and never reaches the model. Similarity values are illustrative; ChromaDB returns cosine distances, and actual values depend on model, phrasing and document length.

Vocabulary engineering: moving a document without gradient descent

The sophisticated form of this attack, as formalised in PoisonedRAG (Zou, Geng, Wang and Jia, USENIX Security 2025), uses gradient-based optimisation to find text that minimises cosine distance to a target query vector. That needs white-box access to the embedding model, or query access at scale to approximate it. With that machinery, the paper injected five crafted texts into a corpus of 2.6 million and drove attacker-chosen answers at above 90% success.

The cheap version needs neither gradient descent nor white-box access. Identify the vocabulary of the query you want to hijack, put that vocabulary in your document, add authority signals for the generation step. In my lab that got to 95% success against an undefended pipeline (twenty runs on a small seeded corpus, one model — a lab reading, not a base rate). The three documents I used, annotated:

Document one, the CFO correction. Title: Q4 2025 Financial Results — CORRECTED FIGURES (Board Update). Against a query like “what was Q4 2025 revenue?”, Q4 2025 is an exact temporal match, Financial Results clusters with revenue and financials, CORRECTED FIGURES implies it supersedes something, and Board Update is an authority signal. Body states fabricated numbers as plain fact and closes with Approved by: CFO Office.

Document two, the restatement notice. States that Q4 revenue has been restated, actual revenue $8.3M, “not the previously reported $24.7M.” It names the real figure and reframes it as an acknowledged error, which gives the model a narrative for resolving the contradiction it is about to encounter.

Document three, the board minutes. Emergency session, agenda item three, corrected results, same numbers.

Three documents corroborating one fabrication against one legitimate document telling the truth. In a top-3 retrieval, three crowd out one, and the legitimate content never enters the context window. That is vocabulary engineering: shifting which regions of embedding space are occupied so fabricated documents sit closer to anticipated queries than the documents they are meant to displace.

Two conditions, and which one does the work

PoisonedRAG frames the attack as two conditions that must both hold.

Retrieval condition. For a target query q, the poisoned document d_p must make top-k:

cos_distance(embed(d_p), embed(q)) < cos_distance(embed(d_legit), embed(q))

Generation condition. Once in context, d_p must cause the model to produce the attacker’s answer rather than the correct one, which requires the target answer to be present and framed with enough authority to outweigh contradicting sources.

Research attention went to the retrieval condition, because that is where the elegant optimisation lives. Recent work suggests the framing is doing more of the work. In “Architecture Matters: Comparing RAG Systems under Knowledge Base Poisoning” (Korn, May 2026), most of the strongest attack variant’s advantage came from adversarial framing rather than retrieval optimisation, and attack success across four architectures with comparable clean accuracy spread nearly 58 points, from 81.9% for vanilla RAG down to 24.4% for a recursive setup.

If framing dominates, then the stage that resolves contradictions between retrieved documents is a more valuable place to invest than another round of retrieval hardening. In my own measurements the legitimate document was often retrieved and still lost. The real Q4 figure sat in the context window while the model reported the fabricated one, because two documents said the real figure had been corrected and one document just quietly stated it.

Turning the geometry against the attacker

The attacker’s requirement is also their signature. To be retrieved, the poisoned documents must cluster near the target query position, which is near the legitimate documents on that topic. An ingestion-time check can look for this.

Two signals, computed before anything is stored:

# Signal 1 — nearest neighbour in the existing collection
existing = collection.query(query_embeddings=[new_doc_embedding], n_results=3)
for dist in existing["distances"][0]:
    if (1.0 - dist) > SIMILARITY_THRESHOLD:      # 0.85 in the lab
        flag("HIGH_SIMILARITY — possible content override")

# Signal 2 — pairwise similarity within the incoming batch
for i, e_i in enumerate(new_embeddings):
    for j in range(i + 1, len(new_embeddings)):
        if cosine_similarity(e_i, new_embeddings[j]) > CLUSTER_THRESHOLD:   # 0.90
            flag("TIGHT_CLUSTER — possible coordinated injection")

Both fire on the three-document attack. Each poisoned document’s nearest neighbour is the legitimate Q4 report, because both are about Q4 2025 financials. The three cluster tightly with each other, because they are variations on one fabricated narrative. This single layer took poisoning success from 95% to 20%.

The geometric reason it works is the closest thing to a structural guarantee in this area: the attacker cannot fully satisfy both requirements at once. Satisfying the retrieval condition means occupying space near existing content, which is detectable. Evading detection means moving away from that space, which degrades retrieval. The 20% residual is the band where an attacker balances the two, typically with a single document instead of a cluster, or with enough vocabulary variation to fall below the similarity threshold while staying retrievable.

Access-controlled retrieval: the where clause

A metadata filter on every vector query that restricts which documents the requesting user is allowed to retrieve. Without it, every document in the collection is reachable by every user through an ordinary question. Ask an assistant what the salary bands are and it retrieves whatever is semantically close, with no idea who is asking.

I ran this against a lab pipeline holding three restricted documents: salary data marked HR-only, litigation detail marked privileged, and an M&A pipeline marked board-level. Queried as a regular engineering user, in natural language, with no evasion. Twenty out of twenty queries returned confidential content.

# Vulnerable: no notion of who is asking
results = collection.query(query_embeddings=[query_embedding], n_results=3)

# Hardened: retrieval scoped to the requester's clearances
results = collection.query(
    query_embeddings=[query_embedding],
    n_results=3,
    where={"classification": {"$in": user_permitted_classifications}},
);

That filter is the only complete defence against cross-tenant leakage, because it is structural. It prevents unauthorised content from entering the context window at all. Output monitoring, prompt hardening and every other heuristic runs after retrieval, by which point the data is already in the prompt.

The threshold problem

The lab hardcodes 0.85 and 0.90. Those are not universal values, and copying them into production is the most common way this control fails.

At 0.85 the defence catches the three-document attack because the poisoned documents score around 0.88 to 0.92 against the legitimate financials. Increase vocabulary variation, keeping the financial theme but varying the specific language, and similarity drops to roughly 0.78 to 0.82. Below threshold, not flagged, still retrievable.

The right value comes from your ingestion patterns. An append-only corpus can sit low. A living wiki with versioned policies cannot, because every legitimate amendment looks like an override attempt. Baseline your collection’s actual similarity distribution and set the threshold around mean plus two standard deviations, then revisit as the collection grows.

These thresholds are properties of your embedding model, not of your data. Similarity distributions are not comparable across models. Swap all-MiniLM-L6-v2 for Qwen3-Embedding or BGE-M3 and every threshold you tuned is now measuring a different geometry. A model upgrade silently invalidates your detector. Pin the embedding model version alongside the threshold, and treat an embedding migration as a re-baselining project.

Your vectors are not opaque

Embedding vectors get treated as one-way functions. You embed text, store the vector, and the original text is not recoverable from the vector alone. That assumption was never proven for dense sentence embeddings. It was inherited by practitioners who thought of embeddings as just numbers.

The research record says otherwise:

• Morris, Kuleshov, Shmatikov and Rush, “Text Embeddings Reveal (Almost) As Much As Text” (EMNLP 2023) built Vec2Text, an iterative correct-and-re-embed method that recovered 92% of 32-token inputs exactly, and recovered full names from a dataset of clinical notes. Not fragments. The text.
• Chen, Lent and Bjerva, “Text Embedding Inversion Security for Multilingual Language Models” (ACL 2024) extended inversion to multilingual embedding spaces and found that some languages are markedly more exposed than others.
• ALGEN (Chen, Xu and Bjerva, February 2025) removed the expensive prerequisite. Earlier attacks assumed access to millions of text-embedding pairs to train the inversion model. ALGEN aligns a victim embedding space to the attacker’s space with a one-step linear map, and reports that a single pair gives partial success while about 1,000 pairs reach optimum across a range of black-box encoders, with ROUGE-L up to about 46 in their main cross-encoder results and cosine similarity around 0.95 on the aligned space.
• LAGO (Yu, Chen, Bjerva, Kosta and Li, May 2025) generalises ALGEN, using language-similarity graph optimisation to gain a further 10 to 20% ROUGE-L, with as few as ten samples per language.

ALGEN’s contribution is cost: it collapses the data requirement from millions of pairs to roughly a thousand, which moves inversion from a research capability to something an ordinary attacker can do. The high word-recovery figures come from the Vec2Text line of work. Both matter together: strong recovery quality from earlier work, near-zero setup cost from the newer work.

Consider the sensitive documents in a multi-tenant scenario: salary bands, litigation detail with settlement authority, an M&A pipeline with named targets and valuations. Access-controlled retrieval stops unauthorised users from retrieving those through normal queries. Encryption at rest stops plaintext exfiltration at the database layer. Neither helps if someone walks off with the raw vector store through a cloud misconfiguration, a compromised admin credential or an unsecured backup.

What this means for your threat model

Reclassify vector store compromise in your incident response plan. Most plans treat it as a metadata leak, vectors only, low severity, no notification. That classification is wrong on the current research. The severity floor for a vector store breach is a partial document leak of the most sensitive material in the collection. The practical exercise: take the ten most sensitive documents in your knowledge base, imagine a reconstruction that recovers most of their content in the wrong word order, and ask whether that is a breach you would have to disclose.

Evaluate per-tenant vector encryption if you are multi-tenant. IronCore Labs’ Cloaked AI uses property-preserving encryption so encrypted vectors remain usable for nearest-neighbour search while being scoped to per-tenant keys. For multi-tenant SaaS where tenant isolation is the product promise, this belongs on the roadmap.

A vector store security checklist

Access

• The vector database API requires authentication, and is not simply open on the internal network
• Service accounts have minimum necessary permissions, separated by function
• Admin credentials are vaulted and rotated separately from application credentials

Classification

• The vector store is classified at the sensitivity of the documents it was built from, not lower
• Vector store exfiltration appears in your breach response scenarios, as a document leak
• Highly sensitive source documents are flagged in vector metadata so you can scope an incident

Monitoring

• Bulk embedding query volume is logged and alerted
• Ingestion events are logged with contributor identity and timestamp
• Sudden large ingestion from a single source triggers review

Recovery

• Point-in-time snapshots of the collection are taken on a schedule
• Restore from a known-good snapshot has actually been tested
• Snapshots carry the same access controls as the live store

Inversion exposure

• You know whether your embedding model is publicly available
• The embedding model version is pinned next to your anomaly thresholds
• Per-tenant vector encryption has been evaluated, with a documented decision

Where this leaves you

Cosine similarity measures an angle. It has no notion of truth, authority or provenance, and when you build a RAG system you inherit every security assumption of the embedding space along with its retrieval quality.

Poisoning works because vocabulary engineering can move a document’s position without touching the model’s internals. Anomaly detection works because the geometric requirement that makes the attack effective is also what makes it visible at ingestion. Inversion works because dense vectors carry enough of the source text to reconstruct much of it, and the assumption that they did not was folklore rather than a result.