Semantic Vector Search

Use lakebase_vector for approximate nearest-neighbor retrieval over embeddings. It retains pgvector's vector types, distance operators, and query syntax; the index access method is lakebase_ann.

Contents

• Create the extension — enable lakebase_vector and its pgvector dependency
• Prepare embeddings — generate vectors with the AI Gateway and match the column dimension
• Build the index — match the distance metric, operator class, and query operator
• Tune the index — configure index-build options and concurrent index management
• Query — rank by vector distance or filter by a similarity radius
• Tune search — inspect the index and tune recall against latency
  • Use prefilter selectively — apply selective filters before ANN scoring

Create the Extension

Lakebase Search requires Postgres 16 or later. Enable the extension before creating vector columns or indexes:

CREATE EXTENSION IF NOT EXISTS lakebase_vector CASCADE;

lakebase_vector installs pgvector through CASCADE. It relies on a preloaded library that Neon enables by default; if the project customized its preloaded-library list, confirm the library remains enabled.

Prepare Embeddings

Neon's AI Gateway embeddings endpoint is an OpenAI-compatible way to generate vectors for Lakebase Search. This table uses 1024 dimensions as an example; set the column dimension to the output of the model and any dimensions parameter you choose:

CREATE TABLE documents (
  id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
  title text NOT NULL,
  body text NOT NULL,
  embedding vector(1024)
);

The AI Gateway is available on paid Neon plans only (Launch and Scale) and draws down prepaid AI Gateway credits. If a model shows enabled: false in GET /v1/models, the branch's AI Gateway page in the Neon Console lists the reason and the fix. See the neon-ai-gateway skill.

Enable the AI Gateway on the branch and get NEON_AI_GATEWAY_TOKEN and the bare-host NEON_AI_GATEWAY_BASE_URL:

• With neon.ts, declare aiGateway: true, run neon deploy, then run neon env pull --file .env.
• Without neon.ts, run neon env pull --service ai-gateway --file .env from a directory linked to the intended project and branch. This requests the gateway variables explicitly and skips other services.
• Alternatively, get both values from the Neon Console. Neon Functions receive them automatically.

AI SDK (Recommended)

For TypeScript apps, use Neon's @neon/ai-sdk-provider 1.0.0+ with the Vercel AI SDK (ai, 6+) (npm install @neon/ai-sdk-provider ai dotenv). The provider reads NEON_AI_GATEWAY_TOKEN and NEON_AI_GATEWAY_BASE_URL directly and sends embedding requests to /v1/embeddings. Load .env in a standalone app; Neon Functions inject the variables automatically.

import "dotenv/config";
import { neon } from "@neon/ai-sdk-provider";
import { embed, embedMany } from "ai";

const embeddingModel = neon.embeddingModel("qwen3-embedding-0-6b");
const providerOptions = { neon: { dimensions: 1024 } };
// Qwen3 query prompt adapted for Neon documentation search; documents stay unprefixed.
const queryPrompt =
  "Instruct: Given a question about Neon, retrieve documentation passages that answer the question\nQuery: ";

const { embeddings: documentEmbeddings } = await embedMany({
  model: embeddingModel,
  values: documentBodies, // non-empty document batch; store vectors by input index
  providerOptions,
  maxParallelCalls: 1,
});
const { embedding: queryEmbedding } = await embed({
  model: embeddingModel,
  value: `${queryPrompt}${queryText}`,
  providerOptions,
}); // bind to $1 below

Skip empty imports before calling embedMany. It returns vectors in input order and splits requests according to the provider's item limit. For large imports, also split document text within the endpoint's token limits and keep database writes bounded.

For Neon documentation search, the example adapts the task instruction in Qwen3's stored query prompt from config_sentence_transformers.json. Its document prompt is empty, so document text stays unprefixed. Neither SDK applies SentenceTransformers prompts automatically. Keep the original queryText for BM25, and use the selected model's own prompt conventions if you switch models.

OpenAI SDK Alternative

For an existing OpenAI SDK integration, install openai and dotenv (npm install openai dotenv) and configure the client explicitly:

import "dotenv/config";
import OpenAI from "openai";

const apiKey = process.env.NEON_AI_GATEWAY_TOKEN;
const gatewayBaseUrl = process.env.NEON_AI_GATEWAY_BASE_URL;
if (!apiKey || !gatewayBaseUrl) {
  throw new Error("Missing Neon AI Gateway credentials");
}

const client = new OpenAI({
  apiKey,
  baseURL: `${gatewayBaseUrl}/v1`,
});

async function embedBatch(inputs: string[]): Promise<number[][]> {
  if (inputs.length === 0) return [];
  const response = await client.embeddings.create({
    model: "qwen3-embedding-0-6b",
    input: inputs,
    dimensions: 1024,
    encoding_format: "float",
  });
  const vectors: number[][] = [];
  for (const item of response.data) vectors[item.index] = item.embedding;
  return inputs.map((_, index) => {
    const embedding = vectors[index];
    if (!embedding) throw new Error(`Missing embedding for input ${index}`);
    return embedding;
  });
}

const documentEmbeddings = await embedBatch(documentBodies);
const [queryEmbedding] = await embedBatch([`${queryPrompt}${queryText}`]);
if (!queryEmbedding) throw new Error("Missing query embedding");

Use the same queryPrompt defined above, only for query inputs. Keep each batch within the endpoint's input and rate limits, and pair each vector with its source row using the response index. The OpenAI JavaScript SDK v6 defaults to requesting base64 and decodes the response as base64. If a compatible endpoint returns float arrays instead, that decoder can silently turn them into zeros. Pass encoding_format: "float" explicitly for the AI Gateway; the SDK then returns the float arrays directly. Base64 can reduce response size when an endpoint actually returns base64, but do not assume that format for this gateway.

Use the same model and output dimensions for stored documents and queries. Follow the model's input conventions: some models recommend a query instruction while document text remains unprefixed. If the model supports shorter vectors, compare retrieval quality, storage, and latency before changing dimensions; set the column and both embedding requests to the chosen size. Switching models requires re-embedding the corpus; changing dimensions also requires migrating the vector column and index. Generate embeddings in the application, outside SQL, unless the architecture already provides an in-database embedding function. Other embedding providers are also valid when their output matches the schema and index.

Build the Index

Choose the operator class and query operator as a matched pair:

Choose the metric recommended for the selected model, and keep its index operator class and query operator aligned. The Qwen3 Embedding model card demonstrates cosine similarity and normalized dot product for retrieval. For unit-normalized vectors, cosine, inner product, and L2 have the same exact ranking; approximate indexes can still differ in recall and latency, so benchmark on your corpus.

CREATE INDEX documents_embedding_ann ON documents
  USING lakebase_ann (embedding vector_cosine_ops);

Tune the Index

The default index options suit most workloads:

• build_mode = 'standard' balances recall and index build time. Use quality for better recall when a longer build is acceptable.
• lists = 'auto' chooses the IVF partition layout from the number of indexed vectors. Choose between auto and a manual value case by case: test both on the target dataset and use the value that better meets recall and performance targets.

To prioritize recall over index build time:

CREATE INDEX documents_embedding_ann_quality ON documents
  USING lakebase_ann (embedding vector_cosine_ops)
  WITH (build_mode = 'quality');

To override the automatic partition layout instead:

CREATE INDEX documents_embedding_ann_lists ON documents
  USING lakebase_ann (embedding vector_cosine_ops)
  WITH (lists = '1024');

For a large table, use CREATE INDEX CONCURRENTLY to avoid locking out writes while creating the index. For a frequently changing table, periodically use REINDEX INDEX CONCURRENTLY to rebuild the index with minimal write locking.

Query

Generate the query embedding with the same model and dimensions used for stored documents, following the model's query-input conventions (using the Qwen3 query prompt shown above), then bind it as a parameter:

SELECT id, title, embedding <=> $1::vector AS distance
FROM documents
ORDER BY distance
LIMIT $2;

Distance sorts ascending: a smaller value is a closer match. Keep the query operator consistent with the index operator class.

To filter by a similarity radius, use the matching boolean range operator in WHERE and the distance operator in ORDER BY:

SELECT id, title
FROM documents
WHERE embedding <<=>> sphere($1::vector, 0.5)
ORDER BY embedding <=> $1::vector
LIMIT $2;

The cosine range operator <<=>> returns a boolean; do not use it as the ranking expression.

Tune Search

Inspect the index before overriding defaults:

SELECT lakebase_ann_index_info('documents_embedding_ann');

This reports lists, default_probes, and default_epsilon. Small datasets use exact flat search before IVF lists are built. In that state, lists and default_probes are empty. Leave lakebase_ann.probes set to its default of 'auto'; lakebase_ann.epsilon still controls full-precision reranking during flat search.

For an IVF index, lakebase_ann.probes controls how many partitions are searched at each level. Higher values generally improve recall at the cost of speed. Its default is 'auto'. When lists is not empty, the shape of probes must match the shape of lists: use one value for a one-level index or two comma-separated values for a two-level index. At each level, the probes value must be no larger than the corresponding lists value. A mismatched or out-of-range value causes an error.

lakebase_ann.epsilon controls how many candidates are reranked using full-precision distances. Higher values rerank more candidates and take longer. Its default is 'auto', which works well for most workloads.

Use Prefilter Selectively

By default, PostgreSQL applies non-vector filters after the ANN index returns candidate rows. Enable prefilter when a filter is cheap to evaluate and removes most rows. Leave it off for loose or expensive filters.

BEGIN;
SET LOCAL lakebase_ann.prefilter = on;

SELECT id, title
FROM documents
WHERE id % 100 = 0
ORDER BY embedding <=> $1::vector
LIMIT $2;
COMMIT;

Start with probes and epsilon set to 'auto'. Benchmark manual probe values against representative query embeddings and choose the smallest values that satisfy recall and tail-latency targets. Keep session settings and the query in the same transaction when using a connection pool or stateless driver.

Source: lakebase_vector documentation.