pg_infer 1.0.0 released -- transformer model knowledge as SQL relations

Back to Devoured - May 25, 2026

DEVOURED

pg_infer 1.0.0 released -- transformer model knowledge as SQL relations

Data databaseaibackendopensourcepostgresql PostgreSQL

pg_infer 1.0.0 is a new PostgreSQL 18+ extension that exposes small transformer model internals as SQL-queryable relations, enabling efficient, costed, and parallelized inference directly within the database.

What: Greg Burd released pg_infer 1.0.0, a PostgreSQL 18+ extension that allows users to query transformer model internals (gate activations, feature labels, learned associations, embeddings) as SQL relations and index data. It supports CPU-efficient inference, including Microsoft BitNet b1.58 models, and can offload inference to PostgreSQL replicas, leveraging existing idle hardware.

Why it matters: This extension fundamentally changes how small LLMs can be integrated with data, moving inference from an external service to a first-class, queryable database operation, which aligns model behavior with traditional relational data management principles.

Takeaway: If you use PostgreSQL 18+ and need to integrate small language models, explore pg_infer to leverage existing database infrastructure for inference and gain SQL-level visibility into model knowledge.

Deep dive

pg_infer 1.0.0 is a PostgreSQL 18+ extension that exposes transformer model internals as SQL-queryable relations.
It treats the model as a first-class data source, allowing the PostgreSQL planner to cost, schedule, and parallelize inference as an operator within a query plan.
The extension provides functions like describe(entity) to get learned relations, walk(prompt) for per-layer activations, and implies(a, b) for directional support.
It includes a custom index access method supporting ORDER BY <~> for model-aware document ranking without pre-computed embeddings.
Unlike pgvector or RAG-style integrations, pg_infer stores the model itself in WAL-logged 8KB pages, enabling full database backup, replication, and point-in-time recovery for model state.
Optimized for CPU execution using BLAS (OpenBLAS) and f16 gate vectors, it specifically supports Microsoft BitNet b1.58 models, which are efficient on commodity CPUs.
A remote backend (larql-server) allows offloading inference to idle PostgreSQL replica hosts, utilizing existing hardware capacity.
The project is based on Chris Hayuk's LARQL project, which pioneered the idea of queryable transformer internals.

Decoder

Transformer model: A deep learning model architecture, particularly effective for processing sequential data like natural language, known for its attention mechanism.
Gate activations: Internal numerical values within a neural network layer that determine the flow of information.
Feature labels: Metadata associated with learned features in a model.
Embeddings: Numerical representations of concepts, words, or entities in a continuous vector space.
BitNet b1.58: A family of "two-bit / 1.58-bit" ternary-weight transformer models developed by Microsoft, designed for high quality on commodity CPUs with dramatically lower memory and power costs.
vindex: A format for extracting and storing transformer model knowledge (gate vectors, feature activations, learned associations) developed by the LARQL project.
WAL-logged pages: Data stored in PostgreSQL's Write-Ahead Log, ensuring durability and recoverability.
Index Access Method (AM): A PostgreSQL mechanism that defines how a specific type of index is stored and accessed.
BLAS (Basic Linear Algebra Subprograms): A specification that defines a set of low-level routines for common linear algebra operations, optimized for performance.

Original article

Full article content is not available for inline reading.

Read the original article →

Explore all 62 articles in today's digest