DEVOURED
Xerj (GitHub Repo)
Xerj is a local search engine that replaces grep with tree-sitter-based indexing to reduce agent token usage by up to 2.7x.
What: Xerj uses tree-sitter to index code, logs, PDFs, and SQL data into an Elasticsearch-compatible local API. It is designed to minimize context window bloat for AI agents by retrieving precise code passages rather than full files.
Why it matters: This indicates a shift toward 'reference coding,' where agents perform local semantic/lexical search across related codebases to solve problems, rather than relying solely on the LLM's training data or full-file context injection.
Takeaway: Install the binary and index a local repository via 'xerj autoindex ' to enable your coding agent to retrieve symbols and code snippets via MCP.
Deep dive
- Indexes various file formats including code, logs, PDFs, and SQLite databases.
- Compatible with the Elasticsearch API for search, hybrid, and vector queries.
- Uses tree-sitter for AST-aware indexing rather than plain text search.
- Acts as an MCP server for integration with IDE agents like Cursor or Claude Code.
- Operates on a local-only trust model, intentionally lacking TLS/auth in the default --insecure mode.
- Significant token reduction demonstrated for coding agents during retrieval tasks.
- Incremental indexing skips unchanged files using content hashes.
Decoder
- RAG (Retrieval-Augmented Generation): The process of providing an AI model with external data to reference during its generation process to improve accuracy.
- Tree-sitter: An incremental parsing tool used to build concrete syntax trees for source code, allowing for language-aware indexing and search.
- MCP (Model Context Protocol): An open standard for connecting AI assistants to systems, data, and tools.
- AST (Abstract Syntax Tree): A tree representation of the abstract syntactic structure of source code.
Original article
Full article content is not available for inline reading.
Explore all 51 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi