Back to Devoured - November 18, 2026
DEVOURED
Accelerating development for massive Git repositories

Accelerating development for massive Git repositories

Tech Google Developers
Google open-sourced ABFS, a system that reduces Android build times from 80 minutes to two by using content-addressable storage and lazy loading.
What: Another Build File Service (ABFS) is now available on GitHub. It uses a kernel filesystem module (CASFS) to stream source files only when needed and applies a two-phase caching system to verify build steps. This tool allows for massive Git repositories to be managed with minimal overhead, enabling parallel agentic workflows.
Why it matters: As build graphs for massive software projects like Android grow, traditional timestamp-based build systems become a bottleneck; content-addressability is the necessary evolution for high-scale development.
Takeaway: If you are managing massive, multi-repo builds, evaluate ABFS to see if it can replace or augment your current Ninja or Bazel pipelines.
Deep dive
  • Content-addressable storage: A method where data is identified by its hash (usually SHA) rather than its location, ensuring build reproducibility.
  • Lazy-loading: A design pattern where a resource or file is not loaded or downloaded until it is specifically requested by the build process.
  • Tapping: The process of recording exactly which files a build process reads and writes, allowing for accurate dependency tracking even when build rules are incomplete.
  • Ninja-go: A specialized implementation of the Ninja build tool that integrates with ABFS caching.
Decoder
  • Content-Addressable Storage (CAS): A mechanism where data is stored and retrieved based on a unique hash of its content rather than its file path, allowing for efficient deduplication and caching.
Original article

Accelerating development for massive Git repositories

Android’s source code is big – the Android 17 open source release has 1,084 Git repositories, 1.62 million files, and takes 300 GB for a built workspace. Additionally, in a typical Android source tree with hardware vendor code and first-party applications, the size increases by 2-3x.

Traditionally, it took over 30 minutes to fetch Android Open Source Project (AOSP) source code and well over an hour to build on even highly capable developer workstations. As software development evolved over the last 2 years, modern coding harnesses and LLMs have accelerated the development of many projects. These tools are used today for Android operating system development, but the long sync and build times make them less efficient. Both human developers and agents need quick setup, parallel workspaces, and fast build-and-test iteration times.

We built Another Build File Service (ABFS) to address these bottlenecks. ABFS is a full suite of software across a Linux kernel filesystem module, user-space daemons, and server-side infrastructure. ABFS uses content-addressability across the entire stack to lazy-load and cache source code, developer workspaces, intermediate build files, and build results.

Today, we are excited to announce that ABFS is open source on GitHub.

The impact of ABFS to Android is:

  • Reduced checkout times from ~36 minutes to under 8 seconds.
  • Reduced full build times from ~80 minutes to just over 2 minutes.
  • Enabled parallel agentic workflows across multiple workspaces and agent orchestrators.
  • Shared cache reuses up to 100% of build actions across continuous integration (CI) and developer workspaces.

ABFS already powers parallel workflows for developers and agents in Android, but it is designed to scale to other massive Git-based projects.

Case Study: Android

In the Android build system, developers define build rules in Blueprint files (Android.bp), Soong compiles them into Ninja files, and Ninja executes the final build graph. Similar to the Make build tool, Ninja is timestamp-based. If a build step’s input file is newer than the existing output file, Ninja rebuilds the output file.

Code Size

Analyzing the AOSP code for Android 17 shows the sheer disk footprint and file count involved in a single checkout and build.

In a typical Android tree that includes hardware-specific repositories, board support packages, first-party applications, form-factor-specific code, etc., the code size is 2–3 times the size of AOSP (over 3,000 Git repositories), compounded across multiple active SKU and legacy maintenance branches.

Build Cacheability

Building the Cuttlefish emulator target executes a build graph with 192,249 Ninja actions. While local incremental builds are fast once a tree is already built, every new workspace, branch switch, or ephemeral agent session starts cold.

Even though typical developer changes touch only 1-10 different Git projects, someone building locally without any remote execution might waste up to 80 minutes on a 64-core machine repeating work their CI system already built. Even if a developer edits a core library like system/libbase, 97% of the build graph is identical to the CI build.

Unspecified Dependencies

Sharing cached build outputs across CI and local machines is also tricky in a codebase of Android's age and rate of change. Build rules occasionally introduce hidden dependencies (such as passing an undeclared schema file inside raw compiler flags), over-declared inputs, or undeclared outputs. In timestamp-based builds, hidden dependencies silently produce stale incremental artifacts, forcing developers to wipe output directories and run clean builds.

Solution: ABFS

ABFS makes building Android fast by replacing massive local source checkouts with a virtual, content-addressable filesystem (CASFS). ABFS streams files on demand only when a build actually touches them and utilizes shared build caching to help eliminate redundant downloads and compilation.

On the server side, Git-pushers continuously ingest new Git objects from Gerrit or other Git hosts into a horizontally scalable gRPC ABFS Content-Addressable Storage (CAS) Server that indexes Git content, workspace snapshots, and build step results. On the developer host, a local Least Recently Used (LRU) Cacheman daemon, Mount Daemon, and Tap utility connect user-space build tools to the kernel filesystem so checkouts and builds stream only what they need.

Filesystem

At the core of the client is a virtual filesystem called Content-Addressable Storage Filesystem (CASFS), a Linux kernel module that exposes Git SHA content hashes via extended attributes. When a file's hash is set, CASFS populates its metadata immediately and downloads file bytes via Cacheman only when needed:

$ touch hello
$ abfs hash --set cc628ccd10742baea8241c5924df992b5c019f71 hello   # metadata only
$ ls -l hello      # name and size are available; content is not downloaded yet
-rw-r----- 1 user primarygroup 6 Nov  3 00:11 hello
$ cat hello        # content is lazy-loaded on first read
world

Initial checkouts download only lightweight directory metadata prepopulated with Git hashes. CASFS uses Git's object model natively, allowing standard git and repo operations to work transparently.

Correct two-phase build caching

CASFS can also tap build processes. Tapping records every file actually read, written, or deleted into lists called patches, which are stored alongside cached build outputs. Ninja-go replaces timestamp checks with two-phase content-addressable caching:

  1. Phase 1 - Declared key lookup: Ninja-go hashes a step’s command, environment, and declared input hashes to fetch candidate results from prior CI or developer builds.
  2. Phase 2 - Read-patch verification: Before accepting and applying a candidate result, ABFS verifies the candidate’s recorded read patch against the current CASFS state. In the common case, this read patch will match the filesystem state and then the write patch is applied. If even a hidden dependency has a different hash on disk, ABFS rejects the stale candidate and reruns the step with tapping enabled.

Unlocking agent development

Because ABFS does lazy loading and local cache deduplication, developers and their agents can now mount multiple concurrent Android workspaces on a single machine, benefit from a shared local build cache, and achieve massive scale in development time.

# Mount 5 distinct Android release branches side-by-side in under 8 seconds each
$ abfs mount -b android-latest-release   /ws/a17
$ abfs mount -b android16-qpr2-release   /ws/a16qpr2
$ abfs mount -b android16-release        /ws/a16
$ abfs mount -b android15-release        /ws/a15
$ abfs mount -b android14-release        /ws/a14

An agent can check out a full Android tree in under 10 seconds and perform a full build in under 10 minutes. Agent swarms are empowered to iterate, review, and test in parallel, which can lead to higher quality solutions and speed.

Extending beyond Google and Android

We released ABFS as open source because we want developers, Android partners, and any large-scale Git-based project to benefit from what we have learned and be able to scale.

While Android’s Soong/Ninja pipeline was the first acceleration target, ABFS was built to be extensible to other build systems. Content-addressable build systems like Bazel map naturally onto CASFS and there is already a proof-of-concept integration with Bazel. Chromium, the Android Common Kernel, and other Google projects are exploring ABFS compatibility.

Acknowledgements

We’d like to extend a special thanks to our significant contributors for their work on this project:

Haamed Gheibi, Erik Sanders, Rob Seymour, Taylor Santiago, Andrew Hsieh, Eric Taseski, Kiyoung Kim, Hung-Wei Chen, Jose “Pepe” Galmes, Ronald Li, Yu-Che Wang, Roger Ellis, Sandeep Patil, Amith Dsouza and the entire Android Build team

Explore all 55 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi