DEVOURED
Progress on provable inference for large language models
Anthropic released FlashProve, an open-source research prototype that uses zkSNARKs to generate cryptographic proofs that specific model weights produced an output.
What: FlashProve allows a prover to generate a 19MB proof for a 400B-parameter MoE model (Llama 4 Maverick) that a verifier can check in seconds on a CPU. It is 70-95x slower than standard vLLM inference but provides cryptographic assurance of inference integrity.
Why it matters: As model tampering and cost-cutting via secret model-routing become industry concerns, software-only cryptographic verification provides an auditable way to enforce model compliance without relying on trusted hardware.
Deep dive
- Methodology: Uses zkSNARKs to create a succinct certificate of computation without revealing model weights.
- Performance: Benchmarked at 110 tokens/second on a single H100, with verification taking ~20 seconds on a CPU.
- Components: Includes a prover, a C++ verifier, a Lean-formalized verifier for correctness, and an interactive visualizer for security audits.
- Post-hoc vs In-line: Uses a post-hoc approach to avoid the performance trade-offs of real-time proving.
- Formal Verification: Used the Lean proof assistant to mathematically verify completeness, soundness, and zero-knowledge properties.
- Constraints: Currently limited to 1,024-token windows and relies on integer-based weight quantization.
Decoder
- zkSNARK: A zero-knowledge succinct non-interactive argument of knowledge; a cryptographic proof that allows one party to prove they know specific information or performed a computation correctly without revealing the underlying data.
- Lean: A functional programming language and theorem prover used for formalizing mathematical proofs and verifying software correctness.
- Post-hoc proving: Generating proofs after an inference request has been completed, rather than in real-time during the model's generation pass.
- Mixture of Experts (MoE): A model architecture where only a subset of network parameters (experts) is active for any given input, improving inference efficiency.
Original article
Full article content is not available for inline reading.
Explore all 53 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi