The RipGrep Musl Segfault That Led to a One-Line Linux Kernel Patch

Ripgrep 15.2.0's musl build occasionally segfaults during very-large multi-threaded searches. That was the bug report, filed July 26 against the ripgrep issue tracker. What followed is the best kind of debugging saga: the reporter pinned the crash to an exact instruction sequence, eliminated every in-process explanation, and landed on a suspected Linux kernel bug introduced in the 7.0 series. On August 1, a kernel-hardening maintainer posted a one-line patch candidate based on an upstream kernel list analysis, and the reporter is testing it as of this writing.

The Crash: Deterministic Reproducer, Musl Only#

The original report came from a developer who first hit the crash in the ripgrep binary bundled inside a popular AI coding tool, then reproduced it with the vanilla ripgrep 15.2.0 musl release - the bundled binary was byte-for-byte identical. The crash signature is a SIGSEGV inside musl's mallocng allocator, in a get_meta() integrity assertion, reached via calloc called from opendir while ripgrep's directory walker spawns threads.

Reproduction needs a genuinely large tree: about 20 GiB across 1.84 million files, searched in a loop for a literal string that is never present. On the reporter's 24-core Threadripper machine running openSUSE Tumbleweed's 7.0.12 kernel, the crash landed every 1 to 3 minutes, always early in the walk (about 1.6 seconds in, versus 7.6 seconds for a clean run). Two things stood out immediately: glibc builds never crashed, and four other machines - including a Threadripper 9970X that is a microarchitectural match for the original CPU - running kernels 6.8 through 6.19 never reproduced it. The bug tracked the kernel version, not the hardware.

The Investigation: Pinning a Vanishing Store#

The reporter then published a full analysis repository with instrumented builds, patches, and scripts. The key finding is a "self-tear": a thread stores a value into a freshly-faulted anonymous page inside musl's allocator, then reloads the same address about ten instructions later and reads zero.

The evidence chain is unusually rigorous:

• An immediate re-read probe one instruction after the store sees the value correctly, ruling out a store that never committed - the store landed, then the mapping underneath it changed.
• A pagemap read at the moment of the mismatch reports the page as present and soft-dirty but backed by PFN 0, the kernel's zero page.
• A core dump captured without tracing (so the race is not perturbed) confirms the crashing thread's frame pointer matches the torn slot address exactly, while another worker thread is concurrently in closedir -> free -> munmap.
• Control builds that pre-fault the page before the allocator writes to it eliminate the crash completely; a comparable timing perturbation that does not fault the page crashes at the full rate. The crash disappears exactly when the first write to a freshly-faulted page is removed.

The conclusion: the per-VMA-lock anonymous-fault fast path publishes a new PTE while a concurrent munmap is mid-teardown, and the TLB-shootdown IPI lands after the publish, transiently exposing a zero-page translation to a thread that just wrote there. A source-level diff across v6.19, v7.0, v7.1 and mainline pins the change on the munmap side: v7.0 (January 2026) reworked PTE-table reclaim during zap in three commits (4c640eb4181c, fb4ddf208511, eda8c5e77622), and that rework is absent from 6.19, present through 7.2-rc1, and matches the reproduce/no-reproduce machine split. The analysis is honest about confidence: the observed behavior is kernel-caused with high confidence; the specific commit is a strong correlation awaiting kernel review.

The One-Line Fix Candidate#

On August 1, a maintainer known for a prominent kernel-hardening patch series posted a patch candidate on the issue, based on an upstream kernel list find by Andy Lutomirski: in mm/memory.c, the PTE-table-free call inside zap_pte_range passes the current PTE address where it should pass the range start. One line changes from pte_free_tlb(tlb, pmd_pgtable(pmdval), addr) to pte_free_tlb(tlb, pmd_pgtable(pmdval), start). The reporter is testing it against the 7.0 kernel now. As of publication the bug is not fixed in mainline.

What Developers Are Saying#

The discussion around this story split into four distinct camps.

The first debated the writeup itself. The report and analysis are visibly LLM-assisted, and readers divided sharply: one side called the style unreadable and overwrought, the other called it close to the ideal bug report - detailed, organized, and self-verifying. The argument itself became a signal about how technical communication is changing.

The second camp argued kernel specifics. Several readers pointed out that an extra TLB flush is never an error - the CPU may flush whenever it likes - so the flush itself is not the bug; the mystery is how a zero-page PTE ends up present where it should not be. The reporter's prefault control experiment answered the deeper question of specificity, but the source-level mechanism remains the open item.

The third camp tried to make it a Rust-versus-C argument. That did not survive contact: the crash is in musl (C) and the kernel (C), the allocator that failed is C, and the exchange ended with the blunt observation that the bug is in the C code Rust interacts with.

The fourth camp surfaced the practical allocator gotcha: ripgrep sets jemalloc as its global allocator, but that override only covers Rust-side allocations. C library calls like opendir's calloc still run through musl mallocng, so the crash lived in an allocator most users never think about. And the original symptom - broken search inside an AI coding tool that bundles musl ripgrep - made the reach of static musl binaries concrete.

Why It Matters#

Three lessons worth keeping.

First, this is a masterclass in debugging an invisible race: instrument the allocator, split the store-to-reload window with probes, read pagemap as ground truth, use pre-faulting as a mechanism-based control, capture a core without tracing, then diff kernel versions. The whole playbook is in the analysis repo, reusable by anyone chasing a heisenbug. The investigation is a model of what a good bug report looks like when the reporter controls every variable.

Second, shipping static musl binaries means inheriting kernel bugs through libc paths you do not audit. Ripgrep is a fast, mature, massively deployed tool; its musl build is the default for containers and bundled tools. A race in kernel page-table teardown surfaced as a crash in a search tool, not in anything near the kernel. If you ship musl static builds, the calloc inside opendir is part of your surface area.

Third, for the Linux crowd, the v7.0 PTE-reclaim rework is the suspect and the fix may be one line. Kernel regressions this deep usually surface as heisenbugs months later, in userspace tools nobody expects to be crashy. When the reproducer is this tight and the kernel-version correlation this clean, the fix follows fast - the test is running now.

Continue Reading#

• Best CLI Tools for AI Development in 2026 - where ripgrep sits in the modern developer toolkit
• GitHub Casefold: A Branchless Rust Crate From the RipGrep Author - more systems-level Rust from the same author
• GhostLock: A 15-Year Linux Kernel Vulnerability - another kernel bug with a long tail
• Bun's Rust Rewrite: Status Check - Rust in performance-critical developer tools
• Zig's Incremental Compilation Deep Dive - more compiler and systems engineering analysis

Sources#

• RipGrep issue #3494: x86_64-unknown-linux-musl binaries occasionally segfault during very-large searches
• Analysis repository: ripgrep-3494-analysis
• RipGrep 15.2.0 release (musl binary)
• Linux kernel list thread referenced in the patch candidate