Case study

KERNHEAP: Linux kernel heap tampering detection

KERNHEAP was the first integral solution against vulnerabilities abusing the Linux kernel heap. It pioneered concepts applicable to dynamic memory allocators in other operating systems, their kernels and toolchains.

KERNHEAP: Linux kernel heap tampering detection
  • Field of use

    Information Assurance

  • Initial Development


  • First Functional Milestone


  • EOL

    2011 (superseded)

  • Distribution

    Open Source (2009-2011)

  • Languages
    • C

During a fateful day, sometime summer 2009, the grsecurity developer (B. Spengler) suggested the kernel heap as an area worth investigating for developing new mitigations. The Linux kernel suffered of a complete lack of security mitigations against all vulnerability classes involving dynamic memory. With several allocators supported (including but not limited to multiple variants of a Slab-type allocator), the attack surface was a significant challenge. Previous efforts included those of S. Eren (BSD, not publicly released and details unknown). At the time, exploits abusing kernel heap-related vulnerabilities were well known in the offensive security community, and most of them were relatively easy to implement reliably. The lack of mitigations was pervasive across all operating systems, not just Linux.


Subreption dedicated a significant amount of time and effort investigating the internals of the Linux kernel allocators and the exploitation techniques known at the time. Subsystems like the IPC semaphore system were modified to deter known techniques, and the principles learned in the process were applied to other subsystems exhibit similar potential for abuse. As for the allocators, a novel technique was designed to provide meta-data integrity assurance in an inter-dependent fashion across all the objects used by the allocator. The technique, cascade canaries or cascade guards, was legitimately original research not seen, published or implemented by anyone before.


The modifications made to the Linux kernel materialized as a patch for the 2.6 and 3.2 series of the kernel. A seminal paper published in the well-known hacker zine Phrack Magazine. Between 2009 and late 2010, KERNHEAP patches were distributed as copyrighted free software. Distribution in this form ceased due to intellectual property and etiquette violations by software vendors and members of the Linux kernel development community. KERNHEAP became a DARPA funded technology under the auspices of the Cyber Fast Track program. The original KERNHEAP code and DYMASEC were registered with the US Copyright office between 2011 and 2012. Some of the techniques pioneered by KERNHEAP were introduced in Microsoft Windows (in Vista and newer versions) and other operating systems.


Zeroday protection
  • Detection and prevention of use-after-free, double-free and uninitialized heap access conditions.
  • Full multi-allocator meta-data integrity protection pioneering a concept never used before.
  • Randomization of slab layouts, preventing predictable memory ordering for kernel heap objects.
  • Pioneered the concept of slab cache guard pages to isolate caches against intra-cache overwrites.
Confidentiality and miscelaneous features
  • Sanitization of sensitive objects.
  • Integrated self-testing capabilities.
  • Safe unlinking for kernel linked lists.
  • Measured impact with synthetic benchmarks.
  • Optimized hot code paths.
  • Configuration-free (after kernel compilation).
“Projects such as Subreption’s KERNHEAP provide a strong framework for implementing heap hardening, and future work should continue to refine these defensive techniques.”
“Must reads: – tioctl/sctp_houdini.c [12,5] – Larry H's Phrack 66 article (KERNHEAP Heap Protection [14])”
Copyright and external trademarks

Trademarks, logos and brand names are the property of their respective owners. All company, product and service names referenced or mentioned in this page are for identification and fair-use purposes only, with no endorsement implied.