Why Linux security has failed (for the past 10 years)

Introduction

Apparently there’s been an increased interest in bringing Linux kernel security issues to attention, for the past few months. It is a natural reaction to a policy which has been long time tacitly agreed upon by mostly all people involved in Linux kernel development (and more so, those with security-relevant roles, particularly a specific vendor). That is, a policy of silence. It is no surprise that Linux security currently looks much better on paper and marketing propaganda than it does in reality.

It takes decent amounts of will and dedication to summarize, categorize and review every potential security vulnerability for such a huge project, requiring collaboration between different vendors, who might or might not have agendas of their own, conflicting the interests of the users, or the rest of vendors themselves. It takes approximately ten minutes for an average computer user to write a summary of why SELinux can help your organization cut down security risks.

What you don’t know is that you will have to go through the learning curve of writing policies, reviewing all software being used (including commercial applications which might not conflict with any ’learning’ mode at kernel level, but consistently prevent targeted reverse engineering or make it even more tiresome), testing the setup and adapting its architecture to the real needs of your organization. MLS is rarely used out of US government environments.

But there’s a targeted policy. Suddenly all problems seem like they’ve gone away for good. Again, reality hurts and the targeted policy is to true containment and least-privilege policies what system call policies are to discretionary access control. They are useless. They work as a step towards a real policy, and might contain specific pieces of software, while leaving the rest of the system exposed to abuse. The targeted policy is the result of trying to push the wrong technology to the wrong audience. SELinux is an extremely versatile system for certain organizations, but is largely overkill for the rest of mere mortals.

Most of the work on SELinux for the past few years has been done on userland tools. No real innovation, no further protections for the Linux kernel, besides some tiny improvements here and there (again, useful to very few people, like network packet marking). Most people don’t need to separate classified network traffic from unclassified data. Yet, with the extensive resources of the organizations and people involved on developing SELinux (who could maintain it as an out-of-tree patch without much trouble), they have pushed, and continue pushing these changes into the code base everyone uses. It might be possible to disable it in the kernel configuration but it is still there. This isn’t the immediate problem, though…

Linux kernel echo chambers and plagiarism

Meanwhile, other even less efficient approaches are merged into the kernel (initially, the kernel development team rejected any Linux Security Modules based approaches, arguing people should use SELinux, but now we have TOMOYO merged…), and others keep being rejected or plagiarized. For instance, it is a matter of fact that quite a few features of PaX have been “ported” over the past few years, in numerous times with flaws that rendered them useless or inefficient when compared to the PaX counterpart. In this list of dubious merit, we have DEBUG__RODATA_ (PaX provides KERNEXEC), and the almost infinite amount of structures with function or file pointers declared without const modifier (therefore, making RODATA useless for those). Many of these have been fixed by Brad Spengler and the PaX Team.

By plagiarized we mean (while acknowledging that free software essentially permits this behavior without the full spectrum of legal ramifications of other licensing frameworks, intellectual property and patent infringement notwithstanding) ’re-implemented’ or ‘ported’ in fully (or partially verbatim) manner, without mentioning previous work. ExecShield was flawed back in the day and instead of working towards making it consistent, any opposition or critics have seen a less than productive (and often civil) response.

Who loses in this picture? The users. Histrionic egos apart (the only possible justification for not solving a conflict that could easily end by crediting someone else’s efforts accordingly), it is the users who always lose in this sort of fight over who is right or wrong about matters which are technically refutable. There is little room for subjectivity about security, except for that brought up by the people managing it.

The situation is aggravated by the fact that most kernel developers have absolutely no real security insight, and can’t grasp even the simplest concepts, for example why dereferencing an userland address from kernel-land might represent a security problem, or resetting pointers to addresses reachable by an userland process. This led to the discussions and posterior development of a flawed mmap_min_address, which might fix a potential venue of attack (if implemented correctly, it has been proven broken by different people at different times) but misses the point completely.

This is only the tip of the icerberg. Perhaps, it is because of the dubious practices of some parties involved with kernel development and security, that many skilled individuals have retreated from any public forum to avoid discussing technical information, which may be surreptitiously eavesdropped and relayed privately to closed mailing lists or private communication channels. In the long term, the kernel development team will likely end in a bubble isolated from any realistic security discussion. Again, those who will suffer the consequences are the users.

We can only hope that PaX and grsecurity keep being plagiarized (and KERNHEAP in the near future), but this time with a proper implementation that actually works. Otherwise we will keep seeing the same window dressing, and the same usual maladies between people who do understand kernel security, and those who are in a position to manage it but unfortunately don’t understand it.

And we’ve come to this point without directly mentioning the normal course of action for dealing with potential security vulnerabilities in the Linux kernel: patching them silently. While it is true that it is costly to review every patch and every bug report for potential security implications, when a large organization markets a product as secure, and has a large staff to work on such areas, it doesn’t seem reasonable to avoid dedicating some resources for this purpose. Whether this is to “save face” or avoid additional costs, is again moot, because it is the users who suffer the consequences of a silently patched issue.

Internet hosting providers have their servers compromised, information is lost or damaged, services turn unreliable. And it all has an associated cost which incurs in immediate or near future losses for the businesses affected. This isn’t about the ego of the “security researcher” or kernel developer of the moment and his short lived five minutes of fame.

What does the Linux kernel need for real security?

If we want to improve the security of the Linux kernel effectively, without further time wasting on useless discussions and diatribes:

1. It needs a working KERNEXEC implementation, which truly protects read-only data and prevents execution of data within kernel regions. At the moment only PaX provides one that really does what it is supposed to do.
2. It needs a functional, yet simple RBAC system, which let’s developers work on its API to create more complicated security models. SELinux is simply overkill and bloated at this point.
3. It needs a functional, effective NX implementation for stack, heap/anonymous mappings, etc. If it doesn’t provide the level of coverage of PaX’s PAGEXEC/SEGMEXEC, it is no good.
4. It needs a functional, simplistic, yet effective heap protection implementation. This is the gap that KERNHEAP intended to fill, but it is unlikely the kernel developers would ever bother themselves to review and consider it for inclusion. Proposing it is likely a waste of precious time, like previous attempts have proven. A more likely fate is one similar to grsecurity: cannibalization of features, resulting in grossly ineffective “mitigations” we will need to constantly patch-away to maintain our codebase.
5. It needs to stop receiving as many mostly-useless scheduler changes, implementations or functionality which isn’t already required by a large enough user base. It is ridiculous to see the volume of changes related to scheduler or memory management code in the 2.6 kernel. There should be a separate tree for this sort of features.
6. The kernel developers must rely less on Coverity (and similar static code analysis tools) and learn to review code manually a bit better. This could trim down the list of glaring security bugs that get merged into the kernel by an order of magnitude. Complaining about comments style, indentation, API naming, etc, is definitely part of the job, but reviewing the code for potential issues should be part of it too.
7. If an organization wants to have functionality merged into mainline kernel, it should require truly consistent reasons to justify any potential overhead. If KERNHEAP is (under objective, unbiased, assessment and judgment) deemed overkill for the security needs for the majority of users, it shouldn’t be merged. If a patch is proposed by someone with more candid relations with the kernel development team, and bears the same conditions, it shouldn’t be merged either. If you don’t want TOMOYO, or grsecurity, you don’t want SELinux! There’s no consensus about which security system to use among users, though most people prefer to disable SELinux shortly after installation. There’s likely a reason for this, even if users might or might not understand security implications to their full extent. The unrealistic, bubble-wrapped, tree-hugging reality of some of the kernel developers gets us nowhere.

This article might inspire polarized responses, and the point will most definitely be lost in the process: Linux kernel security must improve, and it depends on respecting and crediting (where it is due) the work of external developers, even if they ever so lightly bruised the egos of Linux kernel developers and business involved with them. Otherwise, it is the majority of users that will suffer the consequences, while a conscious minority will continue doing everything at hand to run Linux safely on their systems. The effect of constant plagiarizing (through cannibalization of features) and ad hominem attacks against developers of Linux kernel mitigations will surely result in drastic changes, eventually pushing them towards a closed development process, depriving the users of access to their talent, and we all know who should be blamed for it.