Ksplice, Rebootless Linux Kernel Security Updates

Submitted by Jeremy
on April 25, 2008 - 1:20pm

"I've put together an automatic system for applying kernel security patches to the Linux kernel without rebooting it, and I wanted to share this system with the community in case others find it useful or interesting," said Jeff Arnold, announcing ksplice. He explained, "the system takes as input a kernel security patch (which can be a unified diff taken directly from Linus' GIT tree) and the source code corresponding to the running kernel, and it automatically creates a set of kernel modules to perform the update. The running kernel does not need to have been customized in advance in any way." The project's website notes, "ksplice cannot handle semantic changes to data structures—that is, changes that would require existing instances of kernel data structures to be transformed." With this limitation, Jeff suggested ksplice is still able to automatically apply 84% of the kernel security patches released between May 2005 and December 2007. He continued:

"I've been pursuing this project because I don't like dealing with reboots whenever a new local kernel security vulnerability is discovered. The rebootless update practices/systems that are already out there require manually constructing an update (through a process that can be tricky and error-prone), and they tend to have other disadvantages as well (such as requiring a custom kernel, not handling inline functions properly, etc). This new system works on existing kernels, and it simply takes a unified diff as input and does the rest on its own."


From: Jeff Arnold
Subject: A system for rebootless kernel security updates
Date: Apr 23, 11:59 am 2008

Hello,

I've put together an automatic system for applying kernel security patches 
to the Linux kernel without rebooting it, and I wanted to share this 
system with the community in case others find it useful or interesting.

Here's the summary:  The system takes as input a kernel security patch 
(which can be a unified diff taken directly from Linus' GIT tree) and the 
source code corresponding to the running kernel, and it automatically 
creates a set of kernel modules to perform the update.  The running kernel 
does not need to have been customized in advance in any way.  To be fully 
automatic, the system cannot be used to apply patches that introduce 
semantic changes to data structures, but most Linux kernel security 
patches don't make these kinds of changes.  I've evaluated the system 
against various kernel versions and security vulnerabilities, and the 
system can automatically apply 84% of the significant kernel security 
patches from May 2005 through December 2007.

I've been pursuing this project because I don't like dealing with reboots 
whenever a new local kernel security vulnerability is discovered.  The 
rebootless update practices/systems that are already out there require 
manually constructing an update (through a process that can be tricky and 
error-prone), and they tend to have other disadvantages as well (such as 
requiring a custom kernel, not handling inline functions properly, etc). 
This new system works on existing kernels, and it simply takes a unified 
diff as input and does the rest on its own.

The system's website is http://web.mit.edu/ksplice.

The GIT repository, code tarball, and binary tarballs are available here:
http://web.mit.edu/ksplice/ksplice.git
http://web.mit.edu/ksplice/dist/ksplice-src.tar.gz
http://web.mit.edu/ksplice/dist/ksplice-bin-i386.tar.gz
http://web.mit.edu/ksplice/dist/ksplice-bin-x86_64.tar.gz

A document describing how the system works is available here: 
http://web.mit.edu/ksplice/doc/ksplice.pdf

Any feedback would be appreciated.

Jeff Arnold
jbarnold@mit.edu
--

From: FD Cami Subject: Re: A system for rebootless kernel security updates Date: Apr 23, 2:37 pm 2008 On Wed, 23 Apr 2008 14:59:05 -0400 (EDT) Jeff Arnold <jbarnold@MIT.EDU> wrote: > Hello, Hi Jeff, > I've put together an automatic system for applying kernel security patches > to the Linux kernel without rebooting it, and I wanted to share this > system with the community in case others find it useful or interesting. (reading on) > Here's the summary: The system takes as input a kernel security patch > (which can be a unified diff taken directly from Linus' GIT tree) and the > source code corresponding to the running kernel, and it automatically > creates a set of kernel modules to perform the update. The running kernel > does not need to have been customized in advance in any way. To be fully > automatic, the system cannot be used to apply patches that introduce > semantic changes to data structures, but most Linux kernel security > patches don't make these kinds of changes. I've evaluated the system > against various kernel versions and security vulnerabilities, and the > system can automatically apply 84% of the significant kernel security > patches from May 2005 through December 2007. Awesome. Please note that reading this, I thought at first that the set of kernel modules were in fact, updated kernel modules (i.e. necessary unloading/ loading of modules) which I understood was not the case after reading your PDF. After checking with a friend of mine, he understood it like I did. Perhaps : - it automatically creates a set of kernel modules to perform the update. + it automatically creates a set of kernel modules containing the kernel + functions touched by the update, and arranges for the running kernel to + use the new functions from now on. would be better. > I've been pursuing this project because I don't like dealing with reboots > whenever a new local kernel security vulnerability is discovered. The > rebootless update practices/systems that are already out there require > manually constructing an update (through a process that can be tricky and > error-prone), and they tend to have other disadvantages as well (such as > requiring a custom kernel, not handling inline functions properly, etc). > This new system works on existing kernels, and it simply takes a unified > diff as input and does the rest on its own. It really looks like a non intrusive way of achieving superior uptime. Congrats ! Best, Francois --

Wow.

APz
on
April 25, 2008 - 8:30pm

If this actually works, this might be one of the greatest things in the Linux development lately.

again kernel-only?

olecom
on
April 25, 2008 - 10:34pm

How long those developers will sit inside kernel box?

http://kerneltrap.org/node/15996#comment-298437

Do they actually respect, that without userspace kernel panics?

Oh, well....

What are you talking about?

Anonymous (not verified)
on
April 26, 2008 - 12:58pm

What are you talking about? Userspace updates don't require reboots, so there is no reason to use a strange mechanism like this for them.

Linux development cult

olecom
on
April 26, 2008 - 4:43pm

I'm talking about it. I.e. Huge, never ending kernel development and almost nothing more (from system point of view).

Wana try bisect to understand (:?
____

This is just useless

Anonymous (not verified)
on
April 27, 2008 - 11:23am

This is just useless off-topic rambling, go troll somewhere else

Oh, come on, dear anonymous

olecom
on
April 28, 2008 - 12:27pm

Oh, come on, dear anonymous linux fanboy!

It's evident that commodity hardware had reached the end, anything new are fancy toasters or time killers.

OTOH software has almost nothing to propose, except bare bones full of holes as from security, as from development infrastructure, as from userspace points of view. No creativity, no imagination, bare kernel with stupidly huge development.

I respect Linus, he started kernel, sparse, git all is on different stages of development. But where others, where their ideas as fundamental as those? New KDE, new Firefox, new C compiler, new Perl? Come on! There is nothing new.

Simple idea of having general plain text interface for GPL drivers to generate source for any GPL kernel. How about that? Your new and cool kernel will have hardware support Linux has! Then filesystems, then anything else. Then let Linux bloat itself as its developers wish.

Fanboys listening to Linux hype and compiling kernels every day and night. All programmers can, is coding, sometime interesting but most of the time without much look on bigger picture.

Even that kernel: vt102 terminal, slow fbdev, no reliable security of source code (e.g. static audit of C lang crap, API usage crap, etc.) as well as binaty (e.g. PaX + move to hardened userspace).

Ask yourself next time, why binary of the vmlinux went up, but no useful, innovative features are in? Megs of kernel, hundreds of megs of page chache and finally only kernel in all RAM, and everybody's happy.

Yes, i'm a troll, i have such blog entry here. This doesn't change present situation.
_____

Please note that I never

Anonymous (not verified)
on
April 28, 2008 - 10:30pm

Please note that I never decided if I agree or disagree with you, but this is still completely off-topic rambling, and I refuse to even consider it's validity.

You can complain all you will on your blog, and I might even read it, but here it is just trolling.

You're complaining that

Anonymous (not verified)
on
April 30, 2008 - 6:50am

You're complaining that there's nothing new, yet you don't state what actually needs to be created. Maybe there isn't that much "new" stuff being created because we have created most of what anyone actually wants, and everyone is just gradually improving those.

If there's such a need for "new" stuff, please give an example of what, or start creating yourself...

Also, that crap about plain text kernel driver interfaces doesn't even make sense.

Useful innovative features go into the kernel every day -- but they're only useful to people working at the kernel level, because, well, they're kernel features.

If you can state a cohesive argument, it might help your credibility.

> Also, that crap about

olecom
on
April 30, 2008 - 1:01pm

> Also, that crap about plain text kernel driver interfaces doesn't even make sense.

OK, what is POSIX+configure+forests of #ifdef/#endif in userspace applications, that were developed not only for Linux?

If Linux kernel developers don't bother much to document own APIs even in human-readable form, then what can i say about automated source processing?

Forcing to read sources, is kind of slavery: "i did job, care to read it". Examples that this is not working: vmsplice() security hole as well as many comments from top developers (e.g. Al), that there is no that much review. There is nothing "new" WRT automated tools for audit/maintaining of the code, not saying about generating it for quite clear and simple interfaces of hardware (take a look at LabView or SPICE models for analog components for instance). Please, be my guest, look at my stuff here on site and comment there.

This is off-topic indeed here. And take care of what you are calling crap, because real one can hit you in your face right from holes in Linux.
____

I agree. Those pesky kernel

Anonymous (not verified)
on
April 28, 2008 - 12:20am

I agree.
Those pesky kernel developers hacking away million lines of code and I don't see anything new on my desktop no matter what kernel I reboot into.

At least with Windows I see the development: a Clippy here, ohhh ribbons there, heeyy: Bob!, of course we need a BIG clock; I almost missed my coffee break due to that tiny little clock they used to have, and look at that cute little dog that show up whenever I want to search for something.

Are Linus and his minions so inept they can't even code a cute little dog??

- Peder

You miss the point about

Anonymous (not verified)
on
April 29, 2008 - 4:56pm

You miss the point about Linux.

But certainly we should towards a more user friendly Linux system.
For me, I still have a dual-boot because of games. This is a must to so many people (game is huge stuff these days).
So, this is definitely an area Linux should work on to attract more users.

I think you missed the point

Anonymous (not verified)
on
April 30, 2008 - 10:35pm

I think you missed the point of the post you are replying to :)

Hint: sarcasm

It's not new; there have

Anonymous (not verified)
on
April 27, 2008 - 1:44am

It's not new; there have been various hacks like this before mostly for Telco applications. It's also a common technique outside Linux. But he seems to be the first guy to actually post it to the Linux kernel mailing list instead of hiding somewhere.

Yeah! Shame on him! Err-

Anonymous (not verified)
on
April 28, 2008 - 8:56pm

Yeah! Shame on him! Err- what?

Pretty cool

Anonymous (not verified)
on
April 26, 2008 - 1:18pm

Pretty cool.
Then you can run your system year in, year out, for years without reboot and still keep the kernel up-to-date. :)

But what about evil hacker use this to load rootkit into kernel?

But what about evil hacker use this to load rootkit into kernel?

Tomasz Chmielewski (not verified)
on
April 26, 2008 - 2:12pm

> But what about evil hacker use this to load rootkit into kernel?

If an evil hacker uses this to load a rootkit, he already has all root privileges and can load any other harmful module/rootkit anyway.

No You can't run for years

anon (not verified)
on
April 27, 2008 - 12:41pm

No You can't run for years and have kernel up to date, You can only fix bugs, for new features You still need to reboot to new kernel

Modules

on
April 27, 2008 - 3:52pm

Depending on what the new features are, you can often manage it by simply installing a module that implements the feature. This is mostly true of drivers for new hardware, but there are various subsystems that are / have been hot-plugable with modules...

Cheers & God bless
Sam "SammyTheSnake" Penny

Further approaching the 100% mark

Anonymous (not verified)
on
April 26, 2008 - 3:50pm

One step further for linux servers towards a 100 % uptime. Now security updates won't need reboots anymore. Amazing.

And then just imagine: The distributors could automatically distribute and apply updates through the kernel's modules and apply them right away in the running kernel. Awesome...

This should be good for distro kernels.

David Pottage (not verified)
on
April 29, 2008 - 7:11am

This should be good for distro kernels.

Just think if you can prepare a special kernel module that will apply security patches to a running kernel, then so can your favorite distro. In future when there is a security update, instead of downloading a ~20Mb kernel package from security.debian.org or the like, and then waiting until a suitable time to install it and reboot the system, you can download a small package containing patching modules for the standard kernels from that distro, and install it immediately.

After installing the patching modules you can then download the full kernel package and install at a suitable time, but there will not be any hurry. If necessary the system with the old kernel can be rebooted, and the patching modules loaded at bootup.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.