Linux: DeskOpt, "Completely Unfair Scheduling"

Submitted by Jeremy
on September 5, 2007 - 9:01pm

"Completely fair scheduling is [a] really good thing, but if you want the best performance for certain applications you need to tune up some things," explained Michal Piotrowski in his announcement for the fifth version of his DeskOpt daemon. The daemon is a Python script that helps to automatically tune the I/O scheduler and the process scheduler to offer better performance for certain applications such as games or audio applications. The script supports the default CFS process scheduler and CFQ I/O scheduler, as well as the anticipatory I/O scheduler and the deadline I/O scheduler.

The small script utilizes an XML configuration file, deskopt.conf, used to define scheduler classes each supporting their own scheduler tunings. One or more applications can then be added to each scheduler class, and when any of the specified applications starts the daemon will automatically tune the schedulers per the settings in that scheduler class. As examples in the provided sample configuration file Michal defines a "games" scheduler class defining two games receiving the highest scheduler priority and an "audio" scheduler class receiving not quite as high of a scheduler priority.


From:	Michal Piotrowski [email blocked]
Subject: [ANNOUNCE] DeskOpt 005 (completely unfair scheduling ;))
Date:	Wed, 05 Sep 2007 14:55:49 +0200

Hi,

Completely fair scheduling is really good thing, but if you want the best
performance for certain application you need to tune up some things.

DeskOpt is designed to help people tune up their systems for certain tasks.

Here are the results from the Nexuiz benchmark

|------------------------------------------------------------------------|
|                          |    CFS + CFQ      |   CFS + CFQ + DeskOpt * |
|------------------------------------------------------------------------|
| timedemo demos/demo2     | 41,395896833 fps  |   41,460051233 fps      |
|--------------------------|-------------------|-------------------------|
| timedemo demos/demo2     | 18,343223633 fps  |   37,814504933 fps      |
| + massive_intr 4 1000    |                   |                         |
|--------------------------|-------------------|-------------------------|
| timedemo demos/demo2     | 9,334974533 fps   |   37,376611267 fps      |
| + massive_intr 8 1000    |                   |                         |
|------------------------------------------------------------------------|

* default class for games

Software requirements:
- Python 2.x (http://python.org/)
- elementtree (http://effbot.org/downloads/#elementtree)

DeskOpt installation:
cp deskopt /usr/local/bin/
mkdir /etc/deskopt/
cp deskopt.conf /etc/deskopt/
cp deskopt.rc /etc/init.d/

The configuration file must be a valid XML file.

CFQ tuning:
back_seek_max
back_seek_penalty
fifo_batch_expire
fifo_batch_async
fifo_batch_sync
quantum
slice_sync
slice_idle
slice_async
slice_async_rq

http://lwn.net/Articles/101029/
http://lwn.net/Articles/114273/

Anticipatory tuning:
antic_expire
read_batch_expire
read_expire
write_batch_expire
write_expire

Documentation/block/as-iosched.txt

Deadline tuning:
fifo_batch
front_merges
read_expire
write_expire
writes_starved

Documentation/block/deadline-iosched.txt
http://kerneltrap.org/node/431


Usage: deskopt [options]

Options:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config-file=CONFIG_FILE
                        Configuration file
  -d, --debug           Debug
  -H HDD, --hard-drives=HDD
                        List of hard drives
  -l LOOP_TIME, --loop-time=LOOP_TIME
                        Loop time
  -q, --quit            Quit
  -v, --version         Show version number

Example:
# deskopt -c /path/to/my/config/file.conf --hard-drives="sda hdb" -l 60
or
# deskopt -c /path/to/my/config/file.conf

The latest version of DeskOpt can be downloaded from 
http://www.stardust.webpages.pl/files/tools/deskopt/

Changelog:
- support for Anticipatory and Deadline scheduler (requested by Chris Snook)
- support for a different hard drives (previous version was limited to hdd
  with /usr partition)
- something like a documentation (requested by Satyam Sharma)
- the code looks a bit better now

diff -uprN deskopt-004/deskopt deskopt-005/deskopt | diffstat
 deskopt |  870 +++++++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 562 insertions(+), 308 deletions(-)

TODO:
- support for device queue length tuning (suggested by Andi Kleen)
- kill signal handling, bug fixes etc.

Comments and suggestions are welcome.

It still has many flaws :)

Regards,
Michal

--
LOG
http://www.stardust.webpages.pl/log/

Sample configuration file:

<cf>
<sched_class name="games" prio="10">

	<scheduler value="cfs">
		<cfs_sched_nice value="-20" />
		<cfs_sched_batch_wakeup_granularity_ns value="10000000" />
		<cfs_sched_min_granularity_ns value="1000000" />
		<cfs_sched_runtime_limit_ns value="16000000" />
		<cfs_sched_stat_granularity_ns value="0" />
		<cfs_sched_wakeup_granularity_ns value="2000000" />
	</scheduler>

	<scheduler value="cfq">
		<cfq_sched_nice value="-c1 -n0" />
		<cfq_sched_back_seek_max value="16384" />
		<cfq_sched_back_seek_penalty value="2" />
		<cfq_sched_fifo_expire_async value="250" />
		<cfq_sched_fifo_expire_sync value="125" />
		<cfq_sched_quantum value="4" />
		<cfq_sched_slice_async value="40" />
		<cfq_sched_slice_async_rq value="2" />
		<cfq_sched_slice_idle value="8" />
		<cfq_sched_slice_sync value="100" />
	</scheduler>
<!--
	<scheduler value="anticipatory">
		<anticipatory_sched_antic_expire value="6" />
		<anticipatory_sched_read_batch_expire value="500" />
		<anticipatory_sched_read_expire value="125" />
		<anticipatory_sched_write_batch_expire value="125" />
		<anticipatory_sched_write_expire value="250" />
	</scheduler>
-->
	<scheduler value="deadline">
		<deadline_sched_fifo_batch value="16" />
		<deadline_sched_front_merges value="1" />
		<deadline_sched_read_expire value="500" />
		<deadline_sched_write_expire value="5000" />
		<deadline_sched_writes_starved value="2" />
	</scheduler>

	<app bin="nexuiz-glx" />
	<app bin="nexuiz-sdl" />
</sched_class>

<sched_class name="audio" prio="7">

	<scheduler value="cfs">
		<cfs_sched_nice value="-10" />
		<cfs_sched_batch_wakeup_granularity_ns value="20000000" />
		<cfs_sched_min_granularity_ns value="1000000" />
		<cfs_sched_runtime_limit_ns value="16000000" />
		<cfs_sched_stat_granularity_ns value="0" />
		<cfs_sched_wakeup_granularity_ns value="2000000" />
	</scheduler>

	<scheduler value="cfq">
		<cfq_sched_nice value="-c1 -n4" />
		<cfq_sched_back_seek_max value="16384" />
		<cfq_sched_back_seek_penalty value="2" />
		<cfq_sched_fifo_expire_async value="250" />
		<cfq_sched_fifo_expire_sync value="125" />
		<cfq_sched_quantum value="4" />
		<cfq_sched_slice_async value="40" />
		<cfq_sched_slice_async_rq value="2" />
		<cfq_sched_slice_idle value="8" />
		<cfq_sched_slice_sync value="100" />
	</scheduler>

	<app bin="amarokapp" />
</sched_class>

<sched_class name="video" prio="5">

	<scheduler value="cfs">
		<cfs_sched_nice value="-10" />
		<cfs_sched_batch_wakeup_granularity_ns value="20000000" />
		<cfs_sched_min_granularity_ns value="1000000" />
		<cfs_sched_runtime_limit_ns value="16000000" />
		<cfs_sched_stat_granularity_ns value="0" />
		<cfs_sched_wakeup_granularity_ns value="2000000" />
	</scheduler>

	<scheduler value="cfq">
		<cfq_sched_nice value="-c1 -n4" />
		<cfq_sched_back_seek_max value="16384" />
		<cfq_sched_back_seek_penalty value="2" />
		<cfq_sched_fifo_expire_async value="250" />
		<cfq_sched_fifo_expire_sync value="125" />
		<cfq_sched_quantum value="4" />
		<cfq_sched_slice_async value="40" />
		<cfq_sched_slice_async_rq value="2" />
		<cfq_sched_slice_idle value="8" />
		<cfq_sched_slice_sync value="100" />
	</scheduler>

	<scheduler value="anticipatory">
		<anticipatory_sched_antic_expire value="6" />
		<anticipatory_sched_read_batch_expire value="500" />
		<anticipatory_sched_read_expire value="125" />
		<anticipatory_sched_write_batch_expire value="125" />
		<anticipatory_sched_write_expire value="250" />
	</scheduler>

	<app bin="kaffeine" />
</sched_class>
</cf>

Related Links:

Now all we need to match

Anonymous (not verified)
on
September 5, 2007 - 10:02pm

Now all we need to match Windows performance is a window manager that gives the foreground app a priority boost.

You've got to be kidding! At

Anonymous (not verified)
on
September 5, 2007 - 11:02pm

You've got to be kidding!

At work I have to use a windows machine (XP) for rietveld refinement and I've never seen a scheduler as bad as this. No matter what the active program is - as long as the refinement is running, the machine becomes unuseable for anything else. Moving windows? Forget it.

On Linux OTOH I can have multiple 100%-CPU tasks and still move windows around, use gui-programs etc., without problem.

Sarcasm?

Anonymous (not verified)
on
September 5, 2007 - 11:32pm

Hmmm, maybe I'm a bit naive, but I think the OP was being sarcastic, leaving out any hint to provoke posters and see if anyone notices.

Well, maybe that's far-fetched... ;)

I don't know what rietveld

Anonymous (not verified)
on
September 6, 2007 - 1:45am

I don't know what rietveld refinement is, but if it's a machinery control software or in any case, it seems like it's trying to run with the highest priority, so if everything else is slow actually the scheduler is doing its job... For comparison try running some busy loop on Linux at realtime priority; and then just attempt to even move the mouse! Not a fair comparison because X is userspace...

Well, it isn't

Anonymous (not verified)
on
September 6, 2007 - 4:38am

Well, it isn't machine-control software, it's a numeric method for refinement of crystal structures. And I'd like to surf^Wwork while a refinement is running. If I'd want cooperative multitasking, I'd switch to Windows 3.1, thank you very much.

BCyDBu

BCyDBu (not verified)
on
November 10, 2010 - 11:21am

vuLdAeIb

What's the difference

Anonymous (not verified)
on
September 6, 2007 - 11:53am

What's the difference between this and using nice?

You have to be superuser to

Anonymous (not verified)
on
September 6, 2007 - 4:17pm

You have to be superuser to give a process higher-than-normal priority, so, you'd have to run something like this:

su -c "nice -10 su -c 'media-player' youser" root

It makes no sense to enter the superuser password for playing media.

i run pgrep player | xargs

eMBee, (not verified)
on
September 7, 2007 - 5:49am

i run

pgrep player | xargs renice -20

as root to raise the priority.
that way the player itself does not get to run as root.

doesn't help much if something is compiling in the background though.

greetings, eMBee

It's a PITA to use nice all

Anonymous (not verified)
on
September 7, 2007 - 12:50am

It's a PITA to use nice all the time?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.