Product SiteDocumentation Site

2.3. Diskdevstat and netdevstat

Diskdevstat and netdevstat are SystemTap tools that collect detailed information about the disk activity and network activity of all applications running on a system. These tools were inspired by PowerTOP, which shows the number of CPU wakeups by every application per second (refer to Section 2.2, “PowerTOP”). The statistics that these tools collect allow you to identify applications that waste power with many small I/O operations rather than fewer, larger operations. Other monitoring tools that measure only transfer speeds do not help to identify this type of usage.
Install these tools with SystemTap with the command:
yum install systemtap tuned-utils kernel-debuginfo
Run the tools with the command:
diskdevstat
or the command:
netdevstat
Both commands can take up to three parameters, as follows:
diskdevstat update_interval total_duration display_histogram
netdevstat update_interval total_duration display_histogram
update_interval
The time in seconds between updates of the display. Default: 5
total_duration
The time in seconds for the whole run. Default: 86400 (1 day)
display_histogram
Flag whether to histogram for all the collected data at the end of the run.
The output resembles that of PowerTOP. Here is sample output from a longer diskdevstat run on a Fedora 10 system running KDE 4.2:
  PID   UID DEV     WRITE_CNT WRITE_MIN WRITE_MAX WRITE_AVG    READ_CNT  READ_MIN  READ_MAX  READ_AVG COMMAND        
 2789  2903 sda1          854     0.000   120.000    39.836           0     0.000     0.000     0.000 plasma            
15494     0 sda1            0     0.000     0.000     0.000         758     0.000     0.012     0.000 0logwatch         
15520     0 sda1            0     0.000     0.000     0.000         140     0.000     0.009     0.000 perl              
15549     0 sda1            0     0.000     0.000     0.000         140     0.000     0.009     0.000 perl              
15585     0 sda1            0     0.000     0.000     0.000         108     0.001     0.002     0.000 perl              
 2573     0 sda1           63     0.033  3600.015   515.226           0     0.000     0.000     0.000 auditd            
15429     0 sda1            0     0.000     0.000     0.000          62     0.009     0.009     0.000 crond             
15379     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15473     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15415     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15433     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15425     0 sda1            0     0.000     0.000     0.000          62     0.007     0.007     0.000 crond             
15375     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15477     0 sda1            0     0.000     0.000     0.000          62     0.007     0.007     0.000 crond             
15469     0 sda1            0     0.000     0.000     0.000          62     0.007     0.007     0.000 crond             
15419     0 sda1            0     0.000     0.000     0.000          62     0.008     0.008     0.000 crond             
15481     0 sda1            0     0.000     0.000     0.000          61     0.000     0.001     0.000 crond             
15355     0 sda1            0     0.000     0.000     0.000          37     0.000     0.014     0.001 laptop_mode       
 2153     0 sda1           26     0.003  3600.029  1290.730           0     0.000     0.000     0.000 rsyslogd          
15575     0 sda1            0     0.000     0.000     0.000          16     0.000     0.000     0.000 cat               
15581     0 sda1            0     0.000     0.000     0.000          12     0.001     0.002     0.000 perl              
15582     0 sda1            0     0.000     0.000     0.000          12     0.001     0.002     0.000 perl              
15579     0 sda1            0     0.000     0.000     0.000          12     0.000     0.001     0.000 perl              
15580     0 sda1            0     0.000     0.000     0.000          12     0.001     0.001     0.000 perl              
15354     0 sda1            0     0.000     0.000     0.000          12     0.000     0.170     0.014 sh                
15584     0 sda1            0     0.000     0.000     0.000          12     0.001     0.002     0.000 perl              
15548     0 sda1            0     0.000     0.000     0.000          12     0.001     0.014     0.001 perl              
15577     0 sda1            0     0.000     0.000     0.000          12     0.001     0.003     0.000 perl              
15519     0 sda1            0     0.000     0.000     0.000          12     0.001     0.005     0.000 perl              
15578     0 sda1            0     0.000     0.000     0.000          12     0.001     0.001     0.000 perl              
15583     0 sda1            0     0.000     0.000     0.000          12     0.001     0.001     0.000 perl              
15547     0 sda1            0     0.000     0.000     0.000          11     0.000     0.002     0.000 perl              
15576     0 sda1            0     0.000     0.000     0.000          11     0.001     0.001     0.000 perl              
15518     0 sda1            0     0.000     0.000     0.000          11     0.000     0.001     0.000 perl              
15354     0 sda1            0     0.000     0.000     0.000          10     0.053     0.053     0.005 lm_lid.sh
The columns are:
PID
the process ID of the application
UID
the user ID under which the applications is running
DEV
the device on which the I/O took place
WRITE_CNT
the total number of write operations
WRITE_MIN
the lowest time taken for two consecutive writes (in seconds)
WRITE_MAX
the greatest time taken for two consecutive writes (in seconds)
WRITE_AVG
the average time taken for two consecutive writes (in seconds)
READ_CNT
the total number of read operations
READ_MIN
the lowest time taken for two consecutive reads (in seconds)
READ_MAX
the greatest time taken for two consecutive reads (in seconds)
READ_AVG
the average time taken for two consecutive reads (in seconds)
COMMAND
the name of the process
In this example, three very obvious applications stand out:
  PID   UID DEV     WRITE_CNT WRITE_MIN WRITE_MAX WRITE_AVG    READ_CNT  READ_MIN  READ_MAX  READ_AVG COMMAND
 2789  2903 sda1          854     0.000   120.000    39.836           0     0.000     0.000     0.000 plasma
 2573     0 sda1           63     0.033  3600.015   515.226           0     0.000     0.000     0.000 auditd
 2153     0 sda1           26     0.003  3600.029  1290.730           0     0.000     0.000     0.000 rsyslogd
These three applications have a WRITE_CNT greater than 0, which means that they performed some form of write during the measurement. Of those, plasma was the worst offender by a large degree: it performed the most write operations, and of course the average time between writes was the lowest. Plasma would therefore be the best candidate to investigate if you were concerned about power-inefficient applications.
Use the strace and ltrace commands to examine applications more closely by tracing all system calls of the given process ID. In the present example, you could run:
strace -p 2789
In this example, the output of the strace contained a repeating pattern every 45 seconds that opened the KDE icon cache file of the user for writing followed by an immediate close of the file again. This led to a necessary physical write to the hard disk as the file metadata (specifically, the modification time) had changed. The final fix was to prevent those unnecessary calls when no updates to the icons had occurred.