6.7. ftrace

The ftrace framework provides users with several tracing capabilities, accessible through an interface much simpler than SystemTap's. This framework uses a set of virtual files in the debugfs file system; these files enable specific tracers. The ftrace function tracer simply outputs each function called in the kernel in real time; other tracers within the ftrace framework can also be used to analyze wakeup latency, task switches, kernel events, and the like.

You can also add new tracers for ftrace, making it a flexible solution for analyzing kernel events. The ftrace framework is useful for debugging or analyzing latencies and performance issues that take place outside of user-space. Unlike other profilers documented in this guide, ftrace is a built-in feature of the kernel.

6.7.1. Using ftrace

The Red Hat Enterprise Linux 6 kernels have been configured with the CONFIG_FTRACE=y option. This option provides the interfaces needed by ftrace. To use ftrace, mount the debugfs file system as follows:

mount -t debugfs nodev /sys/kernel/debug

All the ftrace utilities are located in /sys/kernel/debug/tracing/. View the /sys/kernel/debug/tracing/available_tracers file to find out what tracers are available for your kernel:

cat /sys/kernel/debug/tracing/available_tracers

power wakeup irqsoff function sysprof sched_switch initcall nop

To use a specific tracer, write it to /sys/kernel/debug/tracing/current_tracer. For example, wakeup traces and records the maximum time it takes for the highest-priority task to be scheduled after the task wakes up. To use it:

echo wakeup > /sys/kernel/debug/tracing/current_tracer

To start or stop tracing, write to /sys/kernel/debug/tracing/tracing_on, as in:

echo 1 > /sys/kernel/debug/tracing/tracing_on (enables tracing)

echo 0 > /sys/kernel/debug/tracing/tracing_on (disables tracing)

The results of the trace can be viewed from the following files:

/sys/kernel/debug/tracing/trace: This file contains human-readable trace output.
/sys/kernel/debug/tracing/trace_pipe: This file contains the same output as /sys/kernel/debug/tracing/trace, but is meant to be piped into a command. Unlike /sys/kernel/debug/tracing/trace, reading from this file consumes its output.