Solaris Dynamic Tracing Guide

snv_23

The contents of this Documentation are subject to the Public Documentation License Version 1.01 (the "License"); you may only use this Documentation if you comply with the terms of this License. A copy of the License is available at http://www.opensolaris.org/os/community/documentation/license.

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.

Sun, Sun Microsystems, the Sun logo, docs.sun.com, AnswerBook, AnswerBook2, Java, StarOffice and Solaris are trademarks, registered trademarks, or service marks of Sun Microsystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

The OPEN LOOK and Sun™ Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun's licensees who implement OPEN LOOK GUIs and otherwise comply with Sun's written license agreements.

Federal Acquisitions: Commercial Software–Government Users Subject to Standard License Terms and Conditions.

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l'utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l'autorisation préalable et écrite de Sun et de ses bailleurs de licence, s'il y en a. Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.

Des parties de ce produit pourront être dérivées du système Berkeley BSD licenciés par l'Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d'autres pays et licenciée exclusivement par X/Open Company, Ltd.

Sun, Sun Microsystems, le logo Sun, docs.sun.com, AnswerBook, AnswerBook2, et Solaris sont des marques de fabrique ou des marques déposées, ou marques de service, de Sun Microsystems, Inc. aux Etats-Unis et dans d'autres pays. Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d'autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.

L'interface d'utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sun reconnaît les efforts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d'utilisation visuelle ou graphique pour l'industrie de l'informatique. Sun détient une licence non exclusive de Xerox sur l'interface d'utilisation graphique Xerox, cette licence couvrant également les licenciés de Sun qui mettent en place l'interface d'utilisation graphique OPEN LOOK et qui en outre se conforment aux licences écrites de Sun.

CETTE PUBLICATION EST FOURNIE “EN L'ETAT” ET AUCUNE GARANTIE, EXPRESSE OU IMPLICITE, N'EST ACCORDEE, Y COMPRIS DES GARANTIES CONCERNANT LA VALEUR MARCHANDE, L'APTITUDE DE LA PUBLICATION A REPONDRE A UNE UTILISATION PARTICULIERE, OU LE FAIT QU'ELLE NE SOIT PAS CONTREFAISANTE DE PRODUIT DE TIERS. CE DENI DE GARANTIE NE S'APPLIQUERAIT PAS, DANS LA MESURE OU IL SERAIT TENU JURIDIQUEMENT NUL ET NON AVENU.

September 2005

Abstract

DTrace is a comprehensive dynamic tracing framework for the Solaris™ Operating System. DTrace provides a powerful infrastructure to permit administrators, developers, and service personnel to concisely answer arbitrary questions about the behavior of the operating system and user programs. The Solaris Dynamic Tracing Guide describes how to use DTrace to observe, debug, and tune system behavior. This book also includes a complete reference for bundled DTrace observability tools and the D programming language.


Table of Contents

Preface
Who Should Use This Book
How This Book Is Organized
Related Information
Accessing Sun Documentation Online
Ordering Sun Documentation
Typographic Conventions
Shell Prompts in Command Examples
1. Introduction
Getting Started
Providers and Probes
Compilation and Instrumentation
Variables and Arithmetic Expressions
Predicates
Output Formatting
Arrays
External Symbols and Types
2. Types, Operators, and Expressions
Identifier Names and Keywords
Data Types and Sizes
Constants
Arithmetic Operators
Relational Operators
Logical Operators
Bitwise Operators
Assignment Operators
Increment and Decrement Operators
Conditional Expressions
Type Conversions
Precedence
3. Variables
Scalar Variables
Associative Arrays
Thread-Local Variables
Clause-Local Variables
Built-in Variables
External Variables
4. D Program Structure
Probe Clauses and Declarations
Probe Descriptions
Predicates
Actions
Use of the C Preprocessor
5. Pointers and Arrays
Pointers and Addresses
Pointer Safety
Array Declarations and Storage
Pointer and Array Relationship
Pointer Arithmetic
Generic Pointers
Multi-Dimensional Arrays
Pointers to DTrace Objects
Pointers and Address Spaces
6. Strings
String Representation
String Constants
String Assignment
String Conversion
String Comparison
7. Structs and Unions
Structs
Pointers to Structs
Unions
Member Sizes and Offsets
Bit-Fields
8. Type and Constant Definitions
Typedef
Enumerations
Inlines
Type Namespaces
9. Aggregations
Aggregating Functions
Aggregations
Printing Aggregations
Data Normalization
Clearing Aggregations
Truncating aggregations
Minimizing Drops
10. Actions and Subroutines
Actions
Default Action
Data Recording Actions
trace
tracemem
printf
printa
stack
ustack
jstack
Destructive Actions
Process Destructive Actions
Kernel Destructive Actions
Special Actions
Speculative Actions
exit
Subroutines
alloca
basename
bcopy
cleanpath
copyin
copyinstr
copyinto
dirname
msgdsize
msgsize
mutex_owned
mutex_owner
mutex_type_adaptive
progenyof
rand
rw_iswriter
rw_write_held
speculation
strjoin
strlen
11. Buffers and Buffering
Principal Buffers
Principal Buffer Policies
switch Policy
fill Policy
ring Policy
Other Buffers
Buffer Sizes
Buffer Resizing Policy
12. Output Formatting
printf
Conversion Specifications
Flag Specifiers
Width and Precision Specifiers
Size Prefixes
Conversion Formats
printa
trace Default Format
13. Speculative Tracing
Speculation Interfaces
Creating a Speculation
Using a Speculation
Committing a Speculation
Discarding a Speculation
Speculation Example
Speculation Options and Tuning
14. dtrace ( 1M ) Utility
Description
Options
Operands
Exit Status
15. Scripting
Interpreter Files
Macro Variables
Macro Arguments
Target Process ID
16. Options and Tunables
Consumer Options
Modifying Options
17. dtrace Provider
BEGIN Probe
The END Probe
ERROR Probe
Stability
18. lockstat Provider
Overview
Adaptive Lock Probes
Spin Lock Probes
Thread Locks
Readers/Writer Lock Probes
Stability
19. profile Provider
profile- n probes
tick- n probes
Arguments
Timer Resolution
Probe Creation
Stability
20. fbt Provider
Probes
Probe arguments
entry probes
return probes
Examples
Tail-call Optimization
Assembly Functions
Instruction Set Limitations
x86 Limitations
SPARC Limitations
Breakpoint Interaction
Module Loading
Stability
21. syscall Provider
Probes
System Call Anachronisms
Subcoded System Calls
Large File System Calls
Private System Calls
Arguments
Stability
22. sdt Provider
Probes
Examples
Creating SDT Probes
Declaring Probes
Probe Arguments
Stability
23. sysinfo Provider
Probes
Arguments
Example
Stability
24. vminfo Provider
Probes
Arguments
Example
Stability
25. proc Provider
Probes
Arguments
lwpsinfo_t
psinfo_t
Examples
exec
start and exit
lwp-start and lwp-exit
signal-send
Stability
26. sched Provider
Probes
Arguments
cpuinfo_t
Examples
on-cpu and off-cpu
enqueue and dequeue
sleep and wakeup
preempt, remain-cpu
change-pri
tick
Stability
27. io Provider
Probes
Arguments
bufinfo_t structure
devinfo_t
fileinfo_t
Examples
Stability
28. mib Provider
Probes
Arguments
Stability
29. fpuinfo Provider
Probes
Arguments
Stability
30. pid Provider
Naming pid Probes
Function Boundary Probes
entry Probes
return Probes
Function Offset Probes
Stability
31. plockstat Provider
Overview
Mutex Probes
Reader/Writer Lock Probes
Stability
32. fasttrap Provider
Probes
Stability
33. User Process Tracing
copyin and copyinstr Subroutines
Avoiding Errors
Eliminating dtrace ( 1M ) Interference
syscall Provider
ustack Action
uregs[] Array
pid Provider
User Function Boundary Tracing
Tracing Arbitrary Instructions
34. Statically Defined Tracing for User Applications
Choosing the Probe Points
Adding Probes to an Application
Defining Providers and Probes
Adding Probes to Application Code
Building Applications with Probes
35. Security
Privileges
Privileged Use of DTrace
dtrace_proc Privilege
dtrace_user Privilege
dtrace_kernel Privilege
Super User Privileges
36. Anonymous Tracing
Anonymous Enablings
Claiming Anonymous State
Anonymous Tracing Examples
37. Postmortem Tracing
Displaying DTrace Consumers
Displaying Trace Data
38. Performance Considerations
Limit Enabled Probes
Use Aggregations
Use Cacheable Predicates
39. Stability
Stability Levels
Dependency Classes
Interface Attributes
Stability Computations and Reports
Stability Enforcement
40. Translators
Translator Declarations
Translate Operator
Process Model Translators
Stable Translations
41. Versioning
Versions and Releases
Versioning Options
Provider Versioning
Glossary
Index

List of Figures

1.1. Overview of the DTrace Architecture and Components
5.1. Scalar Array Representation
5.2. Pointer and Array Storage

List of Tables

1. Typographic Conventions
2. Shell Prompts
2.1. D Keywords
2.2. D Integer Data Types
2.3. D Integer Type Aliases
2.4. D Floating-Point Data Types
2.5. D Character Escape Sequences
2.6. D Binary Arithmetic Operators
2.7. D Relational Operators
2.8. D Logical Operators
2.9. D Bitwise Operators
2.10. D Assignment Operators
2.11. D Operator Precedence and Associativity
3.1. DTrace Built-in Variables
4.1. Probe Name Pattern Matching Characters
6.1. D Relational Operators for Strings
9.1. DTrace Aggregating Functions
13.1. DTrace Speculation Functions
15.1. D Macro Variables
16.1. DTrace Consumer Options
18.1. Adaptive Lock Probes
18.2. Spin Lock Probes
18.3. Thread Lock Probe
18.4. Readers/Writer Lock Probes
19.1. Valid time suffixes
21.1. sycall Large File Probes
22.1. SDT Probes
23.1. sysinfo Probes
24.1. vminfo Probes
25.1. proc Probes
25.2. proc Probe Arguments
25.3. pr_flag Values
25.4. pr_stype Values
25.5. pr_state Values
26.1. sched Probes
26.2. sched Probe Arguments
27.1. io Probes
27.2. io Probe Arguments
27.3. b_flags Values
28.1. mib probes
28.2. ICMP mib Probes
28.3. IP mib Probes
28.4. IPsec mib Probes
28.5. IPv6 mib Probes
28.6. Raw IP mib Probes
28.7. SCTP mib Probes
28.8. TCP mib Probes
28.9. UDP mib Probes
29.1. fpuinfo Probes
31.1. Mutex Probes
31.2. Readers/Writer Lock Probes
33.1. SPARC uregs[] Constants
33.2. x86 uregs[] Constants
33.3. amd64 uregs[] Constants
33.4. Common uregs[] Constants
40.1. procfs.d Translators
41.1. DTrace Release Versions

List of Examples

1.1. hello.d: Hello, World from the D Programming Language
1.2. trussrw.d: Trace System Calls with truss ( 1 ) Output Format
1.3. rwtime.d: Time read ( 2 ) and write ( 2 ) Calls
3.1. rtime.d: Compute Time Spent in read ( 2 )
3.2. clause.d: Clause-local Variables
5.1. badptr.d: Demonstration of DTrace Error Handling
7.1. rwinfo.d: Gather read ( 2 ) and write ( 2 ) Statistics
7.2. ksyms.d: Trace read ( 2 ) and uiomove ( 9F ) Relationship
7.3. kstat.d: Trace Calls to kstat_data_lookup ( 3KSTAT )
9.1. renormalize.d: Renormalizing an Aggregation
13.1. specopen.d: Code Flow for Failed open ( 2 )
17.1. error.d: Record Errors
33.1. userfunc.d: Trace User Function Entry and Return
33.2. errorpath.d: Trace User Function Call Error Path
34.1. myserv.d: Statically Defined Application Probes