PS RESULTS USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND pg82 30087 0.0 0.4 83844 5564 ? S May16 0:00 /usr/local/pg82/bin/postgres pg82 30088 0.0 0.1 10940 1664 ? Ss May16 0:02 postgres: logger process pg82 30090 0.0 5.5 83984 71284 ? Ss May16 0:01 postgres: writer process pg82 30091 0.0 0.1 11280 1736 ? Ss May16 0:00 postgres: stats collector process pg82 10461 1.4 2.0 84888 26296 ? Ds 16:51 4:31 postgres: pg82 bench [local] SELECT pg82 10462 1.2 2.0 84888 26348 ? Ds 16:51 4:31 postgres: pg82 bench [local] SELECT pg82 10463 1.3 2.0 84888 26020 ? Ds 16:51 4:31 postgres: pg82 bench [local] SELECT pg82 10464 1.4 2.0 84888 26544 ? Ds 16:51 4:31 postgres: pg82 bench [local] SELECT pg82 10502 0.0 0.7 85280 8984 ? Ss 17:54 3:27 postgres: pg82 bench [local] idle in transaction PID/NLWP MEM USR SYS SLP LAT VCX ICX SCL SIG PROCESS 1721/1 20M 1.8 4.4 44 3.9 1K 141 3K 0 postgres: postgres postgres [ 1720/1 6M 2.1 2.6 87 0.1 312 46 4K 0 pgbench -i -t 10 -c 10 1705/1 3M 0.0 0.0 100 0.1 27 1 83 0 -bash 1715/1 20M 0.0 0.0 100 0.3 126 0 147 0 postgres: writer process 1713/1 20M 0.0 0.0 100 0.0 1 3 9 0 /usr/bin/postmaster 1716/1 9M 0.0 0.0 100 0.0 4 0 7 0 postgres: stats buffer proces 1717/1 8M 0.0 0.0 100 0.0 1 1 3 0 postgres: stats collector pro PG_TOP RESULTS last pid: 5402; load avg: 1.33, 0.56, 0.36; up 1+15:07:27 14:17:58 16 processes: 14 sleeping, 2 uninterruptable CPU states: 1.7% user, 0.0% nice, 0.7% system, 69.3% idle, 28.3% iowait Memory: 3854M used, 137M free, 47M buffers, 3638M cached Swap: 5128K used, 1979M free, 404K cached PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 4872 postgres 15 0 817M 654M sleep 17:11 0.96% 1.37% postgres: webtech_rw webtech 10.2.74.41(56860) idle 4898 postgres 15 0 816M 652M sleep 17:09 0.79% 0.98% postgres: webtech_rw webtech 10.2.74.33(40337) idle 4875 postgres 15 0 817M 655M sleep 17:09 1.04% 0.98% postgres: webtech_rw webtech 10.2.74.41(56863) idle 4897 postgres 15 0 816M 652M sleep 17:08 0.83% 0.98% postgres: webtech_rw webtech 10.2.74.33(40336) idle 4873 postgres 15 0 817M 655M sleep 17:07 0.99% 0.98% postgres: webtech_rw webtech 10.2.74.41(56861) idle 5358 postgres 18 0 816M 46M disk 0:00 0.34% 0.98% postgres: webtech_rw webtech 10.2.81.36(39999) SELECT 4874 postgres 15 0 817M 654M sleep 17:11 1.01% 0.78% postgres: webtech_rw webtech 10.2.74.41(56862) idle 4899 postgres 15 0 816M 652M sleep 17:08 0.79% 0.59% postgres: webtech_rw webtech 10.2.74.33(40338) idle 5276 postgres 18 0 816M 132M disk 0:00 0.67% 0.59% postgres: webtech_rw webtech 10.2.81.36(34670) SELECT 4867 postgres 15 0 815M 47M sleep 5:07 0.29% 0.20% postgres: webtech_rw webtech 10.2.74.41(56859) idle 4862 postgres 15 0 815M 47M sleep 3:38 0.16% 0.20% postgres: webtech_rw webtech 10.2.74.33(40333) idle 4880 postgres 16 0 815M 43M sleep 3:56 0.14% 0.00% postgres: webtech_rw webtech 10.2.74.33(40334) idle 5161 postgres 15 0 815M 36M sleep 3:09 0.01% 0.00% postgres: webtech_rw webtech 10.2.74.33(53935) idle 4881 postgres 17 0 815M 25M sleep 0:05 0.00% 0.00% postgres: webtech_rw webtech 10.2.74.33(40335) idle in tran 5403 postgres 20 0 815M 4464K sleep 0:00 0.00% 0.00% postgres: postgres postgres [local] idle MPSTAT RESULTS 08:02:37 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 08:02:40 PM all 42.24 0.00 7.66 12.98 0.00 0.79 0.00 36.33 1533.55 08:02:40 PM 0 58.80 0.00 8.64 4.32 0.00 0.00 0.00 28.24 1005.98 08:02:40 PM 1 73.00 0.00 7.00 4.67 0.00 3.33 0.00 12.00 391.36 08:02:40 PM 2 72.43 0.00 8.31 2.33 0.00 0.00 0.00 16.94 11.96 08:02:40 PM 3 8.94 0.00 9.60 74.50 0.33 2.98 0.00 3.64 124.58 08:02:40 PM 4 22.00 0.00 2.67 3.67 0.00 0.00 0.00 71.67 0.00 08:02:40 PM 5 16.28 0.00 2.33 0.33 0.00 0.00 0.00 81.06 0.00 08:02:40 PM 6 32.33 0.00 4.33 4.00 0.00 0.00 0.00 59.33 0.00 08:02:40 PM 7 53.67 0.00 18.67 9.67 0.00 0.00 0.00 18.00 0.00 08:10:17 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 08:10:20 PM all 12.58 0.00 1.79 22.54 0.04 0.46 0.00 62.58 1249.00 08:10:20 PM 0 0.00 0.00 1.33 17.67 0.00 0.00 0.00 81.00 1002.67 08:10:20 PM 1 3.67 0.00 1.00 35.67 0.00 0.00 0.00 59.67 50.67 08:10:20 PM 2 0.00 0.00 0.66 1.99 0.00 0.00 0.00 97.34 12.00 08:10:20 PM 3 2.33 0.00 4.00 89.67 0.33 3.67 0.00 0.00 184.33 08:10:20 PM 4 0.00 0.00 0.33 14.05 0.00 0.00 0.00 85.62 0.00 08:10:20 PM 5 16.67 0.00 1.67 6.67 0.00 0.00 0.00 75.00 0.00 08:10:20 PM 6 0.33 0.00 0.67 6.33 0.00 0.00 0.00 92.67 0.00 08:10:20 PM 7 78.00 0.00 5.00 7.67 0.00 0.00 0.00 9.33 0.00 VMSTAT RESULTS kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr s1 s2 s3 sd in sy cs us sy id 0 0 0 13760072 10132088 133 23 233 31 31 0 0 0 0 0 206 25375 30254 32511 21 4 75 0 0 0 12871016 9223456 24 76 4 42 42 0 0 2 0 0 1073 73439 79977 88597 58 12 30 1 0 0 12857664 9210312 29 96 6 30 30 0 0 0 0 0 873 76979 88981 95395 68 13 19 1 0 0 12842128 9195224 27 70 7 66 66 0 0 0 0 0 1006 76829 92117 96983 70 14 17 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 3 4 328932 880336 5676 249040 0 1 5 7 1 34 1 0 99 0 0 4 328888 879468 5676 249852 44 0 724 0 343 808 1 4 0 95 0 4 328848 878724 5676 250612 0 0 640 0 370 823 1 2 0 97 0 4 328828 877980 5676 251396 32 0 712 0 340 787 2 2 0 96 0 4 328816 877112 5708 252168 4 0 676 48 380 806 3 4 0 93 0 4 328804 876368 5708 252972 0 0 704 0 343 826 2 3 0 95 0 4 328768 875252 5708 253872 36 0 796 0 384 965 2 2 0 96 0 5 328740 874384 5708 254828 64 0 904 0 349 861 2 1 0 97 0 5 328424 872772 5724 255976 508 0 1408 0 393 822 1 2 0 97 0 5 328092 871904 5748 257056 104 0 780 24 344 796 1 3 0 96 total used free shared buffers cached Mem: 1282796 1270748 12048 0 8260 970192 -/+ buffers/cache: 292296 990500 Swap: 524624 27504 497120 total used free shared buffers cached Mem: 1282796 1273120 9676 0 3756 196756 -/+ buffers/cache: 1072608 210188 Swap: 524624 508568 16056 IOSTAT RESULTS avg-cpu: %user %nice %sys %iowait %idle 9.11 0.00 9.61 7.62 73.66 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util cciss/c0d0 0.00 0.00 0.00 3.00 0.00 24.00 0.00 12.00 8.00 0.01 3.33 2.33 0.70 sda 0.00 20563.00 0.00 283.00 0.00 165904.00 0.00 82952.00 586.23 77.95 655.58 2.60 73.50 avg-cpu: %user %nice %sys %iowait %idle 9.61 0.00 6.74 46.62 37.03 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util cciss/c0d0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 17277.00 0.00 1516.00 0.00 149448.00 0.00 74724.00 98.58 80.94 59.99 0.57 86.60 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.1 12.1 8.8 709.5 0.0 0.0 0.0 3.4 0 2 c2t40d1 0.0 210.6 0.0 3463.2 0.0 0.3 0.0 1.6 0 33 c2t40d0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 progdb:vold(pid508) extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.0 11.0 7.7 760.8 0.0 0.0 0.0 3.2 0 2 c2t40d1 0.0 210.8 0.0 3446.9 0.0 0.3 0.0 1.5 0 33 c2t40d0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 progdb:vold(pid508) extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.9 1219.3 7.2 21509.5 1.1 21.0 0.9 17.2 3 42 c2t40d1 0.0 177.6 0.0 3059.2 0.0 0.5 0.0 3.0 0 54 c2t40d0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 progdb:vold(pid508) extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.1 162.6 9.1 3302.2 0.0 0.9 0.0 5.2 0 9 c2t40d1 0.1 212.8 0.1 3502.0 0.0 0.4 0.0 1.7 0 36 c2t40d0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 progdb:vold(pid508) extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.2 18.0 9.6 691.8 0.0 0.1 0.0 4.5 0 2 c2t40d1 0.0 211.7 0.0 3457.4 0.0 0.3 0.0 1.6 0 33 c2t40d0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 progdb:vold(pid508) SAR EXAMPLES -bash-3.2$ sar -P ALL -s 11:00:00 -e 11:30:00 11:00:01 AM CPU %user %nice %system %iowait %steal %idle 11:10:02 AM all 0.38 0.00 0.09 1.78 0.00 97.75 11:10:02 AM 0 2.79 0.00 0.53 7.84 0.00 88.84 11:10:02 AM 1 0.00 0.00 0.04 1.55 0.00 98.41 11:10:02 AM 2 0.18 0.00 0.08 2.62 0.00 97.12 11:10:02 AM 3 0.03 0.00 0.05 2.23 0.00 97.69 11:10:02 AM 4 0.01 0.00 0.01 0.03 0.00 99.95 11:10:02 AM 5 0.01 0.00 0.00 0.01 0.00 99.98 11:10:02 AM 6 0.00 0.00 0.01 0.00 0.00 99.99 11:10:02 AM 7 0.00 0.00 0.00 0.00 0.00 99.99 11:20:01 AM all 0.39 0.00 0.09 1.48 0.00 98.05 11:20:01 AM 0 2.88 0.00 0.51 7.48 0.00 89.13 11:20:01 AM 1 0.00 0.00 0.03 0.42 0.00 99.55 11:20:01 AM 2 0.19 0.00 0.09 3.02 0.00 96.70 11:20:01 AM 3 0.02 0.00 0.03 0.49 0.00 99.46 11:20:01 AM 4 0.01 0.00 0.01 0.04 0.00 99.94 11:20:01 AM 5 0.00 0.00 0.01 0.36 0.00 99.63 11:20:01 AM 6 0.00 0.00 0.00 0.00 0.00 100.00 11:20:01 AM 7 0.00 0.00 0.01 0.00 0.00 99.99 Average: all 0.38 0.00 0.09 1.63 0.00 97.90 Average: 0 2.84 0.00 0.52 7.66 0.00 88.98 Average: 1 0.00 0.00 0.03 0.98 0.00 98.98 Average: 2 0.18 0.00 0.09 2.82 0.00 96.91 Average: 3 0.02 0.00 0.04 1.36 0.00 98.57 Average: 4 0.01 0.00 0.01 0.03 0.00 99.94 Average: 5 0.01 0.00 0.01 0.18 0.00 99.80 Average: 6 0.00 0.00 0.00 0.00 0.00 99.99 Average: 7 0.00 0.00 0.01 0.00 0.00 99.99 -bash-3.2$ sar -r -s 11:00:00 -e 11:30:00 11:00:01 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad 11:10:02 AM 87148 16351708 99.47 276840 15398144 18459916 21228 0.11 19932 11:20:01 AM 98212 16340644 99.40 276640 15387672 18459908 21236 0.11 19876 Average: 92680 16346176 99.44 276740 15392908 18459912 21232 0.11 19904 -bash-3.2$ sar -b -s 11:00:00 -e 11:30:00 11:00:01 AM tps rtps wtps bread/s bwrtn/s 11:10:02 AM 200.19 22.66 177.53 419.20 2517.63 11:20:01 AM 214.19 0.68 213.51 7.63 3295.18 Average: 207.18 11.68 195.50 213.59 2906.07 DTRACE EXAMPLES dtrace $DESTRUCTOR -q -n ' #pragma D option strsize='"$STRSIZE"' self int starttime; self string sql; '"$LOG"' postgres*:::statement-start /self->starttime && timestamp > self->starttime/ { @q[zonename] = quantize((timestamp - self->starttime)/1000000); } postgres*:::statement-start '"$ZONE"' { self->sql = copyinstr(arg0); self->starttime = (strstr(self->sql, "") == NULL) ? timestamp : 0; } '"$QUANT_END" global value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@ 9 1 |@@ 1 2 |@@@@@@ 3 4 |@@ 1 8 |@@ 1 16 |@@ 1 32 |@@@@ 2 64 | 0 128 | 0 256 | 0 512 | 0 1024 | 0 2048 |@@ 1 4096 | 0 ./pgtps.d ******** Transactions Per Second ********* Txn Type Count ========================================== New 192 Commit 192 Abort 1 ******** Transactions Per Second ********* Txn Type Count ========================================== New 175 Commit 172 Abort 0 FILENAME/DBOBJECT READS WRITES # min avg max # min avg max alldata1__idx_remove_domain_external 1 12 12 12 398 0 0 0 slowdata1__pg_rewrite 1 12 12 12 0 0 0 0 slowdata1__pg_class_oid_index 1 0 0 0 0 0 0 0 slowdata1__pg_attribute 2 0 0 0 0 0 0 0 alldata1__mv_users 0 0 0 0 4 0 0 0 slowdata1__pg_statistic 1 0 0 0 0 0 0 0 slowdata1__pg_index 1 0 0 0 0 0 0 0 slowdata1__pg_index_indexrelid_index 1 0 0 0 0 0 0 0 alldata1__remove_domain_external 0 0 0 0 502 0 0 0 alldata1__promo_15_tb_full_2 19 0 0 0 11 0 0 0 slowdata1__pg_class_relname_nsp_index 2 0 0 0 0 0 0 0 alldata1__promo_177intaoltest_tb 0 0 0 0 1053 0 0 0 slowdata1__pg_attribute_relid_attnum_index 2 0 0 0 0 0 0 0 alldata1__promo_15_tb_full_2_pk 2 0 0 0 0 0 0 0 alldata1__all_mailable_2 1403 0 0 423 0 0 0 0 alldata1__mv_users_pkey 0 0 0 0 4 0 0 0 DD TEST RESULTS 2000000+0 records in 2000000+0 records out 16384000000 bytes (16 GB) copied, 147.161 seconds, 111 MB/s real 4m0.231s user 0m0.924s sys 0m40.334s -bash-3.2$ time dd if=ddfile of=/dev/null bs=8k 4000000+0 records in 4000000+0 records out 32768000000 bytes (33 GB) copied, 408.695 seconds, 80.2 MB/s real 6m48.729s user 0m1.756s sys 0m22.951s BONNIE++ RESULTS # ./bonnie++ Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thud 2G 33324 60 46554 13 24155 5 47561 81 55243 5 182.4 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP slowserver 2G 20583 49 20830 9 10314 3 34440 72 57350 7 447.0 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 3021 97 +++++ +++ +++++ +++ 2938 93 +++++ +++ 7080 100 -bash-3.2$ ~/sbin/bonnie++ -d . -c 3 -n 100 Version 1.95 ------Sequential Output------ --Sequential Input- --Random- Concurrency 3 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dm-breakpad-stag 8G 301 99 103000 51 41581 14 829 99 92119 16 892.4 25 Latency 36263us 870ms 677ms 18949us 727ms 32248us Version 1.95 ------Sequential Create------ --------Random Create-------- dm-breakpad-stagedb -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 100 35400 80 +++++ +++ 42303 86 38625 85 +++++ +++ 43017 88 Latency 25613us 236us 272us 204us 10us 180us Version 1.95 ------Sequential Output------ --Sequential Input- --Random- Concurrency 8 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP tm-breakpad0 32000M 723 97 71362 18 37543 7 2082 97 85594 7 291.3 13 Latency 16383us 410ms 1131ms 26032us 57126us 743ms Version 1.95 ------Sequential Create------ --------Random Create-------- tm-breakpad01-maste -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 100 31132 53 +++++ +++ 63174 69 42784 72 +++++ +++ 61995 69 Latency 34699us 482us 624us 644us 500us 538us IOZONE RESULTS Run began: Thu Apr 9 22:16:09 2009 Excel chart generation enabled Record Size 8 KB File size set to 4194304 KB Command line used: iozone -R -i 0 -i 1 -i 2 -i 3 -i 4 -i 5 -i 8 -l 6 -u 6 -r 8k -s 4G -F f1 f2 f3 f4 f5 f6 Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Min process = 6 Max process = 6 Throughput test with 6 processes Each process writes a 4194304 Kbyte file in 8 Kbyte records Children see throughput for 6 initial writers = 108042.81 KB/sec Parent sees throughput for 6 initial writers = 31770.90 KB/sec Min throughput per process = 13815.83 KB/sec Max throughput per process = 35004.07 KB/sec Avg throughput per process = 18007.13 KB/sec Min xfer = 1655408.00 KB Children see throughput for 6 rewriters = 91648.66 KB/sec Parent sees throughput for 6 rewriters = 66735.31 KB/sec Min throughput per process = 13177.17 KB/sec Max throughput per process = 15879.56 KB/sec Avg throughput per process = 15274.78 KB/sec Min xfer = 3491968.00 KB Children see throughput for 6 readers = 72270.04 KB/sec Parent sees throughput for 6 readers = 72269.06 KB/sec Min throughput per process = 11686.53 KB/sec Max throughput per process = 12506.65 KB/sec Avg throughput per process = 12045.01 KB/sec Min xfer = 3919344.00 KB Children see throughput for 6 re-readers = 74551.41 KB/sec Parent sees throughput for 6 re-readers = 74549.21 KB/sec Min throughput per process = 12063.23 KB/sec Max throughput per process = 12961.55 KB/sec Avg throughput per process = 12425.24 KB/sec Min xfer = 3903600.00 KB Children see throughput for 6 reverse readers = 17313.57 KB/sec Parent sees throughput for 6 reverse readers = 17313.52 KB/sec Min throughput per process = 2569.21 KB/sec Max throughput per process = 3101.18 KB/sec Avg throughput per process = 2885.60 KB/sec Min xfer = 3474840.00 KB Children see throughput for 6 stride readers = 12257.68 KB/sec Parent sees throughput for 6 stride readers = 12257.62 KB/sec Min throughput per process = 997.85 KB/sec Max throughput per process = 3153.15 KB/sec Avg throughput per process = 2042.95 KB/sec Min xfer = 1327344.00 KB Children see throughput for 6 random readers = 22279.35 KB/sec Parent sees throughput for 6 random readers = 22278.80 KB/sec Min throughput per process = 1150.86 KB/sec Max throughput per process = 11009.47 KB/sec Avg throughput per process = 3713.22 KB/sec Min xfer = 438448.00 KB Children see throughput for 6 mixed workload = 1190338.77 KB/sec Parent sees throughput for 6 mixed workload = 28622.30 KB/sec Min throughput per process = 1641.03 KB/sec Max throughput per process = 881036.56 KB/sec Avg throughput per process = 198389.80 KB/sec Min xfer = 7816.00 KB Children see throughput for 6 random writers = 84817.14 KB/sec Parent sees throughput for 6 random writers = 53555.93 KB/sec Min throughput per process = 12304.32 KB/sec Max throughput per process = 15492.32 KB/sec Avg throughput per process = 14136.19 KB/sec Min xfer = 3331680.00 KB PG_STAT_DATABASE and PG_DATABASE_SIZE postgres=# select * from pg_stat_database; datid | datname | numbackends | xact_commit | xact_rollback | blks_read | blks_hit | tup_returned | tup_fetched | tup_inserted | tup_updated | tup_deleted ---------+-----------+-------------+-------------+---------------+-----------+-----------+--------------+-------------+--------------+-------------+------------- 1 | template1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 11510 | template0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 11511 | postgres | 1 | 28890 | 0 | 415 | 2062876 | 4412746 | 843757 | 8 | 344 | 0 1116384 | webbreak | 12 | 4734145 | 9998 | 4271990 | 273811423 | 476079051 | 186765595 | 5365840 | 3798804 | 642751 postgres=# select pg_size_pretty(pg_database_size('breakpad')); pg_size_pretty ---------------- 58 GB PG_TABLES and PG_RELATION_SIZE schemaname | tablename | tableowner | tablespace | hasindexes | hasrules | hastriggers ------------+------------------------+------------+------------+------------+----------+------------- public | c_customer | spec | | t | f | f public | c_customerinventory | spec | | t | f | f public | c_parts | spec | | t | f | f public | c_site | spec | | t | f | f public | c_supplier | spec | | t | f | f public | cp_stat_user_indexes | spec | | f | f | f public | cp_stat_user_tables | spec | | f | f | f public | cp_statio_user_indexes | spec | | f | f | f schemaname | tablename | tableowner | tablespace | hasindexes | hasrules | hastriggers ------------+--------------------+------------+------------+------------+----------+------------- public | cash_transaction | pg82 | | t | f | f public | account_permission | pg82 | | t | f | t public | zip_code | pg82 | | t | f | t public | charge | pg82 | | t | f | t public | commission_rate | pg82 | | t | f | t public | company_competitor | pg82 | | t | f | t public | broker | pg82 | | t | f | t specdb=# select tablename, pg_size_pretty(pg_relation_size(tablename)) from pg_tables where schemaname = 'public' order by tablename; tablename | pg_size_pretty ----------------------+---------------- c_customer | 401 GB c_customerinventory | 13 GB c_parts | 242 MB c_site | 8192 bytes c_supplier | 8192 bytes specdb=# select tablename, indexname, pg_size_pretty(pg_relation_size(indexname)) from pg_indexes where schemaname = 'public' order by tablename, indexname; tablename | indexname | pg_size_pretty ----------------------+---------------------------+---------------- c_customer | c_customer_pkey | 1117 MB c_customerinventory | c_custci_idx | 9905 MB c_customerinventory | c_customerinventory_pkey | 7757 MB c_parts | c_parts_pkey | 60 MB c_site | c_site_pkey | 16 kB c_supplier | c_supplier_pkey | 16 kB PG_STAT_ACTIVITY datid | datname | procpid | usesysid | usename | current_query | waiting | query_start | backend_start | client_addr | client_port -------+----------+---------+----------+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+-------------------------------+-------------------------------+-------------+------------- 16700 | specdb | 3494 | 16385 | spec | select t0.ol_o_id, t0.ol_id, t0.ol_total_value, t0.ol_i_id, t0.ol_qty, t0.ol_ship_date, t0.ol_status, t0.ol_msrp from o_orderline t0 where t0.ol_o_id = $1 for update | f | 2007-05-20 18:51:04.405881-07 | 2007-05-20 18:49:36.200397-07 | 16.1.2.2 | 55902 16700 | specdb | 3495 | 16385 | spec | select t0.in_p_id, t0.in_ordered, t0.in_location, t0.in_qty, t0.in_act_date, t0.in_acc_code from m_inventory t0 where t0.in_p_id = $1 for update | f | 2007-05-20 18:51:04.400477-07 | 2007-05-20 18:49:36.202136-07 | 16.1.3.2 | 51431 16700 | specdb | 3496 | 16385 | spec | in transaction | f | 2007-05-20 18:51:03.850862-07 | 2007-05-20 18:49:36.221088-07 | 16.1.2.2 | 55903 16700 | specdb | 3497 | 16385 | spec | | f | 2007-05-20 18:51:04.285035-07 | 2007-05-20 18:49:36.22856-07 | 16.1.3.2 | 51432 16700 | specdb | 3498 | 16385 | spec | | f | 2007-05-20 18:51:04.355993-07 | 2007-05-20 18:49:36.247287-07 | 16.1.2.2 | 55904 16700 | specdb | 3499 | 16385 | spec | update m_inventory set in_ordered= $1, in_location= $2, in_qty= $3, in_act_date= $4, in_acc_code= $5 where in_p_id = $6 | f | 2007-05-20 18:51:04.36266-07 | 2007-05-20 18:49:36.249247-07 | 16.1.3.2 | 51433 16700 | specdb | 3525 | 16385 | spec | select t0.c_id, t0.c_contact, t0.c_credit_limit, t0.c_state, t0.c_zip, t0.c_phone, t0.c_credit, t0.c_since, t0.c_ytd_payment, t0.c_street1, t0.c_balance, t0.c_first, t0.c_last, t0.c_street2, t0.c_country, t0.c_city from c_customer t0 where t0.c_id = $1 for update | f | 2007-05-20 18:51:04.4098-07 | 2007-05-20 18:49:36.502272-07 | 16.1.1.2 | 53720 16700 | specdb | 3526 | 16385 | spec | select t0.o_id, t0.o_discount, t0.o_ol_cnt, t0.o_c_id, t0.o_total, t0.o_entry_date, t0.o_status, t0.o_ship_date from o_orders t0 where t0.o_c_id = $1 and (t0.o_status = $2 or t0.o_status = $3) for update 16700 | specdb | 3500 | 16385 | spec | | f | 2007-05-20 18:51:04.161188-07 | 2007-05-20 18:49:36.268272-07 | 16.1.2.2 | 55905 16700 | specdb | 3501 | 16385 | spec | | f | 2007-05-20 18:51:04.155349-07 | 2007-05-20 18:49:36.275246-07 | 16.1.3.2 | 51434 16700 | specdb | 3502 | 16385 | spec | update m_inventory set in_ordered= $1, in_location= $2, in_qty= $3, in_act_date= $4, in_acc_code= $5 where in_p_id = $6 | f | 2007-05-20 18:51:04.383896-07 | 2007-05-20 18:49:36.294189-07 | 16.1.2.2 | 55906 16700 | specdb | 3503 | 16385 | spec | COMMIT | f | 2007-05-20 18:51:04.38584-07 | 2007-05-20 18:49:36.29596-07 | 16.1.3.2 | 51435 16700 | specdb | 3504 | 16385 | spec | | f | 2007-05-20 18:51:04.200409-07 | 2007-05-20 18:49:36.31463-07 | 16.1.2.2 | 55907 16700 | specdb | 3505 | 16385 | spec | insert into c_customerinventory(ci_customerid, ci_value, ci_itemid, ci_id, ci_quantity) values ( $1, $2, $3, $4, $5) | f | 2007-05-20 18:51:04.406401-07 | 2007-05-20 18:49:36.32219-07 | 16.1.3.2 | 51436 bench=# select * from pg_stat_activity where query_start < ( now() - interval '10 minutes' ); datid | datname | procpid | usesysid | usename | current_query | waiting | query_start | backend_start | client_addr | client_port -------+---------+---------+----------+---------+----------------------------------------------------+---------+-------------------------------+-------------------------------+-------------+------------- 50071 | bench | 10848 | 10 | pg82 | DELETE FROM accounts WHERE aid = 4436589; | t | 2007-05-20 22:01:46.782499-07 | 2007-05-20 22:00:15.590171-07 | | -1 50071 | bench | 10846 | 10 | pg82 | DELETE FROM accounts WHERE aid = 6643194; | t | 2007-05-20 22:01:46.678033-07 | 2007-05-20 22:00:15.583226-07 | | -1 PG_LOCKS locktype | database | relation | page | tuple | transactionid | classid | objid | objsubid | transaction | pid | mode | granted ---------------+----------+----------+------+-------+---------------+---------+-------+----------+-------------+------+------------------+--------- transactionid | | | | | 43287074 | | | | 43287074 | 3685 | ExclusiveLock | t relation | 16700 | 16749 | | | | | | | 43286964 | 3629 | RowExclusiveLock | t relation | 16700 | 16740 | | | | | | | 43286086 | 3498 | RowShareLock | t relation | 16700 | 16740 | | | | | | | 43286086 | 3498 | RowExclusiveLock | t relation | 16700 | 16703 | | | | | | | 43286967 | 3668 | AccessShareLock | t relation | 16700 | 16708 | | | | | | | 43287068 | 3741 | AccessShareLock | t relation | 16700 | 16728 | | | | | | | 43286405 | 3688 | AccessShareLock | t transactionid | | | | | 43287007 | | | | 43287007 | 3572 | ExclusiveLock | t transactionid | | | | | 43287096 | | | | 43287096 | 3737 | ExclusiveLock | t relation | 16700 | 16706 | | | | | | | 43286961 | 3499 | RowShareLock | t locktype | database | relation | page | tuple | transactionid | classid | objid | objsubid | transaction | pid | mode | granted ---------------+----------+----------+------+-------+---------------+---------+-------+----------+-------------+------+------------------+--------- transactionid | | | | | 43287074 | | | | 43287074 | 3685 | ExclusiveLock | t relation | 16700 | 16749 | | | | | | | 43286964 | 3629 | RowExclusiveLock | t relation | 16700 | 16740 | | | | | | | 43286086 | 3498 | RowShareLock | t relation | 16700 | 16740 | | | | | | | 43286086 | 3498 | RowExclusiveLock | f relation | 16700 | 16728 | | | | | | | 43286405 | 3688 | AccessShareLock | t transactionid | | | | | 43287007 | | | | 43287007 | 3572 | ExclusiveLock | t transactionid | | | | | 43287096 | | | | 43287096 | 3737 | ExclusiveLock | f relation | 16700 | 16706 | | | | | | | 43286961 | 3499 | RowShareLock | t relation | 16700 | 16726 | | | | | | | 43286248 | 3492 | AccessShareLock | t relation | 16700 | 16740 | | | | | | | 43286236 | 3615 | RowExclusiveLock | f relation | 16700 | 16742 | | | | | | | 43286654 | 3618 | AccessShareLock | t relation | 16700 | 16740 | | | | | | | 43286190 | 3624 | RowShareLock | t transactionid | | | | | 43286996 | | | | 43286996 | 3591 | ExclusiveLock | f relation | 16700 | 16732 | | | | | | | 43287040 | 3573 | AccessShareLock | t relation | 16700 | 16730 | | | | | | | 43286608 | 3524 | RowExclusiveLock | t relation | 16700 | 16728 | | | | | | | 43287014 | 3491 | AccessShareLock | t relation | 16700 | 16706 | | | | | | | 43286992 | 3715 | RowShareLock | t relation | 16700 | 16706 | | | | | | | 43286992 | 3715 | RowExclusiveLock | f PG_STAT[IO]_TABLES & PG_STAT[IO]_USER_INDEXES relid | 16385 schemaname | public relname | pgbench_branches seq_scan | 4843 seq_tup_read | 42030 idx_scan | 0 idx_tup_fetch | 0 n_tup_ins | 10 n_tup_upd | 4200 n_tup_del | 0 n_tup_hot_upd | 4200 n_live_tup | 10 n_dead_tup | 0 last_vacuum | 2009-05-18 16:29:17.967476-04 last_autovacuum | 2009-05-18 16:30:08.625902-04 last_analyze | 2009-05-18 16:25:32.490258-04 last_autoanalyze | 2009-05-18 16:30:08.625902-04 -[ RECORD 3 ]----+------------------------------ relid | 16388 schemaname | public relname | pgbench_tellers seq_scan | 4281 seq_tup_read | 420100 idx_scan | 0 idx_tup_fetch | 0 n_tup_ins | 100 n_tup_upd | 4200 n_tup_del | 0 n_tup_hot_upd | 4200 n_live_tup | 100 n_dead_tup | 0 last_vacuum | 2009-05-18 16:29:17.969264-04 last_autovacuum | 2009-05-18 16:30:08.66861-04 relid | schemaname | relname | heap_blks_read | heap_blks_hit | idx_blks_read | idx_blks_hit | 16706 | public | c_customer | 4104 | 1408738 | 1385 | 1967149 | 16701 | public | c_customerinventory | 2744 | 3499081 | 4064 | 1769999 | 16718 | public | c_parts | 0 | 0 | 0 | 0 | 16714 | public | c_site | 0 | 0 | 0 | 0 | 16710 | public | c_supplier | 0 | 0 | 0 | 0 | 16726 | public | m_bom | 2412 | 59717 | 1397 | 159789 | 16740 | public | m_inventory | 25807 | 3554679 | 1447 | 5149310 | 16734 | public | m_largeorder | 1860 | 440494 | 424 | 199792 | 16722 | public | m_parts | 3971 | 616926 | 768 | 1870367 | 16766 | public | s_supplier | 7021 | 43344 | 0 | 0 | relid | indexrelid | schemaname | relname | indexrelname | idx_scan | idx_tup_read | idx_tup_fetch -------+------------+------------+----------------------+---------------------------+----------+--------------+--------------- 16782 | 16787 | public | txn_log_table | txn_log_table_pkey | 45880 | 45880 | 0 16789 | 16791 | public | u_sequences | u_sequences_pkey | 415 | 5782 | 415 16706 | 16708 | public | c_customer | c_customer_pkey | 458615 | 931471 | 458615 16701 | 16705 | public | c_customerinventory | c_custci_idx | 215996 | 5394054 | 0 16701 | 16705 | public | c_customerinventory | c_custname_idx | 204 | 371 | 175 16701 | 16703 | public | c_customerinventory | c_customerinventory_pkey | 87718 | 87718 | 87716 16718 | 16720 | public | c_parts | c_parts_pkey | 0 | 0 | 0 16710 | 16712 | public | c_supplier | c_supplier_pkey | 0 | 0 | 0 SELECT idstat.relname AS table_name, indexrelname AS index_name, idstat.idx_scan AS times_used, pg_size_pretty(pg_relation_size(idstat.relname)) AS table_size, pg_size_pretty(pg_relation_size(indexrelname)) AS index_size, n_tup_upd + n_tup_ins + n_tup_del as num_writes, indexdef AS definition FROM pg_stat_user_indexes AS idstat JOIN pg_indexes ON indexrelname = indexname JOIN pg_stat_user_tables AS tabstat ON idstat.relname = tabstat.relname WHERE idstat.idx_scan < 200 AND indexdef !~* 'unique' ORDER BY idstat.relname, indexrelname; -[ RECORD 1 ]------------------------------------------------------------------------------- table_name | routers index_name | routers_position times_used | 0 table_size | 12 MB index_size | 2 MB num_writes | 5 definition | CREATE INDEX routers_position ON routers USING btree ("position") -[ RECORD 2 ]------------------------------------------------------------------------------- table_name | errors index_name | errors_severity times_used | 79 table_size | 62 GB index_size | 4 GB num_writes | 281576 definition | CREATE INDEX errors_severity ON errors USING btree (severity) -[ RECORD 3 ]------------------------------------------------------------------------------- table_name | hosts index_name | hosts_name times_used | 0 table_size | 7 GB index_size | 640 MB num_writes | 1210 definition | CREATE INDEX hosts_name ON hosts USING btree (name) PG_STAT_BGWRITER pgbench=# select * from pg_stat_bgwriter; checkpoints_timed | checkpoints_req | buffers_checkpoint | buffers_lru | buffers_all | maxwritten_lru | maxwritten_all | buffers_client | buffers_alloc -------------------+-----------------+--------------------+-------------+-------------+----------------+----------------+----------------+--------------- 93 | 100 | 1737158 | 343753 | 61611 | 342 | 12139 | 1520167 | 5464709 PG_STAT_FUNCTIONS funcid | schemaname | funcname | calls | total_time | self_time --------+------------+-------------+-------+------------+----------- 18671 | public | update_time | 1012 | 502679 | 502679 18672 | public | new_user | 1001 | 1499513 | 564532 18673 | public | get_uid | 1001 | 916377 | 916377 PG_STAT_STATEMENTS postgres=# SELECT * FROM pg_stat_statements ORDER BY total_time DESC LIMIT 3; -[ RECORD 1 ]------------------------------------------------------------ userid | 10 dbid | 63781 query | UPDATE branches SET bbalance = bbalance + $1 WHERE bid = $2; calls | 3000 total_time | 20.716706 rows | 3000 -[ RECORD 2 ]------------------------------------------------------------ userid | 10 dbid | 63781 query | UPDATE tellers SET tbalance = tbalance + $1 WHERE tid = $2; calls | 3000 total_time | 17.1107649999999 rows | 3000 -[ RECORD 3 ]------------------------------------------------------------ userid | 10 dbid | 63781 query | UPDATE accounts SET abalance = abalance + $1 WHERE aid = $2; calls | 3000 total_time | 0.645601 rows | 3000 PG_LOG EXAMPLES CONNECTION MANAGEMENT select thehour, noconn, nodisc from ( select extract('hour' from log_time) as thehour, count(*) as noconn from marchlog where message ~* 'connection authorized' group by thehour order by thehour ) as connect NATURAL JOIN ( select extract('hour' from log_time) as thehour, count(*) as nodisc from marchlog where message ~* 'connection authorized' group by thehour order by thehour ) as disconnect order by thehour; thehour | noconn | nodisc ---------+--------+-------- 0 | 2 | 2 1 | 1 | 1 2 | 1 | 1 3 | 1 | 1 4 | 1 | 1 5 | 2 | 2 6 | 1 | 1 7 | 1 | 1 8 | 22 | 22 9 | 117 | 114 10 | 59 | 60 11 | 149 | 133 12 | 50 | 52 13 | 92 | 90 14 | 1440 | 766 15 | 371 | 408 16 | 83 | 89 (17 rows) QUERIES BAD ROWCOUNT ESTIMATE Aggregate (cost=31019.00..31019.00 rows=1 width=0) (actual time=43252.40..43252.40 rows=1 loops=1) -> Seq Scan on requisicaoanalise (cost=0.00..30965.24 rows=21503 width=0) (actual time=8.43..43224.01 rows=9248 loops=1) Filter: ((ra_datacolh >= '2003-06-01'::date) AND (ra_datacolh < '2003-07-01'::date) AND (isactive = 0)) Total runtime: 43252.57 msec Aggregate (cost=10660.84..10660.84 rows=1 width=0) (actual time=172.41..172.41 rows=1 loops=1) -> Index Scan using requisicaoanalise_datacolh on requisicaoanalise (cost=0.00..10654.06 rows=2711 width=0) (actual time=0.13..145.50 rows=9105 loops=1) Index Cond: ((ra_datacolh >= '2003-04-01'::date) AND (ra_datacolh < '2003-05-01'::date)) Filter: (isactive = 0) Total runtime: 172.62 msec Limit (cost=18954.93..18954.95 rows=10 width=136) (actual time=169038.823..169038.851 rows=10 loops=1) -> Sort (cost= 18954.93..18959.75 rows=1929 width=136) (actual time=169038.816..169038.825 rows=10 loops=1) Sort Key: table_one."timestamp" -> Nested Loop Left Join (cost= 0.00..18849.66 rows=1929 width=136) (actual time=862.661..169036.636 rows=179 loops=1) Join Filter: ("outer".id = "inner".id) -> Index Scan using table_one_type_key on table_one (cost=0.00..17.70 rows=3 width=72) (actual time=0.083..864.386 rows=179 loops=1) Index Cond: (src_ip = '192.168.10.128 '::inet) -> Seq Scan on table_two (cost=0.00..4669.92 rows=128592 width=68) (actual time=842.166..933.496 rows=4202 loops=179) BAD COST ESTIMATES Limit (cost=5016.54..5019.04 rows=1000 width=8) (actual time=9560.471..9562.201 rows=1000 loops=1) -> Sort (cost=5016.54..5019.47 rows=1174 width=8) (actual time=9560.469..9561.065 rows=1000 loops=1) Sort Key: calc_rating -> Index Scan using apa_item_common_x1 on apa_item_common aic (cost=0.00..4956.68 rows=1174 width=8) (actual time=19.854..9557.606 rows=1226 loops=1) Index Cond: (apa_user = 704) Filter: (status = 30) Total runtime: 9563.016 ms SLOW INDEX SCAN Limit (cost=400.38..400.39 rows=4 width=3338) (actual time=48377.682..48390.377 rows=500 loops=1) -> Sort (cost=400.38..400.39 rows=4 width=3338) (actual time=48377.663..48381.944 rows=500 loops=1) Sort Key: public.reports.date_processed Sort Method: top-N heapsort Memory: 215kB -> Result (cost=0.00..400.34 rows=4 width=3338) (actual time=775.064..48350.142 rows=1609 loops=1) -> Append (cost=0.00..400.34 rows=4 width=3338) (actual time=775.042..48318.587 rows=1609 loops=1) -> Seq Scan on reports (cost=0.00..10.22 rows=1 width=3338) (actual time=0.009..0.009 rows=0 loops=1) Filter: ((signature IS NOT NULL) AND (date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone) AND ((signature)::text = 'js_PutBlockObject'::text) AND ((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) -> Bitmap Heap Scan on reports_20090223 reports (cost=153.35..157.38 rows=1 width=333) (actual time=775.003..48219.733 rows=1468 loops=1) Recheck Cond: (((signature)::text = 'js_PutBlockObject'::text) AND (date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone) AND ((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) -> BitmapAnd (cost=153.35..153.35 rows=1 width=0) (actual time=737.677..737.677 rows=0 loops=1) -> Bitmap Index Scan on reports_20090223_signature_date_processed_key (cost=0.00..15.53 rows=222 width=0) (actual time=733.183..733.183 rows=1506 loops=1) Index Cond: (((signature)::text = 'js_PutBlockObject'::text) AND (date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone)) -> Bitmap Index Scan on reports_20090223_product_version (cost=0.00..137.58 rows=5318 width=0) (actual time=4.015..4.015 rows=5912 loops=1) Index Cond: (((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) -> Bitmap Heap Scan on reports_20090302 reports (cost=116.41..120.44 rows=1 width=360) (actual time=27.446..30.359 rows=129 loops=1) Recheck Cond: (((signature)::text = 'js_PutBlockObject'::text) AND ((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) Filter: ((date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone)) -> BitmapAnd (cost=116.41..116.41 rows=1 width=0) (actual time=27.378..27.378 rows=0 loops=1) -> Bitmap Index Scan on reports_20090302_signature_key (cost=0.00..19.11 rows=327 width=0) (actual time=25.581..25.581 rows=232 loops=1) Index Cond: ((signature)::text = 'js_PutBlockObject'::text) -> Bitmap Index Scan on reports_20090302_product_version (cost=0.00..97.05 rows=3665 width=0) (actual time=1.699..1.699 rows=2577 loops=1) Index Cond: (((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) -> Bitmap Heap Scan on reports_20090309 reports (cost=108.28..112.31 rows=1 width=349) (actual time=37.607..37.931 rows=12 loops=1) Recheck Cond: (((signature)::text = 'js_PutBlockObject'::text) AND (date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone) AND ((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) -> BitmapAnd (cost=108.28..108.28 rows=1 width=0) (actual time=37.569..37.569 rows=0 loops=1) -> Bitmap Index Scan on reports_20090309_signature_date_processed_key (cost=0.00..9.80 rows=95 width=0) (actual time=37.036..37.036 rows=30 loops=1) Index Cond: (((signature)::text = 'js_PutBlockObject'::text) AND (date_processed >= '2009-02-24 17:53:41'::timestamp without time zone) AND (date_processed <= '2009-03-10 17:53:41'::timestamp without time zone)) -> Bitmap Index Scan on reports_20090309_product_version (cost=0.00..98.23 rows=2984 width=0) (actual time=0.491..0.491 rows=623 loops=1) Index Cond: (((product)::text = 'Widget'::text) AND ((version)::text = '3.1b3pre'::text)) Total runtime: 48394.988 ms (31 rows)