Difference between pages "Run vlc as root" and "I/O sniffing"

From Linuxintro
(Difference between pages)
imported>ThorstenStaerk
(Created page with "vlc always exits when I call it as root. Here is how I change this behavior: gdb /usr/bin/vlc (gdb) info functions All defined functions: Non-debugging symbols: [....")
 
imported>ThorstenStaerk
 
Line 1: Line 1:
[[vlc]] always exits when I call it as root. Here is how I change this behavior:
+
Sometimes, for example when [[troubleshooting]] performance problems you need to understand how your system interacts with its [[hard disk]]s. You may have used [[ioMeter]] to measure your disks' speed and [[ioStat]] and [[sar]] to gather statistics. However sometimes you need to drill down deeper into the analysis, you need to understand what happens on the block level. I call this I/O sniffing. The goal is to observe packets like this:
  [[gdb]] /usr/bin/vlc
+
 
(gdb) info functions
+
initiator XYZ requests block 4711 from device 0815
All defined functions:
+
initiator BLA writes block 1234 to device 9876
+
  block 4711 arrives from device 0815
  Non-debugging symbols:
+
 
 +
It pays to have a good understanding about [[Bandwidth, latency and throughput]], [[block sizes]], caches, buffers, IOPS, async I/O vs sync I/O and buffered I/O vs direct I/O.
 +
 
 +
You can do I/O sniffing using the command blktrace. blktrace will show you every request that goes to the disk.
 +
 
 +
Example:
 +
 
 +
  # blktrace -d /dev/sdg -o - | blkparse -i -  
 
  [...]
 
  [...]
  0x0000000000400f40  geteuid
+
  <abbr title="device (major/minor number)">8,96</abbr>  <abbr title="processor">7</abbr>      <abbr title="sequence number">106</abbr>    <abbr title="timestamp">0.373952974</abbr> <abbr title="PID">11364</abbr> <abbr title="Action/Event">D</abbr>  <abbr title="RWBS (D) field">W</abbr> <abbr title="starting block + blocks">0 + 8</abbr> <abbr title="process">[kworker/7:2]</abbr>
  [...]
+
  8,96  7      107    0.374456639    47 C  W 0 + 8 [0]
Let's break in the function to get the effective user identity:
+
 
(gdb) break geteuid
+
 
  Breakpoint 1 at 0x400f40
+
The RWBS(D) field can be a combination of
Let's start the program to run till the first breakpoint:
+
  R : Read
  (gdb) run
+
W : Write
  Starting program: /usr/bin/vlc
+
  D : Block discard
   
+
  B : Barrier operation
Breakpoint 1, 0x00007ffff71cfc70 in geteuid () from /lib64/libc.so.6
+
  S : Synchronous operations
ok, let's trace the program one command at a time:
+
 
  (gdb) stepi
+
= Exploring IOtrace =
0x00007ffff71cfc75 in geteuid () from /lib64/libc.so.6
+
Using the command
(gdb) stepi
+
  blktrace -d /dev/sdg -o - | blkparse -i - |grep -v kworker
  0x00007ffff71cfc77 in geteuid () from /lib64/libc.so.6
+
shows me all operations but the kernel ones.
(gdb)
+
 
0x0000000000401103 in ?? ()
+
Now the command
(gdb)
+
  dd if=/dev/sdg count=1
0x0000000000401105 in ?? ()
+
results in a lot of operations when I do it for the first time, but only "N" operations when I do it subsequently: The block is already in the files system cache. So let's omit the file system cache:
(gdb)
+
  dd if=/dev/sdg count=1 iflag=direct
Ok, let's look at this program part with a disassembler:
+
Now this results in a read operation every time I do it.  
  [[objdump]] -d -M intel /usr/bin/vlc
+
 
[...]
+
= See also =
  4010f9:      e8 32 0a 00 00          call  401b30 <unsetenv>
+
* [http://linux.die.net/man/8/blktrace blktrace man page]
  4010fe:      e8 3d fe ff ff          call  400f40 <geteuid@plt>
+
* [http://linux.die.net/man/1/blkparse blkparse man page]
  401103:       85 c0                  test  eax,eax
+
* http://unix.stackexchange.com/questions/105610/how-does-blktrace-work
  401105:       0f 84 04 06 00 00      je    40170f <fflush@plt+0x66f>
+
* http://unix.stackexchange.com/questions/105612/why-does-blktrace-only-write-blocks-of-8
  40110b:      be ca 1f 40 00          mov    esi,0x401fca
 
  401110:      bf 06 00 00 00          mov    edi,0x6
 
[...]
 
Wow, it seems as if 4010fe calls geteuid, 401103 prepares a conditional jump and 401105 jumps if equal somewhere. So we call a [[hexeditor]]:
 
okteta /usr/bin/vlc
 
and replace
 
0f 84 04 06 00 00
 
by some instructions to wait:
 
90 90 90 90 90 90
 
When calling vlc now as root, it does not abort :)
 
  
Also, once the following worked:
+
[[Category:geeky]][[Category:analysis]]
sed -<abbr title="in-place edit in the file">i</abbr><abbr title="extended regular expressions">r</abbr> "s/\x0f\x84..../\x90\x90\x90\x90\x90\x90/g" vlc
 

Revision as of 10:00, 18 January 2014

Sometimes, for example when troubleshooting performance problems you need to understand how your system interacts with its hard disks. You may have used ioMeter to measure your disks' speed and ioStat and sar to gather statistics. However sometimes you need to drill down deeper into the analysis, you need to understand what happens on the block level. I call this I/O sniffing. The goal is to observe packets like this:

initiator XYZ requests block 4711 from device 0815
initiator BLA writes block 1234 to device 9876 
block 4711 arrives from device 0815

It pays to have a good understanding about Bandwidth, latency and throughput, block sizes, caches, buffers, IOPS, async I/O vs sync I/O and buffered I/O vs direct I/O.

You can do I/O sniffing using the command blktrace. blktrace will show you every request that goes to the disk.

Example:

# blktrace -d /dev/sdg -o - | blkparse -i - 
[...]
  8,96   7      106     0.373952974 11364  D   W 0 + 8 [kworker/7:2]
  8,96   7      107     0.374456639    47  C   W 0 + 8 [0]


The RWBS(D) field can be a combination of

R : Read
W : Write
D : Block discard
B : Barrier operation
S : Synchronous operations

Exploring IOtrace

Using the command

blktrace -d /dev/sdg -o - | blkparse -i - |grep -v kworker

shows me all operations but the kernel ones.

Now the command

dd if=/dev/sdg count=1

results in a lot of operations when I do it for the first time, but only "N" operations when I do it subsequently: The block is already in the files system cache. So let's omit the file system cache:

dd if=/dev/sdg count=1 iflag=direct

Now this results in a read operation every time I do it.

See also