1

I have a problem with the monitoring of an I/O stream in a particular directory: the amount of read and written bytes during a specified period.

I have been trying to use iostat, inotifywait and inotifywatch, but unfortunately they measure only activity upon directory (open, close, modify, etc), they do not give info about the bytes read and written.

Josip Rodin
  • 1,138
user1856533
  • 111
  • In Unix/Linux there is no "bytes written to the directory's files", remember that the directory is just a list of links to files, a file doesn't belong to a directory. What are yo trying to do? Perhaps there are better ways of getting there. – vonbrand Jan 22 '13 at 09:50

2 Answers2

4

An option is to use the audit system. Something like:

sudo auditctl -a exit,always -F dir=/the/directory -p rw

And then analyse the audit.log for the I/O system calls.

Stéphane Chazelas
  • 544,893
  • As I know, auditctl will create a log in the /var/log/audit folder which have to parse later by using some tools. It is not so convenient way especially in the monitor that i'm going to build. Maybe you can recommend some other not so complex approach? – user1856533 Jan 24 '13 at 07:54
  • You can read the log on-the-fly instead of parsing it afterwards. Just wait for changes (using the inotify system) and read/parse the newly appended lines. – scai Jan 28 '13 at 06:42
3

As far as I know there is no standard tool capable of doing this. Yet I can think of several ways of achieving this, each with their drawbacks.

If you know which process will access the file/directory and you are sure this process does only few or no other I/O you can parse /proc/PID/io and determine the amount of read/written bytes by looking at rchar/wchar (any I/O) or read_bytes/write_bytes (disk I/O) or even syscr/syscw (number of read()/write()-like system calls).

If you don't know which process will access the file/directory (but you are sure these processes do only few or no other I/O) you have to watch the location using the inotify feature, determine the PIDs and then do the mechanism described in the previous paragraph.

If the process determined does other I/O it gets a little bit more complicated. You have to strace it and sum up the offsets of all read/write calls of all file descriptors pointing to your file/directory.

All of these solutions aren't accurate to a single byte but can give you a good estimate.

scai
  • 10,793
  • Thank you very much for really valuable and complete answer. I have been thinking already to use a such way but don't venturing to do it because it is quite complexly. By the way, how I can get all processes IPs that have activity on defined folder? You mention that I can use inotify. I will be very appreciate if you say little bit more? – user1856533 Jan 23 '13 at 07:13
  • Do you mean PID instead of IP? It seems like the inotify interface doesn't report the PID to a specific event so you still have to run lsof on the file belonging to a reported event.

    If you have full access to the system better use auditd as explained by Stephane. If not your only choice is a workaround like the one I described.

     – scai Jan 23 '13 at 07:30
  • Ok, thanks. The first approach is clear - yep, if some processes has an activity on some folder/file it is no problem how to measure. But your second recommendation is can be problematic because if I'm even detected all processes that has an activity on interesting folder/file I cannot be sure that they are reading and writing only there. – user1856533 Jan 24 '13 at 07:35
  • That's true. If you can't be sure you have to look at all their read and write operations. This can be done by using strace as explained in my answer. Additionally you need something like lsof in order to see which file descriptor points to which file. Unfortunately this isn't a very elegant solution and requires some fiddling. – scai Jan 24 '13 at 08:15