du2rrd displays strange directory sizes

[human]

Here is my setup:

command: /usr/bin/du /usr/local/www /usr/home /hosting | /path/to/bin/du2rrd --rrd_root=/path/to/rrd/du2rrd "Sites=/usr/local/www/{Sites}" "Users=/usr/home/{Users}" "Hostings=/hosting/{Hostings}"

system: FreeBSD 6.x, RRDtool 1.3.7, GNU bash version 3.2.25

Here are some size informations:

/hosting real used size is 61G (df -h), du2rrd html interface reads a "Size MB" of 124,157, and the graph reads "Total disk usage Hostings: 83 GByte" /usr/home real size is 21G (du -h /usr/home), but du2rrd html interface reads a "Size MB" of 43,876, and the graph reads "Total disk usage Users: 29 GByte" ...

Is there something wrong in my setup? It looks like the size in the "Size MB" column is twice the real value, and it's also twice the yesterday value (2 day of running, so far). I'm quite sure the "Size MB" will increase again by the same amount tomorrow. Any help would be greatly appreciated.

regards patpro

[human]

display of the same data in firefox 3.x and safari 4.x

[human]

I've added a snapshot of Firefox and Safari both displaying the same data. You'll see a strange calculation problem in the chart. The graph part is the same on both browser, probably because it's computed server-side.

[oetiker]

hi patpro,

looking at the code, it is a complete mystery how we can end up with doubles ... the factor 2 does not even occure in the code ... my fear is that this is a problem burried in the qooxdoo library somewhere, todo with the rendering of the numbers ... :-( I will check once I get safari 4.0 installed here ...

cheers tobi

[human]

Hi oetiker,

I think you can forget about the double thing between FF and Safari, I guess… Here is what I've done yesterday:

• uncheck "javascript" in security preferences of Safari
• force reload my du2rrd web page
• check "javascript" in security preferences of Safari
• reload my du2rrd web page

-> boom. The figures in Safari now match the figures in Firefox. So strange.

Today I have another problem. There has been a new cron cycle this morning, updating data. Now, both firefox and Safari display a size for Hostings of 124,157 MB (chart), 112 GB (graph), but real size (du -hc) is 61G. The exact size of /hosting is 63568472 K, so 124,157 MB is exactly twice the correct size.

Yesterday I thought it might comes from the BLOCKSIZE env. not being set for cron. So I've added BLOCKSIZE=K in my script. This morning, the size was still twice what it should be.

Here is the relevant part of rrdtool dump Hostings.rrd | grep -v NaN

		<database>
			<!-- 2009-06-12 02:00:00 CEST / 1244764800 --> <row><v> 8.9017706719e+10 </v></row>
			<!-- 2009-06-13 02:00:00 CEST / 1244851200 --> <row><v> 1.3018822656e+11 </v></row>
			<!-- 2009-06-14 02:00:00 CEST / 1244937600 --> <row><v> 1.3018822656e+11 </v></row>
			<!-- 2009-06-15 02:00:00 CEST / 1245024000 --> <row><v> 1.3018822656e+11 </v></row>
		</database>

For tomorrow I've added the -k flag to the du command, I hope it will solve the problem.

[oetiker]

it seems your du returns 512byte block unless you call it with du -k ... try this ... du2rrd assumes to get du -k output

[human]

You must be right, but:

• BLOCKSIZE=K is set in my .bashrc, and du behaves like if it's called with -k flag. I wonder why cron would not use a BLOCKSIZE=K set in a script…

• it does not explain the difference between the real directory size (from du) and the size displayed at the bottom of the graph. From the snapshot at http://oss.oetiker.ch/optools/attachment/ticket/6/du2rrd_firefox_vs_safari.jpg your can see that the graph shows a 108 GByte value at the bottom, and a less than 122GByte value in the graph itself, where it should be 124.1 GByte

Am I missing something here?

[human]

I'm replying to myself

Replying to human:

• it does not explain the difference between the real directory size (from du) and the size displayed at the bottom of the graph. From the snapshot at http://oss.oetiker.ch/optools/attachment/ticket/6/du2rrd_firefox_vs_safari.jpg your can see that the graph shows a 108 GByte value at the bottom, and a less than 122GByte value in the graph itself, where it should be 124.1 GByte

the graph itself is ok, 124000 MB gives around 121 GB. So, the only piece of mystery remains in the footer of the graph, showing a strange "out of nowhere" value.

[oetiker]

this is Giga byte vs Mega byte ...

[human]

Replying to oetiker:

this is Giga byte vs Mega byte ...

You're right, for the chart vs. graph. But what about the GB value in the footer of the graph? (see second attached jpeg)

[oetiker]

hmmm I guess this is an rrdtool issue ...

cheers tobi

[human]

Hi tobi,

Ok, I've finally found out what's going on here. This morning, I've done a dump of one of the rrd files, to check raw values. It appears that the "Total Diskusage" displayed at the bottom of the graph is an average of daily values.

This morning I have these values for my /hosting directory: (89017706719+130188226560+130188226560+130188226560+82058336514)/(5*1024*1024*1024)=104.6 GB The "Total Diskusage" value on my graph is 105 GB, so I guess it's 104.6 rounded to the nearest integer.

thanks for your help.

[oetiker]

yes ... it is ... this is caused by using the --base option when drawing the graph ... I guess now that --base 1024 does not get properly considred for the actual drawing operations in rrdtool ...

cheers tobi