Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#6 closed defect (fixed)

du2rrd displays strange directory sizes

Reported by: human Owned by: oetiker
Priority: major Milestone:
Component: component1 Version:
Keywords: Cc:

Description

Here is my setup:

command: /usr/bin/du /usr/local/www /usr/home /hosting | /path/to/bin/du2rrd --rrd_root=/path/to/rrd/du2rrd "Sites=/usr/local/www/{Sites}" "Users=/usr/home/{Users}" "Hostings=/hosting/{Hostings}"

system: FreeBSD 6.x, RRDtool 1.3.7, GNU bash version 3.2.25

Here are some size informations:

/hosting real used size is 61G (df -h), du2rrd html interface reads a "Size MB" of 124,157, and the graph reads "Total disk usage Hostings: 83 GByte" /usr/home real size is 21G (du -h /usr/home), but du2rrd html interface reads a "Size MB" of 43,876, and the graph reads "Total disk usage Users: 29 GByte" ...

Is there something wrong in my setup? It looks like the size in the "Size MB" column is twice the real value, and it's also twice the yesterday value (2 day of running, so far). I'm quite sure the "Size MB" will increase again by the same amount tomorrow. Any help would be greatly appreciated.

regards patpro

Attachments (2)

du2rrd_firefox_vs_safari.jpg (190.4 KB) - added by human 11 years ago.
display of the same data in firefox 3.x and safari 4.x
size_differences.jpg (52.6 KB) - added by human 11 years ago.

Download all attachments as: .zip

Change History (13)

Changed 11 years ago by human

display of the same data in firefox 3.x and safari 4.x

comment:1 Changed 11 years ago by human

I've added a snapshot of Firefox and Safari both displaying the same data. You'll see a strange calculation problem in the chart. The graph part is the same on both browser, probably because it's computed server-side.

comment:2 Changed 11 years ago by oetiker

  • Owner changed from somebody to oetiker

hi patpro,

looking at the code, it is a complete mystery how we can end up with doubles ... the factor 2 does not even occure in the code ... my fear is that this is a problem burried in the qooxdoo library somewhere, todo with the rendering of the numbers ... :-( I will check once I get safari 4.0 installed here ...

cheers tobi

comment:3 Changed 11 years ago by human

Hi oetiker,

I think you can forget about the double thing between FF and Safari, I guess… Here is what I've done yesterday:

  • uncheck "javascript" in security preferences of Safari
  • force reload my du2rrd web page
  • check "javascript" in security preferences of Safari
  • reload my du2rrd web page

-> boom. The figures in Safari now match the figures in Firefox. So strange.

Today I have another problem. There has been a new cron cycle this morning, updating data. Now, both firefox and Safari display a size for Hostings of 124,157 MB (chart), 112 GB (graph), but real size (du -hc) is 61G. The exact size of /hosting is 63568472 K, so 124,157 MB is exactly twice the correct size.

Yesterday I thought it might comes from the BLOCKSIZE env. not being set for cron. So I've added BLOCKSIZE=K in my script. This morning, the size was still twice what it should be.

Here is the relevant part of rrdtool dump Hostings.rrd | grep -v NaN

		<database>
			<!-- 2009-06-12 02:00:00 CEST / 1244764800 --> <row><v> 8.9017706719e+10 </v></row>
			<!-- 2009-06-13 02:00:00 CEST / 1244851200 --> <row><v> 1.3018822656e+11 </v></row>
			<!-- 2009-06-14 02:00:00 CEST / 1244937600 --> <row><v> 1.3018822656e+11 </v></row>
			<!-- 2009-06-15 02:00:00 CEST / 1245024000 --> <row><v> 1.3018822656e+11 </v></row>
		</database>

For tomorrow I've added the -k flag to the du command, I hope it will solve the problem.

comment:4 Changed 11 years ago by oetiker

  • Resolution set to fixed
  • Status changed from new to closed

it seems your du returns 512byte block unless you call it with du -k ... try this ... du2rrd assumes to get du -k output

comment:5 follow-up: Changed 11 years ago by human

You must be right, but:

  • BLOCKSIZE=K is set in my .bashrc, and du behaves like if it's called with -k flag. I wonder why cron would not use a BLOCKSIZE=K set in a script…

Am I missing something here?

comment:6 in reply to: ↑ 5 Changed 11 years ago by human

I'm replying to myself

Replying to human:

the graph itself is ok, 124000 MB gives around 121 GB. So, the only piece of mystery remains in the footer of the graph, showing a strange "out of nowhere" value.

comment:7 Changed 11 years ago by oetiker

this is Giga byte vs Mega byte ...

comment:8 Changed 11 years ago by human

Replying to oetiker:

this is Giga byte vs Mega byte ...

You're right, for the chart vs. graph. But what about the GB value in the footer of the graph? (see second attached jpeg)

Changed 11 years ago by human

comment:9 Changed 11 years ago by oetiker

hmmm I guess this is an rrdtool issue ...

cheers tobi

comment:10 Changed 11 years ago by human

Hi tobi,

Ok, I've finally found out what's going on here. This morning, I've done a dump of one of the rrd files, to check raw values. It appears that the "Total Diskusage" displayed at the bottom of the graph is an average of daily values.

This morning I have these values for my /hosting directory: (89017706719+130188226560+130188226560+130188226560+82058336514)/(5*1024*1024*1024)=104.6 GB The "Total Diskusage" value on my graph is 105 GB, so I guess it's 104.6 rounded to the nearest integer.

thanks for your help.

comment:11 Changed 11 years ago by oetiker

yes ... it is ... this is caused by using the --base option when drawing the graph ... I guess now that --base 1024 does not get properly considred for the actual drawing operations in rrdtool ...

cheers tobi

Note: See TracTickets for help on using tickets.