Using CDEF statement on ARM-based systen causes malloc error.

[human]

If I have a CDEF defined when generating graphs I get the following error:

"ERROR: malloc im->gdes[gdi].data"

elf:~# rrdtool graph tmp.png --imgformat PNG --start '-120s' --end 'now' --width 200 --height 200 -E DEF:temp_inne=temp_fukt.rrd:temp_inne:MAX CDEF:temp100=temp_inne,100,* LINE1:temp_inne#FF0000 LINE1:temp100#0000FF ERROR: malloc im->gdes[gdi].data

elf:~# uname -a Linux elf 2.6.18-4-ixp4xx #1 Sun Apr 22 08:34:11 UTC 2007 armv5tel GNU/Linux

elf:~# rrdtool -v RRDtool 1.2.23 Copyright 1997-2007 by Tobias Oetiker <tobi@…>

Compiled Sep 30 2007 15:11:56

The system is an NSLU2 running debian.

[oetiker]

Hi,

could it be that your application is simply running out memory

the code is not doing anything special on ARM. cheers tobi

[human]

It's possible that rrdtool is running out of RAM, but not likely.

If I create a very small db (~85kb) I get the same error. Everything else works perfect on the system.

[human]

And today everything works flawlessly. Nothing is changed on the box thou, not even rebooted.

[human]

RRD database file

[human]

I have the same problem on a NSLU2 under Debian. My version of rrdtool is 1.2.15, this is what apt-get delivered.

The problem is reproducible for any value below "end-48800" in start time of the following graph. No problem without the CDEF.

rrdtool graph temperatur4.png \ --end now --start end-48799 -a PNG -t "Temperaturverlauf 4h" \ --vertical-label "Temperatur °C" \ --width 800 --height 400 \ DEF:aussen=temperatur.rrd:AF:AVERAGE \ CDEF:aus=aussen,10,/ \ LINE1:aus#ff0000:"Aussen"

[oetiker]

Without the CDEF you use less memory ... have a look at ulimit -a

[human]

Replying to oetiker:

Without the CDEF you use less memory ... have a look at ulimit -a

Yes, but why does it work for higher values of "start"? I can produce a graph that contains the last 12 hours but not one that only contains the last 4 hours! Is it not correct to assume that creating a graph for a longer time span needs more resources than one for a shorter time span?

ulimit -a gives me "unlimited" for almost everything.

[oetiker]

Sorry I don't really know about specifics of memory management on ARM based CPUs running Linux ... As the error message indicates, malloc does not succeed here ...

You may want to raise the topic again on the mailing list. Maybe there are users who have experiance with this platform.

[human]

Creating graphs are extremely slow on my NSLU2

I call graph.sh 30

Here is the graph.sh script :

let nbsecond=3600*$1 rrdtool graph /var/www/temp-$1.gif -s -$nbsecond --title=Temperature \

DEF:tempe=/tmp/frogd.rrd:temp:AVERAGE -v temperature\ LINE2:tempe#00a000:"Deg Celsi" \ GPRINT:tempe:LAST:"Actuel\: %.1lf -" \ GPRINT:tempe:MIN:"Mini\: %.1lf -" \ GPRINT:tempe:MAX:"Maxi\: %.1lf -"\ GPRINT:tempe:AVERAGE:"Moyen\: %.1lf"

[human]

Replying to oetiker:

Sorry I don't really know about specifics of memory management on ARM based CPUs running Linux ... As the error message indicates, malloc does not succeed here ...

You may want to raise the topic again on the mailing list. Maybe there are users who have experiance with this platform.

I too have experienced this problem. It appears that when using CDEF functions on this architecture with compilation optimization enabled, the following code from the data_calc function in rrd_graph.c does not evaluate correctly and an incorrect (very large) number is passed into malloc.

if((im->gdes[gdi].data = malloc((

(im->gdes[gdi].end-im->gdes[gdi].start)

/ im->gdes[gdi].step)

• sizeof(double)))==NULL){

rrd_set_error("malloc im->gdes[gdi].data"); rpnstack_free(&rpnstack); return -1;

Unless you have tons of swap space on the NSLU2 the malloc will fail. I removed the -O2 from CFLAGS and compiled on the NSLU2. This solves the problem. The apt-get and ipkg packages should be updated to reflect this solution.

Robb

[oetiker]

Have you tried making the expression less complex in the C source, like assigning intermediate results to variables. Maybe the optimizer will NOT remove it all again and then it will work ... maybe ...

[human]

Replying to oetiker:

Have you tried making the expression less complex in the C source, like assigning intermediate results to variables. Maybe the optimizer will NOT remove it all again and then it will work ... maybe ...

Yes... I tried calculating intermediate results and it works with the -O2.

rpnend=im->gdes[gdi].end; rpnstart=im->gdes[gdi].start; rpnstep=im->gdes[gdi].step; doublesize=sizeof(double); rpndiff=rpnend-rpnstart; rpnpoints=rpndiff/rpnstep; rpnsize=rpnpoints*doublesize;

if((im->gdes[gdi].data = malloc(rpnsize))==NULL){ /* if((im->gdes[gdi].data = malloc((

• (im->gdes[gdi].end-im->gdes[gdi].start)
• / im->gdes[gdi].step)
• * sizeof(double)))==NULL){ */

rrd_set_error("malloc im->gdes[gdi].data"); rpnstack_free(&rpnstack); return -1;

}

[oetiker]

Is there some ifdef we can use to identify your environment ? Then we could just integrate this in the normal version of rrdtool .... can you do a diff -u between your patched and the original source and send it to me?

cheers tobi

[human]

I can confirm that this bug still occurs in rrdtool 1.3.4. It even occurs with the changes suggested in comment 7, so currently this is a showstopper for rrdtool on arm linux.

rrdtool -v
RRDtool 1.3.4  Copyright 1997-2008 by Tobias Oetiker <tobi@oetiker.ch>
               Compiled Nov 19 2008 04:47:13

uname -a
Linux slug 2.6.26-1-ixp4xx #1 Thu Aug 21 09:59:59 UTC 2008 armv5tel GNU/Linux

gcc -v
Using built-in specs.
Target: arm-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.1-9' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --disable-libssp --enable-checking=release --build=arm-linux-gnu --host=arm-linux-gnu --target=arm-linux-gnu
Thread model: posix
gcc version 4.3.1 (Debian 4.3.1-9)

Maybe disabling optimization alltogether on arm or only using -O1 would work. Regarding the conditionals, try ifdef __arm__ or #machine(arm)

Though I really don't know linux well enough, could this maybe even be a bug in gcc itself?

[oetiker]

You can set the priority as high as you want, I do not have an arm based machine to test this on and all indications are that the problems are related to the way floating point maths is handles on ARM. If you want this resolved, you have to help ... also carrying this to the mailing list will probably help since there may be other ARM users who can tell you how they approach the issue.

I am resetting the prio to minor since it only affects one platform and not even thre the reports are consistent.

cheers tobi

[oetiker]

Another note, have you tried what is suggested in human ?

[human]

Replying to oetiker:

Another note, have you tried what is suggested in human ?

Well, I mistyped my initial post:

I can confirm that this bug still occurs in rrdtool 1.3.4. It even occurs with the changes suggested in comment 7, so currently this is a showstopper for rrdtool on arm linux.

What I actually meant were human's changes. And since even the workaround does not help, this does indeed 'block' rrdtool on arm machines.

You can set the priority as high as you want...

Well, from my point of view an unusable program warants a 'blocker' priority, but ultimately this is your playground, and if you say it's only minor (since it only affects one architecture), that's fine with me.

Also floating point performance on my ARM is abysmal (15-20 minutes to build one graph), so even if the bug was fixed the gained usability is quite small. And with the ARMEL architecture almost ready for use (which offers increased floating point performance upwards of a factor of 20), I'm content to let this bug stay unresolved.