Memory leak in rrd_graph() function.

[human]

I'm writing a program that continuously draws graphs every 10 seconds, but memory usage keeps growing until the program crashes. I tracked down the culprit with valgrind and it turned out to be the rrd_graph() function I was calling.

I understand the problem might be with another library rrdtool uses, but the memory leak does not exist when compiling my program with the 1.2.27 library. Here are the valgrind results for both:

Using the 1.2.27 libraries:

==5825== 
==5825== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 111 from 3)
==5825== malloc/free: in use at exit: 61,418 bytes in 18 blocks.
==5825== malloc/free: 59,678 allocs, 59,660 frees, 10,746,740 bytes allocated.
==5825== For counts of detected errors, rerun with: -v
==5825== searching for pointers to 18 not-freed blocks.
==5825== checked 930,232 bytes.
==5825== 
==5825== LEAK SUMMARY:
==5825==    definitely lost: 0 bytes in 0 blocks.
==5825==      possibly lost: 0 bytes in 0 blocks.
==5825==    still reachable: 61,418 bytes in 18 blocks.
==5825==         suppressed: 0 bytes in 0 blocks.
==5825== Reachable blocks (those to which a pointer was found) are not shown.
==5825== To see them, rerun with: --leak-check=full --show-reachable=yes

Using the 1.3.6 libraries:

==5393== 156 (36 direct, 120 indirect) bytes in 1 blocks are definitely lost in loss record 17 of 78
==5393==    at 0x4025D2E: malloc (vg_replace_malloc.c:207)
==5393==    by 0x448FC30: (within /lib/tls/i686/cmov/libc-2.8.90.so)
==5393==    by 0x4490565: __nss_database_lookup (in /lib/tls/i686/cmov/libc-2.8.90.so)
==5393==    by 0x6024F5B: ???
==5393==    by 0x6025CBE: ???
==5393==    by 0x4436B61: getpwnam_r (in /lib/tls/i686/cmov/libc-2.8.90.so)
==5393==    by 0x4938745: (within /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x493A23C: g_get_home_dir (in /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x47F6050: pango_config_key_get (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47DE3C5: pango_find_map (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47E1F5F: (within /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47E265E: pango_itemize_with_base_dir (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393== 
==5393== 
==5393== 148 (128 direct, 20 indirect) bytes in 1 blocks are definitely lost in loss record 28 of 78
==5393==    at 0x4025D2E: malloc (vg_replace_malloc.c:207)
==5393==    by 0x59909F6: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x59913E7: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x59914FE: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x5996FDE: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x5B86EC3: (within /usr/lib/libexpat.so.1.5.2)
==5393==    by 0x5B87C10: (within /usr/lib/libexpat.so.1.5.2)
==5393==    by 0x5B895EE: (within /usr/lib/libexpat.so.1.5.2)
==5393==    by 0x5B89CE6: (within /usr/lib/libexpat.so.1.5.2)
==5393==    by 0x5B8068B: XML_ParseBuffer (in /usr/lib/libexpat.so.1.5.2)
==5393==    by 0x5994F1D: FcConfigParseAndLoad (in /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x5995265: FcConfigParseAndLoad (in /usr/lib/libfontconfig.so.1.3.0)
==5393== 
==5393== 
==5393== 854 bytes in 23 blocks are possibly lost in loss record 42 of 78
==5393==    at 0x4023DE2: calloc (vg_replace_malloc.c:397)
==5393==    by 0x490BCEB: g_malloc0 (in /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x48A6423: (within /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48A6495: (within /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48AA171: g_type_register_fundamental (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48B4CDE: (within /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48A8BDC: g_type_init_with_debug_flags (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48A8C61: g_type_init (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x47C6466: pango_cairo_font_map_new (in /usr/lib/libpangocairo-1.0.so.0.2202.0)
==5393==    by 0x47C64BC: pango_cairo_font_map_get_default (in /usr/lib/libpangocairo-1.0.so.0.2202.0)
==5393==    by 0x40E4A7E: rrd_graph_init (rrd_graph.c:3767)
==5393==    by 0x40EE920: rrd_graph_v (rrd_graph.c:3597)
==5393== 
==5393== 
==5393== 79,776 bytes in 76 blocks are possibly lost in loss record 73 of 78
==5393==    at 0x4023C4A: memalign (vg_replace_malloc.c:460)
==5393==    by 0x4023CFE: posix_memalign (vg_replace_malloc.c:569)
==5393==    by 0x4921332: (within /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x4922B12: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x4922C14: g_slice_alloc0 (in /usr/lib/libglib-2.0.so.0.1800.2)
==5393==    by 0x48ADE0A: g_type_create_instance (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x48923D4: (within /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x4892C05: g_object_newv (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x4893801: g_object_new_valist (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x489394D: g_object_new (in /usr/lib/libgobject-2.0.so.0.1800.2)
==5393==    by 0x47C8B02: (within /usr/lib/libpangocairo-1.0.so.0.2202.0)
==5393==    by 0x47C9171: (within /usr/lib/libpangocairo-1.0.so.0.2202.0)
==5393== 
==5393== 
==5393== 227,188 (80,896 direct, 146,292 indirect) bytes in 219 blocks are definitely lost in loss record 76 of 78
==5393==    at 0x4025E4C: realloc (vg_replace_malloc.c:429)
==5393==    by 0x5990971: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x59913E7: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x5991A2B: (within /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x598DAD8: FcFontRenderPrepare (in /usr/lib/libfontconfig.so.1.3.0)
==5393==    by 0x58E2EBA: (within /usr/lib/libpangoft2-1.0.so.0.2202.0)
==5393==    by 0x47E4539: pango_font_map_load_fontset (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47E217B: (within /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47E265E: pango_itemize_with_base_dir (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47EB288: (within /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47EC79A: (within /usr/lib/libpango-1.0.so.0.2202.0)
==5393==    by 0x47EE3ED: pango_layout_get_pixel_extents (in /usr/lib/libpango-1.0.so.0.2202.0)
==5393== 
==5393== LEAK SUMMARY:
==5393==    definitely lost: 81,060 bytes in 221 blocks.
==5393==    indirectly lost: 146,432 bytes in 7,238 blocks.
==5393==      possibly lost: 80,630 bytes in 99 blocks.
==5393==    still reachable: 1,262,587 bytes in 9,149 blocks.
==5393==         suppressed: 0 bytes in 0 blocks.
==5393== Reachable blocks (those to which a pointer was found) are not shown.
==5393== To see them, rerun with: --leak-check=full --show-reachable=yes

[human]

Whoops! My function wasn't drawing anything when I used the 1.2.27 library, so ignore whatever I said about it not having any leaks. I'll try to figure out why it isn't drawing.

[human]

Just got the code fixed (It didn't like TEXTALIGN in 1.2.*) I tried the 1.2.30 and indeed, it wasn't leaking.

==30323== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 105 from 3)
==30323== malloc/free: in use at exit: 61,418 bytes in 18 blocks.
==30323== malloc/free: 336,459 allocs, 336,441 frees, 106,434,896 bytes allocated.
==30323== For counts of detected errors, rerun with: -v
==30323== searching for pointers to 18 not-freed blocks.
==30323== checked 721,368 bytes.
==30323== 
==30323== LEAK SUMMARY:
==30323==    definitely lost: 0 bytes in 0 blocks.
==30323==      possibly lost: 0 bytes in 0 blocks.
==30323==    still reachable: 61,418 bytes in 18 blocks.
==30323==         suppressed: 0 bytes in 0 blocks.
==30323== Reachable blocks (those to which a pointer was found) are not shown.
==30323== To see them, rerun with: --leak-check=full --show-reachable=yes

I'm guessing the issue stems from pango or libfontconfig?

[oetiker]

The way rrd_graph is implemented, some memory will not get freed at the end of the run, this is due to pangos internal fontcacheing which brings major performance gains when draing multiple graphs in one run. This amount of memory does not grow though.

Can you please provide a sample perl script showing the growth? I have already spent a lot of quality time with valgrind and rrdtool ... :-)

cheers tobi

[human]

I can confirm this problem in version 1.3.9 and am attaching a simple C program with a RRD file that simply runs 10000 iterations of rrd_graph and it leaks around 1mb per 1000 iterations. I tried it with pango versions 1.24.5 and 1.26.0 with the same results.

[human]

Simple C program that shows the memory leak

[human]

I've modified the above debug test case to get a bit more information. The problem exists in both 1.3.9 and 1.4.4. 1.2.30 is working fine.

I've tested with these libraries:

• pango-1.28.0-2.7
• cairo-1.8.10-3.1

The output for different versions of RRDtool is:

~/tmp/rrddebug> ldd rrdtest-1.2 | grep rrd
        librrd.so.2 => /home/td/tmp/rrdtool-1.2.30/src/.libs/librrd.so.2 (0x00007f3387405000)
~/tmp/rrddebug> ./rrdtest-1.2 
   0: memtotal: 4440; memres: 130; memshr: 93; memstack: 81
1000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
2000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
3000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
4000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
5000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
6000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
7000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
8000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387
9000: memtotal: 4817; memres: 715; memshr: 361; memstack: 387


~/tmp/rrddebug> ldd rrdtest-1.3 | grep rrd
        librrd.so.4 => /home/td/tmp/rrdtool-1.3.9/src/.libs/librrd.so.4 (0x00007ff9d3a1c000)
~/tmp/rrddebug> ./rrdtest-1.3 
   0: memtotal: 17286; memres: 455; memshr: 340; memstack: 96
1000: memtotal: 19677; memres: 2725; memshr: 1827; memstack: 856
2000: memtotal: 20147; memres: 3204; memshr: 1827; memstack: 1326
3000: memtotal: 20626; memres: 3683; memshr: 1827; memstack: 1805
4000: memtotal: 21099; memres: 4161; memshr: 1827; memstack: 2278
5000: memtotal: 21581; memres: 4640; memshr: 1827; memstack: 2760
6000: memtotal: 22067; memres: 5118; memshr: 1827; memstack: 3246
7000: memtotal: 22546; memres: 5597; memshr: 1827; memstack: 3725
8000: memtotal: 23018; memres: 6076; memshr: 1827; memstack: 4197
9000: memtotal: 23491; memres: 6554; memshr: 1827; memstack: 4670


~/tmp/rrddebug> ldd rrdtest-1.4 | grep rrd
        librrd.so.4 => /home/td/tmp/rrdtool-1.4.4/src/.libs/librrd.so.4 (0x00007f8af86fc000)
~/tmp/rrddebug> ./rrdtest-1.4 
   0: memtotal: 17293; memres: 456; memshr: 341; memstack: 97
1000: memtotal: 19452; memres: 2576; memshr: 1823; memstack: 689
2000: memtotal: 19930; memres: 3054; memshr: 1823; memstack: 1167
3000: memtotal: 20405; memres: 3529; memshr: 1823; memstack: 1642
4000: memtotal: 20884; memres: 4008; memshr: 1823; memstack: 2121
5000: memtotal: 21360; memres: 4484; memshr: 1823; memstack: 2597
6000: memtotal: 21840; memres: 4964; memshr: 1823; memstack: 3077
7000: memtotal: 22317; memres: 5441; memshr: 1823; memstack: 3554
8000: memtotal: 22959; memres: 6061; memshr: 1823; memstack: 4196
9000: memtotal: 23271; memres: 6395; memshr: 1823; memstack: 4508

[oetiker]

I think r2117 should fix this ... check it out!

[human]

Looks good. The test case now says:

~/tmp/rrd/rrddebug-with-stats> ldd rrdtest-1.4-r2117 | grep rrd
        librrd.so.4 => /home/td/tmp/rrd/rrdtool-1.4.4-r2117/src/.libs/librrd.so.4 (0x00007f7062763000)
~/tmp/rrd/rrddebug-with-stats> ./rrdtest-1.4-r2117 
   0: memtotal: 17293; memres: 456; memshr: 341; memstack: 97
1000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
2000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
3000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
4000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
5000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
6000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
7000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
8000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214
9000: memtotal: 18977; memres: 2102; memshr: 1824; memstack: 214