On the way to 1.1...

After making the generic output mulithreading and a stress test with repeated loading of oss/alsa on an active output I started thinking about memory leaks again.
The vmsize of dermixd with one active mpg123 ranges from 12MB on my laptop to 45MB on a gentoo nptl system... also this vmsize increases by 2MB on every alsa/oss change, while the resident size "only" increases by 4K (are these really leaking ?).

Using valgrind makes not much fun because it complains about many (repeated) errors in linuxthreads or alsa library that I'm not responsible for (and know nothing about).

OK, I ran alternated ouload oss/alsa in a loop and got up to 165MB vmsize without affecting the used RAM... so, I'll take this "virtual" in VM literally. Ignore it. The resident size is more interesting. It shows an increase of 4K at every change (8K per cycle). This may be a leak...

Switched to my AV machine with Gentoo. Again, going up to 731M vmsize with only 97M RAW really used. RES increasing, too; while here the amount is between 16 and 18K per cycle. Can this be a fault on my side?


I innocently tried other actions: remout, getstat, addin... on every action, there is lost memory! Without outputs. I thought I had this!

According to valgrind, the definitely lost bytes stem from port_watcher:

==13267== 2702 (328 direct, 2374 indirect) bytes in 2 blocks are definitely lost in loss record 19 of 24
==13267==    at 0x1B901701: operator new(unsigned) (in /usr/lib/valgrind/vgpreload_memcheck.so)
==13267==    by 0x804DBCC: port_watcher(void*) (in /home/thomas/dermixd/dermixd_debug)
==13267==    by 0x1B91626D: start_thread (in /lib/libpthread-2.3.5.so)
==13267==    by 0x1BB8B69D: clone (in /lib/libc-2.3.5.so)
==13267== 

And still there is all this babbling about invalid reads and uninitialized ioctl stuff in libpthread and libasound...

Another chapter: 2006-03-26

valgrind:

start & shutdown (with one fullstat)

 LEAK SUMMARY:
==10897==    definitely lost: 8176 bytes in 5 blocks.
==10897==      possibly lost: 9376 bytes in 293 blocks.
==10897==    still reachable: 22238 bytes in 931 blocks.
==10897==         suppressed: 0 bytes in 0 blocks.

start & 1000+2 fullstats & shutdown

==10953== LEAK SUMMARY:
==10953==    definitely lost: 8176 bytes in 5 blocks.
==10953==      possibly lost: 9376 bytes in 293 blocks.
==10953==    still reachable: 22238 bytes in 931 blocks.
==10953==         suppressed: 0 bytes in 0 blocks.

So, the fullstat alone doesn't seem to leak... let's try with playing something:

start & play & shutdown

==11004== LEAK SUMMARY:
==11004==    definitely lost: 8440 bytes in 9 blocks.
==11004==      possibly lost: 9376 bytes in 293 blocks.
==11004==    still reachable: 29974 bytes in 933 blocks.
==11004==         suppressed: 0 bytes in 0 blocks.


Ok, the usual 4 allocated pointers that make no sense, the pthread_create block and:

==11004== 24 bytes in 3 blocks are definitely lost in loss record 9 of 19
==11004==    at 0x1B903487: operator new(unsigned) (vg_replace_malloc.c:132)
==11004==    by 0x805AE65: watchlist_add(std::vector<commsemal*, std::allocator<commsemal*> >*, pthread_mutex_t*, comm_data*, unsigned char, bool) (in_mpg123.cxx:59)
==11004==    by 0x805CEA2: mpg123_input::mpg123_input(input_data*) (in_mpg123.cxx:272)
==11004==    by 0x8061008: new_input_device(std::string, input_data&) (input.cxx:20)
==11004==    by 0x806E5D0: inchannel::load(std::string&, comm_data*) (mixer_data.cxx:215)
==11004==    by 0x8065EB5: process_actions(std::vector<action*, std::allocator<action*> >&, bool) (mixer_actions.cxx:567)
==11004==    by 0x806B637: mixer(bool, int, int) (mixer.cxx:255)
==11004==    by 0x8062E80: main (main.cxx:111)

hm, 24 bytes in 3 blocks... 8-byte blocks... Looks like vector getting me again: I push a new pointer and the clear/erase from vector... this doesn't delete the pointer (malloc-wise)!
Changed the commsemal vector to contain the real thing, not pointers - the data structure is small enough...


==11004== 
==11004== 
==11004== 240 bytes in 1 blocks are definitely lost in loss record 14 of 19
==11004==    at 0x1B903487: operator new(unsigned) (vg_replace_malloc.c:132)
==11004==    by 0x1BA92C78: std::__default_alloc_template<true, 0>::allocate(unsigned) (stl_alloc.h:108)
==11004==    by 0x1BA98567: std::string::_Rep::_S_create(unsigned, std::allocator<char> const&) (stl_alloc.h:673)
==11004==    by 0x1BA98638: std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned) (basic_string.tcc:579)
==11004==    by 0x1BA962DC: std::string::reserve(unsigned) (basic_string.h:257)
==11004==    by 0x1BA967E1: std::string::append(char const*, unsigned) (basic_string.tcc:708)
==11004==    by 0x1BA96534: std::string::operator+=(char const*) (char_traits.h:143)
==11004==    by 0x805DA6D: backwatch(void*) (in_mpg123.cxx:472)
==11004==    by 0x1B91C183: pthread_start_thread (manager.c:300)
==11004==    by 0x1BBEBE46: clone (in /lib/libc-2.3.2.so)
==11004== 

That's
	msg += (buf + pos);

Hm, buf is a char array/cstring ... msg is string; I want to copy the cstring buf from pos to end...
What am I doing wrong here? Well... this msg is a static variable in the thread and may just not be destroyed properly... I don't trust these pthread macros! Thinking about this... pthread_cancel without some cleanup handler is _really_ a bad thing! 
I've learned quite some stuff since I coded the in_mpg123 threads...
Question: Is it feasible to avoid pthread_cancel at all? Well, how? I have many threads just blocking at a read. How to end them gracefully without cancel?

I guess I'll just remove all local variables from thread functions... but wait: At least one pointer has to be stored in the function block... hm... it's just the dynamic stuff that makes trouble - and a string wraps some dynamic stuff!
Well, msg to the backdata stuct for proper destruction, then.
(and check all other threads for such stuff!)

==11004== 
==11004== 8160 bytes in 1 blocks are definitely lost in loss record 17 of 19
==11004==    at 0x1B9032FD: malloc (vg_replace_malloc.c:130)
==11004==    by 0x1B91DE2E: __pthread_initialize_manager (pthread.c:584)
==11004==    by 0x1B91E1C7: pthread_create@@GLIBC_2.1 (pthread.c:752)
==11004==    by 0x8072BF7: output_device::build(output_data*, unsigned, void* (*)(void*)) (output_base.cxx:52)
==11004==    by 0x807A053: alsa_output::alsa_output(output_data*, unsigned) (out_alsa.cxx:84)
==11004==    by 0x807339A: new_output_device(std::string, output_data&) (output.cxx:42)
==11004==    by 0x806F3F3: outchannel::load(std::string, std::string, comm_data*) (mixer_data.cxx:364)
==11004==    by 0x806AF8B: mixer(bool, int, int) (mixer.cxx:183)
==11004==    by 0x8062E80: main (main.cxx:111)


Dunno. Have to check if this mounts up on repeated device construct/destruct.


again:

start & play & shutdown

==11834== LEAK SUMMARY:
==11834==    definitely lost: 8176 bytes in 5 blocks.
==11834==      possibly lost: 9376 bytes in 293 blocks.
==11834==    still reachable: 29974 bytes in 933 blocks.
==11834==         suppressed: 0 bytes in 0 blocks.

Well, I fixed two issues. There remains the mystery of phthread_create.

then:

start 
play & remin 0 & remin 0  & addin & bind 0 0 & play & shutdown

==11892==    definitely lost: 8176 bytes in 5 blocks.
==11892==      possibly lost: 9376 bytes in 293 blocks.
==11892==    still reachable: 29974 bytes in 933 blocks.
==11892==         suppressed: 0 bytes in 0 blocks.

Looks good...

Real-life test with a bit of mixplayer (still using mixplayd-ish API) and tplayershell (some fadings)

==12359== LEAK SUMMARY:
==12359==    definitely lost: 8180 bytes in 6 blocks.
==12359==      possibly lost: 9376 bytes in 293 blocks.
==12359==    still reachable: 28186 bytes in 932 blocks.
==12359==         suppressed: 0 bytes in 0 blocks.

Hm. Interesting. I loose 4 bytes in one block more and have a lot less still reachable... but only one block less...

Detail:

That one is a _bit_ different now:
==12359== 8 bytes in 2 blocks are definitely lost in loss record 4 of 16
==12359==    at 0x1B903487: operator new(unsigned) (vg_replace_malloc.c:132)
==12359==    by 0x8064183: _Z9set_statsRSt6vectorIP9inchannelSaIS1_EERS_IP10outchannelSaIS6_EEPS_IPSsSaISA_EE (mixer_actions.cxx:97)
==12359==    by 0x806B60A: mixer(bool, int, int) (mixer.cxx:229)
==12359==    by 0x8062FF4: main (main.cxx:111)
But essentially the same... I don't believe valgrind here!

A simple test case with the vector push, deletion and clear inside main() without threads shows no leakage in valgrind...
Since 1000 fullstats didn't leak additional memory, I suppode valgrind is really mislead here:

I do 
vector.push_back(new string) in mixer_actions (the main mixer thread)
and then in the same thread; the pointers from vector are pushed to another vector from the writer thread,
followed by vector.clear()
...
The writer thread, then, takes these string pointers from its vector and deletes them after write.
I don't see the leak here. And valgrind only sees about 4 leakings at 1000 cycles...
