About the direct interactive mode... it is no child of the socket watch. It is not supposed to quit without dermixd quitting. Shall it be able to spy on others? That would be useful, actually. But then we need a relation to the socket watch, without being part of it. Or should one be part of it?

Then, the clients need to be distinguished. I use the socket ID ... that wouldn't fit anymore. Oh, no. It would. Socket descriptors are file descriptors. Good.


Script handling is dirty ... at least the parts of which residing in inchannel::work_play(). I practice, it's no issue, since mixer thread script work and worker thread playing are separated by mutexes indirectly. But in theory, it's not nice.

Think about giving the inchannel (worker) a persistent string buffer for those watcher messages.

Check the usage of inchannel::sync_state()! I think I got it backwards there... this should always be called by the mixer thread, not the worker. Or not?
... this is twisted: I introduced the separate states to avoid locking in telltime() ... now, with the mixing happening in the worker thread, I don't see a point where to do the cheap incrementing of the normal state's position without sync. The mixer thread is not involved anymore. Do I actually have a problem here? Is there any sense in the separate states plus the syncinc?

extra_cd in inchannel works is totally unused.

With the INWORK_PLAY action ... there isn't a reason for suggesting issuing of multiple instances of that, is there? I really want to avoid to create a new action instance for that every time.


Now that I made failed filter reset non-fatal, I need some better way to communicate that to the clients: There are now 3 categories of responses:
1. All fine.
2. Error, operation aborted.
3. Worked, but there are issues.
I need to work that in, starting at the calls to filter::chain::set_source() and filter::chain::reset().


I want to add real management of the filter chain. Insert and remove filters at specific places. The problem is the handling of trouble: Inserting or removing filters can break the chain because of incompatible format changes. Eh ... does that have to be? What can happen? Change of channel count (mono/stereo only) could be taken care of implicitly. But when there is a sample rate change, that _might_ be fatal. If filters were restricted in their support of sampling rates. But otherwise ... well, I do allow filters to fail their reset, generically. As long as that is the case, I will have to deal with that at any instant of change.
Keeping a healthy filter chain is potentially troublesome. How should one deal with that? Should the channel be paused until the chain is fixed again? That would be one approach. There should be an extra command to reset the filter chain .. for the case that we want to postpone resume of track until we know the chain is fixed.

Another approach: The chain is always ready to process samples. When a filter fails to reset, it switches to bypass mode. One then can mess with the chain to trigger another reset that may fix up the filter.
For that to work, I need a bypass mode. Go for it.


The SC4 plugin is nasty ... actually the whole SWH family -- they insist on calling setlocale(LC_ALL, "")  ... but I want to ensure C locale for dermixd. So I have to do a setenv() to circumvent? Are those setlocale() calls thread-safe, anyway?

Is in-place interleaving feasible?
Shortcut about interleaving: Since DerMixD does format conversion for the mixing anyway (to float), it can also deinterleave on mixer entry and reinterleave before output. Just give the filters (and anyone else in between) non-interleaved data.

Filters & Files: Got to work out the mechanism for file and format switching, on the fly. Gotta be thread-safe, too. Oh, and the usual crap about file offsets, samples stuck in the filter chain...
Hm, and the chesamplify stuff could be a filter...

Idea about reliability: Add a command to lock certain parts of dermixd, like a certain input channel. So a client can ensure that nobody else interferes with the action. That includes inserting/removing channels that would result in the concerned channel changing its ID, unless I devise a more reliable mechanism to identify a channel (well, like using the nicknames).
One can think about different stages of locking, like protecting the actual input source and filter chain, but not the output routing and volume.

Urgent: Clean up that mess with audio buffers. I feel that when I start deriving classes from a template, I'm doing something wrong. Virtual destructor?
The meaning of the template was different audio data types and I'm going to handle that at runtime now anyway. I feel that my first template class ... is going away.

Working on the input filtering, to host LADSPA effects as well as "simple" resampling, or mono/stereo processing. One thing I have to be very aware of is the implications of an active filter chain on position handling.
Those filters can have latency. Some of that I won't be able to avoid... but how much of that needs to be considered? I guess I could live with those buffered samples getting lost. I'd have to.
This brings me right to the next point: Scanning of the input, shall that include filters? That seems insane.

Idea that doesn' work yet: Preventing underruns with live outputs right at the beginning... one has to work out the synchronization between outputs. If every live output gets one buffer ahead, that might work... but then, they're not in sync anyway, as loading happens whenever. So... an extra prefill really should make no trouble...

But: In case I switch output files on an existing channel, I introduce more gap than is necessary. Generally, I think, before messing with such stuff, I should think about synchronizing issues between different live outputs anyway.

Emergency: I think I whitnessed a deadlock during startup, creation of Lena didn't finish. Is saw it only once... I should make sure that there is nothing else...
I already catched an issue with not all scripted actions getting exorcism, but this must be something else...

old dermixd 1.6.2 does that other thing, too. I should fix that in trunk, too.

Getting down to output thread mix-down.

DONE

Getting down to input thread mixing:

DONE.

Another blurp after streamlining the sample format:

There shall be threads for
1. input mixing / filter processing
2. output format conversion / filter processing

And they will be. The input has threads already... just the resampling/mixing work should be added there. What needs thought is the follower business.
I need to decide if I will bring the pieces together in the leader's buffer or ... elsewhere. If I keep the smoothing, that should also apply when a track ends exactly at a buffer boundary. That's not handled, currently.
For that, one would need to detect a track's end beforehand. That's a bit unfortunate... I detect a track's end only when it fails to fill the buffer with the desired number of samples. So, I'd need to order one sample more, at least... and keep that for the next round. That's not entirely impossible.
Or, the smoothing is just done on the follower's side... for that I'd need to store the last sample values that played for the leader. Hey, I'm doing the smoothing on the follower only already. And I already use startval from the resampler. It already works for tracks ending on a buffer boundary! So just keep that.

Interesting... I see an error in the follower mix-in code. The memmove is off for stereo stuff. Do I hear that? Check version 1.6.2 for this.


The really current one:

I got the format conversions done now. On to the work of applying them.
I figured that I need two threads per output: A sound processor and a writer.
The sound processor has the task to take over the summed mixer buffer from the inputs and convert it to the proper output format. Fancy custom sound processing can be added later.

Now, shall this really be a separate thread? One could just do the stuff in mixer time, but that would be lame if there is more than one output. So, the output processor threads process the buffer and send the result to the output writer... then signalling that they're done to the mixer.

So, on the input side... I also need to do the format conversion, best centrally in the generic input_file class. The follower handling needs to be settled... I had something written about that... So, first thing: Implement the format conversions, make DerMixD build and work again... then add threads work.

The current braindump:

Now I want to get down & dirty with the improved inchannel mixing.

1. Move the mixing itself into the channels, synchronized on outputs via a lock.

2. Properly handle follower action. Actually, because the code is intertangled... this is not really a separate step, well not unless the step right before it is "break follower action".

A channel shall have a generic message for "play n samples into your outputs, from offset x in those". The offset is for the followers.
It might be useful to be a bit smart about the locking of output buffers. To minimize senseless waiting, the inputs could use trylock() to find a free output ... or one just shuffles the order of channel binding if there are multiple output channels. In any case, the funny part of mixing, the resampling to mixer rate and amplification can appen in parallel without issues, the final copy/addition to the output buffer being quick.
One thing to think about is who does final clipping. Eventually, dermixd should work in floats, as it's already not using proper integer math for amplification or interpolation anyway. Well... the clipping is the final stage that has to happen once per output... so the output channel worker thread should do it.
Yup. Give the outputs a mixer buffer with float type first... convert the input mixer buffers to float ... then move the clipping into the outputs, then the mixing into the inputs.

Oh, wait: When the outputs do the clipping in the output thread, that somehow interferes with the multibuffered writing. What I can do... the last input thread mixing into the buffer might do the clipping and conversion to the real output buffer... but then, this part needs to be handled if there's no input, too, for live outputs... but there it's trivial. No clipping as such.

I can still keep the optimization for a single channel playing without amplification... I can detect that case and there do a direct copy. Or I could leave that. Hm. Dunno.

I still am not satisfied with the clipping stuff ... that part will be essentially in the main mixer thread time if there's only one output.
I guess I'll do the clipping in mixer time for now (in the outchannel class, though) ... and think of some better place later. The output thread shall not be distracted from just writing the audio data.

Now, as a precursor, I need my proper conversions from input format to float and back. I want it in a way that does not alter a bit on 16 bit input if there's no amplification or resampling going on.
In mpg123, we do have some nice fast hacks for rounding... based on the float being kept scaled to 32767. I won't port the funny hacks yet, but I ponder keeping such a scale factor to avoid division and multiplication when converting 16bit -> float -> 16bit. 
But then, I got output devices that could support float or 32bit integer output. There, scaling would be needed. And once there's normal float input, this whole thing blows up. Well, let's see how smartass I can get. Start out with 16 bit output in mind, but offer a switch to optimize for other output types.

Dammit... this stuff is ugly... heck, I can get float from libmpg123 ... can't I get float from anyone else, too? For actual raw 16 bit files, this unnecessarily blows up the buffers... I want to do the conversion after prebuffering... that means the fifo_buffer needs to be type-agnostic, somewhat, for different input types. I don't want to push all that code into a template header... which also wouldn't help me a lot with deriving how to scale the type, convert it to something else... but I can turn the fifo_buffer into a building block template. The preprocessor gives me all freedom I need, without having to worry how what works in C++ templates.

But what do I need, exactly? The code as such could easily be turned into a template... just need audio_type as parameter, I could also just fix an input format at compile-time... heck, and when I want to get float samples anyway, including funky computations in the inputs (timestretching, whatnot)? It's easier to just do all-float. Make the input file convert as necessary, same with output. But I do have the low-power application in mind, where my Treo 650 should go mixing most effectively. The thing has MMX, even... that means rather efficient 16bit output from mpg123, otherwise, it's efficient 16 bit from the generic ARM assembly. On those cases I do need headroom for mixing (and some optimized integer amplification... fixed-point math, or simpler?).
Gnnn. Pondering ... what's the right thing to do?


Older Braindump:

I want to move the mixing into the input channel worker threads. At minimum the resampling/amplification should be parallelized, but there's no reason that the actual mixing should not also be there.
I got now the input worker thread merged with the input channel. I actually wanted to separate it into an input_worker class again, but for the mixing to be in parallel, the input worker thread needs to have access to the list of output channels; at least to a list of output buffer pointers and mutexes.
And now here comes one actual problem, or is it one? There's more fun involved when a channel ends and one needs to attach the sound of a follower. Hm, when is the follower ready? In case the follower is buffering right now, one might need to wait... gnnn... I cannot access other input channels just like that in parallel. That needs to be worked out.

Also: Instead of the input channel having a mixer buffer that features amplified samples that are to be clipped later, the clipping can happen in the output later on. Clip once per output. But, for resampling/speed handling, the input needs a separate buffer to avoid double work when being coupled to several outputs.
Yes... the input buffers stay that way, the outputs get the scaled mixer buffer (now long it, will be float in future, probably) and does clipping just before playback... even inside the playback thread.

How would clipping work for the playback thead? The buffer pool should consist of mixer buffers, then? The output thread converting to output format just before playback? That would work, but I don't want any extra work happening in the playback thread. So, who should do this clipping/format conversion? It could happen in a random input thread when it sees that it's the last one mixing. Yes, that's possible with semaphores. Or just any counter after taking a mutex, which is a barrier. An optimization would be to indeed ensure that the inputs operate on the outputs in a non-uniform manner to avoid stalling of all inputs trying to access output 1 when they could work on all outputs in parallel.
So all work happens in the parallel phase of input threads being let loose... that's the ideal. Let's go there.

Now just figure out how to handle followers. I need to ensure that one can access the follower safely in the parallel section.

I don't want to do fancy synchronization between input channels, so the follower rule must be that the follower must be done with buffering when the leader breathes its last sample... the buffering should happen a cycle before the ending.
Or... should it run internally, actually? The channel can take care of buffering itself, also re-buffering on invalidated buffer contents. Why should the mixer bother with that?
So the leader channel would need to communicate to the follower channel that it should wake up and fill it's output buffers from the position where the leader left. In that case, the leader wouldn't signal that it's done to the mixer, the follower should do that.

That might work... that way, one even stays consistent in case the follower has different output channels. Hey... I see test case: Two channels, each mixing to separete raw output... observe the gaplessness in direct comparison.


A propos tests: Testing the scanners should be rather easy.
