Implement the core processing method¶
At this stage, and if the previous tests were successfully passed, your processor should be correctly detected by the Auditory front-end framework. However, there is still some work to do. In particular, the core of your processor has to be implemented, which performs the processing of the input signal and returns a corresponding output.
This section will provide guidelines as to how to implement that method.
However, this task is very dependent on the functionality of a particular
processor. You can get insights as to how to perform the signal processing task
by looking at the code of the .processChunk
methods of existing processors.
Note
Some of the challenges in implementing the processing method were already presented in a section of the technical description. It is recommended at that stage to go back and read that section again.
Input and output arguments¶
The processing method should be called processChunk
and be placed in a block
of methods with no attributes (e.g., following the class constructor). The
function takes a single effective input argument, a chunk of input signal and
returns a single output argument, the corresponding chunk of output signal.
Because it is a non-static method of the processor, an instance of the
processor is passed as first input argument. Hence the method definition looks
something like this for a monaural single-output processor:
function out = processChunk(pObj,in)
% The signal processing to obtain "out" from "in" is written here
%
% ...
end
Or, for a binaural single-output processor (such as ildProc
):
function out = processChunk(pObj,in_left,in_right)
% The signal processing to obtain "out" from "in" is written here
%
% ...
end
If your processor is not of one of the two kinds described above, then you are
free to use a different signature for your processChunk
method (i.e.,
different number of input or output arguments). However, you will then have to
override the initiateProcessing
method.
Given an instance of your processor, say p
, this allows you to call this
method (and in general all methods taking an object instance as first argument)
in two different ways:
processChunk(p,in)
p.processChunk(in)
The two calls will of course return the same output.
Note
Having an instance of the processor as an argument means that you can access
all of its properties to carry out the processing. In particular, the external
and internal parameter properties you have defined earlier. For example, the
processing method of a simple “gain” processor could read as out = in *
p.gain
The arguments in
and out
are arrays containing “pure” data. Although
signal-related data is stored as specific signal objects in the Auditory front-end, only the
data is passed around when it comes to processing. It is done internally to
avoid unnecessary copies. So it is not something that has to be addressed in the
implementation of your processing method. Your input is an array whose
dimensionality depends on the type of signal. Dimensions are ordered in the same
way as in the data-storing buffer of the signal
object. For example, the input in
in the gammatoneProc.processChunk
is a
one-dimensional array indexing time. Similarly, the output should be arranged in
the same way than in its corresponding output signal object. For example, the
output out
of modulationProc.processChunk
is a three-dimensional array
where the first dimension indexes time, the second refers to audio frequency and
the third corresponds to modulation frequency. Just like the way data is stored
in the modulationSignal.Data
buffer.
Note
The first dimension for all signals used in the Auditory front-end is always indexing time.
Chunk-based and signal-based processing¶
As the name of the method processChunk
suggests, you should implement the
processing method such that it can process consecutive chunks of input signal,
as opposed to the entire signal at once. This enables “online” processing,
and eventually “real-time” processing once the software has been
sufficiently optimised. This has two fundamental consequences on your
implementation:
- The input data to the processing method can be of arbitrary duration.
- The processing method needs to maintain continuity between input chunks. In other words, when concatenating the outputs obtained by processing individual consecutive chunks of input, one need to obtain the same output as if all the consecutive input were concatenated and processed at once.
Point 1. above implies that depending on the type of processing you are carrying
out, it might be necessary to buffer the input signal. For example, processors
involving framing of the signal, such as ratemapProc
or ildProc
, need to
put the segment of the input signal that went out of bound of the framing
operation in a buffer. This buffer is then appended to the beginning of the next
input chunk. This is illustrated in a section of the
technical description of the framework. This also means that for some processor
(those which lower the sampling rate in general), an input that is too short in
time might produce an empty output. But this input will still be considered in
the next chunk.
Point 2. is the most challenging one because it very much depends on the
processing carried out by the processor. Hence there are no general guidelines.
However, the Auditory front-end comes with some building blocks to help with this task. It
features for instance filter objects that can be used for processing. All
filters manage their internal states themselves, such that output continuity is
ensured. For an example on how to use filters, see e.g.
gammatoneProc.processChunk
. Sometimes however, one need more than simple
filtering operations. One can often find a workaround by using some sort of
“overlap-save” method using smart buffering of the input or output as described
in the technical description. A good example of using
buffering for output continuity can be found in e.g., ildProc.processChunk
.
Reset method¶
To ensure continuity between output chunks, your new processor might include “internal states” (e.g., built-in filter objects or internal buffers). Normally, incoming chunks of input are assumed to be consecutive segments of a same signal. However, the user can decide to process an entirely new signal as input at any time. In this case, your processor should be able to reset its internal states.
This is performed by the reset
method. This method should be implemented in
a method block with no method attributes, just like the constructor. It should
simply reset the filters (if any) by calling all the filters reset
methods,
and/or empty all internal buffers.
If your processor does not need any internal state storage, then the reset
method should still be implemented (as it is an abstract method of the parent
class) but can be left empty (see, e.g., itdProc.reset
).