Implement the core processing method

At this stage, and if the previous tests were successfully passed, your processor should be correctly detected by the Auditory front-end framework. However, there is still some work to do. In particular, the core of your processor has to be implemented, which performs the processing of the input signal and returns a corresponding output.

This section will provide guidelines as to how to implement that method. However, this task is very dependent on the functionality of a particular processor. You can get insights as to how to perform the signal processing task by looking at the code of the .processChunk methods of existing processors.

Note

Some of the challenges in implementing the processing method were already presented in a section of the technical description. It is recommended at that stage to go back and read that section again.

Input and output arguments

The processing method should be called processChunk and be placed in a block of methods with no attributes (e.g., following the class constructor). The function takes a single effective input argument, a chunk of input signal and returns a single output argument, the corresponding chunk of output signal. Because it is a non-static method of the processor, an instance of the processor is passed as first input argument. Hence the method definition looks something like this for a monaural single-output processor:

function out = processChunk(pObj,in)

  % The signal processing to obtain "out" from "in" is written here
  %
  % ...

end

Or, for a binaural single-output processor (such as ildProc):

function out = processChunk(pObj,in_left,in_right)

  % The signal processing to obtain "out" from "in" is written here
  %
  % ...

end

If your processor is not of one of the two kinds described above, then you are free to use a different signature for your processChunk method (i.e., different number of input or output arguments). However, you will then have to override the initiateProcessing method.

Given an instance of your processor, say p, this allows you to call this method (and in general all methods taking an object instance as first argument) in two different ways:

  • processChunk(p,in)
  • p.processChunk(in)

The two calls will of course return the same output.

Note

Having an instance of the processor as an argument means that you can access all of its properties to carry out the processing. In particular, the external and internal parameter properties you have defined earlier. For example, the processing method of a simple “gain” processor could read as out = in * p.gain

The arguments in and out are arrays containing “pure” data. Although signal-related data is stored as specific signal objects in the Auditory front-end, only the data is passed around when it comes to processing. It is done internally to avoid unnecessary copies. So it is not something that has to be addressed in the implementation of your processing method. Your input is an array whose dimensionality depends on the type of signal. Dimensions are ordered in the same way as in the data-storing buffer of the signal object. For example, the input in in the gammatoneProc.processChunk is a one-dimensional array indexing time. Similarly, the output should be arranged in the same way than in its corresponding output signal object. For example, the output out of modulationProc.processChunk is a three-dimensional array where the first dimension indexes time, the second refers to audio frequency and the third corresponds to modulation frequency. Just like the way data is stored in the modulationSignal.Data buffer.

Note

The first dimension for all signals used in the Auditory front-end is always indexing time.

Chunk-based and signal-based processing

As the name of the method processChunk suggests, you should implement the processing method such that it can process consecutive chunks of input signal, as opposed to the entire signal at once. This enables “online” processing, and eventually “real-time” processing once the software has been sufficiently optimised. This has two fundamental consequences on your implementation:

  1. The input data to the processing method can be of arbitrary duration.
  2. The processing method needs to maintain continuity between input chunks. In other words, when concatenating the outputs obtained by processing individual consecutive chunks of input, one need to obtain the same output as if all the consecutive input were concatenated and processed at once.

Point 1. above implies that depending on the type of processing you are carrying out, it might be necessary to buffer the input signal. For example, processors involving framing of the signal, such as ratemapProc or ildProc, need to put the segment of the input signal that went out of bound of the framing operation in a buffer. This buffer is then appended to the beginning of the next input chunk. This is illustrated in a section of the technical description of the framework. This also means that for some processor (those which lower the sampling rate in general), an input that is too short in time might produce an empty output. But this input will still be considered in the next chunk.

Point 2. is the most challenging one because it very much depends on the processing carried out by the processor. Hence there are no general guidelines. However, the Auditory front-end comes with some building blocks to help with this task. It features for instance filter objects that can be used for processing. All filters manage their internal states themselves, such that output continuity is ensured. For an example on how to use filters, see e.g. gammatoneProc.processChunk. Sometimes however, one need more than simple filtering operations. One can often find a workaround by using some sort of “overlap-save” method using smart buffering of the input or output as described in the technical description. A good example of using buffering for output continuity can be found in e.g., ildProc.processChunk.

Reset method

To ensure continuity between output chunks, your new processor might include “internal states” (e.g., built-in filter objects or internal buffers). Normally, incoming chunks of input are assumed to be consecutive segments of a same signal. However, the user can decide to process an entirely new signal as input at any time. In this case, your processor should be able to reset its internal states.

This is performed by the reset method. This method should be implemented in a method block with no method attributes, just like the constructor. It should simply reset the filters (if any) by calling all the filters reset methods, and/or empty all internal buffers.

If your processor does not need any internal state storage, then the reset method should still be implemented (as it is an abstract method of the parent class) but can be left empty (see, e.g., itdProc.reset).