Add your own processors¶
The Auditory front-end framework has been designed in such a way that it can be easily
upgraded. To add a new processor, write its class definition in a new .m
file and add it to the /src/Processors
folder. If correctly written, the
processor should be automatically detected by the framework and be ready to use.
This section documents in details how to correctly write the class definition
of a new processor. It is highly recommended to look into the definition of
existing processors to get a grasp of how classes are defined and written in
Matlab. In the following, we will sometimes refer to a particular existing
processor to illustrate some aspects of the implementation.
Note
- The following descriptions are exhaustive, and adding a processor to the framework is actually easier than the length of this page suggests!
- This tutorial is written assuming limited knowledge about object- oriented programming using Matlab. Hence most OOP concepts involved are briefly explained.
- You can base your implementation on the available
templateProc.m
file which contains a pre-populated list of properties and methods. Simply copy the file, rename it to your processor name, and follow the instructions.
Check-list for adding a new processor¶
To write the class definition for a new processor such that it will be recognised and properly integrated, one has to follow these steps:
- Set up the specific properties of the processor class
- Implement the processor’s static methods
- Implement the processor’s abstract methods (processing and resetting)
- Implementing parameters “getter” methods
- Implement the processor constructor
- Take a break and test your implementation
- Implement the core processing method
- Override parent methods (optional)
- Allowing alternative processing options (optional)
- Add a new type of signal (optional)
- Final testing
Getting started and setting up processor properties¶
The properties of an object are a way to store data used by the object. There are two types of properties for processors, those which:
- store all the parameters needed to integrate the processor into the framework (e.g., the sampling frequency on which it operates, the number of inputs/outputs, ...)
- store parameter values which are used in the actual processing
When writing the class definition for a new processor, it is only necessary to
implement the latter: parameters which are needed in the computation. All
parameters needed for the integration of the processor in the framework are
already defined in the parent Processor
class. Your new processor should
inherit this parent class in order to automatically have access to the
properties and methods of the parent class. Inheritance in Matlab is indicated
by the command < nameOfParentClass
following the name of your new class in
the first line of its definition.
The new processor class definition should be saved in a .m
file that has
the same name as the defined class. In the example below, that would be
myNewProcessor.m
.
There are usually two categories of properties to be implemented for a new processor: external (user-controlled) parameters and internal parameters necessary for the processor but which do not need to be known to the “outside world”.
Note
Only the two types of properties below have been used so far in every processor implementation. However, it is fine to add more if needed for your new processor.
External parameters controllable by the user¶
External parameters are directly related to the parameters the user can access
and change. The actual values for these are stored in a specific object
accessible via the .parameters
property of the processor. Defining them as
individual properties seems redundant, and is therefore optional. However it can
be very convenient in order to simplify the access to the parameter value and to
make your code more readable.
Instead of storing an actual value, the corresponding processor property should
only point to a value in the .parameters
object. This will avoid having two
different values for the same parameter. To do this, external parameters should
be defined as a set of dependent properties. This is indicated by the
Dependent = true
property attribute. If a property is set to Dependent
,
then a corresponding “getter” method has to be implemented for it. This will be
developed in a following section. For example, if
your new processor has two parameters, parA
and parB
, you can define
these as properties as follow:
classdef myNewProcessor < Processor
properties (Dependent = true)
parA;
parB;
end
%...
end
This will allow easier access to these values in your code. For example,
myNewProcessor.parA
will always give the same output as
myNewProcessor.parameters.map('xx_nameTagOfParameterA')
, even if the
parameter value changes due to feedback. This simplifies greatly the code,
particularly when many parameters are involved.
Internal parameters¶
Internal parameters are sometimes (not always) needed for the functioning of the processor. They are typically used to store internal states of the processor (e.g., to allow continuity in block-based processing), filter instances (if your processor involves filtering), or just intermediate parameter values used to make code more readable.
Because they are “internal” to the processor, these parameters are usually
stored as a set of private properties by using the GetAccess = private
property attributes. This will virtually make the property invisible and
inaccessible to all other objects.
Implementing static methods¶
Static methods are methods that can be called without an existing instance of an
object. In the implementation of processors, they are used to store all the
hard-coded information. This can be for example the processor name, the type of
signal it accepts as input, or the names and default values of its external
parameters. A static method is implemented by defining it in a method block with
the (Static)
method attribute:
classdef myNewProcessor < Processor
% ... Properties and other methods definition
methods (Static)
function out = myStaticMethod_1(in)
%...
end
function out = myStaticMethod_2(in)
%...
end
end
end
Static methods share the same structure and names across processors, so they can easily be copy/pasted from an existing processor and then modified to reflect your new processor. The following three methods have to be implemented.
.getDependency()
: Returns the type of input signal by its user request name.getParameterInfo()
: Returns names, default values, and descriptions of external parameters.getProcessorInfo()
: Returns information about the processor as a Matlab structure
As they are used to hard-code and return information, none of these methods accept input arguments.
getDependency
¶
This method returns the type of input signal your processor should accept:
function name = getDependency()
name = 'requestNameOfInputSignal';
end
where 'requestNameOfInputSignal'
is the request name of the signal that
should be used as input. “Request name” corresponds to the request a user would
place in order to obtain a particular signal. For example, the inner hair-cell
envelope processor requires as input the output of e.g., a gammatone filterbank.
The request name for this signal is 'filterbank'
which should therefore be
the output of the static method ihcProc.getDependency()
. You can also check
the list of currently valid request names by typing requestList
in Matlab’s
command window.
If you are unsure about which name should be used, consider which processor
would come directly before your new processor in a processing chain (i.e., the
processor your new processor depends on). Say it is named dependentProc
.
Then typing:
dependentProc.getProcessorInfo.requestName
in Matlab’s command window will return the corresponding request name you should
output in your getDependency
method.
getParameterInfo
¶
This method hard-codes all information regarding the (external) parameters used
by your processor, i.e., lists of their names, default values, and description.
These are used to populate the output of the helper script parameterHelper
and to give a default value to parameters when your processor is instantiated.
The lists are returned as cell arrays of strings (or any other type for the default parameter values). They should follow the same order, such that the n-th member of each of the three lists relate to the same parameter.
Parameter names need not be the same as the parameter property name you
defined earlier. This will become apparent in
the next section. In fact, names should be
changed to at least include a two or three letters prefix that is unique to your
new processor. You can make sure it is not already in use by browsing through
the output of the parameterHelper
script.
The method should look something like this:
function [names,defValues,description] = getParameterInfo()
names = {'xx_par1','xx_par2','xx_par3'};
defValues = {0.5, ...
[1 2 3 4], ...
'someStringValue'};
description = {'Tuning factor of dummy example (s)',...
'Vector of unused frequencies (Hz)',...
'Model name (''someStringValue'' or ''anotherValue'')'}
end
This dummy example illustrates the following important points:
- Use a unique prefix in the name of the parameters (
xx_
above) that abbreviates the name or task of the processor. - Find a short, but self-explanatory parameter name (not like
parX
above). If it makes sense, you can re-use the same name as a parameter involved in another processor. The prefix will make the name unique. - Default values can be of any type (e.g., float number, array, strings,...)
- Descriptions should be as short as possible while still explanatory. Mention if applicable the units or the different alternatives.
getProcessorInfo
¶
This method stores the properties of the processor that are needed to integrate it in the framework. It outputs a structure with the following fields:
.name
: A short, self-explanatory name for the processor.label
: A name for the processor that is used as a label. It can the same as.name
if that is sufficient, or a bit longer if needed..requestName
: The name tag of the request that a user should input when calling the.addProcessor
method of the manager. This has to be a valid Matlab name (e.g., it cannot include spaces)..requestLabel
: A longer name for the signal this processor produces, used e.g., as plot labels.outputType
: The type of signal object (name of the class) this processor produces. If none of the existing signals in the framework are suitable, you will need to implement a new one.isBinaural
: Set to 0 if your processor operates on a single channel (e.g., an auditory filterbank) or to 1 if it needs a binaural input (e.g., the inter-aural level differences processor). If your processor can operate on both mono and stereo signals (such as the pre-processorpreProc.m
), set it to 2.
Your method should initialise the structure that will be returned as output and give a value to all of the above-mentioned fields:
%...
function pInfo = getProcessorInfo
pInfo = struct;
pInfo.name = 'MyProcessor';
pInfo.label = 'Processor doing things';
% etc...
end
Implementing parameters “getter” methods¶
As described in an earlier section, external
parameters of the processor, i.e., those that can be modified by the user, are
implemented as Dependent
properties of your processor class. For your
implementation to be valid, a “getter” method needs to be implemented for each
of these parameters. If not, Matlab will generate an error when trying to access
that parameter value. If a property is set as Dependent
, then its getter
method will be called whenever the program tries to access that property. In
general, this can be useful for a property that depends on others and that
need to be recomputed whenever accessed. In the present case, we will set the
getter method to read the corresponding parameter value in the parameter object
associated with your processor. If the value of the parameter has changed
throughout the processing (e.g., in response to feedback), then we are sure to
always get the updated value.
“Getter” methods for parameters are implemented without any method attribute and always follow the same structure. Hence they can easily be copy/pasted and adjusted:
methods
function value = get.parName(pObj)
value = pObj.parameters.map('xx_parNameTag')
end
% ... implement one get. method for each parameter
end
In the above example, parName
is the name of the parameter as a dependent
property of your processor class, and xx_parNameTag
is the name of the
parameter defined in the static
.getParameterInfo
method. pObj
represents an instance of your processor
class, it does not need to be changed across methods.
Implement the processor constructor¶
For any possible application, every class should implement a very specific method: a class constructor. A class constructor is a function that has the exact same name as your class. It can take any combination of input arguments but can return only a single output: an “instance” of your class.
In the Auditory front-end architecture however, the input arguments to the constructor of all
processors have been standardised, such that all processor constructors can be
called using the exact same arguments. The input arguments should be (in this
order) the sampling frequency of the input signal to the processor and an
instance of a parameter object returned e.g. by the script genParStruct.m
.
The constructor’s role is then to create an object of the class, and often to
initialise all its properties. Most of this initialisation step is the same
across all processors (e.g., setting input/output sampling frequencies,
indicating the type of processor, ...). Hence all processor constructors rely
heavily on the constructor of their parent class (or super-constructor),
Processor(...)
which defines these across-processors operations. This allows
to have all this code in one place which reduces the code you have to write for
your processor, as well as reducing chances for bugs and increasing
maintainability. This concept of “inheritance” will be discussed in a
further section.
In practice, this means that the constructor for your processor will be very short:
1 2 3 4 5 6 7 8 9 10 11 12 13 | function pObj = myNewProcessor(fs,parObj)
%myNewProcessor ... Provide some help here ...
if nargin<2||isempty(parObj); parObj = Parameters; end
if nargin<1; fs = []; end
% Call super-constructor
pObj = pObj@Processor(fs, fsOut,'myNewProcessor',parObj);
% Additional code depending on your processor
% ...
end
|
Note
The constructor method should be placed in a “method” block with no method attributes.
Let us break down the constructor structure line by line:
- Line 1: As stated earlier, all processor constructors take two input and
return a single output, your processor instance
pObj
. Matlab restricts all constructors to return a single output. If for any reason you need additional outputs, you would have to place them in a property of your processor instead of a regular output. Input arguments are the input sampling frequency, i.e., the sampling frequency of the signal at the input of the processor, and a parameter objectparObj
. - Line 2: This is where you will place help regarding how to call this constructor. Because they have a generic form across all processors, you can easily copy/paste it from another processor.
- Lines 4 and 5: An important aspect in this implementation is that the constructor should be called with no input argument and still return a valid instance of the processor, without any error. Hence these two lines define default values for inputs if none were specified.
- Line 8: This line generates a processor instance by calling the class
super-constructor. The super-constructor takes four inputs:
- the input sampling frequency
fs
- the output sampling frequency. If your processor does not modify the
sampling rate, then you can replace
fsOut
withfs
. If the output sampling rate of your processor if fixed, i.e., not depending on external parameters, then you can specify it here, in place offsOut
. Lastly, if the output sampling rate depends on some external parameters (i.e., susceptible to change via feedback from the user), then you should leave thefsOut
field empty:[]
. The output sampling rate will be defined in another method that is called every time feedback is involved. - the name of the children processor, here
myNewProcessor
. - the parameter object
parObj
already provided as input.
- the input sampling frequency
- Line 11: Your processor might need additional initialisation. All extra code
should go there. To ensure that no error is generated when calling the
constructor with no arguments (which Matlab sometimes does implicitly), the
code should be embedded in a
if nargin > 0 ... end
block. Here you can for example initialise buffers or internal properties.
Warning
The initialisation of anything that depends on external parameters (e.g., filters, framing windows, ...) is not performed here on line 11. When parameters change due to feedback, these properties need to be re-initialised. Hence their initialisation is performed in another method that will be described in a following section.
Preliminary testing¶
At this stage of the implementation, your processor should be correctly instantiated and recognised by the framework. In some cases (e.g., your processor is a simple single input / single output processor), it might even be correctly integrated and routed to other processors. In any case, now is a good time to take a break from writing code and do some preliminary testing. We will go through a few example tests you can run, describe which problems could arise and suggest how to solve them. Try to run these tests in the order they are listed below, as this will help troubleshooting. They should run as expected before you go further in your implementation.
Note
You will not be able to instantiate your processor before you have written a
concrete implementation to Processor
abstract methods. To carry out the
tests below, just write empty processChunk
and reset
methods. In this
way, Matlab will not complain about trying to instantiate a class that
contains abstract methods. The actual implementation of these methods will be
described in later sections.
Default instantiation¶
As mentioned when implementing the constructor, you should be able to get a valid instance of your processor by calling its constructor without any input arguments:
>> p = myNewProcessor
If this line returns an error, then you have to revise your implementation of the constructor. The error message should indicate where the problem is located, so that you can easily correct it. If your processor cannot be instantiated with no arguments, then it will not be listed as a valid processor.
If on the other hand this line executed without error, then there are two things you should control:
- The line above (if not ended by a semicolon) should display the visible,
public properties of the processor. Check that this list corresponds to the
properties you defined in your implementation. The property values should be
the default values you have defined in your
getParameterInfo
static method. If a property is missing, then you forgot to list it in the beginning of your class definition (or you defined it asHidden
orPrivate
). If a value is incorrect, or empty, then it is a mistake in yourgetParameterInfo
method. In addition, theType
property should refer to thename
field returned bygetProcessorInfo
static method. - Inspect the external parameters of the processor by typing
p.parameters
. This should return a list of all external parameters. Control that all parameters are there and that their default value is correct.
To test that your external properties are indeed dependent, you can change the
value of one or more of them directly in your parameter
processor property
and see if that change is reflected in the dependent property. For example if
you type:
p.parameters.map('xx_par1') = someRandomValue
then this should be reflected in the property associated with that parameter.
Note
The input and output frequency properties of your processor, FsHzIn
and
FsHzOut
are probably incorrect, but that is normal as you did not specify
the sampling frequency when calling the constructor with no arguments.
Is it a valid processor?¶
To test whether your processor is recognised as a valid processor, run the
requestList
script. The signal request name corresponding to your processor
should appear in the list (i.e., the name defined in
getProcessorInfo.requestName
). If not (and the previous test did work), then
maybe your class definition file is not located in the correct folder. Move it
to the src/Processors
folder. Another possibility is that you made your
processor hidden (which should not happen if you followed these instructions).
Setting explicitly the bHidden
property of your processor to 1
will
hide it from the framework. This is used in order to allow “sub-processors” in
the framework, but it is probably not the case for you here so you should not
enable this option.
Are parameters correctly described?¶
If your processor is properly recognised, then you can call the
parameterHelper
script from the command window. There you should see a new
category corresponding to your processor. Clicking on it will display a list of
user-controllable parameters for your processor, as well as their descriptions.
Feel free to adjust your getParameterInfo
static method to have a more
suitable description.
Implementing the core processing method¶
At this stage, and if the previous tests were successfully passed, your processor should be correctly detected by the Auditory front-end framework. However, there is still some work to do. In particular, the core of your processor has to be implemented, which performs the processing of the input signal and returns a corresponding output.
This section will provide guidelines as to how to implement that method.
However, this task is very dependent on the functionality of a particular
processor. You can get insights as to how to perform the signal processing task
by looking at the code of the .processChunk
methods of existing processors.
Note
Some of the challenges in implementing the processing method were already presented in a section of the technical description. It is recommended at that stage to go back and read that section again.
Input and output arguments¶
The processing method should be called processChunk
and be placed in a block
of methods with no attributes (e.g., following the class constructor). The
function takes a single effective input argument, a chunk of input signal and
returns a single output argument, the corresponding chunk of output signal.
Because it is a non-static method of the processor, an instance of the
processor is passed as first input argument. Hence the method definition looks
something like this for a monaural single-output processor:
function out = processChunk(pObj,in)
% The signal processing to obtain "out" from "in" is written here
%
% ...
end
Or, for a binaural single-output processor (such as ildProc
):
function out = processChunk(pObj,in_left,in_right)
% The signal processing to obtain "out" from "in" is written here
%
% ...
end
If your processor is not of one of the two kinds described above, then you are
free to use a different signature for your processChunk
method (i.e.,
different number of input or output arguments). However, you will then have to
override the initiateProcessing
method.
Given an instance of your processor, say p
, this allows you to call this
method (and in general all methods taking an object instance as first argument)
in two different ways:
processChunk(p,in)
p.processChunk(in)
The two calls will of course return the same output.
Note
Having an instance of the processor as an argument means that you can access
all of its properties to carry out the processing. In particular, the external
and internal parameter properties you have defined earlier. For example, the
processing method of a simple “gain” processor could read as out = in *
p.gain
The arguments in
and out
are arrays containing “pure” data. Although
signal-related data is stored as specific signal objects in the Auditory front-end, only the
data is passed around when it comes to processing. It is done internally to
avoid unnecessary copies. So it is not something that has to be addressed in the
implementation of your processing method. Your input is an array whose
dimensionality depends on the type of signal. Dimensions are ordered in the same
way as in the data-storing buffer of the signal
object. For example, the input in
in the gammatoneProc.processChunk
is a
one-dimensional array indexing time. Similarly, the output should be arranged in
the same way than in its corresponding output signal object. For example, the
output out
of modulationProc.processChunk
is a three-dimensional array
where the first dimension indexes time, the second refers to audio frequency and
the third corresponds to modulation frequency. Just like the way data is stored
in the modulationSignal.Data
buffer.
Note
The first dimension for all signals used in the Auditory front-end is always indexing time.
Chunk-based and signal-based processing¶
As the name of the method processChunk
suggests, you should implement the
processing method such that it can process consecutive chunks of input signal,
as opposed to the entire signal at once. This enables “online” processing,
and eventually “real-time” processing once the software has been
sufficiently optimised. This has two fundamental consequences on your
implementation:
- The input data to the processing method can be of arbitrary duration.
- The processing method needs to maintain continuity between input chunks. In other words, when concatenating the outputs obtained by processing individual consecutive chunks of input, one need to obtain the same output as if all the consecutive input were concatenated and processed at once.
Point 1. above implies that depending on the type of processing you are carrying
out, it might be necessary to buffer the input signal. For example, processors
involving framing of the signal, such as ratemapProc
or ildProc
, need to
put the segment of the input signal that went out of bound of the framing
operation in a buffer. This buffer is then appended to the beginning of the next
input chunk. This is illustrated in a section of the
technical description of the framework. This also means that for some processor
(those which lower the sampling rate in general), an input that is too short in
time might produce an empty output. But this input will still be considered in
the next chunk.
Point 2. is the most challenging one because it very much depends on the
processing carried out by the processor. Hence there are no general guidelines.
However, the Auditory front-end comes with some building blocks to help with this task. It
features for instance filter objects that can be used for processing. All
filters manage their internal states themselves, such that output continuity is
ensured. For an example on how to use filters, see e.g.
gammatoneProc.processChunk
. Sometimes however, one need more than simple
filtering operations. One can often find a workaround by using some sort of
“overlap-save” method using smart buffering of the input or output as described
in the technical description. A good example of using
buffering for output continuity can be found in e.g., ildProc.processChunk
.
Reset method¶
To ensure continuity between output chunks, your new processor might include “internal states” (e.g., built-in filter objects or internal buffers). Normally, incoming chunks of input are assumed to be consecutive segments of a same signal. However, the user can decide to process an entirely new signal as input at any time. In this case, your processor should be able to reset its internal states.
This is performed by the reset
method. This method should be implemented in
a method block with no method attributes, just like the constructor. It should
simply reset the filters (if any) by calling all the filters reset
methods,
and/or empty all internal buffers.
If your processor does not need any internal state storage, then the reset
method should still be implemented (as it is an abstract method of the parent
class) but can be left empty (see, e.g., itdProc.reset
).
Override parent methods¶
The Auditory front-end framework was developed to maximise code reusing. Many of the existing
processors, although they carry out different processing tasks, have common
attributes in terms of e.g., number of inputs, number of outputs, how to call
their processing methods, ... Hence all aspects of initialisation (and re-
initialisation following a response to feedback) and input/output routing have
been implemented for common-cases as methods of the parent Processor
class.
If your processor does not behave similarly to others in one of these regards,
then this approach allows you to redefine the specific method in your new
children processor class definition. In the object oriented jargon, this
procedure is called method overriding.
In the following, we list the methods that might need overriding and how to do
so. Subsections for each methods will start with a description of what the
method does and a note explaining in which cases the method needs to be
overridden, such that you can quickly identify if this is necessary for your
processor. Some examples of existing processors that override a given method
will also be given so they can be used as examples. Note that all non- static
methods from the parent Processor
class can be overridden if necessary. The
following list only concerns methods that were written with overriding in mind
to deal with particular cases.
Note
Overridden methods need to have the same method attribute(s) as the parent method they are overriding.
Initialisation methods¶
verifyParameters
¶
This method is called at the end of the Processor
super-constructor. It
ensures that user-provided parameters are valid. The current implementation of
the Auditory front-end relies on the user being responsible and aware of which type or values
are suitable for a given parameter. Therefore, we do not perform a systematic
check of all parameters. Sometimes though, you might want to verify that user-
provided parameters are correct in order to avoid Matlab returning an error at a
later stage. For example, ihcProc.verifyParameters
will check that the inner
hair-cell model name entered by the user is part of the list of valid names.
Another use for the verifyParameters
method is to solve conflicts between
parameters. For example, the auditory filterbank in gammatoneProc
can be
instantiated in three different ways (e.g., by providing a range of frequency
and a number of channels, or directly a vector of centre frequencies). The user-
provided parameters for this processor are therefore potentially “over-
determining” the position of centre frequencies. To make sure that there is no
conflict, some priority rules are defined in gammatoneProc.verifyParameters
to ensure that a unique and non-ambiguous vector of centre frequencies is
generated.
Note
This method does nothing by default. Override it if you need to perform specific checks on external parameters (i.e., the user-provided parameters extended by the default values) before instantiating your processor.
To override this method, place it in a methods block with the
Access=protected
attribute. The method takes only an instance of the
processor object (say, pObj
) as input argument, and does not return any
output.
If you are checking that parameters have valid values, replace those which are
invalid with their default value in pObj.parameters.map
(see e.g.,
ihcProc.verifyParameters
). It is a good practice here to inform the user by
returning a warning, so that he/she knows that the default value is used
instead.
If you are solving conflicts between parameters, set up a priority rule and only
retain user-provided parameters that have higher priority according to this rule
(see e.g., gammatoneProc.verifyParameters
). Mention explicitly this rule in
the help line of your processor constructor.
prepareForProcessing
¶
This method performs the remaining initialisation steps that we purposely did
not include in the constructor as they initialise properties that are
susceptible to change when receiving feedback. It also includes initialisation
steps that can be performed only once processors have been linked together in a
“processing tree”. For example, ildProc
needs to know the original sampling
frequency of the signal before its cross-correlation was computed to provide lag
values in seconds. But to access the cross-correlation processor and request
that value, the two processors need to be linked together already, which does
not happen at the level of instantiation but later. Hence this method will be
called for each processors once they all have been inter-linked, but also
whenever feedback is received.
Note
Override this method if your processor has properties or internal parameters that can be changed via user feedback or that comes directly from preceding processors in the processing tree.
This method should have the Hidden=true
method attribute. Hidden methods are
sometimes used in the Auditory front-end when we need public access to it (i.e., other
objects than the processor itself should be able to call the method) but when it
is not deemed necessary to have the user call it. The user can still call the
method by explicitly writing its name, but the method will not appear in the
list of methods returned by Matlab script methods(.)
nor by Matlab’s
automatic completion.
The method only takes an instance of the processor as input argument and does
not return outputs. In the method, you should initialise all internal parameters
that are susceptible to changes from user feedback. Note that this includes the
processor’s output sampling frequency FsHzOut
if this frequency depends on
the processor parameters. A good example is
ratemapProc.prepareForProcessing
, which initialises internal parameters
(framing windows), the output sampling frequency and some filters.
instantiateOutput
¶
This method is called just after a processor has been instantiated to create a signal object that will contain the output of this new processor and add the signal to the data object.
Note
Override this method if your output signal object constructor needs additional
input arguments (e.g., for a FeatureSignal
), if your processor generates
more than one type of output, or if your processor can generate either mono or
stereo output (e.g., the current preProc
). There is no processor in the
current implementation that generates two different outputs. However, the pre-
processor can generate either mono or stereo outputs depending on the number
of channels in the input signal (see preProc.instantiateOutput
for an
example).
This method should have the Hidden=true
method attribute. It takes as input
an instance of your processor and a instance of a data object to add the signal
to. It returns the output signal object(s) as a cell array with the usual
convention that first column is left channel (or mono) and right column is right
channel. Different lines are for different types of signals.
Warning
Because there is no such processor at the moment, creating a new processor that returns two different types of output (and not just left/right channels) might involve additional changes. This is left to the developers responsibility to test and adjust existing code.
Input/output routing methods¶
When the manager creates a processing “tree”, it also populates the Input
and Output
properties of each processors with handles to their respective
input and output signal objects. The methods defined in the parent Processor
should cover most cases already, and it is unlikely that you will have to
override them for your own processor. For these two methods, it is important to
remember the internal convention when storing multiple signals in a cell array:
columns are for audio channels (first column is left or mono and second column
is right). Different lines are for different types of signals.
The way Input
and Output
properties are routed should be in accordance
with how they are used in the initiateProcessing
method, which will be
described in the next subsection.
addInput
¶
This method takes an instance of the processor and a cell array of handles to
dependent processors (i.e., processors one level below in the processing tree)
and does not return any arguments. Instead, it will populate the Input
property of your processor with a cell array of handles to the signals that are
outputs to the dependent processors. The current implementation of
Processor.addInput
works for three cases, which overall cover all currently
existing processors in the Auditory front-end:
- There is a single dependent processor which has a single output.
- There are two dependent processors each with single output corresponding to the left and right channels of a same input signal.
- There is a single dependent processor which produces two outputs: a left and
a right channel (such as
preProc
for stereo signals).
Note
Override this method if your processor input signals are related to its dependent processors in a different way than the three scenarios listed above.
This method should have the Hidden=true
attribute. You should just route the
output of your dependent processors to the input of your new processor
adequately. Again, it was not necessary thus far to override this method, hence
no examples can be provided here. Additionally, this functionality has not been
tested, so it might imply some minor reworking of other code components.
addOutput
¶
This method adds a signal object (or a cell array of signals) to the Output
property of your processor.
Note
Override this method if your processor has multiple outputs of different types. If your processor returns two outputs as the left and right channel of a same representation, it is not necessary to override this method.
This method should have the Hidden=true
method attribute. It takes as input
an instance of the processor and a single or a cell array of signal objects.
Processing method¶
initiateProcessing
¶
This method is closely linked to the addInput
, addOutput
and
processChunk
methods. It is a wrapper to the actual processing method that
routes elements of the cell arrays Input
and Output
to actual inputs and
outputs of the processChunk
method and call that method. It also appends the
new chunk(s) of output to the corresponding output signal(s).
The parent implementation considers two cases: monaural and binaural (i.e., a “left” and a “right” inputs) which produce single outputs.
Note
Override this method if your processor is not part of the two cases above or
if your implementation of the processChunk
has a different signature than
the standard.
A good example of an overridden initiateProcessing
method can be found in
preProc.initiateProcessing
, as the processing method of the pre-processor
does not have a standard signature as it returns two outputs (left and right
channels).
Allowing alternative processing options¶
Sometimes, two different processors (implemented as two different classes) can
perform the same operation. The choice between such alternative processors is
made depending on a given user-provided (or default) request parameter value.
This is the case for example for the auditory filterbank, which can be performed
by either a Gammatone filterbank (gammatoneProc.m
) or a dual-resonance non-
linear filterbank (drnlProc.m
).
As can be seen when browsing parameterHelper
, the two processors should be
listed under the same request name, and one of the parameters ('fb_type'
in
the example above) should allow to switch between the two (or more)
alternatives. When the manager instantiates the processors and notices that a
given representation has alternative ways of being computed, it will call the
methods isSuitableForRequest
of each alternatives to know which one should
be used.
Therefore, if your processor represents an alternative way of carrying out a
given operation, you should implement its isSuitableForRequest
method, as
well as for its alternative, if it was not already existing.
This method takes as unique input an instance of a processor and will look into
its parameters
property to determine if it is the suitable alternative. It
will return a boolean indicating if it is suitable (true
) or not
(false
). Note that this method is called internally, not from an actual
processor instance that would be used afterwards, but from a dummy, empty
processor generated using the user-provided request and parameters.
See as examples gammatoneProc.isSuitableForRequest
and
drnlProc.isSuitableForRequest
.
Implement a new signal type¶
The Auditory front-end supports already a wide range of signal types:
TimeDomainSignal
: used for single-dimensional signalTimeFrequencySignal
: used for two-dimension signals (time and frequency)CorrelationSignal
: used for three-dimension signals (time, frequency and lags)ModulationSignal
: used for three-dimension signals (time, audio frequency and modulation frequency).FeatureSignal
: used for a labelled collection of time-domain signalsBinaryMask
: used for two-dimensional (time and frequency) binary signals (0 or 1).
If your new processor generates a new type of signal that is not currently supported, you might have to add your own implementation of a new signal. This tutorial will not go in details on how to implement new signal types. However, the following aspects should be considered:
- Your signal class should inherit the parent
Signal
class. - It should implement the abstract
plot
method. If there is no practical way of plotting your signal, this method could be left empty. - Its constructor should take as argument a handle to your new processor (that
generates this signal as output), a buffer size in seconds, and a vector of
size across the other dimensions (
[size_dim2, size_dim3,...]
). If more arguments are needed (as is the case forFeatureSignal
), then this signature can be changed, but theinstantiateOutput
of your processor should also be overridden.
Recommendations for final testing¶
Now the implementation of your new processor should be finalised, and it is important to test it thoroughly. Below are some recommendations with regard to testing:
- Make sure that all aspects of your implementation work. Test for mono as well as stereo input signals, vary your processor parameters and check that the change is reflected accordingly in the output.
- If you have based your implementation on another existing implementation (even better, one that is documented in the literature), then compare your new implementation with the reference implementation and control that both provide the same output up to a reasonable error. A reasonable error, for a processor that does not involve stochastic processes should be around quantisation error, assuming that your new implementation is exactly as the reference.
- Test the online capability of your processor (i.e., maintaining the
continuity of its output) by processing a whole signal and the same signal
cut into chunks. Both runs should provide the same output (up to a
“reasonable error”). You can use the test script
test_onlineVSoffline
to perform that task.