Add your own knowledge sources

If you want to add some new functionality to the Two!Ears Auditory Model the best way to do this is to add a knowledge source. This implies of course that the auditory cues or data that your new knowledge source will need is already available. To check this first, have a look at all the available processors of the Auditory front-end. If the cues you need are not provided then read on how to add them. Otherwise you can go on and create your own knowledge source. For understanding the full potential that is available with knowledge sources like dynamic data initialisation of other knowledge sources or data requests you should familiarise yourself first with the blackboard architecture.

Example of adding a new knowledge source

The following is a step-by-step tutorial on how to implement a new knowledge source to be used within the blackboard architecture. As an example, the implementation of a knowledge source that performs a simple localisation task is considered here. The knowledge source will take interaural time differences computed by the Auditory front-end as inputs and produce a hypothesis about the mostly likely position of a sound source which will be put on the blackboard. The first step of the implementation is to set-up a class SimpleLocalisationKS, including a constructor and a destructor method and two additional methods canExecute and execute. Certain constraints on the execution of a specific knowledge source have to be specified in the canExecute method. The execute method contains the actual algorithm which will be computed if the knowledge source is executed by the blackboard scheduler. As the processing for this example class is dependent on data from the Auditory front-end as inputs, it has to be inherited from the AuditoryFrontEndDepKS superclass:

classdef SimpleLocalisationKS < AuditoryFrontEndDepKS
    % This is the basic skeleton structure to be used for setting
    % up a new KS class.

    properties (SetAccess = private)
        % Properties of the class can be specified here

        function obj = SimpleLocalisationKS()
            % This is the class constructor. Parameters of the KS
            % can be initialized here.

        function delete(obj)
            % Class destructor for cleaning up after processing.
            % This function is optional.

        function [bExecute, bWait] = canExecute(obj)
            % Put constraints here that prevent the KS from being
            % executed by the scheduler. Otherwise just put

            bExecute = true;    % KS can always be executed
            bWait = false;

        function execute(obj)
            % Put algorithm here

After the basic structure of the class has been set up, the individual functions can be implemented. In this example, a simple localisation algorithm based on a geometric model of the head is considered. The parameters that are needed for the computation are the distance between the two microphones, the speed of sound, and the size of one signal block in seconds. These values can be specified within the class properties as follows:

properties (SetAccess = private)
    micDistance;    % Microphone distance in meters
    speedOfSound;   % Speed of sound in m/s
    blockSizeSec;   % Size of one audio block in seconds

These parameters can either be specified by the user in the class constructor, or can be transferred to the class constructor as input arguments. Additionally, the class constructor must contain a specification of the parameters that should be used for data processing within the Auditory front-end, to compute the corresponding ITD values from the available binaural audio data:

function obj = SimpleLocalisationKS(micDistance)
    % Class constructor for the example KS.

    % Assign specified microphone distance to class properties
    obj.micDistance = micDistance;

    % For simplicity, block size and speed of sound will be
    % hard-coded here
    obj.blockSizeSec = 0.1;
    obj.speedOfSound = 343;

    % Generate parameter structure for the AFE
    afeParams = genParStruct( ...
        'fb_type', 'gammatone', ...
        'fb_lowFreqHz', 80, ...
        'fb_highFreqHz', 8000, ...
        'fb_nChannels', 32, ...
        'cc_wSizeSec', 20E-3, ...
        'cc_hSizeSec', 10E-3);

    % Specify requested signal features
    requests{1}.name = 'itd';
    requests{1}.params = param;

    % Initialize AFE
    obj = obj@AuditoryFrontEndDepKS(requests);

The knowledge source is now able to compute ITDs from incoming binaural audio signals and make this data available for further processing within the class. The actual processing happens in the execute method, which can contain arbitrary algorithms and processing steps. In this case, using simple geometric mapping from ITDs to azimuth locations, the execute method can be specified as:

function execute(obj)
    % Get ITDs from the AFE
    afeData = obj.getAFEdata();
    itdObj = afeData(1);
    itds = itdObj.getSignalBlock(obj.blockSizeSec, obj.timeSinceTrigger)';

    % Compute average ITD value over time and frequency
    avgItd = mean(mean(itds));

    % Map ITD to position
    azimuth = asind(avgItd * obj.speedOfSound / obj.micDistance);

    % Generate a new location hypothesis and put it on the blackboard.
    % For simplicity, head orientation is assumed to be fixed to 0°
    % in this case.
    headOrientation = 0;
    aziHyp = SourcesAzimuthsDistributionHypothesis(headOrientation, azimuth, 1);
    obj.blackboard.addData('sourcesAzimuthsDistributionHypotheses', ...
        aziHyp, false, obj.trigger.tmIdx);

    % If desired show some debugging output, if Blackboard is started in
    % verbose mode with BlackboardSystem(1);
    bbprintf(obj, ['[SimpleLocalisationKS:] Created ', ...

    % Trigger the event that is KS has been executed
    notify(obj, 'KsFiredEvent', BlackboardEventData(obj.trigger.tmIdx));

The class destructor and the canExecute method will be left unchanged in this example. An explanation of how these methods work in detail can be found in the blackboard architecture section. Having defined all four methods as described above, the knowledge source is now ready to be used within the Blackboard system.