Identification knowledge sources

This section focuses on implementation of sound identification knowledge sources within the blackboard framework.

Identity knowledge source: IdentityKS

Objects of class IdentityKS implement source type models, by incorporating an instance of a model (which has to implement the models.Base interface) with knowledge about the relationship of auditory cues and certain sound source types. Many Identity knowledge sources can be used concurrently; usually, for each sound class to be identified, you would instantiate an object of class IdentityKS with the respective model. The models get loaded from directories you specify upon construction, and should be created with the Sound identification training pipeline. The model object of IdentityKS can employ any kind of model, such as a linear support vector machine, or a Gaussian mixture model. The IdentityKS needs access to Auditory front-end signals, thus it is a subclass of AuditoryFrontEndDepKS (see Section Auditory signal dependent knowledge source superclass: AuditoryFrontEndDepKS). The model object holds the signal request structure.

The knowledge source predicts, based on the incorporated source model, whether the currently received auditory stream includes an auditory object of the sound type it represents.

binds to AuditoryFrontEndKS.KsFiredEvent
writes data category identityHypotheses
triggers event KsFiredEvent

Have a look at the example Identification of sound types to see IdentityKS in action.

Identity decision knowledge source: IdDecisionKS

The identity knowledge source checks new identity hypotheses. It then decides which of them are valid, by comparison and incorporating knowledge about the number of assumed auditory objects in the scene.

binds to IdentityKS.KsFiredEvent
reads data category identityHypotheses
writes data category identityDecision
triggers event KsFiredEvent

Identity Live Debugging knowledge source: IdTruthPlotKS

This is not really a knowledge source in the sense of the word, but rather a way to enable live-inspection of the identity information in the blackboard system. Upon construction, it takes ground truth information about event labels, onset and offset times; and when triggered, displays this in comparison with the actual hypotheses created by the identity knowledge sources. This figure shows an example of the produced plot.

binds to IdentityKS.KsFiredEvent
reads data category identityHypotheses

Segment Identity knowledge source: SegmentIdentityKS

The segment identity knowledge source checks new stream segregation hypotheses and assigns an identity to each. This is performed by applying the mask estimated by the StreamSegregationKS before performing inference on the source’s identity. segIdentityHypotheses are triggered for each source in the segmentationHypotheses

binds to SegmentationKS.KsFiredEvent
reads data category segmentationHypotheses
writes data category segIdentityHypotheses
triggers event KsFiredEvent