Identification knowledge sources¶
This section focuses on implementation of sound identification knowledge sources within the blackboard framework.
Identity knowledge source: IdentityKS
¶
Objects of class IdentityKS
implement source type models, by incorporating
an instance of a model (which has to implement the models.Base
interface) with knowledge about the relationship of auditory cues and certain
sound source types. Many Identity knowledge sources can be used concurrently;
usually, for each sound class to be identified, you would instantiate an object
of class IdentityKS
with the respective model. The models get loaded from
directories you specify upon construction, and should be created with the
Sound identification training pipeline. The model object of IdentityKS
can
employ any kind of model, such as a linear support vector machine, or a Gaussian
mixture model. The IdentityKS
needs access to Auditory front-end signals, thus it is a
subclass of AuditoryFrontEndDepKS
(see
Section Auditory signal dependent knowledge source superclass: AuditoryFrontEndDepKS). The model object holds the signal
request structure.
The knowledge source predicts, based on the incorporated source model, whether the currently received auditory stream includes an auditory object of the sound type it represents.
binds to | AuditoryFrontEndKS.KsFiredEvent |
writes data category | identityHypotheses |
triggers event | KsFiredEvent |
Have a look at the example Identification of sound types to see IdentityKS
in action.
Identity decision knowledge source: IdDecisionKS
¶
The identity knowledge source checks new identity hypotheses. It then decides which of them are valid, by comparison and incorporating knowledge about the number of assumed auditory objects in the scene.
binds to | IdentityKS.KsFiredEvent |
reads data category | identityHypotheses |
writes data category | identityDecision |
triggers event | KsFiredEvent |
Identity Live Debugging knowledge source: IdTruthPlotKS
¶
This is not really a knowledge source in the sense of the word, but rather a way to enable live-inspection of the identity information in the blackboard system. Upon construction, it takes ground truth information about event labels, onset and offset times; and when triggered, displays this in comparison with the actual hypotheses created by the identity knowledge sources. This figure shows an example of the produced plot.
binds to | IdentityKS.KsFiredEvent |
reads data category | identityHypotheses |
Segment Identity knowledge source: SegmentIdentityKS
¶
The segment identity knowledge source checks new stream segregation hypotheses and
assigns an identity to each. This is performed by applying the mask estimated by
the StreamSegregationKS before performing inference on the source’s identity.
segIdentityHypotheses
are triggered for each source in the
segmentationHypotheses
binds to | SegmentationKS.KsFiredEvent |
reads data category | segmentationHypotheses |
writes data category | segIdentityHypotheses |
triggers event | KsFiredEvent |