Modules of the Two!Ears Auditory Model

Below you see an overview over the architecture of the complete Two!Ears Auditory Model. This architecture allows to retrieve binaural ear signals in two different ways: on the one hand, a dummy head mounted on a Robotic platform can be used to capture real-world audio input for follow-up processing. On the other hand, the Binaural simulator can be used to simulate artificial ear signals. In both cases, our auditory model allows for active listening by, e.g., rotating the listener’s head.

Based on the above, classic auditory cues can be extracted from the sampled ear signals in a completely block-based manner, using the Auditory front-end which could also act as a standalone module. However, our model’s inference capabilities are not only based on such bottom-up features; instead, the Two!Ears system incorporates top-down feedback methods and a Blackboard system to allow for scene understanding in given scenarios. The blackboard coordinates several expert units, so called Knowledge sources that interpret the extracted auditory cues based on internal world knowledge and try to disambiguate complex input by performing sophisticated active exploration patterns.

../../_images/modules.png

Fig. 1 Modules of the Two!Ears Auditory Model.