A deep learning framework for hybrid linguistic-paralinguistic speech systems

There are two aspects to speech: the linguistic aspect and the paralinguistic aspect. Hybrid linguistic-paralinguistic speech systems that employ joint knowledge of both aspects may fundamentally contribute to better speech processing and understanding, and improve the recognition of both the linguistic and paralinguistic aspects. However, formulating such a hybrid system is a non-trivial task due to unsolved challenges in each of its constituting systems as well as challenges that arise from attempting to integrate such systems. We provide a framework to assess the feasibility of the proposed integration using transfer learning and propose a formulation to hybrid linguistic- paralinguistic speech systems using deep learning.