Speech Information Systems

1. Introduction
Why is speech technology important? What are the main components of a speech information system (e.g. intelligent personal assistant)?

Language, speech and text in human communication. The elements of the natural speech chain and how they work. Basic concepts of human speech production, speech perception and understanding. The main features of the acoustic structure of speech. Levels of speech, redundancy, additional information carried.

2, 3, Elementary signal processing, speech coding and compression
The role of speech coding in digital speech storage and in infocommunication network systems. Understanding the elementary signal processing steps required for the transmission, storage and analysis of speech, taking into account the specific properties of speech. Distinguish between speech/silence and other acoustic signals. Basic methods of speech coding, familiarisation with current popular formats. The impact of coding on other speech technologies. Classification of coded speech (intelligibility, naturalness).

4, Speech response systems basics
Basic concepts of machine generated speech (fixed, unlimited and mixed vocabulary). Design aspects and implementation steps for a fixed vocabulary acoustic database. Reasons for designing mixed systems and possible solutions. High fidelity prosody modification algorithms. Construction and basic classes of text-to-speech and concept-to-speech systems.

5, 6 Importance of speech and text databases, Text processing techniques
Description, design and processing methods of databases. The role of the acoustic environment. Understanding the phases of recognition creation. Databases, Vocabulary, Automatic extension of databases, adaptivity. The role of prosody. Design of multilingual systems. Developer environments and tools. Creating databases optimised for machine learning. Punctuation handling, text pre- and post-processing.


7, 8, Advanced speech response systems
Unified text representation, text analysis and transformation tasks and related databases. Design aspects and methods of construction of acoustic databases with unlimited vocabulary. Design of a speech response corpus. Database creation, modification and their algorithms. The importance and implementation of prosody (pitch, volume, rhythm variation). Multi-voice systems and automatic voice conversion. Multilingual systems. Language detection, accentuation. Unified voice annotation systems. Development environments. Algorithms for automated implementation of systems, machine learning based solutions.

9, Speech-based classification
The concept and uses of speech classification. Basic concepts of voice-based speaker identification, speaker identification and verification, UBM, likelihood-ratio framework. Voice activity detection. Introduction to the mathematical basis of the methods used. Machine learning based methods. Use of speaker identification in practice.

10, 11 Speech recognition
Basic concepts and basic architectures of speech recognition. Reference-based pattern matching. Basic equation of MAP for speech recognition. Application of Hidden Markov Models (HMM) in speech-to-text conversion. Acoustic, pronunciation and language modelling, knowledge source integration in the Weighted Finite State Transducer (WFST) framework. Hybrid HMM-neural and end-to-end neural approaches for speech recognition. CTC (Connectionist Temporal Classification) learning. Advanced neural architectures for speech recognition. Self-supervised and unsupervised speech-to-text conversion techniques. Tailoring models for application, fine-tuning. Restoring text to written form. Offline and online speech recognition, dictation process.

12, Design and implementation steps for speech information systems.
Typical application environments, dominant application systems (e.g. customer service automation, healthcare, rehabilitation). The concept of corporate acoustic image and methods to ensure a high quality company image.

13, 14 Application of speech functions in information systems
Basic concepts of speech-based dialogue systems. System driven, user driven and mixed initiative systems. DTMF and speech recognition based control in speech response systems. Uni- and multimodal systems. Modality conversion and its role in global personal communication systems.  

Lab topics:
Lab 1: Basic speech acoustics, Energy measurement, modification, spectrum generation, analysis, F0 measurement
Lab 2: Quantization, Sampling, Speech Compression, Speech Editing
Lab 3: Speech synthesis
Lab 4: Speaker identification
Lab 5: Speech recognition
Lab 6: Dialogue systems  (e.g. chatbot)