Application of Deep Learning in Visual Computing 

Detailed syllabus of the lectures

1. Introduction to computer vision, fundamental tasks and challenges. Mathematical foundations of image processing: convolution, Fourier transform, filtering in the frequency domain.

2. Learning-based vision, evaluation metrics. Linear classification, cost functions, optimization methods. Fundamentals of neural networks: modular backpropagation, multilayer neural networks.

3. Convolutional neural networks, network architectures commonly used in computer vision. Practical problems and techniques in learning-based vision.

4. Deep Learning in practice, ensuring convergence, avoiding overfitting. Hyperparameter search, model compression, pruning, and ensembles.

5. Types of segmentation, semantic segmentation methods, U-Net, upscaling techniques. ASPP and CRF extensions. Datasets, metrics, applications in autonomous vehicles, simulation solutions.

6. Detection architectures, R-CNN variants, YOLO. Important metrics and datasets, anchor-based and anchor-free solutions. Mask R-CNN and other R-CNN extensions. Visual intelligence and sensors of self-driving cars, recognition of driving-relevant objects.

7. Video analytics, event detection in videos, driver assistance systems. Types of attention: spatial and channel attention, self-attention, visual transformers.

8. Deep Learning in 3D, representation of spatial structures: voxels, point clouds, meshes. Volumetric networks, kd-trees, point networks, mesh networks. Multi-view applications.

9. Visualization of convolutional networks. Texture generation, image generation. Colorization of black-and-white images, style and domain transfer with convolutional neural networks.

10. Visual intelligence using reinforcement learning, DQN, REINFORCE, Actor-Critic. Creating agents for computer games, hard visual attention.

11. Disadvantages and limitations of supervised learning, possibilities for learning from limited data: sim2real, few-shot learning. Learning compact representations, self-supervised techniques: filling in missing image regions, object removal.

12. Neural rendering: combining classical computer graphics techniques with deep generative networks to produce controllable and photorealistic images. Differentiable rendering: integrating 3D color-space information into network training.

Detailed syllabus of the practicals/laboratories

• Traffic sign classification using convolutional neural networks.

• Semantic segmentation in a robot soccer environment.

• Object detection in a robot soccer environment.

• Video processing, event recognition, object tracking.

• Image generation using neural networks.

• Application of neural networks in robotics, reinforcement learning for solving control and cooperation problems.