Introduction to Python and Natural Language Technologies

Lecture Topic

Lab Topic

1

Introduction: what is natural language processing, typical applications, history, major areas of NLP/CL, relationship to computer science and linguistics

Setting up - using the provided VMs, git repository, basic NLP tools

2

Introduction to Python, basic syntax, built-in types, operators, functions, file manipulation

Using Jupyter. Writing simple functions. Handling text files.

3

Built-in types in detail (list, set, dict), immutability. Advanced string manipulation, encodings, Regular expressions.

Typical string manipulation exercises. Writing a simple parser with regular expressions.

4

Object-oriented Python, properties, static methods, class methods, magic methods, operator overloading. Iterators, generators. Context managers.

Complex OOP exercise. Writing our own iterator and context manager.

5

Decorators. Functional programming in Python. Writing command line applications in Python. Using Linux command line applications for text processing. IO redirection, pipelines.

Writing a simple command line application. Using it in the terminal, interacting with built-in Linux commands via pipes.

6

Scientific Python. Numpy, scipy. Basic matrix operations. Sparse matrices.

Matrix manipulation. Working with large sparce matrices.

7

Data science. Handling text data. Basic shell commands. Pandas.

Handling text data exercises. Data cleaning. Pandas exercises.

8

Deep learning for NLP. Feed forward neural networks, recurrent neural networks. LSTM, GRU.

PyTorch basics. Defining neural networks, training and evaluation loops

9

Textual sequence modeling: sequence labeling and classification, sequence-to-sequence models. Attention.

Sequence modeling in PyTorch.

10

Language modeling. Word vectors. Contextualized language models. Transformers. BERT.

Exploring pretrained models: word2vec, GloVe, fastText, BERT, ELMo

11

Dependency parsing. Universal dependencies.

Working with Universal Dependencies. Multilingual models and problems.

12

NLP applications I.: neural machine translation, sentiment analysis, question answering, summarization, dialogue.

NLP in practice. Using pretrained models for machine translation, question answering etc.

13

NLP applications II.: knowledge bases.

NLP in practice II. Homework consultation.

14

Practical session

Practical session