A tantárgyleírás hatályossága
Hatályosság kezdete:
2026. March 21.
Hatályosság vége:
—
| Subject name (Hungarian, English) |
Információelmélet
Information Theory
|
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Subject code | BMEVISZMA03 | ||||||||||||
| Subject type | — | ||||||||||||
| Training Level | — | ||||||||||||
| Course types and hours (weekly/semester) |
|
||||||||||||
| Assessment type | félévközi érdemjegy | ||||||||||||
| Credits | 4 | ||||||||||||
| Subject coordinator |
DR. Pintér Márta Barbara
position: egyetemi docens
contact:
pinter.marta@vik.bme.hu
|
||||||||||||
| Responsible department |
Számítástudományi és Információelméleti Tanszék
|
||||||||||||
| Faculty | Villamosmérnöki és Informatikai Kar | ||||||||||||
| Subject website | — | ||||||||||||
| Primary curriculum type | — | ||||||||||||
| Direct prerequisites – Strong prerequisite | none | ||||||||||||
| Direct prerequisites – Weak prerequisite | none | ||||||||||||
| Direct prerequisites – Parallel prerequisite | none | ||||||||||||
| Direct prerequisites – Milestone prerequisite | none | ||||||||||||
| Direct prerequisites – Exclusion | none |
Objectives
Programme
1. Variable length source coding
Unique decodability, prefix coding
2. McMillan's theorem and Kraft's theorem
3. Jensen's inequality
The entropy function and its main properties
4. Shannon-Fano coding
Huffman coding
5. Lempel-Ziv type algorithms
6. The entropy of a source, Markov source
Conditional entropy and its properties
7. Mutual information and its properties
8. Quantization
9. Lloyd-Max algorithm
10. The discrete memoryless channel model
11. Channel capacity
Fano's inequality
12. Converse of the channel coding theorem
Channel coding theorem
13. Basic principles of error correction, Hamming codes
14. Zero-error codes.
Unique decodability, prefix coding
2. McMillan's theorem and Kraft's theorem
3. Jensen's inequality
The entropy function and its main properties
4. Shannon-Fano coding
Huffman coding
5. Lempel-Ziv type algorithms
6. The entropy of a source, Markov source
Conditional entropy and its properties
7. Mutual information and its properties
8. Quantization
9. Lloyd-Max algorithm
10. The discrete memoryless channel model
11. Channel capacity
Fano's inequality
12. Converse of the channel coding theorem
Channel coding theorem
13. Basic principles of error correction, Hamming codes
14. Zero-error codes.
The course deals with the theoretical problems arising during transfer
and storage of information. The theoretical limits of data compression
and reliable information transmission are presented. Basic properties
of Shannon's information measures are covered and several data
compression techniques are taught. Course topics include the main
principles of channel coding along with basic examples of situations
when such coding is required.
Students completing the course are supposed to
(1) know the theoretical limits of efficiency of variable length source coding
(2) know the main codes realizing the above limits
(3) be acquainted with the main principles of lossy source coding
(4) develop a basic understanding of the main concepts of classical
information theory
(5) be able to rightly model situations when the task is information
transmission in a noisy environment.
Learning outcomes
Ez a tantárgy a KKK rendeletben meghatározott, következő kompetenciák fejlesztését szolgálja:
Knowledge
No learning outcomes recorded.
Skills
No learning outcomes recorded.
Attitudes
No learning outcomes recorded.
Autonomy and responsibility
No learning outcomes recorded.
Oktatási módszertan
3 lectures per week
Tanulástámogató anyagok
Online források
Cover - Thomas: Elements of Information Theory, Wiley, 2006.
Recommended preliminary knowledge for completing the subject
Knowledge type competencies
(azon előzetes ismeretek összessége, amelyek megléte nem kötelező, de a tantárgy eredményes teljesítését nagyban elősegíti)
Probability Theory
Skill type competencies
(azon előzetes képességek és készségek összessége, amelyek megléte nem kötelező, de a tantárgy eredményes teljesítését nagyban elősegíti)
nincs
Recommended (non-compulsory) preliminary competencies
(azon ajánlott (nem kötelező) előzetesen megszerzendő kompetenciák összessége, amelyek jelentősen hozzájárulnak a tantárgy eredményes teljesítéséhez)
Probability Theory
General rules
Requirements:
There are 2 midterm tests during the semester. To complete the
course with a valid grade 40% of the total score should be achieved on
both of the midterms. If this requirement is met, the course
grade is calculated by averaging the results of the three midterms
with equal weights.
In the exam period: ---
Additional possibilities:
There will be a make up test for each of the three midterms during the
semester. One more make up test can be written on the week right after
the semester in case one (and only one) midterm is still below 40%.
Assessment methods
In-term assessments
No detailed assessments provided.
Weight of in-term assessments
No weights provided.
Exam-period assessments
No detailed assessments provided.
Weight of exam elements
No weights provided.
Grade calculation
No grade thresholds provided.
Attendance requirements
No attendance requirements provided.
Rules for retake and resubmission
Not provided.
Short description
Not provided.
Detailed description
Not provided.
Recommended courses
Probability Theory
Workload to complete the subject
No workload breakdown provided.
Validity of subject requirements
Requirements valid from:
—
Requirements valid until:
—
Curriculum placement
No curriculum placements recorded for this subject version.