Audio-Video Data Processing
winter semester, 2021-2022
Student groups: Computer-Science in Romanian 3rd year, Computer-Science in English 3rd year
Professors:
Conf. Dr. Adrian Sterca [forest at cs dot ubbcluj dot ro]
Objectives:
At the end of this course, students should have fundamental knowledge in:
- digital audio and video signal representation and processing
- audio-video codec standards like ISO's MPEG and ITU-T H.26[1234]
- audio-video streaming over best-effort networks
- satellite audio-video broadcasting
- computer vision intro
They should also have expertise in designing and implementing an audio-video
processing system.
The quiz re-examination will take place on Microsoft Teams on Monday, 14th February 2022, time 13.00.
The quiz exam will be effectively taken on the moodle.cs.ubbcluj.ro platform.
News [22nd-February-2022]: Final grades PDAV
If you have questions or spot errors in the grades, please drop me an email.
Course description:
- Introduction
- Introduction to multimedia
- Color spaces
- Analog representation of audio and video signals
- Digital teprezentation of still images. The JPEG compression standard
- Digital representation of audio signal
- Audio and video formats (containers), codecs
- AV containers: .avi, .ogg, .mp4, .vob, .3gp, .mkv
- MPEG-1,2,4
- H.264/MPEG-4 AVC and the SVC extension
- H.265, High Efficiency Video Coding (HEVC)
- VP9
- Streaming and signalling protocols
- RTP/RTCP - Real Time Protocol
- RTSP - Real Time Streaming Protocol
- SDP - Session Description Protocol
- SIP - Session Initiation Protocol
- HTTP Streaming, DASH
- Congestion control algorithms for audio-video applications in "best-effort" networks
- TCP AIMD (Additive Increase Multiplicative Decrease)
- DCCP - Datagram Congestion Control Protocol
- TFRC - TCP-Friendly Rate Control
- UTFRC - Utility-driven TCP-Friendly Rate Control
- Voice over IP
- Audio-video communication in satellite networks
- Satellite communications
- Frequency bands
- DBS - Direct Broadcast Satellite (Direct-To-Home)
- DVB standards: DVB-S (satellite), DVB-T (terrestrial), DVB-C (cable)
- The FFMpeg library
- libavcodec
- libavformat
- libswscale
- libavfilter
- Audio-video for the web
- Audio-Video support in HTML 5
- Real-Time Communication on the web (WebRTC)
- Introduction to Computer Vision
- What is Computer Vision ?
- Feature extraction & feature matching
- Image features: SIFT
- Image features: SURF
- Multimedia QoS in Internet
- QoS terminology
- QoS requirements in "best-effort" networks
- P2P TV
- Peer-2-Peer networks
- DHT - Distributed Hash Tables
- Applications
Course Requirements (grading criteria):
There are 3 possible grading criteria and each student is allowed to choose the one that suits him best:
- Project: Students should develop a project related to
audio-video data processing either individually or in small teams of 2 persons. Students must choose
the project theme in the beginning of the semester and they must present the project at the end of the semester.
During the semester, they must also show work progress and intermediate versions of the project at the labs.
The possible grades a student can receive on the project are: 10, 9 and 4. No other grades are possible.
Examples of project ideas are here: List of possible projects (in romanian).
Please note that the theme of
each project and also its developement must be discussed with me at the laboratory because there are
several ways of implementing a specific task (like the ones mentioned in the list of projects from above), some
of them are very easy (using a library) and some of them are more complicated. Generally, I prefer that you
work on your project not relying on an additional external library, but rather do the low-level stuff
yourself.
- Labs: Students must complete 3 laboratory tasks (i.e. 3 labs) during the
semester. The lab tasks require the students to build a part of a video codec
and are detailed here: Laboratory tasks.
The final grade for this course is the average lab grade.
- Quiz test: If the student does not get an average lab grade of at least 5 and he/she
does not do a project, he/she must take the quiz test and the grade he/she receives for this quiz test is the final grade. This grade
can not be greater than 6.