Description

The Advanced Operating Systems and Virtualization course (6 CFU) aims at presenting advanced design/implementation methods and techniques for modern Operating Systems, including the support for system virtualization, and several security aspects. The topics dealt with in the course are bound to case studies oriented to Linux systems and x86 processors. During the course, we will see the internals from Linux 2.4, 2.6, 3.0, 4.0, and 5.0.

The course requires basic knowledge of the organisation and services offered by modern Operating Systems, of data structures, and of C/assembly programming. Some insights on the x86 computing architectures will help at bootstrapping the new topics shown in the course.

News  feed

Lectures

The reference teachers for this course are:

  • Ing. Gabriele Proietti Mattia
  • Prof. Roberto Beraldi, course owner

For attending the course you need to join to the Google Classroom by using this invite link and your Sapienza G Suite account.

Schedule

The course starts on 24th February, 2021 and ends on 28th May, 2021. During the week, two lectures are held in mixed mode (lecture in presence + Google Meet):

  • Wednesday, Room A3, 17-19
  • Friday, Room A3, 8-11

For attending the course in presence you need to follow the COVID19 Sapienza general rules for phase-3 available at this link. Room A3 is located at the Department of Computer, Control and Management Engineering “Antonio Ruberti” (Building RM102), Via Ariosto 25, Roma (00185).

Plan and material

# Date Topics Ref
1 24/02/2021
17.00 ~ 19.00
0. Introduction
  • Course Outline
  • A first glance on the boot process
1. x86 Boot
  • 1. Step1: BIOS/UEFI
  • 1.1 Pre-Boot and Real Mode
  • 1.2 BIOS
  • 2. Step 2: Stage 1 Bootloader
  • 2.1 MBR
  • Slides T#0
  • Slides T#1, up to 27
2 26/02/2021
08.00 ~ 11.00
1. x86 Boot
  • 2.1 MBR
  • 2.2 x86 Protected Mode
  • 2.3 x86 Memory Addressing
  • 2.4 x86 Privileges and Protection
Lab01 - Git: an essential guide
  • Slides T#1, up to 51
  • Slides L#1
3 03/03/2021
17.00 ~ 19.00
1. x86 Boot
  • 2.4 Paging
  • 3. Step 3: Stage 2 Bootloader
  • 3.1 GRUB/UEFI
  • 3.2 Multi-core Support
Slides T#1, up to end
4 05/03/2021
08.00 ~ 11.00
2. Step 4: Kernel Boot
  • 1. Initial Life of the Linux Kernel
  • 2. startup_32()
Lab02 - Building the Kernel
  • Slides T#2, up to 18
  • Slides L#2
5 10/03/2021
17.00 ~ 19.00
2. Step 4: Kernel Boot
  • 3. start_kernel()
  • 3.1 A Primer on Memory Organization
  • 3.2 Bootmem and Memblock Allocators
  • 3.3 Paging Introduction
  • 3.4 Paging Initialization
Slides T#2, up to 62
6 12/03/2021
08.00 ~ 11.00
2. Step 4: Kernel Boot
  • 3.4 Paging Initialization
  • 3.5 TLB
  • 3.6 Final Operations and Recap
3. Memory Management
  • 1. Memory Representation
Lab03 - ASM in C
  • Slides T#2, up to end
  • Slides T#3, up to 12
  • Slides L#3
7 17/03/2021
17.00 ~ 19.00
3. Memory Management
  • 3.2 The Buddy System
  • 3.3 High Memory
  • 3.4 Memory Finalization
  • 3.5 Steady-state memory allocation
  • 3.5.1 Fast Allocations & Quicklists
Slides T#3, up to 50
8 19/03/2021
08.00 ~ 11.00
3. Memory Management
  • 3.5.2 SLAB Allocator
  • 3.5.3 CPU Caches
  • 3.5.4 Large Allocations
  • 3.6 User & Kernel Space
Lab04 - Kernel Modules
  • Slides T#3, up to end
  • Slides L#4
9 24/03/2021
17.00 ~ 19.00
4. System Calls
  • 4.1 Introduction
  • 4.2 Handler / Dispatcher
  • 4.3 Invoking Process
  • 4.3.1 User Space Invoking process
  • 4.3.2 Kernel Wrapper Routines
Slides T#5, up to 34
10 26/03/2021
08.00 ~ 11.00
4. System Calls
  • 4.3.2 Kernel Wrapper Routines
  • 4.3.3 X86_64 Invoking Process
  • 4.4 vDSO
  • 4.5 Conclusions
Lab05 - Kernel Messaging and Debugging
  • Slides T#4, up to end
  • Slides L#5
11 31/03/2021
17.00 ~ 19.00
5. Interrupts
  • 5.1 Introduction
  • 5.2 IRQs and Inter-Processor Interrupts
  • 5.3 The IDT and the Activation Scheme
  • 5.4 Exception Handling
  • 5.4.1 Fixups and Page Fault Handler
Slides T#5, up to 30
- 02/04/2021 No Lecture -
12 07/04/2021
17.00 ~ 19.00
5. Interrupts Management
  • 5.4.1 Fixups and Page Fault Handler
  • 5.5 Interrupts Handling
  • 5.5.1 I/O Interrupts
  • 5.5.2 Inter-Processor Interrupts (IPIs)
  • 5.6 Software Interrupts (SoftIRQs) and Tasklets
Slides T#5, up to 65
13 09/04/2021
08.00 ~ 11.00
5. Interrupts Management
  • 5.6 Software Interrupts (SoftIRQs) and Tasklets
  • 5.7 Work Queues
6. Time Management
  • 6.1 Introduction
  • 6.2 Timekeeping Architecture
  • 6.2.1 Low-resolution Timers
Lab06 - Final project presentation and organization
  • Slides T#5, up to end
  • Slides T#6, up to 13
  • Slides L#6
14 14/04/2021
17.00 ~ 19.00
6. Time Management
  • 6.2.1 Low-resolution Timers
  • 6.2.1 Generic Time Subsystem
  • 6.3 Watchdogs
7. Concurrency in the Kernel
  • 7.1 Introduction
  • 7.2 Synchronization
  • 7.2.1 Per-CPU Variables
  • 7.2.2 Atomic operations
  • 7.2.3 Memory Barriers
  • 7.2.4 Spinlocks
  • 7.2.5 Seqlocks
  • Slides T#6, up to end
  • Slides T#7, up to 31
15 16/04/2021
08.00 ~ 11.00
7. Concurrency in the Kernel
  • 7.2.6 RCU
  • 7.2.7 Semaphores
8. Virtual Filesystem
  • 8.1 Introduction
  • 8.2 The Common File Model
Lab07 - Kernel Data Structures
  • Slides T#7, up to end
  • Slides T#8, up to 23
  • Slides L#7
16 21/04/2021
17.00 ~ 19.00
8. Virtual Filesystem
  • 8.2 The Common File Model
  • 8.2.1 Operations
  • 8.3 Pathname Lookup
  • 8.4 Files
  • 8.5 The /proc filesystem
  • 8.6 The /sys filesystem
  • 8.7 Device Management
  • Slides T#8, up to 65
17 23/04/2021
08.00 ~ 11.00
8. Virtual Filesystem
  • 8.7 Device Management
  • 8.7.1 Char Devices
  • 8.7.2 Block Devices
  • 8.7.3 Devices and VFS
  • 8.7.4 Classes
  • 8.7.5 Udev
9. Userspace Initialization
  • 9.1 init
  • 9.2 runlevels/targets
  • 9.2.1 Systemd
  • 9.3 End of the boot process
Lab08 - Misc devices, ioctl and /proc filesystem
  • Slides T#8, up to end
  • Slides T#9
  • Slides L#8
18 28/04/2021
17.00 ~ 19.00
10. Process Management
  • 10.1 Process Control Block
  • 10.1.1 Accessing the PCB
  • 10.2 The fork()/exec() model
  • 10.2.1 Kernel Threads
  • 10.3 Out Of Memory (OOM) Killer
Slides T#10, up to 46
19 30/04/2021
08.00 ~ 11.00
10. Process Management
  • 10.4 Process Starting
  • 10.4.1 The ELF Format
  • 10.4.2 Dynamic Linking
Lab09 - Function Hooking: Kprobes and ftrace
  • Slides T#10, up to 81
  • Slides L#9
20 05/05/2021
17.00 ~ 19.00
10. Process Management
  • 10.4.3 Initial Steps of Programs’ Life
11. Scheduling
  • 11.1 Introduction
  • 11.2 Priorities and Weights
  • 11.3 Scheduler Core
  • 11.3.1 Wait Queues
  • 11.3.2 Scheduler Entry Point
  • Slides T#10, up to end
  • Slides T#11, up to 36
21 07/05/2021
08.00 ~ 11.00
11. Scheduling
  • 11.3.3 Scheduler Algorithms
  • 11.4 Context Switch
12. Virtualization
  • 12.1 Introduction
  • Slides T#11, up to end
  • Slides T#12, up to 7
22 12/05/2021
17.00 ~ 19.00
12. Virtualization
  • 12.2 Software-based Virtualization
  • 12.2.1 VirtualBox
  • 12.3 Paravirtualization
  • 12.4 Hardware-assisted Virtualization
  • 12.4.1 Virtualization of Memory
  • 12.5 Linux Containers
  • 12.5.1 cgroups
  • 12.5.2 namespaces
Slides T#12, up to 46
23 14/05/2021
08.00 ~ 11.00
12. Virtualization
  • 12.5.2 namespaces
  • 12.5.3 Container Runtimes and Docker
13. Security
  • 13.1 Introduction
  • 13.2 User Authentication
  • 13.3 Internet Security
  • 13.4 Secure Operating Systems
  • Slides T#12, up to end
  • Slides T#13, up to end
24 19/05/2021
17.00 ~ 19.00
14. Epilogue
  • Introduction
  • Linux History
  • Kernels
Slides T#14, up to end
25 21/05/2021
08.00 ~ 11.00
No Lecture -
26 26/05/2021
17.00 ~ 19.00
No Lecture -
27 28/05/2021
08.00 ~ 11.00
No Lecture -

Slides

Slides can be found in the public shared folder of the course. Slides and the content of this site are protected by the Creative Common License (CC NC-BY-SA 4.0).

Theory

The Syllabus of the course can be found here.

  1. Introduction
  2. x86 Boot
  3. Kernel Boot
  4. Memory Management
  5. System Calls
  6. Interrupts Management
  7. Time Management
  8. Concurrency in the Kernel
  9. Virtual Filesystem
  10. User Space Initialization
  11. Process Management
  12. Scheduling
  13. Virtualization
  14. Security
  15. Epilogue

Lab

  1. Git: an essential guide
  2. Building the Kernel
  3. ASM in C
  4. Kernel Modules
  5. Kernel Messaging & Debugging
  6. Final project presentation and organization
  7. Kernel Data Structures
  8. Misc devices, ioctl and /proc filesystem
  9. Function Hooking: Kprobes and ftrace
  10. Analysis of a rootkit

Exams

The exam consists of a written test and a practical project. The final mark is the weighted average of the scores obtained at the written test and at the project. The written test is worth 3/5 of the final mark, while the project is worth 2/5 of the final mark. Rejecting the final mark is possible, but requires the student to pass the written test again and hand over a different project.

Written part

The written part consists of open questions about all the topics presented within the lectures. The following dates refer to the written exam that you need to book on InfoStud. During the written exam no material is allowed, of any kind. The mode of operation (online/mixed/presence) will be updated as soon as available. Please do not book on Infostud but follow the link on the Book column (see also News section), you need to book on Infostud only if you are planning to discuss the project (see next section).

Call Date / Time Room Mode Book Results
0* April 15, 2021 - Online - -
1 June 7, 2021 @ 16.00 A3 Presence** Closed Post
2 July 7, 2021 @ 16.00 B2 Presence** Closed Post
3 September 2, 2021 @ 16:00 B2 Presence** Closed Post
4* October 11, 2021 @ 16:00 - Online Closed Post
5 January 26, 2022 @ 8.30 - Online Closed -
6 February 11 2022 @ 8.30 B2 Presence** Closed -

* extraordinary exam session / past editions of the course;
** exam will be held in presence for everyone except for those who has special needs, every case will be evaluated as specified by Sapienza rules;

Remote exam rules

If your request for taking the exam remotely will be approved you need to prepare the following:

  • your id document with you photograph, that will be checked before the exam
  • your environment with doors closed, that will be checked before the exam
  • a webcam (that can be even your smartphone) that must point at you in which must be clearly visible:
    • the screen of your pc
    • your look
    • your hands

The day of the exam will be the same (if not told differently by the teacher), a Google Meet will be published in the Google Classroom which you will need to join to from your pc (and, if needed, even from your smartphone). The exam will be taken with exam.net, preferably with SEB, if you can install it (Windows or macOS), otherwise you can use any browser but in that case in the webcam the pc screen must be clearly visible. Please try to join at least 15 minutes before the exam starting time.

Project part

The project requires the student to develop some new services within the Linux kernel. Instructions for project assignment will be given during the lectures. A one-year grace period since the last lecture is granted to hand off the project. The project can be handed off only after having passed the written test.

For developing the project you need a GitHub account and to join the course GitHub classroom, see this course news for the track and all of the information that you may need.

When you planned to discuss the project you need to book the proper exam call on Infostud and to send an email to the teacher Proietti Mattia according to the time range in which you want to discuss the project. For instance, if you intend to present the project in June, write an email to [email protected] and book for the #1 exam call before the 3rd June:

Call Date / Time Book Deadline Project discussion range
0* April 15th, 2021 - -
1 June 7th, 2021 @ 16.00 June 3, 2021 June 7 ~ June 30, 2021
2 July 7th, 2021 @ 16.00 July 2, 2021 July 7 ~ July 31, 2021
3 September 2nd, 2021 @ 16:00 August 29, 2021 Sept. 3 ~ Oct. 1, 2021
4* October 11, 2021 @ 16.00 October 5, 2021 Oct. 11 ~ Oct. 31, 2021
5 January 26, 2022 @ 8.30 January 16, 2022 Jan. 10 ~ Jan. 31, 2022
6 February 11, 2022 @ 8.30 February 4, 2022 Feb. 1 ~ Mar. 1, 2022
  • Google Classroom
  • Google Meet - join the Google Classroom then open the meet link under “Classwork”
  • Shared folder with course resources
  • Code Examples - GitHub repository with code examples presented at lectures

References

References and links that I provide here are meant to help the students increasing their understanding of the topics covered in this course. No unique handbook exists which covers all the presented topics. I hope this is an added value for the course, rather than a limitation. You can come to office hours to discuss about your doubts and to ask for clarifications.

  • Daniel P. Bovet, Marco Cesati, Understanding the Linux Kernel. O’Reilly.
  • Robert Love, Linux Kernel Development, Addison-Wesley Professional.
  • Mel Gorman, Understanding the Linux Virtual Memory Manager. Prentice Hall.
  • Alessandro Rubini, Jonathan Corbet, Linux Device Drivers, O’Reilly.
  • Mauerer, Wolfgang. Professional Linux kernel architecture. John Wiley & Sons, 2010.