Description

The Advanced Operating Systems and Virtualization course (6 CFU) aims at presenting advanced design/implementation methods and techniques for modern Operating Systems, including the support for system virtualization, and several security aspects. The topics dealt with in the course are bound to case studies oriented to Linux systems and x86 processors. During the course, we will see the internals from Linux 2.4, 2.6, 3.0, 4.0, and 5.0.

The course requires basic knowledge of the organisation and services offered by modern Operating Systems, of data structures, and of C/assembly programming. Some insights on the x86 computing architectures will help at bootstrapping the new topics shown in the course.

News  feed

Lectures

The reference teachers for this course are:

  • Prof. Roberto Beraldi, course owner
  • Ing. Gabriele Proietti Mattia

For attending the course you need to join to the Google Classroom by using this invite link and your Sapienza G Suite account.

Schedule

The course starts on 24th February, 2021 and ends on 28th May, 2021. During the week, two lectures are held in mixed mode (lecture in presence + Google Meet):

  • Wednesday, Room A3, 17-19
  • Friday, Room A3, 8-11

For attending the course in presence you need to follow the COVID19 Sapienza general rules for phase-3 available at this link. Room A3 is located at the Department of Computer, Control and Management Engineering “Antonio Ruberti” (Building RM102), Via Ariosto 25, Roma (00185).

Plan and material

# Date Topics Ref
1 24/02/2021
17.00 ~ 19.00
0. Introduction
  • Course Outline
  • A first glance on the boot process
1. x86 Boot
  • 1. Step1: BIOS/UEFI
  • 1.1 Pre-Boot and Real Mode
  • 1.2 BIOS
  • 2. Step 2: Stage 1 Bootloader
  • 2.1 MBR
  • Slides T#0
  • Slides T#1, up to 27
2 26/02/2021
08.00 ~ 11.00
1. x86 Boot
  • 2.1 MBR
  • 2.2 x86 Protected Mode
  • 2.3 x86 Memory Addressing
  • 2.4 x86 Privileges and Protection
Lab01 - Git: an essential guide
  • Slides T#1, up to 51
  • Slides L#1
3 03/03/2021
17.00 ~ 19.00
1. x86 Boot
  • 2.4 Paging
  • 3. Step 3: Stage 2 Bootloader
  • 3.1 GRUB/UEFI
  • 3.2 Multi-core Support
Slides T#1, up to end
4 05/03/2021
08.00 ~ 11.00
2. Step 4: Kernel Boot
  • 1. Initial Life of the Linux Kernel
  • 2. startup_32()
Lab02 - Building the Kernel
  • Slides T#2, up to 18
  • Slides L#2
5 10/03/2021
17.00 ~ 19.00
2. Step 4: Kernel Boot
  • 3. start_kernel()
  • 3.1 A Primer on Memory Organization
  • 3.2 Bootmem and Memblock Allocators
  • 3.3 Paging Introduction
  • 3.4 Paging Initialization
Slides T#2, up to 62
6 12/03/2021
08.00 ~ 11.00
2. Step 4: Kernel Boot
  • 3.4 Paging Initialization
  • 3.5 TLB
  • 3.6 Final Operations and Recap
3. Memory Management
  • 1. Memory Representation
Lab03 - ASM in C
  • Slides T#2, up to end
  • Slides T#3, up to 12
  • Slides L#3
7 17/03/2021
17.00 ~ 19.00
3. Memory Management
  • 3.2 The Buddy System
  • 3.3 High Memory
  • 3.4 Memory Finalization
  • 3.5 Steady-state memory allocation
  • 3.5.1 Fast Allocations & Quicklists
Slides T#3, up to 50
8 19/03/2021
08.00 ~ 11.00
3. Memory Management
  • 3.5.2 SLAB Allocator
  • 3.5.3 CPU Caches
  • 3.5.4 Large Allocations
  • 3.6 User & Kernel Space
Lab04 - Kernel Modules
  • Slides T#3, up to end
  • Slides L#4
9 24/03/2021
17.00 ~ 19.00
4. System Calls
  • 4.1 Introduction
  • 4.2 Handler / Dispatcher
  • 4.3 Invoking Process
  • 4.3.1 User Space Invoking process
  • 4.3.2 Kernel Wrapper Routines
Slides T#5, up to 34
10 26/03/2021
08.00 ~ 11.00
4. System Calls
  • 4.3.2 Kernel Wrapper Routines
  • 4.3.3 X86_64 Invoking Process
  • 4.4 vDSO
  • 4.5 Conclusions
Lab05 - Kernel Messaging and Debugging
  • Slides T#4, up to end
  • Slides L#5
11 31/03/2021
17.00 ~ 19.00
5. Interrupts
  • 5.1 Introduction
  • 5.2 IRQs and Inter-Processor Interrupts
  • 5.3 The IDT and the Activation Scheme
  • 5.4 Exception Handling
  • 5.4.1 Fixups and Page Fault Handler
Slides T#5, up to 30
- 02/04/2021 No Lecture -
12 07/04/2021
17.00 ~ 19.00
5. Interrupts Management
  • 5.4.1 Fixups and Page Fault Handler
  • 5.5 Interrupts Handling
  • 5.5.1 I/O Interrupts
  • 5.5.2 Inter-Processor Interrupts (IPIs)
  • 5.6 Software Interrupts (SoftIRQs) and Tasklets
Slides T#5, up to 65
13 09/04/2021
08.00 ~ 11.00
5. Interrupts Management
  • 5.6 Software Interrupts (SoftIRQs) and Tasklets
  • 5.7 Work Queues
6. Time Management
  • 6.1 Introduction
  • 6.2 Timekeeping Architecture
  • 6.2.1 Low-resolution Timers
Lab06 - Final project presentation and organization
  • Slides T#5, up to end
  • Slides T#6, up to 13
  • Slides L#6
14 14/04/2021
17.00 ~ 19.00
6. Time Management
  • 6.2.1 Low-resolution Timers
  • 6.2.1 Generic Time Subsystem
  • 6.3 Watchdogs
7. Concurrency in the Kernel
  • 7.1 Introduction
  • 7.2 Synchronization
  • 7.2.1 Per-CPU Variables
  • 7.2.2 Atomic operations
  • 7.2.3 Memory Barriers
  • 7.2.4 Spinlocks
  • 7.2.5 Seqlocks
  • Slides T#6, up to end
  • Slides T#7, up to 31
15 16/04/2021
08.00 ~ 11.00
7. Concurrency in the Kernel
  • 7.2.6 RCU
  • 7.2.7 Semaphores
8. Virtual Filesystem
  • 8.1 Introduction
  • 8.2 The Common File Model
Lab07 - Kernel Data Structures
  • Slides T#7, up to end
  • Slides T#8, up to 23
  • Slides L#7
16 21/04/2021
17.00 ~ 19.00
8. Virtual Filesystem
  • 8.2 The Common File Model
  • 8.2.1 Operations
  • 8.3 Pathname Lookup
  • 8.4 Files
  • 8.5 The /proc filesystem
  • 8.6 The /sys filesystem
  • 8.7 Device Management
  • Slides T#8, up to 65
17 23/04/2021
08.00 ~ 11.00
8. Virtual Filesystem
  • 8.7 Device Management
  • 8.7.1 Char Devices
  • 8.7.2 Block Devices
  • 8.7.3 Devices and VFS
  • 8.7.4 Classes
  • 8.7.5 Udev
9. Userspace Initialization
  • 9.1 init
  • 9.2 runlevels/targets
  • 9.2.1 Systemd
  • 9.3 End of the boot process
Lab08 - Misc devices, ioctl and /proc filesystem
  • Slides T#8, up to end
  • Slides T#9
  • Slides L#8
18 28/04/2021
17.00 ~ 19.00
10. Process Management
  • 10.1 Process Control Block
  • 10.1.1 Accessing the PCB
  • 10.2 The fork()/exec() model
  • 10.2.1 Kernel Threads
  • 10.3 Out Of Memory (OOM) Killer
Slides T#10, up to 46
19 30/04/2021
08.00 ~ 11.00
10. Process Management
  • 10.4 Process Starting
  • 10.4.1 The ELF Format
  • 10.4.2 Dynamic Linking
Lab09 - Function Hooking: Kprobes and ftrace
  • Slides T#10, up to 81
  • Slides L#9
20 05/05/2021
17.00 ~ 19.00
10. Process Management
  • 10.4.3 Initial Steps of Programs’ Life
11. Scheduling
  • 11.1 Introduction
  • 11.2 Priorities and Weights
  • 11.3 Scheduler Core
  • 11.3.1 Wait Queues
  • 11.3.2 Scheduler Entry Point
  • Slides T#10, up to end
  • Slides T#11, up to 36
21 07/05/2021
08.00 ~ 11.00
11. Scheduling
  • 11.3.3 Scheduler Algorithms
  • 11.4 Context Switch
12. Virtualization
  • 12.1 Introduction
  • Slides T#11, up to end
  • Slides T#12, up to 7
22 12/05/2021
17.00 ~ 19.00
- -
23 14/05/2021
08.00 ~ 11.00
- -
24 19/05/2021
17.00 ~ 19.00
- -
25 21/05/2021
08.00 ~ 11.00
- -
26 26/05/2021
17.00 ~ 19.00
- -
27 28/05/2021
08.00 ~ 11.00
- -

Slides

Slides can be found in the public shared folder of the course. Slides and the content of this site are protected by the Creative Common License (CC NC-BY-SA 4.0).

Theory

  1. Introduction
  2. x86 Boot
  3. Kernel Boot
  4. Memory Management
  5. System Calls
  6. Interrupts Management
  7. Time Management
  8. Concurrency in the Kernel
  9. Virtual Filesystem
  10. User Space Initialization
  11. Process Management
  12. Scheduling
  13. Virtualization
  14. Security
  15. Linux history and other kernels overview

Lab

  1. Git: an essential guide
  2. Building the Kernel
  3. ASM in C
  4. Kernel Modules
  5. Kernel Messaging & Debugging
  6. Final project presentation and organization
  7. Kernel Data Structures
  8. Misc devices, ioctl and /proc filesystem
  9. Function Hooking: Kprobes and ftrace
  10. Analysis of a rootkit

Exams

The exam consists of a written test and a practical project. The final mark is the weighted average of the scores obtained at the written test and at the project. The written test is worth 3/5 of the final mark, while the project is worth 2/5 of the final mark. Rejecting the final mark is possible, but requires the student to pass the written test again and hand over a different project.

Written part

The written part consists of open questions about all the topics presented within the lectures. The following dates refer to the written exam that you need to book on InfoStud. During the written exam no material is allowed, of any kind. The mode of operation (online/mixed/presence) will be updated as soon as available. Please do not book on Infostud but follow the link on the Book column (see also News section), you need to book on Infostud only if you are planning to discuss the project (see next section).

Call Date / Time Room Mode Book Results
0* April 15th, 2021 - Online - -
1 June 7th, 2021 @ 16.00 A3 Presence** Form -
2 July 7th, 2021 @ 16.00 B2 TBD - -
3 September 2nd, 2021 @ 16:00 B2 TBD - -
4* October 2021 - - - -
5 January 2022 - - - -
6 February 2022 - - - -

* extraordinary exam session / past editions of the course;
** exam will be held in presence for everyone except for those who has special needs, every case will be evaluated as specified by Sapienza rules.

Project part

The project requires the student to develop some new services within the Linux kernel. Instructions for project assignment will be given during the lectures. A one-year grace period since the last lecture is granted to hand off the project. The project can be handed off only after having passed the written test.

For developing the project you need a GitHub account and to join the course GitHub classroom, see this course news for the track and all of the information that you may need.

When you planned to discuss the project you need to book the proper exam call on Infostud and to send an email to the teacher Proietti Mattia according to the time range in which you want to discuss the project. For instance, if you intend to present the project in June, write an email to [email protected] and book for the #1 exam call before the 3rd June:

Call Date / Time Book Deadline Project discussion range
0* April 15th, 2021 - -
1 June 7th, 2021 @ 16.00 June 3rd, 2021 June 7th ~ June 30th, 2021
2 July 7th, 2021 @ 16.00 TBD July 7th ~ July 31th, 2021
3 September 2nd, 2021 @ 16:00 TBD Sept. 2nd ~ Oct. 31th, 2021
4* October 2021 - -
5 January 2022 - -
6 February 2022 - -
  • Google Classroom
  • Google Meet - join the Google Classroom then open the meet link under “Classwork”
  • Shared folder with course resources
  • Code Examples - GitHub repository with code examples presented at lectures

References

References and links that I provide here are meant to help the students increasing their understanding of the topics covered in this course. No unique handbook exists which covers all the presented topics. I hope this is an added value for the course, rather than a limitation. You can come to office hours to discuss about your doubts and to ask for clarifications.

  • Daniel P. Bovet, Marco Cesati, Understanding the Linux Kernel. O’Reilly.
  • Robert Love, Linux Kernel Development, Addison-Wesley Professional.
  • Mel Gorman, Understanding the Linux Virtual Memory Manager. Prentice Hall.
  • Alessandro Rubini, Jonathan Corbet, Linux Device Drivers, O’Reilly.
  • Mauerer, Wolfgang. Professional Linux kernel architecture. John Wiley & Sons, 2010.