363 lines
12 KiB
TeX
363 lines
12 KiB
TeX
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
% Copyright (c) 2015, ETH Zurich.
|
|
% All rights reserved.
|
|
%
|
|
% This file is distributed under the terms in the attached LICENSE file.
|
|
% If you do not find this file, copies can be found by writing to:
|
|
% ETH Zurich D-INFK, Universitaetstr 6, CH-8092 Zurich. Attn: Systems Group.
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
|
|
\documentclass[a4paper,11pt,twoside]{report}
|
|
\usepackage{bftn}
|
|
\usepackage{calc}
|
|
\usepackage{verbatim}
|
|
\usepackage{xspace}
|
|
\usepackage{pifont}
|
|
\usepackage{pxfonts}
|
|
\usepackage{textcomp}
|
|
\usepackage{amsmath}
|
|
\usepackage{multirow}
|
|
\usepackage{listings}
|
|
\usepackage[framemethod=default]{mdframed}
|
|
\usepackage[shortlabels]{enumitem}
|
|
\usepackage{parskip}
|
|
\usepackage{xparse}
|
|
|
|
\newcommand{\todo}[1]{[\textcolor{red}{\emph{#1}}]}
|
|
|
|
\title{CPU drivers in Barrelfish}
|
|
\author{Barrelfish project}
|
|
\tnnumber{21}
|
|
\tnkey{CPU drivers}
|
|
|
|
\begin{document}
|
|
\maketitle % Uncomment for final draft
|
|
|
|
\begin{versionhistory}
|
|
\vhEntry{0.1}{01.12.2015}{GZ}{Initial Version}
|
|
\end{versionhistory}
|
|
|
|
% \intro{Abstract} % Insert abstract here
|
|
% \intro{Acknowledgements} % Uncomment (if needed) for acknowledgements
|
|
\tableofcontents % Uncomment (if needed) for final draft
|
|
% \listoffigures % Uncomment (if needed) for final draft
|
|
% \listoftables % Uncomment (if needed) for final draft
|
|
\cleardoublepage
|
|
\setcounter{secnumdepth}{2}
|
|
|
|
\newcommand{\fnname}[1]{\textit{\texttt{#1}}}%
|
|
\newcommand{\datatype}[1]{\textit{\texttt{#1}}}%
|
|
\newcommand{\varname}[1]{\texttt{#1}}%
|
|
\newcommand{\keywname}[1]{\textbf{\texttt{#1}}}%
|
|
\newcommand{\pathname}[1]{\texttt{#1}}%
|
|
\newcommand{\tabindent}{\hspace*{3ex}}%
|
|
\newcommand{\sockeye}{\lstinline[language=sockeye]}
|
|
\newcommand{\ccode}{\lstinline[language=C]}
|
|
|
|
\lstset{
|
|
language=C,
|
|
basicstyle=\ttfamily \small,
|
|
keywordstyle=\bfseries,
|
|
flexiblecolumns=false,
|
|
basewidth={0.5em,0.45em},
|
|
boxpos=t,
|
|
captionpos=b
|
|
}
|
|
|
|
\chapter{Introduction}
|
|
\label{chap:introduction}
|
|
|
|
This document describes the CPU driver, the part of Barrelfish that typically
|
|
runs in privileged mode (also known as kernel) on our supported architectures.
|
|
|
|
Barrelfish currently supports the following CPU drivers for
|
|
different CPU architectures and platforms:
|
|
\begin{itemize}
|
|
\item x86-32
|
|
\item x86-64
|
|
\item k1om
|
|
\item ARMv7
|
|
\item ...
|
|
\item ARMv8
|
|
\end{itemize}
|
|
|
|
\section{General design decisions}
|
|
\begin{itemize}
|
|
\item No dynamic memory allocation
|
|
\item No preemption
|
|
\item ...
|
|
\end{itemize}
|
|
|
|
\section{Code file structure and layout}
|
|
TODO: Should explain things such as naming, where goes architecture dependent, platform specific code?
|
|
What libraries we use in the kernel? Where is the shared code between libbarrelfish and a cpudriver?
|
|
|
|
\chapter{x86-64}
|
|
|
|
The x86-64 implementation of Barrelfish is specific to the AMD64
|
|
and Intel 64 architectures. This text will refer to features of
|
|
those architectures. Those and further features can be found in
|
|
\cite{intelsa} and \cite{amdsa} for the Intel 64 and AMD64
|
|
architectures, respectively.
|
|
|
|
\section{Boot process}
|
|
|
|
We first describe the boot process for the initial BSP core, followed by
|
|
the boot process of an APP core.
|
|
|
|
\subsection{BSP Core}
|
|
|
|
Barrelfish relies on a multiboot v1~\cite{multiboot1} compliant boot-loader to
|
|
load the initial kernel on the BSP core. In our current set-up we use GRUB as
|
|
our boot-loader which contains an implementation of the multiboot standard.
|
|
|
|
On start-up, GRUB will search the supplied kernel module (on x86-64 this is the
|
|
binary called elver in \pathname{tools/elver/}) for a magic byte sequence
|
|
(defined by multiboot) and begin execution just after that sequence appeared
|
|
(see \pathname{tools/elver/boot.S}).
|
|
|
|
\pathname{boot.S} in elver will set-up an preliminary GDT, an IA32-e page-table,
|
|
and stack for execution. \pathname{elver.c} will then search for a
|
|
binary called \keywname{kernel} or \keywname{cpu} in all the multiboot
|
|
modules, relocate that module and then jump to the relocated kernel module. At
|
|
this point, we have set-up a 1 GiB identity mapping of the physical address
|
|
space using 2 MiB pages in order to address everything we need initially.
|
|
|
|
Note that the reason elver exists is because multiboot v1 does not support
|
|
ELF64 images (or setting up long-mode). If we use a bootloader that supports
|
|
loading relocatable ELF64 images into 64-bit mode, elver would be redundant.
|
|
|
|
After \keywname{elver} is done, execution in the proper BSP kernel program
|
|
begins in \pathname{kernel/arch/x86\_64/boot.S} which then calls
|
|
\fnname{arch\_init}, the first kernel C entry point.
|
|
|
|
\subsection{APP Core}
|
|
|
|
APP cores are booted using the coreboot infrastructure in Barrelfish. The
|
|
logic that boots APP cores resides in \pathname{usr/drivers/cpuboot}.
|
|
|
|
The source code responsible for booting a new core on x86 is found in
|
|
\pathname{usr/drivers/cpuboot/x86boot.c}, specifically in the function called
|
|
\fnname{spawn\_xcore\_monitor}. \fnname{spawn\_xcore\_monitor} will load the
|
|
\keywname{kernel} and \keywname{monitor} binary, and relocate the kernel. The
|
|
function called \fnname{start\_aps\_x86\_64\_start} will afterwards map in the
|
|
bootstrap code (which is defined in \pathname{init\_ap\_x86\_64.S}) for booting the
|
|
APP core. One complication for this code is that it has to resides below 1 MiB
|
|
in physical memory since the new APP core starts in protected mode and
|
|
therefore can not address anything above that limit in the beginning. Once the
|
|
mapping is initiated, the entry point address for the new APP kernel will be
|
|
written into this memory region. Finally, a set of system calls are invoked
|
|
in order to send the necessary IPIs to bootstrap the new processor.
|
|
|
|
\section{Virtual Address Space}
|
|
|
|
The page table is constructed by copying VNode capabilities into VNodes to
|
|
link intermediate page tables, and minting Frame / DeviceFrame capabilities
|
|
into leaf VNodes to perform mappings.
|
|
|
|
When minting a frame capability to a VNode, the frame must be at least as
|
|
large as the smallest page size. The type-specific parameters are:
|
|
|
|
\begin{enumerate}
|
|
\item \textbf{Access flags:}
|
|
The permissible set of flags is PTABLE\_GLOBAL\_PAGE
|
|
| PTABLE\_ATTR\_INDEX | PTABLE\_CACHE\_DISABLED |
|
|
PTABLE\_WRITE\_THROUGH. Access flags are set from frame capability
|
|
access flags. All other flags are not settable from user-space (like
|
|
PRESENT and SUPERVISOR).
|
|
|
|
\item \textbf{Number of base-page-sized pages to map:} If non-zero, this
|
|
parameter allows the caller to prevent the entire frame capability from
|
|
being mapped, by specifying the number of base-page-sized pages
|
|
of the region (starting from offset zero) to map.
|
|
\end{enumerate}
|
|
|
|
\todo{address space layout after initialization is done}
|
|
|
|
\section{IO capabilities}
|
|
|
|
IO capabilities provide kernel-mediated access to the legacy IO space of
|
|
the processor. Each IO capability allows access only to a specific range of
|
|
ports.
|
|
|
|
The Mint invocation (see \autoref{sec:mint}) allows the permissible
|
|
port range to be reduced (with the lower limit in the first
|
|
type-specific parameter, and the upper limit in the second type-specific
|
|
parameter).
|
|
|
|
At boot, an IO capability for the entire port space is passed to the
|
|
initial user domain. Aside from being copied or minted, IO capabilities may not
|
|
be created.
|
|
|
|
\section{Global Descriptor Table (GDT)}
|
|
|
|
The GDT table is loaded by the \fnname{gdt\_reset} function during start-up and statically defined.
|
|
|
|
The table contains the following entries:
|
|
|
|
\begin{tabular}{c|l}
|
|
Index & Description \\ \hline
|
|
0 & NULL segment \\
|
|
1 & Kernel code segment \\
|
|
2 & Kernel stack segment \\
|
|
3 & User stack segment \\
|
|
4 & User code segment \\
|
|
5 & Task state segment \\
|
|
6 & Task state segment (cont.) \\
|
|
7 & Local descriptor table \\
|
|
8 & Local descriptor table (cont.) \\
|
|
\end{tabular}
|
|
|
|
\section{Interrupts and Exceptions}
|
|
|
|
The initial (Interrupt Descriptor Table) IDT is set-up by
|
|
\fnname{setup\_default\_idt} in \pathname{irq.c}. The number of entries in the
|
|
IDT is set to 256 entries which are initialized in the following way:
|
|
|
|
\begin{tabular}{c|l}
|
|
Index & Description \\ \hline
|
|
0 & Divide Error \\
|
|
1 & Debug \\
|
|
2 & Nonmaskable External Interrupt \\
|
|
3 & Breakpoint \\
|
|
4 & Overflow \\
|
|
5 & Bound Range Exceeded \\
|
|
6 & Undefined/Invalid Opcode \\
|
|
7 & No Math Coprocessor \\
|
|
8 & Double Fault \\
|
|
9 & Coprocessor Segment Overrun \\
|
|
10 & Invalid TSS \\
|
|
11 & Segment Not Present \\
|
|
12 & Stack Segment Fault \\
|
|
13 & General Protection Fault \\
|
|
14 & Page Fault \\
|
|
15 & Unused \\
|
|
16 & FPU Floating-Point Error \\
|
|
17 & Alignment Check \\
|
|
18 & Machine Check \\
|
|
19 & SIMD Floating-Point Exception \\
|
|
\hline
|
|
32 & \multirow{3}{*}{PIC Interrupts} \\
|
|
\vdots{} & \\
|
|
47 & \\
|
|
\hline
|
|
48 & \multirow{3}{*}{Generic Interrupts} \\
|
|
\vdots{} & \\
|
|
61 & \\
|
|
\hline
|
|
62 & Tracing IPI \\
|
|
63 & Tracing IPI \\
|
|
\hline
|
|
64 & \multirow{3}{*}{Unused} \\
|
|
\vdots{} & \\
|
|
247 & \\
|
|
\hline
|
|
248 & Halt IPI (Stopping a core) \\
|
|
249 & Inter core vector (IPI notifications) \\
|
|
250 & APIC Timer \\
|
|
251 & APIC Thermal \\
|
|
252 & APIC Performance monitoring interrupt \\
|
|
253 & APIC Error \\
|
|
254 & APIC Spurious interrupt \\
|
|
255 & Unused \\
|
|
\end{tabular}
|
|
|
|
The lower 32 interrupts are reserved as CPU exceptions. Except for a
|
|
double fault exception, which is always handled by the kernel
|
|
directly, an exception is forwarded to the dispatcher handling the
|
|
domain on the CPU on which it appeared.
|
|
|
|
Page faults (interrupt 14) are dispatched to the `pagefault` entry
|
|
point of the dispatcher. All other exceptions are dispatched to the
|
|
`trap` entry point of the dispatcher.
|
|
|
|
There are 224 hardware interrupts, ranging from IRQ number 32 to 255.
|
|
The kernel delivers an interrupt that is not an exception and not
|
|
the local APIC timer interrupt to user-space. The local APIC timer
|
|
interrupt is used by the kernel for preemptive scheduling and not
|
|
delivered to user-space.
|
|
|
|
Unused entries will be initialized by a special handler function. The slots
|
|
reserved for generic interrupts can be allocated by user-space applications.
|
|
|
|
\section{Local Descriptor Table (LDT)}
|
|
|
|
The local descriptor table segment in the GDT will
|
|
initially point to NULL as no LDT is installed.
|
|
|
|
User-space applications can install their own LDT table
|
|
which is loaded on context-switching using the
|
|
\fnname{maybe\_reload\_ldt} function.
|
|
|
|
|
|
\section{Registers}
|
|
|
|
\paragraph{Segment registers}
|
|
|
|
Segment registers are initialized by the \fnname{gdt\_reset} function during start-up and each of them points to a GDT entry (index of the GDT table slot for each segment is given in brackets).
|
|
|
|
\begin{itemize}
|
|
\item[cs] Kernel code segment (1)
|
|
\item[ds] NULL segment (0)
|
|
\item[es] NULL segment (0)
|
|
\item[fs] NULL segment (0)
|
|
\item[gs] NULL segment (0)
|
|
\item[ss] Kernel stack segment (2)
|
|
\end{itemize}
|
|
|
|
We also note that the \keywname{fs} and \keywname{gs} segment registers are
|
|
preserved and restored across context switches.
|
|
|
|
\paragraph{General purpose registers}
|
|
\begin{itemize}
|
|
\item \keywname{rcx} contains the start address when running a dispatcher
|
|
for the first time.
|
|
\end{itemize}
|
|
|
|
\todo{Floating point / SIMD}
|
|
\todo{Machine specific registers (MSR)}
|
|
|
|
\section{Hardware devices}
|
|
|
|
\subsection{Serial port}
|
|
On x86, the serial device (a PC16550 compatible controller) is initialized for the first time by the BSP core on boot-up.
|
|
|
|
By default serial port \varname{0x3f8} will be used, but the port can be changed by
|
|
using a command line argument supplied to the kernel.
|
|
|
|
Notable settings for the serial driver are:
|
|
\begin{itemize}
|
|
\item Interrupts are disabled.
|
|
\item FIFOs are enabled.
|
|
\item No stop bit.
|
|
\item 8 data bits.
|
|
\item No parity bit.
|
|
\item BAUD rate is 115200.
|
|
\end{itemize}
|
|
|
|
The serial device is later re-initialized into a different state once the
|
|
serial driver takes over the device. For example, interrupts will then be
|
|
enabled and handled by the driver.
|
|
|
|
\subsection{PIC -- Programmable Interrupt Controller}
|
|
\todo{describe}
|
|
|
|
\subsection{xAPIC -- Advanced Programmable Interrupt Controller}
|
|
\todo{describe}
|
|
|
|
\subsection{System call API}
|
|
This section describe the architectural system calls that are not
|
|
common with other architectures.
|
|
|
|
\begin{itemize}
|
|
\item[7] SYSCALL\_X86\_FPU\_TRAP\_ON: Turn FPU trap on (x86)
|
|
\item[8] SYSCALL\_X86\_RELOAD\_LDT: Reload the LDT register (x86\_64)
|
|
\end{itemize}
|
|
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\bibliographystyle{abbrv}
|
|
\bibliography{barrelfish}
|
|
|
|
\end{document}
|