An update to the ELLCC bare metal experiment last reported here…

Highlights:

• Targets the vexpress-a9 Cortex-A9 based evaluation card.
• SP804 timer supported.
• PL011 Serial port supported for the system console.
• Fully interrupt driven operation for the timers and serial port.
• Vectored interrupts supported using the ARM Generic Interrupt Controller.
• The scheduler now supports multiple thread priorities, with the number of priorities from 1 to N specified at compile time.
• Round robin scheduling supported.
• Tick-less kernel scheduling: no timer interrupts occur unless specifically required for time-slicing.

An example of some tests and commands:

../../bin/qemu-system-arm -M vexpress-a9 -m 128M -nographic -kernel kernel.bin
audio: Could not init `oss' audio driver
kernel started. Type "help" for a list of commands.
kernel % thread1
thread started foo
kernel % thread2
thread2 started
kernel % thread3
unhandled system call (175) args: 1, 1216308, 0, 8, 8, 0
thread3 started
kernel % thread4
thread4 started
kernel % thread5
thread5 started
kernel % thread2 still running
is
unrecognized command: is
kernel % ts
     TID  STATE        PRI NAME       
 0x28d58: RUNNING        1 kernel     
 0x2cd98: IDLE           3 idle0      
 0x2f550: MSGWAIT        1 thread1    
 0x305b0: TIMEOUT        1 thread2    
 0x31670: READY          1 clone1     
 0x326e0: SEMWAIT        1 thread4    
 0x33740: SEMTMO         1 thread5    
kernel % thread5 running
thread2 still running
thread5 running
thread2 still running
thread5 running
thread2 still running
thread5 running
thread2 still running
thread5 running
thread2 still runningindepenent
thread5 running
thread2 still running
kernel % date
Thu Jan  1 00:00:02 1970
kernel % help date
                date: show/set the system clock.
                      The argument used to set the time, if given,
                      is in the form [MMDDhhmm[[CC]YY][.ss]].
kernel % date 050408272014
kernel % date
Sun May  4 08:27:48 2014
kernel % 

The ARM specific source code is available here and the processor independent code is here.

If you want to try this out at home, everything you need except QEMU is packaged as a binary download from ftp://ellcc.org/pub/ choose the tarball appropriate for your host system, untar it, go into the ellcc/baremetal/arm directory and type “make run”.

Make sure you have QEMU installed on your system, as I am not currently able to cross make it for all the hosts.

UPDATE: More info on this little project is here.

So, this thing is getting a little out of hand. “This thing”, in case you haven’t been here before, is my experiment with using a standard Linux C library on a bare metal ARM board. The way that I’m making it work is by writing simple Linux system call handlers to do what the normal Linux system call would do but in a much simplified way. In my last post I described a little about how my test kernel is built and run and how to connect to it with GDB for debugging.

This time I have added some interrupt handlers and implemented a few more system calls. My first attempt at interrupt handling is for the SP804 dual timer to handle the POSIX monotonic and realtime system clocks. After one of my last posts, someone asked what ARM I was targeting. It turned out to be a great question for which I had no clue how to answer. There are so many ways to say ARM: arm7, armv7, cortex, … Diving in to this project at first I had no idea about the nuances. It turns out that that what I’m trying to target at first is the Cortex-a9, specifically as emulated by QEMU on the vexpress-a9 board.

Ah! you say. Then why the heck are you using the SP804 timer for that? You should be using the 64 bit timers in the private memory region! Indeed I should, and that will be an exercise for another day. The a15 is even cooler, with its virtual timer, but I digress.

In this update, I’ve also added some simple command line processing:

[~/ellcc/baremetal/arm] dev% make run
../../bin/qemu-system-arm -M vexpress-a9 -m 128M -nographic -kernel kernel.bin
audio: Could not init `oss' audio driver
kernel started. Type "help" for a list of commands.
kernel % help
                date: show/set the realtime timer.
                time: time the specified command with arguments.
               sleep: sleep for a time period.
Test Commands:
             syscall: test the syscall interface with an unhandled system call.
               yield: yield the current time slice.
             thread1: start the thread1 test case.
               send1: send a message to the thread1 test thread.
             cancel1: cancel the thread1 test thread.
             thread2: start the thread2 test case.
kernel % time date
Thu Jan  1 00:23:50 1970
elapsed time: 0.007252000 sec
kernel %

Kind of cool, right? Well, just to show that some rudimentary threading is going on:

kernel % thread1
unhandled system call (175) args: 1, 1194164, 0, 8, 8, 0
thread started foo
thread self = 0x00029EB0
kernel % send1
thread running 3
kernel % 

Even better, here’s a test case where the thread periodically sleeps:

kernel % thread2
thread2 started
kernel % thread2 still running
thread2 still running
thread2 still running
thread2 still running

The code for thread2 looks like this:

static void *thread2(void *arg)
{
    printf ("thread2 started\n");
    for ( ;; ) {
        // Go to sleep.
        sleep(10);
        printf ("thread2 still running\n");
    }

    return NULL;
}

Unfortunately it will be running forever. I haven’t implemented pthread_cancel() or pthread_kill() support yet. But the command prompt is still active:

thread2 still running
thread2 still running
thread2 still running
timethread2 still running
 sleep 1
elapsed time: 1.000731000 sec
kernel % 

(I had typed in “time sleep 1”)

As usual, the code is available here. Please don’t look at irq.c. The interrupt controller code is a total hack for now.

UPDATE: If you want to try this out at home, everything you need except QEMU is packaged as a binary download from ftp://ellcc.org/pub/ choose the tarball appropriate for your host system, untar it, go into the ellcc/baremetal/arm directory and type “make run”.

Make sure you have QEMU installed on your system, as I am not currently able to cross make it for all the hosts.

I’ve spend much of the weekend (it is a holiday, right?) playing around with my bare metal prototype. In the last post in my Bare Metal ARM series, I did a little context switching. Actually it implemented simple co-routines that could give up execution to each other. In this installment, I’ve added a rudimentary ready list and implemented a simple message passing scheme allowing threads to communicate with each other and block if no messages are available.

The full source of my little prototype is here. A little explanation of how I’m developing this might be in order. init.S is the processor initialization and exception handling code, written in assembly because most of it has to be. init.S calls into the C standard library by calling __libc_start_main() after setting up a few stack pointers and zeroing out the uninitialized data (.bss) area. __libc_start_main() ends up calling main() after initializing the library.

The Makefile can be used to build and run the code.

[~/ellcc/baremetal/arm] dev% make
../../bin/ecc -target arm-ellcc-linux-eabi5 -march=armv7 -mfpu=vfp -mfloat-abi=softfp  -c init.S
../../bin/ecc -target arm-ellcc-linux-eabi5 -march=armv7 -mfpu=vfp -mfloat-abi=softfp -g -MD -MP -Werror -Wall -Wno-unused-function -c main.c
../../bin/ecc -target arm-ellcc-linux-eabi5 -march=armv7 -mfpu=vfp -mfloat-abi=softfp -g -MD -MP -Werror -Wall -Wno-unused-function -c simple_console.c
../../bin/ecc -target arm-ellcc-linux-eabi5 -march=armv7 -mfpu=vfp -mfloat-abi=softfp -g -MD -MP -Werror -Wall -Wno-unused-function -c simple_memman.c
../../bin/ecc -target arm-ellcc-linux-eabi5 -march=armv7 -mfpu=vfp -mfloat-abi=softfp -g -MD -MP -Werror -Wall -Wno-unused-function -c scheduler.c
../../bin/ecc -target arm-ellcc-linux-eabi5 -nostartfiles -T kernel.ld \
    ../../libecc/lib/arm/linux/crtbegin.o \
    init.o main.o simple_console.o simple_memman.o scheduler.o \
    ../../libecc/lib/arm/linux/crtend.o \
    -o kernel.elf -Wl,--build-id=none
../../bin/ecc-objcopy -O binary kernel.elf kernel.bin
[~/ellcc/baremetal/arm] dev% make run
../../bin/qemu-system-arm -M vexpress-a9 -m 128M -nographic -kernel kernel.bin
audio: Could not init `oss' audio driver
kernel: hello world
unhandled system call (0) args: 1, 2, 3, 4, 5, 6
__syscall(0) = -1, No error information
hello from context 42 code = 3
hello from context 42 code = 6809
prompt: hi there
got: hi there
prompt: 

To exit out of QEMU use the key sequence Control-A x.

By the way, the send and receiving of messages (the “hello from context” messages) is done in main.c:

...
Queue queue = {};
static intptr_t context(intptr_t arg1, intptr_t arg2)
{
    for ( ;; ) {
      Message *msg = get_message(&queue);
      printf("hello from context %" PRIdPTR " code = %d\n", arg1, msg->code);
    }
    return 0;
}
...
int main(int argc, char **argv)
{
...
    new_thread(context, 4096, 42, 0);
    Message msg = { { NULL, sizeof(msg) }, 3 };
    send_message(&queue, &msg);
    msg.code = 6809;
    send_message(&queue, &msg);
...

I do debugging by using two windows, one to start QEMU in debug mode and the other to run GDB. The first window:

[~/ellcc/baremetal/arm] dev% make debug
../../bin/qemu-system-arm -s -S -M vexpress-a9 -m 128M -nographic -kernel kernel.bin
audio: Could not init `oss' audio driver

The other window looks like:

[~/ellcc/baremetal/arm] dev% ~/ellcc/bin/ecc-gdb kernel.elf
GNU gdb (GDB) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from kernel.elf...done.
(gdb) target remote :1234
Remote debugging using :1234
0x60000000 in ?? ()
(gdb) break main
Breakpoint 1 at 0x10408: file main.c, line 40.
(gdb) c
Continuing.

Breakpoint 1, main (argc=1, argv=0x100e8 ) at main.c:40
40          printf("%s: hello world\n", argv[0]);
(gdb) 

One aspect of this code that might be confusing is that there is no apparent explicit initialization code in main() for the serial port, memory allocator, or scheduler. Since this is implemented with a full C library, I take advantage of the fact that static constructors work and are extensions to C in both clang and GCC. An example is at the bottom of the scheduler.c file:

/* Initialize the scheduler.
 */
static void init(void)
    __attribute__((__constructor__, __used__));
    
static void init(void)
{   
    // Set up the main and idle threads.
    idle_thread.saved_sp = (Context *)&idle_stack[IDLE_STACK];
    __new_context(&idle_thread.saved_sp, idle, Mode_SYS, NULL,
                  0, 0);
    
    // The main thread is what's running right now.
    main_thread.next = &idle_thread;
    ready = &main_thread;
        
    // Set up a simple set_tid_address system call.
    __set_syscall(SYS_set_tid_address, sys_set_tid_address);
}

Why did I do it this way? Because I don’t have to change any source code if I want to swap out simple_console.c for a hypothetical interrupt_console.c, for example. I just have to change the SRCS macro in the Makefile.

The next step is to implement interrupt handlers, the timer, and preemptive scheduling. By the way, the full POSIX pthread_create() is still a ways off, but it feels like it is getting closer all the time.