Cross Building Linux With ELLCC: Part 2 – Raspberry Pi B+

Update: Cross Building Linux With ELLCC: Part 3 – Other Flavors of Pi
In a previous post I tested the waters to see how close ELLCC is to cross compiling the Linux kernel. That post was mostly a theoretical exercise to see what it would take. Over the holidays I decided that I needed to do something a little more concrete. My son, a 10 year old obsessed with Minecraft, got a Raspberry Pi for Christmas. My nefarious plan is to divert his attention away from Minecraft at least long enough to make his Pi and camera module into something he can use to spy on the family. My other nefarious plan is to use ELLCC to build the Linux kernel that we’ll be running on his Pi.

In the previous post I used the source from the LLVMLinux site. This time I started with the kernel source for the Raspberry Pi. The good news is that the source can be built for the Pi. The bad news is that it doesn’t have all the patches need to compile with a clang based compiler. It turned out that the build wasn’t too bad: I had Christmas Eve and Day off and in between family events and Santa preparations I was able to massage the source enough to boot the kernel on our Pi.

To get started I followed the instructions on Linux From Scratch on the Raspberry Pi.
Here are the exact steps I took. First I created a new directory to hold the ELLCC tool chain and the Raspberry Linux kernel source, then I downloaded and untar’d both:

mkdir ecclinux
cd ecclinux
wget http://ellcc.org/releases/ellcc-x86_64-linux-eng-0.1.23.tgz
tar xvfp ellcc-x86_64-linux-eng-0.1.23.tgz
wget https://github.com/raspberrypi/linux/archive/rpi-4.1.y.tar.gz
tar xvfp rpi-4.1.y.tar.gz

I had to change the kernel sources slightly during my build. I saved a patch
file, here’s how to apply it:

patch -p0 < ellcc/libecc/llvmlinux/ecc-rpi-4.1.y.patch

There is a special little 'm' makefile that helps with the build. Copy
it into the top level of the kernel source directory:

cd linux-rpi-4.1.y/
cp ../ellcc/libecc/llvmlinux/makefile .

Now the actual build is easy:

make bcmrpi_defconfig
make

I have my Raspberry Pi on my home network at 192.124.43.98. I also allowed ssh
and scp root access by follwoing the instructions here. This lets me
just copy the new kernel to my Pi:

scp ./arch/arm/boot/zImage root@192.124.43.98:/boot/kernel.img

Now I can ssh to the Pi and reboot:

[~] dev% ssh root@192.124.43.98
root@192.124.43.98's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@raspberrypi:~# reboot
Connection to 192.124.43.98 closed by remote host.
Connection to 192.124.43.98 closed.
[~] dev%

I ping the Pi until it reboots and log in again:

[~] dev% ssh root@192.124.43.98
root@192.124.43.98's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Dec 26 20:27:15 2015 from 192.124.43.2
root@raspberrypi:~# dmesg | less
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.1.15 (rich@dev) (ecc 0.1.23 based on clang version 3.8.0 (trunk) (based on LLVM 3.8.0svn)) #1 Sat Dec 26 11:32:33 CST 2015
[    0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
...

Very nice!

16 thoughts on “Cross Building Linux With ELLCC: Part 2 – Raspberry Pi B+

  1. Emmanuel Blot

    > I had to change the kernel sources slightly during my build. I saved a patch
    file, here’s how to apply it:
    > patch -p0 < ellcc/libecc/llvmlinux/ecc-rpi-4.1.y.patch

    Where could this patch be downloaded from?
    $ ls -1 ellcc/libecc/
    Makefile
    Makefile.lib
    clang
    config
    config.musl
    doc
    ecclinux
    include
    lib
    mingw
    mkscripts
    share
    src

    (llvmlinux/ directory is not part of ellcc-Mac_OS_X_10.11.2-0.1.24.tgz)

    Thanks.

    Reply
  2. rich Post author

    Hi Emmanuel,
    I see you’ve downloaded 0.1.24. I just put it up last night. The instructions to build Linux are slightly different. I haven’t had time to do an updated blog post yet, but here is how it works now:
    cd ellcc/libecc/ecclinux
    ./build # Print help information
    ./build # Currently two choices, Pi and Pi 2.
    The build script will build the kernel and also a bunch of userspace programs.
    -Rich

    Reply
  3. Emmanuel Blot

    Hi Rich,

    Thanks for the update.

    There is an issue with the script on OS X, when it retrieves the number of parallel jobs to spawn at once, as `/proc` does not exist on OS X. I may suggest to use something like:
    if [ “${UNAME}” = “Darwin” ]; then
    MAXJOBS=`sysctl hw.ncpu | cut -d: -f2`
    else
    MAXJOBS=`cat /proc/cpuinfo | grep processor | wc -l`
    fi

    There is another issue with missing header files:

    HOSTCC scripts/basic/fixdep
    In file included from /Volumes/Development/Sources/ellcc/libecc/ecclinux/linux-4.1.15/scripts/basic/fixdep.c:106:
    /Volumes/Development/Sources/ellcc/bin/../libecc/include/sys/types.h:57:10: fatal error: ‘bits/alltypes.h’ file not found
    #include
    ^
    1 error generated.

    I already encountered this issue with missing header files on another attempt to build Linux on OS X, I think the work around was to copy a bunch of Linux files into /usr/include OSX dir (I hope there would be a better fix)

    Reply
  4. rich Post author

    Hi Emmanuel,

    Thanks for the Darwib suggestion. I’ll add it.
    I think you could set HOSTCC=clang in the build script to fix your include file problem.

    -Rich

    Reply
  5. rich Post author

    Hi Emmanuel,
    I’ve had a chance to try building on the Mac and I have some good news and some bad news.
    I made the Darwin changes for MAXJOBS and HOSTCC in build, the changes are in the repository and will be in the next binary release. Unfortunately, the changes aren’t complete yet. I’ve run into two problems.

    • After the initial configure build in the source directory, the definition of HOSTCC seems to get lost.
    • A HOSTCC build of linux-4.1.15/scripts/mod/mk_elfconfig.c fails because the Mac doesn’t have an elf.h file (obviously).

    I’m sure figuring out the HOSTCC definition will be fairly simple, but the elf.h this sounds more difficult. I have to understand why building Linux needs information about the hosts’s use of ELF.

    I think a Mac build is very possible, but I don’t think it will happen tonight.
    -Rich

    Reply
  6. Emmanuel Blot

    Fix for the MAXJOBS definition (discard extra space that causes trouble to the Make command line):
    MAXJOBS=`sysctl hw.ncpu | cut -d: -f2 | tr -d [:space:]`

    > After the initial configure build in the source directory, the definition of HOSTCC seems to get lost.
    Isn’t it overridden in “makefile”?
    HOSTCC := $(ELLCC)/bin/ecc

    Include path to copy modified ELF files – temp fix, we need a better fix than messing with the system files :-):
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include

    Next issue is that OS X “ar” does not accept to create empty archives with no member file, whereas GNU ar silently does.
    A solution seems to use ecc-ar instead the system-wide ar. I did not succeed to properly export AR from the “makefile” file, but replacing/hardcoding $(AR) within scripts/Makefile.build does the job, I guess it is now “just” a matter of writing a proper fix.

    Remainings issues:
    * OS X xargs does not support “-r” option
    * objdump is not found
    Nevertheless, a non-empty vmlinux file is built…

    Reply
  7. Emmanuel Blot

    It is actually cleaner to override the $(CROSS_COMPILE) variable in the main Makefile file to solve both the objdump and ar issues (and likely a couple of other ones for targets that require binutils tools).
    A similar fix is required to use ellcc’s “size” utility instead of system-wide one in arch/arm/boot/compressed/Makefile

    Unfortunately, defining CROSS_COMPILE from the command line as with a GCC build breaks the build sequence immediately, as this definition creates some artefacts with CC/AS/LD definition.

    Reply
  8. rich Post author

    Hi Emmanuel,
    I applied your patches. They are now in the source tree. Thanks!
    I built using the elf.h file that comes with ELLCC. I added an osx_include directory and put the a link to elf.h there. I also added a HOSTCFLAGS=-I../osx_include when building on Darwin. Seems to work just fine.

    I tried to build the kernel, but found out you really do need a case sensitive file system, at least for these files:
    ./include/uapi/linux/netfilter_ipv4/ipt_ECN.h
    ./include/uapi/linux/netfilter_ipv4/ipt_TTL.h
    ./include/uapi/linux/netfilter_ipv4/ipt_ttl.h
    ./include/uapi/linux/netfilter_ipv4/ipt_ecn.h
    I was trying to think of a clever way of patching the sources to handle this, but simple fix didn’t occur to me.
    I’ll make a case sensitive file system and start the build again.
    -Rich

    Reply
    1. Emmanuel Blot

      > I tried to build the kernel, but found out you really do need a case sensitive file system
      Very true, I use a dedicated sparse bundle for this on my Mac 🙂

      Reply
  9. rich Post author

    With revision 5821 of ELLCC some of the previous userland builds. Right now I’m having problems with iproute2, e2fsprogs, and vim. I need to work on the cmake build now though. configure is going away soon for a clang/LLVM build.
    -Rich

    Reply

Leave a Reply to Emmanuel Blot Cancel reply

Your email address will not be published.