Switching to ELF

MINIX3 is switching to ELF as its new default binary format. a.out executables will still run for a while of course. But switching to ELF means taking into account a new toolchain and library (object) format, both mutually incompatible with the a.out toolchain and library format. This page describes the assumptions and mechanics of the switch.

Compilers

The compilers in the game are gcc, clang and ack. ack will only generate a.out, and will be obsolete in the future. gcc and clang are switching to generating ELF objects.

The Problem

The problem is that ELF-generating compilers require ELF binutils (i.e. assembler, linker), and these are mutually incompatible with the current a.out-producing binutils. So binutils and both (gcc, clang) compilers have to be synchronized.

The next problem is that linking ELF executables requires ELF libraries, again mutually incompatible with a.out, so libraries in the base system and binary packages in pkgsrc have to be synchronized with versions of compilers and binutils.

The Solution

The solution is that, starting at Minix 3.2.0, the base system detects whether ELF-generating binutils and clang are installed. If so, existing top-level Make targets gcc-libraries and clang-libraries won't work any more, they are obsolete. The elf-libraries changes from being a crossbuilt curiosity to a first class citizen; elf-libraries is the new gcc-libraries (built by clang, and used by both clang and gcc). As clang is available in the base system, elf-libraries is then also linked with world, thus promoting gcc-only libraries to a first class citizen.

Starting at the Minix 3.2.0 packages repository, all libraries are in ELF format. So the Minix version number is used to synchronize that switch. If you upgrade to 3.2.0, de-install the library packages you want to use, and re-install them from 3.2.0.

The Switch

The easiest option by far is reinstall using a 3.2.0 .iso.

If you prefer upgrading from source instead:

Upgrade to HEAD, paying due attention to src/docs/UPDATING, as usual. Nothing should change until you install the new binutils and clang.
Rebuild world and reboot so your system identifies itself as 3.2.0
```
# cd /usr/src
# make clean world
```

then reboot with the new image.

Check your machine identifies itself as 3.2.0, and get the new packages index.

# uname -a
Minix hostname 3.2.0 i686
# pkgin up                                 
cleaning database from ftp://ftp.minix3.org/pub/minix/packages/3.1.9/i686/All entries...
downloading pkg_summary.bz2:    0Bbps 100%
processing remote summary (ftp://ftp.minix3.org/pub/minix/packages/3.2.0/i686/All)...
updating database: 100%

(note the 3.2.0 location.)

Wipe the pkgin cache in case it gets confused with cached 3.1.9 packages
```
# rm -r /usr/var/db/pkgin/cache/
```

Remove the currently installed binutils, gcc, and clang, if you have them installed; -f to ignore breaking dependencies.

# pkg_delete -f binutils gcc44       
binutils-2.17nb2: rebuilding run-time library search paths database
gcc44-4.4.3nb3: rebuilding run-time library search paths database

Don't discount the possibility of old tools such as ld, gld, ar, or gar lying around in unexpected places on your filesystem, hitherto without problems. Delete all remaining binutils-like commands at this step to be sure. There are two 'as' binaries installed under /usr/lib, nothing else. Example:
```
# find / -name ld -o -name gld -o -name ar -o -name gar -o -name as -o -name gas -o -name objdump
/usr/lib/i386/as
/usr/lib/i86/as
```
The find should not output any other commands that look like they would be found.

Install the new binutils, clang and gcc packages.

# pkgin in gcc44 clang
calculating dependencies... done.

nothing to upgrade.
4 packages to be installed: binutils-2.17nb3 compiler-rt-r123836nb3 gcc44-4.4.3n
b3 clang-2.9nb2 (108M to download, 282M to install)

proceed ? [y/N] y
downloading packages...
downloading binutils-2.17nb3.tgz:    0Bbps 100%
downloading compiler-rt-r123836nb3.tgz:    0Bbps 100%
downloading gcc44-4.4.3nb3.tgz:   18Mbps 100%
downloading clang-2.9nb2.tgz:   16Mbps 100%
error log can be found in /usr/var/db/pkgin/err.log
installing packages...
installing binutils-2.17nb3...
binutils-2.17nb3: rebuilding run-time library search paths database
installing compiler-rt-r123836nb3...
compiler-rt-r123836nb3: rebuilding run-time library search paths database
installing gcc44-4.4.3nb3...
gcc44-4.4.3nb3: rebuilding run-time library search paths database
[...]
installing clang-2.9nb2...
clang-2.9nb2: rebuilding run-time library search paths database
processing local summary...
updating database: 100%
marking gcc44-4.4.3nb3 as non auto-removable
marking clang-2.9nb2 as non auto-removable

Check: see if your installed binutils supports/generates ELF:

# objdump -i | grep -i minix
elf32-i386-minix
               elf32-i386-minix elf32-little elf32-big srec symbolsrec tekhex 
          i386 elf32-i386-minix elf32-little elf32-big srec symbolsrec tekhex

If not, you have a pre-3.2.0 binutils installed.

Check: do a basic test to see if your compilers generates ELF objects:

# echo 'main() { printf("Hello, World!\n"); } '>hello.c
# rm -f hello-gcc.o hello-clang.o
# gcc -c hello.c -o hello-gcc.o ; clang -c hello.c -o hello-clang.o
# file hello-*.o
hello-clang.o: ELF executable
hello-gcc.o: ELF executable

If not, you have a pre-3.2.0 clang/gcc installed.

This makes your a.out libraries in /usr/lib obsolete, as the top-level Makefile will detect the new packages and assume everything is to be generated in ELF now.

If all went well, you have an 'elf-libraries' target and no more 'gcc-libraries' or 'clang-libraries' targets. Check this:

# cd /usr/src
# make
[..]
        make world         # Compile everything (libraries & commands)
        make includes      # Install include files from src/
        make libraries     # Compile and install libraries (ack)
        make elf-libraries # Compile and install gcc/clang elf libs
[..]

Do another 'make world' to check whether elf-libraries are built now. They should be, with clang. This will update your /usr/lib libraries to ELF, to be used by both clang and gcc.
```
# cd /usr/src
# make clean world
```

Don't be too alarmed if it takes a long time - an extra set of libraries is built now (the ELF ones) with clang, which is a little slower than ack, and the first time around a little slower still because of the extra (quite slowly) one-time generated dependencies.

Example: the first time, 'make clean world' takes 5:34 on a certain test machine; the second time a mere 4:07.

The ACK set of libraries will disappear in the future, alleviating this problem.

Now see if your libraries are ELF format:

# objdump -a /usr/lib/libc.a | head
In archive /usr/lib/libc.a:

abort.o:     file format elf32-i386-minix

Check if you can compile a hello world and it produces ELF:

# rm -f hello-clang hello-gcc
# clang -o hello-clang hello.c ; gcc -o hello-gcc hello.c 
# file hello-*
hello-clang: ELF executable
hello-clang.o: ELF executable
hello-gcc: ELF executable
hello-gcc.o: ELF executable
# ./hello-clang
Hello, World!
# ./hello-gcc
Hello, World!

If so, you are fully switched, congratulations.

a.out, ack and the future

a.out and ack will still be around for a while, partly because we can't boot from ELF executables yet, partly because the boot monitor is a 16-bit program and cannot be build with just clang, and perhaps other reasons. But the goal is to run the base system exclusively with clang, sharing a common object format with gcc, so we have a single library format.