Shared Libraries State

Ending with commit 53f94f8ed431, Minix has full shared libraries support, include runtime loading of shared objects by the runtime linker, ld.elf_so, i.e. dlopen() and friends.

By default binaries are not linked dynamically yet, out of caution. This page describes the interaction with the base system and how to get started with building dynamically linked executables. Everyone is invited to test building dynamic binaries and test the resulting system. And also to help get pkgsrc building dynamically (see Pkgsrc at the end).

But binaries are linked statically by default in the base system. Link executables with LDSTATIC=-dynamic if you want to build them dynamically.

Use 'file' or 'ldd' to examine the resulting executable and which libraries it needs.

# cd /usr/src/commands/wc/
# make clean
# make LDSTATIC=-dynamic
    compile  wc/wc.o
       link  wc/wc
# ldd wc
wc:
        -lminlib.0 => /usr/lib/libminlib.so.0
        -lc.0 => /usr/lib/libc.so.0
        -lcompat_minix.0 => /usr/lib/libcompat_minix.so.0
        -lterminfo.0 => /usr/lib/libterminfo.so.0
# file wc
wc: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), dynamically linked (uses shared libs), not stripped

Don't turn on dynamic linking globally (e.g. with make build) as drivers and boot time processes can't be started if they are dynamically linked.

Building all binaries dynamically makes /bin and /usr/bin 10x smaller.

The rest of this page describes some implementation details that may be of interest.

The dynamic building defaults to no because of two risks: one, functionality (the dynamic linking might produce broken binaries for some reason); two, robustness (dynamic binaries are more fragile; the dynamic linker or a shared library not being available causes the binary to not work, so the whole system can be hosed easily). Also performance is currently lower than equivalent static binaries.

Experience with dynamic linking will determine when to switch.

VM now has full sized text segments so code can be executed anywhere.

There is no logic yet to share the pages between the same instances of shared libraries and executables, so this change was mostly for the functionality of shared libraries; sharing pages is a future performance improvement.

There no longer is an explicit heap region in VM.

Clang needs to be updated for it to be able to link shared libraries, and to be able to link dynamically linked executables. This is clang-2.9nb6 and higher or clang-3.1nb3 and higher. GCC does not need any upgrading.

The presence of a shared-aware clang is detected by the existence of an empty /usr/pkg/bin/clang.dynok file. The buildsystem sets MKPIC to yes by default if this file exists. This triggers the building of shared libraries, as well as the archives. All library code is compiled twice; with and without -fpic. The pic objects are combined into a single _pic.a archive and relinked into a re-linkable shared library.

By default clang and gcc will make dynamically linked binaries if they find shared libraries. The base system build system links all utilities statically unless instructed otherwise .

RS starts drivers and servers using a slightly custom fork and exec. Currently RS does neatly detect dynamically linked executables, but currently can not start them. This could be implemented (although should be combined with merging vfs and RS exec code) but doesn't seem urgent.

All tests are built dynamically if available (detected with MKPIC). This forces the base system to build and run things dynamically.

test63 and an associated module tests basic runtime loading functionality (dlopen() and calling a function in the loaded module).

One of the interesting quirks is that the functioning of the executable is highly dependent on the _end symbol. This _end symbol is added by ld to every elf linking (i.e. binary and shared library). The brk() call needs it to know where the break is. So it relies on the _end symbol being the one from the main executable, not any of the other _ends. If crazy things happen with malloc(), first make sure there is an _end in the dynamic symbols hash table of the main executable first, and whether _end has a sensible value.

Something crazy happening with a dynamically linked executable? A good way to debug it is to watch the relocation happening in the dynamic linker (ld.so). Recompile ld.so with the following uncommented in its Makefile:

CPPFLAGS+=     -DDEBUG
CPPFLAGS+=     -DRTLD_DEBUG
CPPFLAGS+=     -DRTLD_DEBUG_RELOC

To recompile it:

cd /usr/src
make -C libexec/ld.elf_so clean cleandepend
make -C libexec/ld.elf_so
make -C libexec/ld.elf_so install

Then, to enable it, run:

export LD_DEBUG=yes

which will make the linker spit out tons of useful debugging info. from dynamically-linked processes.

Dynamic linking enables runtime loading of modules in e.g. apache, perl, python and clang. But lots of packages are broken because of the dynamic switch. (By default everything in pkgsrc is built dynamically of course). Help with fixing build problems is very welcome.