====== Minix and Pkgsrc ======

pkgsrc has an advanced system to build each package in isolation of the world except for its stated dependencies. This is a good thing and helps package maintenance tremendously. It makes bulk building a little harder. In keeping with its philosophy, the new style (pbulk style building) requires bootstrapping a new pkgsrc instance (i.e. new prefix that is normally /usr/pkg), e.g. /usr/pbulk. The bulk build process needs to work in /usr/pkg, but that's where your currently installed packages live. So we backup the old tools from /usr/pkg and then restore them after. There are a few other things that need to happen, so we created a script to automate the process.

For more info on pbulk style bulk builds, see the following resources:
  * [[http://www.netbsd.org/docs/pkgsrc/bulk.html#bulk.pbulk|pbulk section in the pkgsrc guide]]
  * [[http://www.dragonflybsd.org/docs/developer/PbulkBuilding/|DragonFlyBSD pBulk Building Guide]]
  * [[http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/doc/HOWTO-pbulk?rev=1.7&content-type=text/x-cvsweb-markup&only_with_tag=MAIN|pbulk HOWTO]]

===== pbulk.sh =====

As for the minix specifics, there are a few steps that involve some variables that shouldn't be made up on the spot all the time, so there is a script in pkgsrc that takes care of this for Minix. The script is /usr/pkgsrc/minix/pbulk.sh It executes the build in several phases detailed below. Through the use of command line options, you can run any phase individually with the //--phase-name// argument or run them all with the //--all// argument.
  * //bootstrap// - a new pkgsrc instance is bootstraped into /usr/pbulk using /usr/pkgsrc/bootstrap/bootstrap
  * //backup// - as mentioned above, the bulk build process will clobber your current package source installation if you aren't careful. This phase backs up your /usr/pkg directory as well as your package database.
  * //bin-kit// - bootstraps a binary kit used by pbulk.
  * //build// - executes the bulk build.
  * //restore// - restores system's original /usr/pkg and the package database.
  * //all// - executes all of the above steps in order.

===== Configuration =====

We are going to run the pbulk process //in a minix chroot jail.// This has a few advantages:
  * Sterile, reproducible environment of building
  * Easy to automatically make an up-to-date userland (except for the image of course) to test compiling in
  * The pbulk process wipes out /usr/pkg, which is tricky to keep a live machine working on (sshd etc.), but in a jail that's no problem.

This needs quite a lot of disk space though. At least 3GB extra for the jail currently.

===== Bulk Build In Jail =====

Assuming pkgsrc is created in /usr/pkgsrc:
<code>
# cd /usr/pkgsrc/minix/
# sh pbulk-jail.sh -A
</code>

This will invoke /usr/src/tools/release.sh, use git to checkout the latest version of the minix tree, build it in a jail
hierarchy at /usr/pbulk-jail, copy the host pkgsrc tree there to re-use its objects, and retrieve the latest pkgsrc tree from the pkgsrc repository, and invoke pbulk.sh --all in the chrooted jail.

When the script finishes, you'll have some packages in /usr/pkgsrc/packages/ in the jail. Note, the bootstrapping and binary kit creation takes a while. To do a bulk build of just the misc/figlet package from scratch takes over an hour.

===== Running the above on a separate filesystem =====

The pbulk-jail script will mkfs and mount a new filesystem for the jail if you want; use the -d option to specify a device, and the jail script will umount it, **mkfs it**, and mount it, and build a jail on it etc.
<code>
# sh pbulk-jail.sh -d/dev/c0d0p2s3 -A
</code>

The script will automatically customize the jail directory name based on the devname, so starting multiple bulk builds at on different filesystems is ok.


===== Bulk build customisation =====

In the jail, there are several opportunities to customize the build. You can decide which packages to attempt building by listing them in /usr/pkgsrc/minix/limited_list.pbulk. You can set various pbulk parameters (package upload location, reports, etc.) in /usr/pkgsrc/minix/pbulk.conf. pbulk.conf is installed to /usr/pbulk/etc/pbulk.conf during the bootstrap step, so if you want to do further modifications later, be sure to edit /usr/pbulk/etc/pbulk.conf. The various /usr/pkgsrc/minix/mk.conf* files can be edited as well.

There isn't good support for saving your customisations though, so you can't update the pkgsrc tree and keep your edits easily currently.


====== TODO ======
  * An automated system that, from any OS, runs a qemu, using the qemu builds a new minix from current svn, then runs that .iso to install a Minix, then boots into that minix, then fetches the latest pkgsrc tree, and does the above to bulk-build all known-good packages, and produces pretty html status overviews of it so we can make sure this process, and known-good packages, don't break by base system or pkgsrc changes. And if they do, see in which minix svn / pkgsrc commit things were still OK.

  * Have multiple such instances be able to work together to do parallel pbulk builds.