poudriere

Written by Michael Cole - December 8th 2015


So I know it's been a bit again. Holidays are fun but they definitely cause more down time. I got sick right after Thanksgiving but that gave me some time to work on computer stuff at the house. So I tried to start working on switching my mail server over to FreeBSD. This post won't really be about that too much though, but it's important to the subject of this post. So I installed postfix and immediately noticed that it had no database support. While we are on the subject, quick tip, you can run:

pkg info <package name>
			

And see compile options used for the package. So I had a few options here. I could of course get the ports tree and compile postfix and the necessary dependancies. I could even share the ports tree with nfs or something else. But I wanted to try out poudriere that I have heard so much about.

Before we get too far, I want to remind you this is just what I did, or at least very close to what I did. Do NOT copy and paste blindly from the Internet, try to take some time to understand what commands and configurations do. I take no responsibility for anything you do, or anything that happens as a result of using this guide. USE AT YOUR OWN RISK.

That our of the way, I have to say my first thoughts were that it looked pretty involved. And there were a bunch of guides that seemed pretty different. So I'm going to try to do a start to finish item on it and include every weird thing that I ran into. I will however presume a few things. Firstly that you have ZFS and secondly you already have ZFS setup. If not some of my earliest posts cover this, like Post 5.

So I recommend installing 2 packages:

pkg update
pkg install dialog4ports poudriere
			

The dialog package will be used later for the options stage. Next I recommend setting up your keys that you will sign and verify packages with. On my machine I had not setup any SSL certificates yet, so I started by making my directories and securing the private key directory:

mkdir -p /usr/local/etc/ssl/{keys,certs}
chmod 0600 /usr/local/etc/ssl/keys
			

Then I went into that directory, made the private key, and then created the public certificate using that key.

cd /usr/local/etc/ssl
openssl genrsa -out keys/poudriere.key 4096
openssl rsa -in keys/poudriere.key -pubout -out certs/poudriere.cert
			

You can really use any location or name, as long as your configurations match. You will need the cert file on every FreeBSD box you want to use this repository on. So use the directions above on making the directory and then copy the cert file to those other servers. Do NOT regenerate it on them, or copy the private key to them.

Ok with all the certs out of the way, lets get to a little configuration. This is the part that will vary the most for your setup. If you edit the configuration file:

vi /usr/local/etc/poudriere.conf
			

You will see all of the options, you will need to match things to your system that I don't know. The first setting is your ZPOOL name. This is just the name all by itself. For the purpose of demonstration:

ZPOOL=mypool
			

You can also change your ZROOTFS if you want, or leave it at the default which is:

ZROOTFS=/poudriere
			

Now you have to pick a site to download the jail files from:

FREEBSD_HOST=ftp://ftp.freebsd.org
			

I switched my BASEFS to match my ZROOTFS:

BASEFS=/poudriere
			

A lot of the settings I left at default. The next one can be pretty important for a few reasons. It can effect the speed of compiling packages and it can effect memory usage. They give some great examples in the configuration file for USE_TMPFS. You can control not only if you are using it, but the different type of things you want to use it for. As I stated about it's RAM so it's faster to use it for more things, but it also can limit your system resources. There is also a TMPFS_LIMIT option where you can specify the limit in GB to overcome some of those issues.

The next options were pretty important to me because I was customizing the packages. And I really wanted to make sure I knew if Options or Dependencies were changing.

CHECK_CHANGED_OPTIONS=verbose
CHECK_CHANGED_DEPS=yes
			

Now it's time to tell it where your private key is:

PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key
			

You can also control how many parallel jobs to run. Again this can effect speed and system resources.

I disabled Linux support:

NOLINUX=yes
			

I was also going to setup a website to distribute the packages in my house. The next setting related to that:

URL_BASE=http://packages.mydomain.mine/logs/
			

For the most part I think that's what I adjusted. So save all of that, and exit.

My next recommendation is to figure out what your ABI strings are:

pkg -vv | grep -i abi 
			

Currently if you are on stable it will return:

ABI = "FreeBSD:10:amd64";
ALTABI = "freebsd:10:x86:64";
			

These are strings that will make your life easier later, if you use them. So now we tell poudriere to get the ports tree:

A lot of people at this stage will make a custom make file. The general recommendation seems to be to put that file in /usr/local/etc/poudriere.d/. I didn't bother because for the most part the defaults will work fine, and I'm not too worried about saving space with the documentation, etc. As well later I will manually setup all the options I want for the packages I want.

poudriere ports -c
			

Once that's done, I make my jail:

poudriere jail -c -j FreeBSD:10:amd64 -v 10.2-RELEASE -a amd64
			

Notice I used the ABI string from above. Your ports tree should be updated because you just got it. But I am going to include this, because this next part will be repeated pretty much every time you update your repository. I start by making sure my ports tree is updated:

poudriere ports -p default -u
			

Then I update my jail:

poudriere jail -u -j FreeBSD:10:amd64
			

And file we are on the subject, you can make multiple jails and compile multiple different versions. I'm not going to cover that. Also to list your jails run:

poudriere jail -l
			

Of course I didn't want to create every package in the repository, I only wanted a few for mail. So start by editing a file:

vi /usr/local/etc/poudriere.d/pkglist.txt
			

And then add the package names one per line. You can have comments as well. So our very basic postfix example:

mail/postfix
			

Notice the complete path. It matched the ports path without the /usr/ports at the start. Now I set my compile options. This is where the dialog package comes in handy.

poudriere options -cf /usr/local/etc/poudriere.d/pkglist.txt
			

In that interface customize what you need. Again I can't really tell you what these will be for your setup. Once you are complete, you can look in /usr/local/etc/poudriere.d/options directory and see them. This process also figures out dependencies and asks you questions on them as well. So even though you list 1 file you may be compiling 30.

Now we are getting closer to being done. I'd like to take a moment to recommend using tmux to run poudriere especially if you have a ton of packages. That said, to compile the items run:

poudriere bulk -f /usr/local/etc/poudriere.d/pkglist.txt -j FreeBSD:10:amd64
			

The important thing to note here, is that these builder jails actually use 127.0.0.1 as their IP Address. If you have any other jails with that IP assigned, it may fail with an IP Clash error.

Now that you will have a little time on your hands, lets to the website setup. Honestly it doesn't matter what server you use. You just need to alias /data to /data/logs/bulk and /packages to /data/packages under your zroot directory in your configuration. Also make sure you have Indexes on for the data/packages directory. And make sure your web document root is set to /usr/local/share/poudriere/html. You may also wish to lock down the web server, so only certain hosts/users can access different parts of the site. I actually used a jail and nullfs mounted some of these. Look up your specific web server for exact documentation.

Now that the repository is complete, lets cover some client configuration. Hopefully you already copied your public cert file. Now you need to make your repository directory:

mkdir -p /usr/local/etc/pkg/repos
			

For the sake of an example I'm going to call my repository local:

vi /usr/local/etc/pkg/repos/local.conf
			

Inside there put your local repository settings:

local: {
        url: "http://packages.mydomain.mine/packages/${ABI}-default/",
        mirror_type: "http",
        signature_type: "pubkey",
        pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
        fingerprints: "/usr/share/keys/pkg",
        enabled: yes
}
			

Make sure your base url matches, and because you named your jail the ABI string you can use a variable in there. This will prevent you from having to custom edit the file every time the version of FreeBSD changes. Also notice your certificate location. Make sure everything matches your setup.

And that's almost it. I'm actually going to mention that as of the current time and not necessarily any point beyond now, that the package system doesn't really allow you to prioritize repositories or pin packages to specific repositories. This is something Debian definitely did better. Again this comment is based on this snapshot of time. To get around this you can add the -r option and the name of your repository to the pkg command. For example:

pkg search -r local postfix
pkg install -r local postfix
			

And now you can use all this information, the update and build commands above and compile any package you like. There are also packages like the dvd library that can't be distributed in binary package mode. This comes in handy for that too. Or if your impatient and want a package now and the port is available, but the weekly build isn't done yet. There are many scenarios where having your own repository can be handy.