This post is the latest in a series of posts documenting the process of converting from using a chroot to using a full proper jail.
I spent about 2 hours of this fine Saturday morning writing this up and carrying out the steps.
As a result, both dev and test are now using a FreeBSD jail to extract data from ports in order to populate the database.
These are working notes, not so much a tutorial. However, you might find it useful too. I know I will when it comes to to convert stage tomorrow.
When you see mkjail below, that’s from sysutils/mkjail.
In this post:
- on the jail – means the parent jail
- on the child jail – means the child jail
- on the host – means the server, the main server, hosting all the jails
These are the steps used to update test-ingress01 from using ~freshports/ports-jail to /jails/freshports
2021-08-14
on the host:
sudo zfs snapshot system/jails/test-ingress01@before-moving-to-jail-freshports
on the jail:
sudo service freshports stop
sudo service ingress stop
sudo sysrc freshports_enable="NO"
sudo sysrc ingress_enable="NO"
on the host:
sudo service jail stop test-ingress01
sudo mv /etc/fstab.test-ingress01 /etc/fstab.test-ingress01.NOT-IN-USE
sudo zfs create -o mountpoint=none nvd/freshports/jailed/test-ingress01
sudo zfs create -o mountpoint=none nvd/freshports/jailed/test-ingress01/jails
sudo zfs create -o mountpoint=none nvd/freshports/jailed/test-ingress01/mkjail
sudoedit /etc/jail.conf
Add this to test-ingress01 jail definition:
allow.chflags;
# added when trying to get devfs in subjails
allow.mount.fdescfs;
allow.mount;
allow.mount.devfs;
allow.mount.linprocfs;
allow.mount.nullfs;
allow.mount.procfs;
allow.mount.tmpfs = 1;
allow.mount.zfs;
allow.raw_sockets;
allow.socket_af;
children.max=6;
exec.created+="zfs set jailed=on nvd/freshports/jailed/test-ingress01";
exec.created+="zfs jail $name nvd/freshports/jailed/test-ingress01";
exec.poststart += "jail -m allow.mount.linprocfs=1 name=$name";
enforce_statfs=1;
EDIT: 2021-09-09 - enforce_statfs was missing from the above - re https://twitter.com/DLangille/status/1435965521959215109
Start the jail:
sudo service jail start test-ingress01
in the jail:
verify zfs
[dan@test-ingress01:~] $ zfs list
[dan@test-ingress01:~] $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
nvd 81.7G 133G 23K none
nvd/freshports 81.5G 133G 23K none
nvd/freshports/jailed 2.48G 133G 24K none
nvd/freshports/jailed/test-ingress01 72K 133G 24K none
nvd/freshports/jailed/test-ingress01/jails 24K 133G 24K none
nvd/freshports/jailed/test-ingress01/mkjail 24K 133G 24K none
[dan@test-ingress01:~] $
Set mount points:
sudo zfs set mountpoint=/jails nvd/freshports/jailed/test-ingress01/jails
sudo zfs set mountpoint=/var/db/mkjail nvd/freshports/jailed/test-ingress01/mkjail
[dan@test-ingress01:~] $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
nvd 81.7G 133G 23K none
nvd/freshports 81.5G 133G 23K none
nvd/freshports/jailed 2.48G 133G 24K none
nvd/freshports/jailed/test-ingress01 72K 133G 24K none
nvd/freshports/jailed/test-ingress01/jails 24K 133G 24K /jails
nvd/freshports/jailed/test-ingress01/mkjail 24K 133G 24K /var/db/mkjail
[dan@test-ingress01:~] $
Creating the jail within a jail:
sudo pkg install mkjail
sudoedit /usr/local/etc/mkjail.conf :
ZPOOL="nvd"
JAILDATASET="freshports/jailed/test-ingress01/jails"
sudo mkjail create -a amd64 -j freshports -v 12.2-RELEASE
If that fails with:
tar: could not chdir to '/12.2-RELEASE/'
Then run the same command again.
sudo sysrc jail_enable="YES"
su to root, enter a bash shell, and run this:
cat << EOF > /etc/jail.conf
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
path = /jails/\$name;
allow.raw_sockets;
securelevel = 2;
exec.consolelog="/var/tmp/jail-\$name";
host.hostname = "\$name\$(hostname)";
persist;
freshports {
host.hostname = "freshports";
ip4 = inherit;
persist;
devfs_ruleset=0;
allow.mount=true;
enforce_statfs=1;
allow.mount.devfs;
allow.mount.procfs;
}
EOF
mkdir -p /jails/freshports/usr/local/etc/pkg/repos
echo "FreeBSD: { enabled: no }" > /jails/freshports/usr/local/etc/pkg/repos/FreeBSD.conf
cat << EOF > /jails/freshports/usr/local/etc/pkg/repos/local.conf
local: {
url: "pkg+http://fedex.unixathome.org/packages/122amd64-default-master-list/"
mirror_type: "srv",
signature_type: "PUBKEY",
pubkey: "/etc/ssl/slocum.unixathome.org.cert",
enabled: true
}
EOF
cat << EOF > /jails/freshports/etc/ssl/slocum.unixathome.org.cert
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAuveHTXwrwGmdWG6oFWgN
R/7bOQiQIE9iFmXmy4/MX9+01zMh2mjTfIOFe8IE1QqOK7X5EkVqhIdHCFg1Cg/x
Gl5oxxQEhp/HQbtVN7pNcpXKIGl0t5sFcjwXQqXS1wb87JP+v6KuOSTCFzS3l/X2
XtOo5QJZtxuF7+IM2PZlYb8MDmhVxriPw0pWRiP8lyW2LV1dSrN+VFpllZYinfIv
Sr7sUArIVlHH+Ddrm5MWjqTsHid36og26NDmAQnfX96IBF1sXadSTxKy3YwCnEcv
L7mBJhNyTIULuSbknM9zP9amkrlyLhWl+SdRGRkcOmXwHgzbdZsL62OlkXYIgkB+
tK099ARziCSe+sclhgfjoixnXxk0h9gUU6h5BDafATvtP4KDmwDYQEXO/7OPS0/H
vHFLEWLExcbW6hF0fyy2aA/HTlX83bBqJ+evsFtcvyxNfDp7tnjus/oAmJ82IU4F
0ZmWMoJDeNznO+3iokHH0J8vxa4kd1hjoSaDh1+qOZCXGRcsqyDTgWQEdQ5Uy76j
c9NgCJqHqJyjPqsiIZmPJNGUh1f7VaSUT8a131X5dwj6kx02s55UGJeIlK6F1d1f
7+7pdfv4rzHK3iPlP1eXQlv8szGAxdDbkSFqLK6gIC8V/Mf0B0zN0aU1JZkaAvoH
GopjN8IyF6fx/5yN9EAp8GUCAwEAAQ==
-----END PUBLIC KEY-----
EOF
cat << EOF > /jails/freshports/etc/resolv.conf
search unixathome.org int.unixathome.org
nameserver 10.55.0.1
nameserver 10.55.0.73
nameserver 10.55.0.13
EOF
cat << EOF > /jails/freshports/etc/rc.conf
cron_enable="NO"
syslogd_enable="NO"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
EOF
service jail start
jexec freshports
Nothing but this should be running:
root@freshports:/ # ps auwwx
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 8106 0.0 0.0 13196 3800 5 SJ 14:11 0:00.01 /bin/csh -i
root 8376 0.0 0.0 11768 2776 5 R+J 14:12 0:00.00 ps auwwx
pkg install pkg
exit
Now, back in the parent jail
cd /jails/freshports/usr
sudo git clone https://git.FreeBSD.org/ports.git
sudo mv ~freshports/ports-jail ~freshports/ports-jail.NO.LONGER.IN.USE
sudo pkg upgrade
cd /usr/local/etc/freshports
reconcile the configuration files:
sudoedit config.ini config.ini.sample
sudoedit config.pm config.pm.sample
sudo diff -ruN config.pm config.pm.sample | less
sudoedit config.sh config.sh.sample
sudo diff -ruN config.sh config.sh.sample | less
On ansible:
ansible-playbook freshports-ingress-git.yml --limit=test-ingress01.int.unixathome.org --tags=sudoers
on the parent jail:
sudo sysrc ingress_enable="YES"
sudo service ingress start
Watch the logs, wait for some commits.
sudo service ingress stop
cd /var/db/ingress/message-queues/incoming
sudo mv -i * ../holding/
sudo sysrc freshports_enable="YES"
sudo service freshports start
cd ../holding/
Move one of the xml files into incoming:
sudo mv -i BLAH.xml ../incoming/
check for errors, repeat until confidence is high.
