This is the first of several post reviewing and documenting the subversion to git conversion process.
In short, FreshPorts is split into two parts, which are explained by this high level overview:
- ingress – getting stuff into the database
- web – getting stuff from the database
That division of labor is not strictly delimited. Typically, each part runs in it’s own FreeBSD jail, usually named:
- ingress01
- nginx01
In this post, I’ll concentrate on the ingress daemon, a shell script.
What runs on an ingress01?
The term ingress refers to the part of FreshPorts which loads data into the database. This is usually a jail called ingress01, or in my dev host, dev-ingress01, test-ingress01, etc. It is the ingress node which also runs the reporting for the users (new ports, updated ports, expired ports, etc).
The FreshPorts packages used within an ingress node include:
[11:27 test-ingress01 dvl ~] % pkg info -x fresh freshports-freshports-2.0.16 freshports-helper_scripts-0.0.6 freshports-ingress-2.0.4_15 freshports-ingress-metaport-0.1.1 freshports-periodics-1.1.1_1 p5-freshports-modules-2.2.1 p5-freshports-scripts-2.2.6 p5-freshports-verify-2.2.0 py312-freshports-git-proc-commit-0.2.3
All of those SHOULD be found in the FreshPorts ports repo. The freshports-ingress-metaport is used to install other packages. Install freshports-ingress-metaport, and you should have everything you need in an ingress node.
Let’s start with the freshports-ingress package, now at version 2.0.4_15
Background: the jails dvl-ingress01 & dev-ingress01 both run on mostly-checked out code, not packages. That is, I have a working copy of the code and that is what is use to run the scripts, etc. However, test-ingress01 runs on packages; there are no working copies at all.
What does the freshports-ingress package install?
[12:09 test-ingress01 dvl ~/tmp] % pkg info -l freshports-ingress freshports-ingress-2.0.4_15: /usr/local/etc/freshports/ingress.sh.sample /usr/local/etc/newsyslog.conf.d/ingress.conf.sample /usr/local/etc/rc.d/ingress /usr/local/etc/syslog.d/ingress-daemon.conf.sample /usr/local/libexec/freshports-service/ingress.sh
OK, looking on dvl-ingress01, let’s compare working copy against installed package
Background: this is a typical ingress node:
[12:12 dvl-ingress01 dvl ~] % ps auwwx USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND ingress 20472 0.0 0.0 14068 2428 - SCJ 12:13 0:00.00 sleep 3 freshports 22107 0.0 0.0 14068 2428 - SCJ 12:13 0:00.00 sleep 3 root 27576 0.0 0.0 25708 12540 - IsJ 12:11 0:00.03 sshd-session: dvl [priv] (sshd-session) dvl 27719 0.0 0.0 25708 12840 - SJ 12:11 0:00.12 sshd-session: dvl@pts/3 (sshd-session) root 59483 0.0 0.0 14432 3296 - SCsJ 2May26 0:17.76 /usr/sbin/syslogd -s root 59488 0.0 0.0 14432 3124 - IJ 2May26 0:00.06 syslogd: syslogd.casper (syslogd) root 59490 0.0 0.0 14432 3096 - IsJ 2May26 0:00.00 syslogd: system.net (syslogd) root 59690 0.0 0.0 14280 2812 - SsJ 2May26 0:05.29 /usr/sbin/cron -s nagios 59752 0.0 0.0 21796 9492 - SsJ 2May26 0:17.12 /usr/local/sbin/nrpe -c /usr/local/etc/nrpe.cfg -d root 59922 0.0 0.0 25200 11908 - SsJ 2May26 0:17.92 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups (sshd) freshports 60639 0.0 0.0 14184 2588 - IsJ 2May26 0:00.27 daemon: freshports[60640] (daemon) freshports 60640 0.0 0.0 14744 3152 - SJ 2May26 0:45.53 /bin/sh /usr/local/libexec/freshports-service/freshports.sh ingress 60712 0.0 0.0 14184 2576 - IsJ 2May26 0:00.63 daemon: ingress[60713] (daemon) ingress 60713 0.0 0.0 14744 3028 - SJ 2May26 0:31.40 /bin/sh /usr/local/libexec/freshports-service/ingress.sh root 72878 0.0 0.0 25708 12536 - IsJ 11:15 0:00.03 sshd-session: dvl [priv] (sshd-session) dvl 72950 0.0 0.0 25708 12836 - IJ 11:15 0:00.13 sshd-session: dvl@pts/5 (sshd-session) dvl 22271 0.0 0.0 14824 3344 3 R+J 12:13 0:00.00 ps auwwx dvl 27786 0.0 0.0 18896 7632 3 SsJ 12:11 0:00.08 -zsh (zsh) dvl 72976 0.0 0.0 21852 8096 5 Is+J 11:15 0:00.10 -zsh (zsh)
There are two main daemons, both shell scrips, which (in short) do this:
- ingress – pulls data from git, creates xml files
- freshports – extracts data from xml file, loads it into the database
The freshports daemon has been around for decades. The ingress daemon was create to handle git.
dvl-ingress01 is part of dvl.freshports.org – I used to do all my development work on dev.freshports.org – but I kept breaking stuff while people were reviewing, so I created dvl.freshports.org so I could break that instead.
Where can I find that main daemon code?
[12:11 dvl-ingress01 dvl ~] % locate ingress.sh /usr/home/dvl/src/fp-listen/branches/git/ingress/config/ingress.sh /usr/home/dvl/src/fp-listen/branches/git/ingress/ingress.sh /usr/home/dvl/src/fp-listen/tags/fp-ingress-2.0.0/fp-ingress.sh /usr/home/dvl/src/fp-listen/tags/fp-ingress-2.0.1/fp-ingress/fp-ingress.sh /usr/home/dvl/src/fp-listen/tags/fp-ingress-2.0.1/fp-ingress.sh /usr/home/dvl/src/fp-listen/tags/fp-ingress-2.0.2/fp-ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.3/config/ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.3/ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.4/config/ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.4/ingress/config/ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.4/ingress/ingress.sh /usr/home/dvl/src/fp-listen/tags/ingress-2.0.4/ingress.sh /usr/local/etc/freshports/ingress.sh /usr/local/etc/freshports/ingress.sh.sample /usr/local/libexec/freshports-service/ingress.sh
Let’s compare the package code with the working copy code:
[12:11 dvl-ingress01 dvl ~] % diff -ruN /usr/home/dvl/src/fp-listen/branches/git/ingress/ingress.sh /usr/local/libexec/freshports-service/ingress.sh
Ahh, that working copy is git:
[12:50 dvl-ingress01 dvl ~/src/fp-listen/branches/git/ingress] % git remote -v origin git@github.com:FreshPorts/fp-listen.git (fetch) origin git@github.com:FreshPorts/fp-listen.git (push)
As found at https://github.com/FreshPorts/fp-listen/blob/main/branches/git/ingress/ingress.sh
Let me compare:
[12:50 dvl-ingress01 dvl ~/src/fp-listen/branches/git/ingress] % fetch -o ~/tmp/ingress.sh https://raw.githubusercontent.com/FreshPorts/fp-listen/refs/heads/main/branches/git/ingress/ingress.sh /usr/home/dvl/tmp/ingress.sh 1387 B 3021 kBps 00s [12:51 dvl-ingress01 dvl ~/src/fp-listen/branches/git/ingress] % diff -ruN ~/tmp/ingress.sh ingress.sh
That matches.
Looking at the 2.0.4 directory, that matches too:
[12:54 dvl-ingress01 dvl ~/src/fp-listen/tags/ingress-2.0.4] % diff -ruN ~/tmp/ingress.sh ingress.sh [12:54 dvl-ingress01 dvl ~/src/fp-listen/tags/ingress-2.0.4] % diff -ruN ~/tmp/ingress.sh ~/src/fp-listen/branches/git/ingress [12:55 dvl-ingress01 dvl ~/src/fp-listen/tags/ingress-2.0.4] % diff -ruN ~/tmp/ingress.sh /usr/local/libexec/freshports-service/ingress.sh [12:55 dvl-ingress01 dvl ~/src/fp-listen/tags/ingress-2.0.4] %
Installed versions
All three hosts have the same installed, via package:
[15:17 dvl-ingress01 dvl ~] % md5 /usr/local/libexec/freshports-service/ingress.sh MD5 (/usr/local/libexec/freshports-service/ingress.sh) = 5ab777b489f34cf0c6824e0c7fc4c5e2 [15:17 dev-ingress01 dvl ~] % md5 /usr/local/libexec/freshports-service/ingress.sh MD5 (/usr/local/libexec/freshports-service/ingress.sh) = 5ab777b489f34cf0c6824e0c7fc4c5e2 [15:17 test-ingress01 dvl ~] % md5 /usr/local/libexec/freshports-service/ingress.sh MD5 (/usr/local/libexec/freshports-service/ingress.sh) = 5ab777b489f34cf0c6824e0c7fc4c5e2
dvl-ingress01 is still using svn
dev-ingress01 is still using subversion, but nothing there has changed recently.
[15:22 dev-ingress01 dvl ~/src/daemon-ingress/config] % svn log | head ------------------------------------------------------------------------ r5498 | dan | 2020-11-29 16:32:25 +0000 (Sun, 29 Nov 2020) | 2 lines Correct FLAGDIR ------------------------------------------------------------------------ r5425 | dan | 2020-07-19 16:32:34 +0000 (Sun, 19 Jul 2020) | 3 lines Mention that our MSGDIR is also defined in the freshports configuration file [15:22 dev-ingress01 dvl ~/src/daemon-ingress/config] % [15:22 dev-ingress01 dvl ~/src/daemon-ingress/config] % svn info Path: . Working Copy Root Path: /usr/home/dvl/src/daemon-ingress URL: svn+ssh://svnusers@svn.int.unixathome.org/freshports-1/daemontools/branches/git/ingress/config Relative URL: ^/daemontools/branches/git/ingress/config Repository Root: svn+ssh://svnusers@svn.int.unixathome.org/freshports-1 Repository UUID: 46e82423-29d8-e211-989e-002590a4cdd4 Revision: 5955 Node Kind: directory Schedule: normal Last Changed Author: dan Last Changed Rev: 5498 Last Changed Date: 2020-11-29 16:32:25 +0000 (Sun, 29 Nov 2020)
Conclusion
I think FreshPorts/fp-listen is the repo to use for ingress.
[15:21 dvl-ingress01 dvl ~] % cd /usr/home/dvl/src/fp-listen/branches/git/ingress [15:24 dvl-ingress01 dvl ~/src/fp-listen/branches/git/ingress] % git remote -v origin git@github.com:FreshPorts/fp-listen.git (fetch) origin git@github.com:FreshPorts/fp-listen.git (push)
Work to be done
We need to tag releases for ingress, pull the release tarballs into a FreeBSD build host, and create the package.
We have this:
[15:27 dvl-ingress01 dvl ~/src/fp-listen/branches/git] % ls -l total 20 drwxr-xr-x 3 dvl dvl 6 2025.07.06 11:44 fp-daemon/ drwxr-xr-x 4 dvl dvl 6 2025.07.06 11:44 fp-listen/ drwxr-xr-x 3 dvl dvl 5 2025.07.06 11:44 freshports/ drwxr-xr-x 3 dvl dvl 5 2026.05.12 13:06 ingress/ drwxr-xr-x 3 dvl dvl 5 2025.07.06 11:44 ingress_svn/
The same may apply to freshports and fp-listen. They have not been reviewed yet.
