FreshPorts News

The July 4th tweet announced “dev, test, and stage .freshports.org are now running Python 3.9”. Three days later, “Is it coincidental that dev, test, and stage are all stuck on commits from July 4?”

I took notes as I explored the problem. The first clue was:

2022.07.07 00:12:03 git-delta.sh /usr/local/libexec/freshports/git-to-freshports-xml.py –repo ports –path /var/db/ingress/repos/ports –branch main –commit-range e92af65a1220ad4e3a567f67e3ab0ff259910631..271ce919763ea8e3bad7f642f13e4cc13fbad5ce –spooling /var/db/ingress/message-queues/spooling –output /var/db/ingress/message-queues/incoming
/usr/local/libexec/freshports/git-delta.sh: /usr/local/libexec/freshports/git-to-freshports-xml.py: not found

That file exists:

[dev-ingress01 dan /var/log/freshports] % pkg which /usr/local/libexec/freshports/git-to-freshports-xml.py                                                                                                                         19:27:27
/usr/local/libexec/freshports/git-to-freshports-xml.py was installed by package py39-freshports-git-proc-commit-0.1.1

The problem was the shebang:

[dev-ingress01 dan /var/log/freshports] % head /usr/local/libexec/freshports/git-to-freshports-xml.py                                                                                                                              19:27:46
#!/usr/local/bin/python3.8


Yeah, I shouldn't be doing that.

The code was fixed. Later, I modified the code to raise alarms which would be noticed.

Catching up on missing code

The code is now catching up on commits made after July 7th, because that’s when commit processing was manually stopped. Next, the system needs to process commits between July 4th and July 7th.

When a commits are processed, the hash of the last commit found is stored as a tag in the local copy of the repo. This tag is used as a reference when processing next occurs. It is the starting point. The old code was updating the even though processing failed to create the required XML file[s]. The new code does not not make this error.

My job now is is locate the hash of the missing code and reset it.

Looking at the data

Here is the SQL I ran. I should put this into a page on FreshPorts. It might be useful.

freshports.devgit=# select * from repo;
 id | name  |      description       |   repo_hostname    | path_to_repo | repository | db_root_prefix 
----+-------+------------------------+--------------------+--------------+------------+----------------
  1 | ports | The FreeBSD Ports tree | svnweb.freebsd.org | /ports       | subversion | 
  2 | doc   | The FreeBSD doc tree   | svnweb.freebsd.org | /doc         | subversion | 
  3 | src   | The FreeBSD src tree   | svnweb.freebsd.org | /base        | subversion | 
  9 | src   | The FreeBSD src tree   | cgit.freebsd.org   | /src         | git        | /base
  8 | doc   | The FreeBSD doc tree   | cgit.freebsd.org   | /doc         | git        | /doc
  7 | ports | The FreeBSD Ports tree | cgit.freebsd.org   | /ports       | git        | /ports
(6 rows)


freshports.devgit=# select id, message_id from commit_log where repo_id = 7 order by message_date desc limit 1;
   id   |                message_id                
--------+------------------------------------------
 911534 | db5eeb201b93c19c42724ec98f8ca1be9e88db15
(1 row)

freshports.devgit=# select id, message_id from commit_log where repo_id = 8 order by message_date desc limit 1;
   id   |                message_id                
--------+------------------------------------------
 911515 | 21de22b400a3970b2d3d4b2831c44702698f57e7
(1 row)

freshports.devgit=# select id, message_id from commit_log where repo_id = 9 order by message_date desc limit 1;
   id   |                message_id                
--------+------------------------------------------
 911537 | 03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29
(1 row)

freshports.devgit=#

I ran similar queries on test and stage to see how that works. They had different id values for repo_id, but the results were the same. I expect this, but verified it. This means I can run the same git tag commands on each database instance.

I know that I can do that in one query, but I can’t recall how. I have it in code somewhere.

Confirming the dates

I confirmed the dates of the above commits and compared it against the Python upgrade.

1. src: 03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29 2022.07.04 20:45:04
2. doc: 21de22b400a3970b2d3d4b2831c44702698f57e7 2022.07.04 16:51:02
3. ports: db5eeb201b93c19c42724ec98f8ca1be9e88db15 2022.07.04 20:30:03

When was Python updated?

[dev-ingress01 dan /usr/local/libexec/freshports] % grep python39 /var/log/messages
Jul  4 20:49:21 dev-ingress01 pkg[66393]: python39-3.9.13 installed

Those dates look right.

Where I started to understand the tag

This is where I got confused and was not sure of the exact approach. The code did this

git tag -m “last known commit of ${refname}” -f freshports/${refname} ${refname}

The confusing part was parsing the above:

git tag -m “last known commit of main/origin” -f freshports/origin/main origin/main

I expected this command to take a commit hash. In a way, it is. It’s saying: the last commit I processed is the most recent commit (the one at the top).

Ahh! OK, but I can also supply a hash.

Let’s review the command:

1. -m Use the given tag message
2. -f Replace an existing tag with the given name (instead of failing)

What this does: it tags the last commit with freshports/origin/main.

The code uses that later. See my annotation of the code where I explain the code to myself. See also my attempt to find an online code reviewer and my earlier misunderstanding and reading of that code. That confusion lasted about 2 hours but I’m much better for it now.

Setting the tag

Let’s do it for doc first, because that’s a much lower volume repo.

Setting this will cause duplicate commits to be processed, but if a commit is already in FreshPorts, it is not reloaded into the database. I could invoke git-to-freshports-xml.py with a range, but I’d rather do it this way.

This would be:

git tag -m “last known commit of main/origin” -f freshports/origin/main 21de22b400a3970b2d3d4b2831c44702698f57e7

That hash value is from an earlier section.

[dev-ingress01 dan ~ingress/repos/doc] % echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main 21de22b400a3970b2d3d4b2831c44702698f57e7' | sudo su -fm ingress
Updated tag 'freshports/origin/main' (was 6a70cd2969)

Now I wait…

There we go:

[dev-ingress01 dan ~ingress/repos/ports] % sudo xtail /var/log/freshports

*** /var/log/freshports/freshports.log ***
Jul  9 16:51:00 dev-ingress01 FreshPorts[10344]: into /usr/local/etc/periodic/everythreeminutes/100.fp_system_status
Jul  9 16:51:00 dev-ingress01 FreshPorts[10392]: into /usr/local/etc/periodic/everythreeminutes/215.fp_check_git_for_commits

*** /var/log/freshports/ingress-daemon.log ***
Jul  9 16:51:00 dev-ingress01 check_for_git_commits.sh[10398]: touching ~ingress/signals/check_git ~ingress/signals/job_waiting
Jul  9 16:51:00 dev-ingress01 check_for_git_commits.sh[10428]: done touching, going away now
Jul  9 16:51:02 dev-ingress01 ingress[71055]: yes, there is a job waiting
Jul  9 16:51:02 dev-ingress01 ingress[71055]: running /usr/local/bin/perl ./job-waiting.pl
Jul  9 16:51:02 dev-ingress01 ingress[71055]: from directory  /usr/local/libexec/freshports
Jul  9 16:51:02 dev-ingress01 ingress[71055]: -rw-r--r--  1 dan  dan  2571 Apr 24 18:40 ./job-waiting.pl

*** /var/log/freshports/freshports.log ***
Jul  9 16:51:03 dev-ingress01 FreshPorts[11369]: running job-waiting.pl 
Jul  9 16:51:03 dev-ingress01 FreshPorts[11369]: starting ./job-waiting.pl (/usr/local/libexec/freshports) 
Jul  9 16:51:03 dev-ingress01 FreshPorts[11369]: running ./job-waiting.pl as user = 'ingress' (/usr/local/libexec/freshports) 
Jul  9 16:51:03 dev-ingress01 FreshPorts[11369]: checking jobs for ingress (/usr/local/libexec/freshports) 
Jul  9 16:51:03 dev-ingress01 FreshPorts[11369]: /var/db/ingress/signals/check_git exists.  About to run check_git.sh (/usr/local/libexec/freshports) 
Jul  9 16:51:03 dev-ingress01 check_git.sh[11437]: /usr/local/libexec/freshports/check_git.sh has started
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '16f7503065af6905cf0c89f6bbe0c5f4e22d7058 website/themes/beastie/i18n/fr.toml: Fix some typos'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit 'd0e5718b926292f7e85b47a28c05fe7f5a143554 website/content/fr/where.adoc: Fix translation'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '21b5dd496e13ab140eee747d64e9dc4b477a222c - traditional Chinese Translation of the news items (March 2022)'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '5286308fc42391f43fa675580dd67e12521490bb fix build'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '763a2c6fbdf6f219a8c4d2880c25212d3af7467f - traditional Chinese Translation of the news items (April 2022)'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '19586a4dd1e7c2b65f9563e7c49c04814f8ab153 - traditional Chinese Translation of the news items (May 2022)'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '47dcae406e2e0ee34e3766dd9ee04ae4d10642b8 - traditional Chinese Translation of the news items (June 2002)'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit 'a6a3fc9c74a1a055bff161440220ca819caa15a8 fix typo and an untranslated item.'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit 'f814150c1af932259a239ad01c0237f58bdce402 New committer (doc): Graham Perrin, grahamperrin@'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '3a4370a41f988ad0b30f1f256c6ac73aa6e9b248 - traditional Chinese Translation of the latest news item (2022-06-01)'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit 'd9b55a60fba211836e97d39920c55a83cc4443b6 Update information about the clusteradm team'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit 'e321dc3829ee30d5eebdd4d14027dba139d85fc1 [doc-es][articles/contributors]: Keep up with latest changes'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '346a108dfb5f7bbfc2ccbe844c6a6a1aff449442 [doc-es][articles/contributing] Keep up with latest changes'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '43995b87b0ab9b5250283f5338ba824ee9bf32a8 Document FreeBSD version 1400062 and 1400063'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '8ae6611622a426406377d55a0b75ad9965a74ac2 Document FreeBSD version 1301505'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '393a57488dec89f58455ffdb9d3f94dcb03bf080 Document FreeBSD version 1203506 and 1203507'
Jul  9 16:51:03 dev-ingress01 git-to-freshports-xml.py[11506]:[INFO] Processing commit '31fe9dc1198f997305ace049b055254007fc7963 Update CORE Team liaison'

Checking… that first commit is from 4 July.

Looking in the logs, here’s one which was processed but already in the database:

[dev-ingress01 dan ~freshports/message-queues/recent] % cat 2022.07.08.20.04.11.000016.31fe9dc1198f997305ace049b055254007fc7963.log                                                      16:53:36
grabbing categories from disk
'accessibility arabic archivers astro audio base benchmarks biology cad chinese comms converters databases deskutils devel distfiles dns editors emulators finance french ftp games german graphics hebrew hungarian irc japanese java korean lang mail math misc multimedia net net-im net-mgmt net-p2p news polish ports-mgmt portuguese print russian science security shells sysutils textproc ukrainian vietnamese www x11 x11-clocks x11-drivers x11-fm x11-fonts x11-servers x11-themes x11-toolkits x11-wm'
categories already loaded
about to process
finished setting up the Parser
Processing file [/var/db/ingress/message-queues/incoming/2022.07.08.20.04.11.000016.31fe9dc1198f997305ace049b055254007fc7963.xml]...
parsing file now


 *** start of all updates ***


 *** end of all updates *** 
source in getSource is 'git'
That commit is of Type: 'git'
EOF
invoking XML_Munge_git because this is a git commit
about to process
Observer has noticed that processing has begun.
finished setting up the Parser
Processing file [/var/db/ingress/message-queues/incoming/2022.07.08.20.04.11.000016.31fe9dc1198f997305ace049b055254007fc7963.xml]...
dbname = freshports.devgit
parsing file now


 *** start of all updates ***

 --- start of an update --- 

 --- end of OS --- 
OS is 'FreeBSD' : branch = 'main' for git
after converting '$Updates{branch_git}' we have 'head'
next we need to strip any leading 'branches/' prefix
OS is 'FreeBSD' : branch = 'main' for git
OS is 'FreeBSD' : branch = 'head' for git
OS is 'FreeBSD' : branch = 'head' for database names
sql = 'select SystemIDGet('FreeBSD')'
SystemBranchIDGetOrCreate has converted 'head' to 'head' which will be used in the database
sql = 'select SystemBranchIDGet(1, 'head')'
OS is 'FreeBSD' (1) : branch = head (1)
found Committer= [bofh]
found Author= [Muhammad Moinur Rahman]
OS                   = [FreeBSD]
Branch git           = [main]
branch_database_name = [head]
branch_for_files     = [head]
Committer            = [bofh]
Date                 = [2022/07/08 20:04:11 UTC]
Repository     = [doc]
Revision       = [31fe9dc1198f997305ace049b055254007fc7963]
MessageId      = [31fe9dc1198f997305ace049b055254007fc7963]
short hash     = [31fe9dc]
Subject        = [Update CORE Team liaison]
Log            = [Update CORE Team liaison

The new CORE.12 has decided on their roles about liaisons with other
administrative teams.

Approved by:	core (bofh with core-secretary@ hat on)]
into handle_message_end, let's save that message now!

xml_munge_git.pm::SaveUpdateToDB --- start
GetExistingMessageID => sql=select id from commit_log where message_id = '31fe9dc1198f997305ace049b055254007fc7963'
message 31fe9dc1198f997305ace049b055254007fc7963 has already been added to the database
no commit id returned.  we'll just exit now shall we?

Good. Now I feel better. Let’s try src next.

src

First, the magic:

[dev-ingress01 dan ~ingress/repos/src] % echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main 03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29' | sudo su -fm ingress
Updated tag 'freshports/origin/main' (was ea1d45c89a1)

There we go:

*** /var/log/freshports/freshports.log ***
Jul  9 16:57:00 dev-ingress01 FreshPorts[43668]: into /usr/local/etc/periodic/everythreeminutes/100.fp_system_status
Jul  9 16:57:00 dev-ingress01 FreshPorts[43694]: into /usr/local/etc/periodic/everythreeminutes/215.fp_check_git_for_commits

*** /var/log/freshports/ingress-daemon.log ***
Jul  9 16:57:00 dev-ingress01 check_for_git_commits.sh[43698]: touching ~ingress/signals/check_git ~ingress/signals/job_waiting
Jul  9 16:57:00 dev-ingress01 check_for_git_commits.sh[43724]: done touching, going away now
Jul  9 16:57:02 dev-ingress01 ingress[71055]: yes, there is a job waiting
Jul  9 16:57:02 dev-ingress01 ingress[71055]: running /usr/local/bin/perl ./job-waiting.pl
Jul  9 16:57:02 dev-ingress01 ingress[71055]: from directory  /usr/local/libexec/freshports
Jul  9 16:57:02 dev-ingress01 ingress[71055]: -rw-r--r--  1 dan  dan  2571 Apr 24 18:40 ./job-waiting.pl

*** /var/log/freshports/freshports.log ***
Jul  9 16:57:02 dev-ingress01 FreshPorts[43818]: running job-waiting.pl 
Jul  9 16:57:02 dev-ingress01 FreshPorts[43818]: starting ./job-waiting.pl (/usr/local/libexec/freshports) 
Jul  9 16:57:02 dev-ingress01 FreshPorts[43818]: running ./job-waiting.pl as user = 'ingress' (/usr/local/libexec/freshports) 
Jul  9 16:57:02 dev-ingress01 FreshPorts[43818]: checking jobs for ingress (/usr/local/libexec/freshports) 
Jul  9 16:57:02 dev-ingress01 FreshPorts[43818]: /var/db/ingress/signals/check_git exists.  About to run check_git.sh (/usr/local/libexec/freshports) 
Jul  9 16:57:02 dev-ingress01 check_git.sh[43836]: /usr/local/libexec/freshports/check_git.sh has started

*** /var/log/freshports/git.log ***
2022.07.09 16:57:02 git-delta.sh has started. Will check these repos: 'doc ports src'
2022.07.09 16:57:02 git-delta.sh XML dir is /var/db/ingress/message-queues/incoming
2022.07.09 16:57:02 git-delta.sh Now processing repo: doc ---------------
2022.07.09 16:57:02 git-delta.sh REPODIR='/var/db/ingress/repos/doc' exists
2022.07.09 16:57:02 git-delta.sh Repodir is /var/db/ingress/repos/doc
2022.07.09 16:57:02 git-delta.sh Running: /usr/local/bin/git fetch:
2022.07.09 16:57:02 git-delta.sh fetch completed.
2022.07.09 16:57:02 git-delta.sh working on 'origin/main'
2022.07.09 16:57:02 git-delta.sh Is freshports/origin/main defined on the repo 'doc'?
2022.07.09 16:57:02 git-delta.sh running: git rev-parse -q --verify freshports/origin/main^{}
31fe9dc1198f997305ace049b055254007fc7963
2022.07.09 16:57:02 git-delta.sh the latest commit we have for freshports/origin/main is:
31fe9dc1198f997305ace049b055254007fc7963
2022.07.09 16:57:02 git-delta.sh Running: /usr/local/bin/git rev-list freshports/origin/main..origin/main
2022.07.09 16:57:02 git-delta.sh Done.
2022.07.09 16:57:02 git-delta.sh No commits were found
2022.07.09 16:57:02 git-delta.sh Now processing repo: ports ---------------
2022.07.09 16:57:02 git-delta.sh REPODIR='/var/db/ingress/repos/ports' exists
2022.07.09 16:57:02 git-delta.sh Repodir is /var/db/ingress/repos/ports
2022.07.09 16:57:02 git-delta.sh Running: /usr/local/bin/git fetch:
2022.07.09 16:57:03 git-delta.sh fetch completed.
2022.07.09 16:57:03 git-delta.sh working on 'origin/2021Q2'
2022.07.09 16:57:03 git-delta.sh Is freshports/origin/2021Q2 defined on the repo 'ports'?
2022.07.09 16:57:03 git-delta.sh running: git rev-parse -q --verify freshports/origin/2021Q2^{}
d1da14bab7a800be62786aeb321b781179ea8b3f
2022.07.09 16:57:03 git-delta.sh the latest commit we have for freshports/origin/2021Q2 is:
d1da14bab7a800be62786aeb321b781179ea8b3f
2022.07.09 16:57:03 git-delta.sh Running: /usr/local/bin/git rev-list freshports/origin/2021Q2..origin/2021Q2
2022.07.09 16:57:03 git-delta.sh Done.
2022.07.09 16:57:03 git-delta.sh No commits were found
2022.07.09 16:57:03 git-delta.sh working on 'origin/2021Q3'
2022.07.09 16:57:03 git-delta.sh Is freshports/origin/2021Q3 defined on the repo 'ports'?
2022.07.09 16:57:03 git-delta.sh running: git rev-parse -q --verify freshports/origin/2021Q3^{}
5d6eb9d394c16d3e3ff4816090470feb64b39239
2022.07.09 16:57:03 git-delta.sh the latest commit we have for freshports/origin/2021Q3 is:
5d6eb9d394c16d3e3ff4816090470feb64b39239
2022.07.09 16:57:03 git-delta.sh Running: /usr/local/bin/git rev-list freshports/origin/2021Q3..origin/2021Q3
2022.07.09 16:57:03 git-delta.sh Done.
2022.07.09 16:57:03 git-delta.sh No commits were found
2022.07.09 16:57:03 git-delta.sh working on 'origin/main'
2022.07.09 16:57:03 git-delta.sh Is freshports/origin/main defined on the repo 'ports'?
2022.07.09 16:57:03 git-delta.sh running: git rev-parse -q --verify freshports/origin/main^{}
f46a28a1965dcaae036a25b7f387cc5304a3890a
2022.07.09 16:57:03 git-delta.sh the latest commit we have for freshports/origin/main is:
f46a28a1965dcaae036a25b7f387cc5304a3890a
2022.07.09 16:57:03 git-delta.sh Running: /usr/local/bin/git rev-list freshports/origin/main..origin/main
2022.07.09 16:57:03 git-delta.sh Done.
2022.07.09 16:57:03 git-delta.sh No commits were found
2022.07.09 16:57:03 git-delta.sh Now processing repo: src ---------------
2022.07.09 16:57:03 git-delta.sh REPODIR='/var/db/ingress/repos/src' exists
2022.07.09 16:57:03 git-delta.sh Repodir is /var/db/ingress/repos/src
2022.07.09 16:57:03 git-delta.sh Running: /usr/local/bin/git fetch:
2022.07.09 16:57:03 git-delta.sh fetch completed.
2022.07.09 16:57:03 git-delta.sh working on 'origin/main'
2022.07.09 16:57:03 git-delta.sh Is freshports/origin/main defined on the repo 'src'?
2022.07.09 16:57:03 git-delta.sh running: git rev-parse -q --verify freshports/origin/main^{}
03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29
2022.07.09 16:57:03 git-delta.sh the latest commit we have for freshports/origin/main is:
03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29
2022.07.09 16:57:03 git-delta.sh Running: /usr/local/bin/git rev-list freshports/origin/main..origin/main
2022.07.09 16:57:03 git-delta.sh Done.
2022.07.09 16:57:03 git-delta.sh The commits found are:
2022.07.09 16:57:03 git-delta.sh dff31ae1c59cab9437e88bfd0f2abd35ddaa98f1
2022.07.09 16:57:03 git-delta.sh 9ef1127008ce94cf626daed346a3c1ee03063617
2022.07.09 16:57:03 git-delta.sh eec3290266bc09b4c4b4d875d2269d611adc0016
2022.07.09 16:57:03 git-delta.sh 8f9972075cb3864d47a5796eb1abdb0f4d1be8fc
2022.07.09 16:57:03 git-delta.sh aeb6948d4319fdaef8a7a6ea72717968cf5ef79b

…

ports

The main event:

[dev-ingress01 dan ~ingress/repos/ports] % echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main db5eeb201b93c19c42724ec98f8ca1be9e88db15' | sudo su -fm ingress
Updated tag 'freshports/origin/main' (was 8f48d2bcbf66)

As I type this, the system is both creating new XML files from the ports git repo while it’s processing the above commits from the src repo. I like queues, even if it’s only implemented via files on disk in a directory. Here I am checking the queue:

[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     16:59:40
      13
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:07
      26
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:09
      37
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:11
     105
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:23
     117
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:25
     145
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:30
     156
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:00:32
     318
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:01:01
     492
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:01:30
     552
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:02:49
     552
[dev-ingress01 dan ~ingress/repos/ports] % ls ~ingress/message-queues/incoming | wc -l     17:02:51
     551
[dev-ingress01 dan ~ingress/repos/ports] %                                                 17:02:56

test

I’m collecting the above commands here for future reference, when I do stage.

cd doc
echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main 21de22b400a3970b2d3d4b2831c44702698f57e7' | sudo su -fm ingress
cd -
cd src
echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main 03473e8ec8fa8d0f1ea30f85d8796ea9bf94bf29' | sudo su -fm ingress
cd -
cd ports
echo 'git tag -m "last known commit of main/origin" -f freshports/origin/main db5eeb201b93c19c42724ec98f8ca1be9e88db15' | sudo su -fm ingress

I did a sudo pkg upgrade on test, then ran the above commands, then started ingress and freshports services.

All three FreshPorts nodes are now updating.