Jun 272021

This outlines the steps for putting the new git-delta.sh script into use. These steps will be required on each ingress node. stop ingress commit processing Look at the output of sudo tail /var/log/freshports/git.log and you should see something like this: [dan@devgit-ingress01:~] $ sudo tail /var/log/freshports/git.log 2021.06.27 17:21:05 git-delta.sh Repodir is /var/db/ingress/repos/src 2021.06.27 17:21:05 git-delta.sh Running: […]

Jun 272021

Following on from last weekend’s work, today I’m learning more about git, and the difference between Plumbing and Porcelain commands. The original intent was to use git tag -l freshports/$refname to detect if a tag exists. The problem there, it always returns 0. [dan@mydev:/var/db/ingress/repos/ports] $ git tag -l | head 10-eol 4-eol 5-eol 6-eol 7-eol […]

Jun 262021

When upgrading packages on a FreshPorts ingress node (that is usually a jail which processes incoming commits and loads them into the database), I believe there is a possibility of interrupting commit processing. It’s a race condition between pkg removing a file and that file being required by the commit processing. This post serves to […]

Jun 212021

FreshPorts has three main daemons: ingress – waits for incoming commits and creates XML. freshports – processes the XML into the database. fp-listen – listens for events and clears cache The first two run on the backend (the ingress node, or jail), the last run runs on the webserver (nginx jail). The list of events […]

Jun 212021

Following on from yesterdays Quarterly branches: what’s next? and Processing commits on branches with git, I’ve started in on clearing commits from cache. The code First step, create a table which looks like this: freshports.devgit=# \d cache_clearing_commits Table “public.cache_clearing_commits” Column | Type | Collation | Nullable | Default —————–+—————————–+———–+———-+——— commit_to_clear | text | | not […]

Jun 202021

With today’s work on Processing commits on branches with git, good progress was made. You can see commits on the quarterly branch (not up to date, because it’s not yet automated). You can see the port on the quarterly branch, if there’s a commit on the branch. Want to see a particular commit on a […]

Jun 202021

Transitioning from subversion to git was both technically and personally challenging. During the COVID-19 pandemic there was a lot of work to carry out. A new server was created on AWS with a new layout and structure. For a few months it was nothing but my day job and FreshPorts in all my non-working hours. […]

May 312021

Today I discovered a problem which required the reprocessing of about 400 commits. How is that accomplished? delete the commits from the database tell the ingress daemon to start processing from commit x Deleting This works so long as all the commits have been process in order. The canonical method would query git to get […]

May 292021
lack of index brings website to a crawl

Today at about 1:00 pm Philadelphia time, the website went offline. It was still there, it just could not answer any queries. In this post: Website and database server running in AWS FreeBSD 12.2 on a t3.medium EC2 instance PostgreSQL 12.5 running on a db.t2.large RDS instance Initial look Looking at load, it was acceptable: […]

May 092021

This post is a how-to and reminder for myself. I’m working on finding a better way for FreshPorts to know a new physical ports category when it finds it. I think the only way is to look at the directories in the FreeBSD ports repo. A discussion on IRC led to this shell script: [dan@devgit-ingress01:/var/db/ingress/repos/ports] […]