When upgrading packages on a FreshPorts ingress node (that is usually a jail which processes incoming commits and loads them into the database), I believe there is a possibility of interrupting commit processing. It’s a race condition between pkg removing a file and that file being required by the commit processing. This post serves to […]
The proper way to upgrade a FreshPorts ingress node Read More »
FreshPorts has three main daemons: ingress – waits for incoming commits and creates XML. freshports – processes the XML into the database. fp-listen – listens for events and clears cache The first two run on the backend (the ingress node, or jail), the last run runs on the webserver (nginx jail). The list of events
adding a new event to fp-listen Read More »
Following on from yesterdays Quarterly branches: what’s next? and Processing commits on branches with git, I’ve started in on clearing commits from cache. The code First step, create a table which looks like this: freshports.devgit=# \d cache_clearing_commits Table “public.cache_clearing_commits” Column | Type | Collation | Nullable | Default —————–+—————————–+———–+———-+——— commit_to_clear | text | | not
Commit cache clearing Read More »
With today’s work on Processing commits on branches with git, good progress was made. You can see commits on the quarterly branch (not up to date, because it’s not yet automated). You can see the port on the quarterly branch, if there’s a commit on the branch. Want to see a particular commit on a
Quarterly branches: what’s next? Read More »
Transitioning from subversion to git was both technically and personally challenging. During the COVID-19 pandemic there was a lot of work to carry out. A new server was created on AWS with a new layout and structure. For a few months it was nothing but my day job and FreshPorts in all my non-working hours.
Processing commits on branches with git Read More »
Today I discovered a problem which required the reprocessing of about 400 commits. How is that accomplished? delete the commits from the database tell the ingress daemon to start processing from commit x Deleting This works so long as all the commits have been process in order. The canonical method would query git to get
Rerunning all commits from a given point Read More »
Today at about 1:00 pm Philadelphia time, the website went offline. It was still there, it just could not answer any queries. In this post: Website and database server running in AWS FreeBSD 12.2 on a t3.medium EC2 instance PostgreSQL 12.5 running on a db.t2.large RDS instance Initial look Looking at load, it was acceptable:
lack of index brings website to a crawl Read More »
This post is a how-to and reminder for myself. I’m working on finding a better way for FreshPorts to know a new physical ports category when it finds it. I think the only way is to look at the directories in the FreeBSD ports repo. A discussion on IRC led to this shell script: [dan@devgit-ingress01:/var/db/ingress/repos/ports]
Generating XML for a single commit Read More »
This post is related to a recent tweet. It was months ago that I wondered why one port installed another as a dependency. I could see no link. Since it FreshPorts ports, earlier this week I did some testing: I removed the package, and it’s own dependencies from dev, then test, and finally stage. Monitoring
Removing unneeded packages Read More »
This post is mostly for personal use as I investigate this topic. It may take a few days and I need to keep notes. Last week, FreshPorts went live with git commits. There are some outstanding issues, such as commits to branches, which I will catch up on in the weeks to come. During that
Removing ‘unused’ tables Read More »