I’m pretty sure I want to implement more caching on FreshPorts. My first thought was to write the files to disk and then remove them when they need updating. I recall a caching program that either works with PostgreSQL is is implemented in PostgreSQL. Putting to cache into a database would be interesting. It’s not […]
The commits.php page has a long history dating back to 2006 in GitHub and before that in another repo. Soon, it is going away. The justification is in issue 337: it’s not used much, and it’s not doing as it should (loading the last 100 commits). In the early days, the front page showed only
/commits.php is going away Read More »
With today’s work on Processing commits on branches with git, good progress was made. You can see commits on the quarterly branch (not up to date, because it’s not yet automated). You can see the port on the quarterly branch, if there’s a commit on the branch. Want to see a particular commit on a
Quarterly branches: what’s next? Read More »
This post is mostly for personal use as I investigate this topic. It may take a few days and I need to keep notes. Last week, FreshPorts went live with git commits. There are some outstanding issues, such as commits to branches, which I will catch up on in the weeks to come. During that
Removing ‘unused’ tables Read More »
This is a cheat sheet for FreshPorts behind-the-scenes administration. If you want one for users, please start with suggested content. Ingress Actions git-single-commit.sh – Take a commit from the repo and insert it into the database. Will fail if commit is already there. REPO is one of src, ports, doc. Hash is the git commit
FreshPorts cheat sheet Read More »
Recently, a new use of ZFS snapshots was introduced for a cache on FreshPorts. This approach involves creating a snapshot on the empty cache, then rolling back to that snapshot when the cache needs to be cleared. This idea came from this Tweet after much discussion on how to properly and safely delete directory contents
ZFS when you’re not doing ZFS Read More »
A new FreshPorts features is being developed. I’m trying to determine if packages for a port can be cached separately from the port itself. I will explain. When you visit a port page, if it’s not cached, the page is built and then cached. Not the whole page, just the parts pertaining to that port.
In the previous post I documented how the packages_raw table is updated to populate the abi_id and port_id columns. This post will discuss how that data is then pushed into the packages table. Thoughts When updates arrive, they arrive in quantity. The latest update was 30474 records. From that, we will have some new packages,
Updating the packages table – INSERT, UPDATE, DELETE – plus some ZFS configuration Read More »
This post is another in the packages series. It documents how the data is transformed from the raw form (packages_raw) into normalized data (packages). The Packages – how is this data stored? post may be useful reading. The primary purpose of this blog post is documentation of the above mentioned transformation process. Along the way
Using DEFERRABLE INITIALLY DEFERRED on constraints Read More »
FreshPorts had a cache clearing issue recently. It lingered. For a long time. It took me a while to figure it out. It turned out to be a Python 3 compatibility issue. In this post: PostgreSQL 12.1 FreeBSD 12.1 I will outline how database changes invoke external scripts asynchronously which then clear the cache. I
Cache clearing issue Read More »