A new FreshPorts features is being developed. I’m trying to determine if packages for a port can be cached separately from the port itself. I will explain.
When you visit a port page, if it’s not cached, the page is built and then cached. Not the whole page, just the parts pertaining to that port. The right hand column is not cached.
If you look at https://dev.freshports.org/editors/joe you’ll see a table listing the available packages, as shown here:
At present, this table is cached with the rest of the port information.
My thinking: ports and packages are updated independently. It would be nice/cool/pretty damn eloquent to cache them separately.
PROBLEM: The ports cache is static HTML. How do you toss another bit of static HTML into the middle of that?
How caching works
When an update to a port occurs, the cache for that port is cleared. The next web request for that port will repopulate that cache.
While generating the HTML for that port, if the package HTML is not found, it can also be generated.
However, packages and port are updated via separate mechanisms:
- port – via an incoming commit
- packages – a new package repo build appears
PROBLEM: the packages cache can disappear right after you look to see if it’s there; it’s a race condition.
Conclusion
It will be far easier to cache the packages for a port with the port itself. To do otherwise creates a mess of problems to solve.
This means each time a new repo is build, the entire ports cache must be cleared. That’s just a bunch of files on disk. mv * ../DELETEME/ && rm -rf ../DELETEME/ will fix that.
Any other ideas
Do you have any ideas? All welcome. Thank you.
I think the best solution is to break up the ports cache into three parts per page
* top part, before packages
* middle part, packages
* bottom part, after packages and onto the commits
At present, if there are multiple pages for the commits, you see this in the cache:
[dan@x8dtu-nginx01:~] $ sudo ls -l ~freshports/cache/ports/security/vuxml
total 130
-rwxrwxr– 1 www freshports 144695 Apr 19 15:20 Detail.head.PageSize100.PageNum1.html
-rwxrwxr– 1 www freshports 144473 Apr 19 15:39 Detail.head.PageSize100.PageNum2.html
[dan@x8dtu-nginx01:~] $
I think that would become:
Detail.head.part1.PageSize100.PageNum1.html
Detail.head.part2.PageSize100.PageNum1.html
Detail.head.part1.PageSize100.PageNum2.html
Detail.head.part2.PageSize100.PageNum2.html
I realize now that there is much overlap in the cache between PageNum1 and PageNum2. I don’t feel like reconciling that. It’s only disk space.
At present, the cache is about 2GB. Trivial. No big gains to be made by reducing the duplication.
Done.