If packages and ports are cached separately, how is one included in the middle of the other?

A new FreshPorts features is being developed. I’m trying to determine if packages for a port can be cached separately from the port itself. I will explain.

When you visit a port page, if it’s not cached, the page is built and then cached. Not the whole page, just the parts pertaining to that port. The right hand column is not cached.

If you look at https://dev.freshports.org/editors/joe you’ll see a table listing the available packages, as shown here:

Packages for joe
Packages for joe

At present, this table is cached with the rest of the port information.

My thinking: ports and packages are updated independently. It would be nice/cool/pretty damn eloquent to cache them separately.

PROBLEM: The ports cache is static HTML. How do you toss another bit of static HTML into the middle of that?

How caching works

When an update to a port occurs, the cache for that port is cleared. The next web request for that port will repopulate that cache.

While generating the HTML for that port, if the package HTML is not found, it can also be generated.

However, packages and port are updated via separate mechanisms:

  • port – via an incoming commit
  • packages – a new package repo build appears

PROBLEM: the packages cache can disappear right after you look to see if it’s there; it’s a race condition.

Conclusion

It will be far easier to cache the packages for a port with the port itself. To do otherwise creates a mess of problems to solve.

This means each time a new repo is build, the entire ports cache must be cleared. That’s just a bunch of files on disk. mv * ../DELETEME/ && rm -rf ../DELETEME/ will fix that.

Any other ideas

Do you have any ideas? All welcome. Thank you.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

2 thoughts on “If packages and ports are cached separately, how is one included in the middle of the other?”

  1. I think the best solution is to break up the ports cache into three parts per page

    * top part, before packages
    * middle part, packages
    * bottom part, after packages and onto the commits

    At present, if there are multiple pages for the commits, you see this in the cache:

    [dan@x8dtu-nginx01:~] $ sudo ls -l ~freshports/cache/ports/security/vuxml
    total 130
    -rwxrwxr– 1 www freshports 144695 Apr 19 15:20 Detail.head.PageSize100.PageNum1.html
    -rwxrwxr– 1 www freshports 144473 Apr 19 15:39 Detail.head.PageSize100.PageNum2.html
    [dan@x8dtu-nginx01:~] $

    I think that would become:

    Detail.head.part1.PageSize100.PageNum1.html
    Detail.head.part2.PageSize100.PageNum1.html
    Detail.head.part1.PageSize100.PageNum2.html
    Detail.head.part2.PageSize100.PageNum2.html

    I realize now that there is much overlap in the cache between PageNum1 and PageNum2. I don’t feel like reconciling that. It’s only disk space.

    At present, the cache is about 2GB. Trivial. No big gains to be made by reducing the duplication.

Leave a Comment

Scroll to Top