cache invalidation

Today, kwm mentioned that was out of date, showing the most recent commit as back in March. My first concern was the system had stopped recording such commits. To check on that, I looked at and searched my email for the most recent commit message. Extracting the message id from that email, I quickly found that FreshPorts was will processing such commits.

Checking the dev server I found that it had all the most recent commits.

This led me to thinking there was code which had not made it into production.

A bit of debugging had me thinking it was outdated SQL which was missing, but then I noticed it was a caching problem.

I cleared the cache on production and everything was appearing correctly.

To get a sample of what was going wrong, I took a recent commit and tested with it. I viewed the following pages to make sure they were cached:


At this point the cache contained this:


Why does the first entry have head in the pathname? Because it’s a non-port. That’s the crux of the issue I think.

To test the cache invalidation, I deleted the commit from the database: begin;
BEGIN delete from commit_log where message_id = '';
DELETE 1 commit;

Then I reran the commit through the system. Here’s the cache after that:


There’s the issue. The entry in head is still present.

Examination of the fp-listen logs shows:

Oct  5 22:24:07 webserver fp-listen: Just woke up! *************
Oct  5 22:24:07 webserver fp-listen: got port_updated and I need to call listen_port
Oct  5 22:24:07 webserver fp-listen: checking for cache entries to remove...
Oct  5 22:24:07 webserver fp-listen: COUNT: 1 entries to process
Oct  5 22:24:07 webserver fp-listen: removing glob /usr/websites/*
Oct  5 22:24:07 webserver fp-listen: DELETE FROM cache_clearing_ports WHERE id = 1827187
Oct  5 22:24:07 webserver fp-listen: finished

It seems like I need to change everything in cache to be branch-aware. fp-listen will also need to allow for a branch, which means updating the cache_clearing_ports table to be branch-aware as well. \d cache_clearing_ports
                                     Table "public.cache_clearing_ports"
   Column   |            Type             |                             Modifiers                             
 id         | integer                     | not null default nextval('cache_clearing_ports_id_seq'::regclass)
 port_id    | integer                     | not null
 category   | text                        | not null
 port       | text                        | not null
 date_added | timestamp without time zone | not null default now()
    "cache_clearing_ports_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
    "cache_clearing_ports_port_id_fkey" FOREIGN KEY (port_id) REFERENCES ports(id) ON UPDATE CASCADE ON DELETE CASCADE

Ironically, I’ve discovered that the website caches the page for Mk/ but never invalidates it, as found in the processing logs for a recent commit:

done RecordPortsTreeButNonPortCommits
No errors found during that commit
Observer has noticed that the update for has finished.
Observer will clear the following ports from cache after the commit:
Observer will clear the following files from cache after the commit:
we are ignoring /ports/head/Mk/ for cache clearing
*** end of items to be cleared

That clearly must change.

Website Pin Facebook Twitter Myspace Friendfeed Technorati Digg Google StumbleUpon Premium Responsive

Leave a Comment

Scroll to Top