FreshPorts has a long history of Sanity Testing each commit to the FreeBSD Ports tree. It has been doing this for two main reasons:
- Ensure the integrity of the FreshPorts database
- Help correct obvious errors as soon as possible
In 2002, FreshPorts allowed committers to opt in to receiving these notifications directly. Prior to this feature, all Sanity Test Failures went directly to the FreshPorts admin (me). I would then forward to the committer in question. By going directly to the source, errors are fixed earlier (I’m not always in front of the keyboard when the email comes in) and it lessens my workload.
Sanity Test Failures are not common. I have no statistics, but I’ll guess that we get about 10 or 12 a year. In the year ending 10 October 2006, FreshPorts saw 59,678 port-related commits. My guess indicates roughly 0.02% of all commits invoke a Sanity Test Failure. Yes, they’re pretty rare.
So rare that I’ve decided to capture and store Sanity Test Failures. The BETA system starting to record all Sanity Test Failures on 10 October 2006. So far, we have one. Actually, it was that one failure that prompted me to add the code to record the details.
It was pretty straight forward, but I had to watch the COMMITs and ROLLBACKs. It’s odd… wanting to store something in the database, but also issue a ROLLBACK after the failure…
Have a look for yourself: Beta Sanity Test Failures. If you click on the X with the red background, you’ll see the details of the failure. Eventually, this red X will appear everywhere that a Sanity Test Failure commit appears. Just give me some time to amend the database queries.
Do you have any ideas or suggestions about this idea? Thanks.
Today we had another Sanity Test Failure. The BETA system grabbed and stored the results as expected. I added some sorting to the SELECT so that the data is displaying in descending DATE order (newest failures at the top of the page).