With the recent move of the FreeBSD ports tree to SVN from CVS, there are some subtle changes on FreshPorts.
Early this morning, I noticed a commit which demonstrated a side effect that I had not anticipated. The issue isn’t obvious from the above link, but I noticed it when the commit failed to properly complete its refresh. That is, the fetching of the files from SVN failed.
What was the problem? FreshPorts tried to fetch a directory. Sorry, that won’t work.
Why fetch a directory? Well, because it was told to do so. The commit in question added a new port (CVS did not list directories when a new port (devel/rubygem-cool.io). With such adds, CVS did not list the added directories. However, SVN does.
For examples of newly added ports, compare these two commit emails:
In the first link, you will see head/devel/rubygem-cool.io/ listed. However, in the second link, you do not see ports/devel/geoip-java/ listed at all.
If you view the full list of files for that rubygem-cool.io commit, you will see that the directory is listed.
The FreshPorts code for processing the list of files affected by a commit assumed all files listed were indeed files (as opposed to directories). Saving the list was not affected. The system would assume an entry in the commit email was a file. However, if a subsequent save created a child under a file, said file would be transparently converted to a directory.
However, FreshPorts fetches each file touched by a commit so it can refresh the port information (e.g. Makefile, pkg-descr). These fetches are done through the web interface to the repository. But, you cannot fetch a directory. Thus, I had to make changes to the code.
The changes were subtle, and hopefully it will continue to work and the SVN commit message does not
change. The patch in question is:
$ cvs di -ur 1.50 verifyport.pm
Index: verifyport.pm
===================================================================
RCS file: /home/repositories/freshports-1/scripts/verifyport.pm,v
retrieving revision 1.50
diff -u -r1.50 verifyport.pm
--- verifyport.pm 26 Jun 2012 12:25:21 -0000 1.50
+++ verifyport.pm 16 Jul 2012 12:34:05 -0000
@@ -346,6 +346,17 @@
print "outside ports tree: ignoring $filename\n";
next;
}
+
+ #
+ # do not fetch directories
+ # THIS MAKES USE OF THE FACT that SVN puts added directories into
+ # the commit message with a trailing /
+ # if that process discontinues, we'll have to find another way to do this.
+ #
+ if ($filename =~ m|^*/$|) {
+ print "this is a directory. Will not fetch $filename\n";
+ next;
+ }
my $directory = File::Basename::dirname ($filename);
my $FILE = File::Basename::basename($filename);
Yes, I’m taking advantage of the trailing / on the directory names. If SVN stops this practice, I have another plan. Each element in the FreshPorts representation of the ports tree is either a directory or a filename. After saving the list of files to the database, retrieve that list from the database, and fetch only the filenames, ignoring the directories. This solution is guaranteed to work but involves slightly more database I/O.
Here is the list of elements (i.e. files/directories) affected by that devel/rubygem-cool.io commit:
freshports.org=# SELECT E.*, element_pathname(E.id) freshports.org-# FROM commit_log_elements CLE, element E freshports.org-# WHERE commit_log_id = (SELECT id freshports.org(# FROM commit_log freshports.org(# WHERE message_id = '201207160844.q6G8iGIa021308@svn.freebsd.org') freshports.org-# AND CLE.element_id = E.id freshports.org-# ORDER BY element_pathname(E.id); id | name | parent_id | directory_file_flag | status | element_pathname --------+-----------------+-----------+---------------------+--------+---------------------------------------- 77295 | Makefile | 84 | F | A | /ports/devel/Makefile 437815 | rubygem-cool.io | 84 | D | A | /ports/devel/rubygem-cool.io 437816 | Makefile | 437815 | F | A | /ports/devel/rubygem-cool.io/Makefile 437817 | distinfo | 437815 | F | A | /ports/devel/rubygem-cool.io/distinfo 437818 | pkg-descr | 437815 | F | A | /ports/devel/rubygem-cool.io/pkg-descr (5 rows) freshports.org=#
In this case, you can see that /ports/devel/rubygem-cool.io is a directory (as indicated by the D in the directory_file_flag column).
This is the first of a series of posts regarding changes made to the FreshPorts system to cater for the SVN repository. More to follow later.
