Baptiste Daroussin (bapt) brought my attention to a new port today. It’s about three months old, but it wasn’t showing up in FreshPorts. If you look at ports-mgmt/pkg now, it should look like this:
bapt speculated that perhaps pkg was a reserved word in FreshPorts. I thought not. I was wrong. It is a special case in FreshPorts. But it took a while to find that out.
First, we checked dev and beta to see if they were having the same issue. They were.
Next, let’s look at the database:
freshports.org=# select * from element where name = 'pkg' and parent_id = (select id from element where name = 'ports-mgmt'); id | name | parent_id | directory_file_flag | status --------+------+-----------+---------------------+-------- 405152 | pkg | 265343 | D | A (1 row) freshports.org=#
OK, pkg is definitely in the database as a directory. Good.
Does pkg have any children?
freshports.org=# select * from element where parent_id = 405152; id | name | parent_id | directory_file_flag | status --------+-------------+-----------+---------------------+-------- 413289 | files | 405152 | D | A 413282 | pkg-plist | 405152 | F | A 406674 | pkg-message | 405152 | F | A 405155 | pkg-descr | 405152 | F | A 405154 | distinfo | 405152 | F | A 405153 | Makefile | 405152 | F | A (6 rows) freshports.org=#
Yes, that looks right. Good.
I started looking through the web code to see if there was a problem there. I found one. The database does not know this is a port:
freshports.org=# select * from elementGet('/ports/ports-mgmt/pkg'); id | name | type | status | iscategory | isport | element_pathname --------+------+------+--------+------------+--------+----------------------- 405152 | pkg | D | A | f | f | /ports/ports-mgmt/pkg (1 row) freshports.org=#
See isport? It should be true.
FreshPorts logs everyone commit it processes. Going to the logs for the initial commit for this port, I find:
STARTING _CompileListOfPorts ................................ FILE ==: Add, ports/Mk/bsd.pkgng.mk, 1.1, ports, Mk, bsd.pkgng.mk, 1785995 YES, this file is in the ports tree ... but is on the list of IgnoredItems! FILE ==: Modify, ports/Mk/bsd.port.mk, 1.704, ports, Mk, bsd.port.mk, 1785996 YES, this file is in the ports tree ... but is on the list of IgnoredItems! FILE ==: Modify, ports/ports-mgmt/Makefile, 1.49, ports, ports-mgmt, Makefile, 1785997 YES, this file is in the ports tree ... but is on the list of IgnoredItems! FILE ==: Add, ports/ports-mgmt/pkg/Makefile, 1.1, ports, ports-mgmt, pkg, Makefile, 1785998 YES, this file is in the ports tree ... but is on the list of IgnoredItems! FILE ==: Add, ports/ports-mgmt/pkg/distinfo, 1.1, ports, ports-mgmt, pkg, distinfo, 1785999 YES, this file is in the ports tree ... but is on the list of IgnoredItems! FILE ==: Add, ports/ports-mgmt/pkg/pkg-descr, 1.1, ports, ports-mgmt, pkg, pkg-descr, 1786000 YES, this file is in the ports tree ... but is on the list of IgnoredItems! ENDING _CompileListOfPorts ................................
IgnoredItems? OK, that sounds promising. Looking in scripts/constants.pm I find:
# # These are the entries within /usr/ports/ which we ignore # and /usr/ports/which FreshPorts does not track # %FreshPorts::Constants::IgnoredItems = ( "Attic" => 1, "distfiles" => 2, "Mk" => 3, "Tools" => 4, "Templates" => 5, "Makefile" => 6, "pkg" => 7, "Makefile.inc" => 8, );
For fun, I removed pkg, and renumberd Makefile.inc to 7. Then I reran the initial commit. Success. We have a port. Then I reran all other commits for the port. Everything was caught up.
However, it’s niggling at me… wondering how IgnoredItems is used and why it caught pkg.
Looking in scripts/verifyport.pm I see:
# is this file is in the ports tree? # e.g. ports/LEGAL won't get through here because $port_name will not be defined. if ($subtree eq $FreshPorts::Config::ports_prefix && defined($category_name) && defined($port_name)) { print "YES, this file is in the ports tree\n"; if (!defined($FreshPorts::Constants::IgnoredItems{$category_name}) && !defined($FreshPorts::Constants::IgnoredItems{$port_name})) { # find the port for this filename.... ... } else { print "... but is on the list of IgnoredItems!\n\n"; } } else { print "that file isn't in the ports tree\n"; }
It sounds like I can be a bit more specific with my tests. Looking for each value in IgnoredPorts, I find these items:
dan@kraken:~] $ find /usr/ports/ -name Attic [dan@kraken:~] $ find /usr/ports/ -name distfiles /usr/ports/distfiles [dan@kraken:~] $ find /usr/ports/ -name Mk /usr/ports/Mk [dan@kraken:~] $ find /usr/ports/ -name Tools /usr/ports/Tools [dan@kraken:~] $ find /usr/ports/ -name Templates /usr/ports/Templates [dan@kraken:~] $ [dan@kraken:~] $ find /usr/ports/ -name Makefile.inc /usr/ports/arabic/Makefile.inc /usr/ports/audio/xmms-faad/Makefile.inc /usr/ports/chinese/Makefile.inc /usr/ports/devel/p4/Makefile.inc /usr/ports/devel/ecore-main/Makefile.inc /usr/ports/devel/eric4/Makefile.inc /usr/ports/devel/subversion16/Makefile.inc /usr/ports/editors/xxe/Makefile.inc /usr/ports/french/Makefile.inc /usr/ports/games/anki/Makefile.inc /usr/ports/german/Makefile.inc /usr/ports/graphics/evas-core/Makefile.inc /usr/ports/hebrew/Makefile.inc /usr/ports/hungarian/Makefile.inc /usr/ports/japanese/Makefile.inc /usr/ports/korean/Makefile.inc /usr/ports/misc/gnome2-reference/Makefile.inc /usr/ports/net-im/licq/Makefile.inc /usr/ports/net/ntp-rc/Makefile.inc /usr/ports/net/ntp-devel/Makefile.inc /usr/ports/net/pcnfsd/files/Makefile.inc /usr/ports/net/ntp/Makefile.inc /usr/ports/net/samba-libsmbclient/Makefile.inc /usr/ports/news/husky-base-devel/Makefile.inc /usr/ports/news/husky-base/Makefile.inc /usr/ports/polish/Makefile.inc /usr/ports/ports-mgmt/pkg_install/files/Makefile.inc /usr/ports/portuguese/Makefile.inc /usr/ports/russian/Makefile.inc /usr/ports/textproc/aspell/Makefile.inc /usr/ports/ukrainian/Makefile.inc /usr/ports/vietnamese/Makefile.inc /usr/ports/x11-themes/irssi-themes/Makefile.inc [dan@kraken:~] $
Not very many items. In effect, I want to look for, and ignore these items:
- ports/distfiles
- ports/Mk
- ports/Tools
- ports/Tempaltes
It may be better to have a list of ignored ‘categories’ (e.g. Mk). Not that Mk is a category…
http://beta.freshports.org/ports-mgmt/pkg/ has been updated.
I notice the Master Sites differs from http://beta.freshports.org/ports-mgmt/pkg/ and I do not know why
http://www.freshports.org/ports-mgmt/pkg/ has also been updated.
I suspect the issue on dev is related to some test setting. I also note that the list of master sites is not sorted.
yeah, ‘make master-sites-all’ in my dev environment is returning duplicates.
pkg was indeed a reserved word as every ports category contained a directory called pkg which provided a COMMENT file with a description of the category.
They were removed 8 years ago: http://www.freebsd.org/cgi/query-pr.cgi?pr=59651
Today a new clue arrived. The script I created to detect ports that do not have entries in the ports_categories table found a candidate:
What is interesting about www/firefox-esr and www/firefox-esr-i18n is the repo-copy. I wonder if that’s what is causing this missed entry from the table.
It appears that sometimes a port is not included in the right categories. e.g.