Mar 312012
 

Baptiste Daroussin (bapt) brought my attention to a new port today. It’s about three months old, but it wasn’t showing up in FreshPorts. If you look at ports-mgmt/pkg now, it should look like this:

Bad port.  Bad bad port.

bapt speculated that perhaps pkg was a reserved word in FreshPorts. I thought not. I was wrong. It is a special case in FreshPorts. But it took a while to find that out.

First, we checked dev and beta to see if they were having the same issue. They were.

Next, let’s look at the database:

freshports.org=# select * from element where name = 'pkg' and parent_id = (select id from element where name = 'ports-mgmt');
   id   | name | parent_id | directory_file_flag | status 
--------+------+-----------+---------------------+--------
 405152 | pkg  |    265343 | D                   | A
(1 row)
freshports.org=# 

OK, pkg is definitely in the database as a directory. Good.

Does pkg have any children?

freshports.org=# select * from element where parent_id = 405152;
   id   |    name     | parent_id | directory_file_flag | status 
--------+-------------+-----------+---------------------+--------
 413289 | files       |    405152 | D                   | A
 413282 | pkg-plist   |    405152 | F                   | A
 406674 | pkg-message |    405152 | F                   | A
 405155 | pkg-descr   |    405152 | F                   | A
 405154 | distinfo    |    405152 | F                   | A
 405153 | Makefile    |    405152 | F                   | A
(6 rows)

freshports.org=# 

Yes, that looks right. Good.

I started looking through the web code to see if there was a problem there. I found one. The database does not know this is a port:

freshports.org=# select * from elementGet('/ports/ports-mgmt/pkg');
   id   | name | type | status | iscategory | isport |   element_pathname    
--------+------+------+--------+------------+--------+-----------------------
 405152 | pkg  | D    | A      | f          | f      | /ports/ports-mgmt/pkg
(1 row)

freshports.org=# 

See isport? It should be true.

FreshPorts logs everyone commit it processes. Going to the logs for the initial commit for this port, I find:

STARTING _CompileListOfPorts ................................
FILE ==: Add, ports/Mk/bsd.pkgng.mk, 1.1, ports, Mk, bsd.pkgng.mk, 1785995
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

FILE ==: Modify, ports/Mk/bsd.port.mk, 1.704, ports, Mk, bsd.port.mk, 1785996
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

FILE ==: Modify, ports/ports-mgmt/Makefile, 1.49, ports, ports-mgmt, Makefile, 1785997
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

FILE ==: Add, ports/ports-mgmt/pkg/Makefile, 1.1, ports, ports-mgmt, pkg, Makefile, 1785998
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

FILE ==: Add, ports/ports-mgmt/pkg/distinfo, 1.1, ports, ports-mgmt, pkg, distinfo, 1785999
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

FILE ==: Add, ports/ports-mgmt/pkg/pkg-descr, 1.1, ports, ports-mgmt, pkg, pkg-descr, 1786000
YES, this file is in the ports tree
... but is on the list of IgnoredItems!

ENDING _CompileListOfPorts ................................

IgnoredItems? OK, that sounds promising. Looking in scripts/constants.pm I find:

#
# These are the entries within /usr/ports/ which we ignore
# and /usr/ports/ which FreshPorts does not track
#
%FreshPorts::Constants::IgnoredItems = (
        "Attic"                 => 1,
        "distfiles"             => 2,
        "Mk"                    => 3,
        "Tools"                 => 4,
        "Templates"             => 5,
        "Makefile"              => 6,
        "pkg"                   => 7,
        "Makefile.inc"  => 8,
);

For fun, I removed pkg, and renumberd Makefile.inc to 7. Then I reran the initial commit. Success. We have a port. Then I reran all other commits for the port. Everything was caught up.

However, it’s niggling at me… wondering how IgnoredItems is used and why it caught pkg.

Looking in scripts/verifyport.pm I see:

# is this file is in the ports tree?
# e.g. ports/LEGAL won't get through here because $port_name will not be defined.
if ($subtree eq $FreshPorts::Config::ports_prefix && defined($category_name) && defined($port_name)) {
    print "YES, this file is in the ports tree\n";

    if (!defined($FreshPorts::Constants::IgnoredItems{$category_name}) && !defined($FreshPorts::Constants::IgnoredItems{$port_name})) {
        # find the port for this filename....
...
    } else {
        print "... but is on the list of IgnoredItems!\n\n";
    }
} else {
    print "that file isn't in the ports tree\n";
}

It sounds like I can be a bit more specific with my tests. Looking for each value in IgnoredPorts, I find these items:

dan@kraken:~] $ find /usr/ports/ -name Attic
[dan@kraken:~] $ find /usr/ports/ -name distfiles
/usr/ports/distfiles
[dan@kraken:~] $ find /usr/ports/ -name Mk       
/usr/ports/Mk
[dan@kraken:~] $ find /usr/ports/ -name Tools
/usr/ports/Tools
[dan@kraken:~] $ find /usr/ports/ -name Templates
/usr/ports/Templates
[dan@kraken:~] $ 
[dan@kraken:~] $ find /usr/ports/ -name Makefile.inc
/usr/ports/arabic/Makefile.inc
/usr/ports/audio/xmms-faad/Makefile.inc
/usr/ports/chinese/Makefile.inc
/usr/ports/devel/p4/Makefile.inc
/usr/ports/devel/ecore-main/Makefile.inc
/usr/ports/devel/eric4/Makefile.inc
/usr/ports/devel/subversion16/Makefile.inc
/usr/ports/editors/xxe/Makefile.inc
/usr/ports/french/Makefile.inc
/usr/ports/games/anki/Makefile.inc
/usr/ports/german/Makefile.inc
/usr/ports/graphics/evas-core/Makefile.inc
/usr/ports/hebrew/Makefile.inc
/usr/ports/hungarian/Makefile.inc
/usr/ports/japanese/Makefile.inc
/usr/ports/korean/Makefile.inc
/usr/ports/misc/gnome2-reference/Makefile.inc
/usr/ports/net-im/licq/Makefile.inc
/usr/ports/net/ntp-rc/Makefile.inc
/usr/ports/net/ntp-devel/Makefile.inc
/usr/ports/net/pcnfsd/files/Makefile.inc
/usr/ports/net/ntp/Makefile.inc
/usr/ports/net/samba-libsmbclient/Makefile.inc
/usr/ports/news/husky-base-devel/Makefile.inc
/usr/ports/news/husky-base/Makefile.inc
/usr/ports/polish/Makefile.inc
/usr/ports/ports-mgmt/pkg_install/files/Makefile.inc
/usr/ports/portuguese/Makefile.inc
/usr/ports/russian/Makefile.inc
/usr/ports/textproc/aspell/Makefile.inc
/usr/ports/ukrainian/Makefile.inc
/usr/ports/vietnamese/Makefile.inc
/usr/ports/x11-themes/irssi-themes/Makefile.inc
[dan@kraken:~] $ 

Not very many items. In effect, I want to look for, and ignore these items:

  1. ports/distfiles
  2. ports/Mk
  3. ports/Tools
  4. ports/Tempaltes

It may be better to have a list of ignored ‘categories’ (e.g. Mk). Not that Mk is a category…

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

  7 Responses to “ports-mgmt/pkg not found”

  1. I notice the Master Sites differs from http://beta.freshports.org/ports-mgmt/pkg/ and I do not know why

  2. http://www.freshports.org/ports-mgmt/pkg/ has also been updated.

    I suspect the issue on dev is related to some test setting. I also note that the list of master sites is not sorted.

  3. yeah, ‘make master-sites-all’ in my dev environment is returning duplicates.

  4. pkg was indeed a reserved word as every ports category contained a directory called pkg which provided a COMMENT file with a description of the category.
    They were removed 8 years ago: http://www.freebsd.org/cgi/query-pr.cgi?pr=59651

  5. Today a new clue arrived. The script I created to detect ports that do not have entries in the ports_categories table found a candidate:

    This is a list of ports that do not have entries in the ports_categorie table
    This can be fixed with this query:
    begin;  insert into ports_categories select id, category_id from ports_active PA
    WHERE NOT EXISTS (SELECT * from ports_categories PC where PC.port_id = PA.id and
    PC.category_id = PA.category_id);
      id   | category_id |       name       | category 
    -------+-------------+------------------+----------
     30785 |           2 | firefox-esr      | www
     30784 |           2 | firefox-esr-i18n | www
    (2 rows)
    

    What is interesting about www/firefox-esr and www/firefox-esr-i18n is the repo-copy. I wonder if that’s what is causing this missed entry from the table.

  6. It appears that sometimes a port is not included in the right categories. e.g.

    dan@ngaio:~/ports/vietnamese/libreoffice] $ make -V CATEGORIES
    editors
    [dan@ngaio:~/ports/vietnamese/libreoffice] $ cd ../../hungarian/libreoffice/
    [dan@ngaio:~/ports/hungarian/libreoffice] $ make -V CATEGORIES
    hungarian editors