Ports come and go. Sometimes they come back. Take a recent commit on 1 July against editors/ved (and others). This port was created on 01 Jan 2001 and removed on 15 May 2014 because it used the deprecated smake. The 1 July 2021 commit brought it back. There was a problem.
Here is the page after the commit:
FreshPorts page for editors/ved
The history shows that the version did not change with this commit.
FreshPorts history for editors/ved showing 1.71_1 for most recent commit and the previous commit
If you go back into the 11 Apr 2014 commit for editors/ved and click on the Subversion link, you’ll wind up at revision 350895, you can see the previous Makefile.
With that 1 July commit, editors/ved became a secondary port and obtains its PORTVERSION information from the primary port devel/schilybase.
Looking at devel/schilybase, its version is 2021.06.07. The maintainer brought this issue to my attention.
Logs
In the logs I found:
[dan@devgit-ingress01:~/modules] $ cd ~freshports/message-queues/archive/2021_07/2021_07_01/
[dan@devgit-ingress01:/var/db/freshports/message-queues/archive/2021_07/2021_07_01] $ less *2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3*.log
...
# # # # Resurrecting deleted ports # # # #
...
found a deleted port: port='ved', port_id='4397', element_id='48745'
...
inspecting: 'Add' , '/ports/head/editors/ved/Makefile' , 'editors/ved/Makefile' , '2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3' , 'editors' , 'ved' , 'Makefile'
...found a file from that port
...hmmm, we are modifying a file for a port which is deleted... ........ I will resurrect that port for you
Port editors/ved needs to be Resurrected
........ resurrection done!
we will not examine any more files for this port!
finished looking through the files
# # # # Finished resurrecting deleted ports # # # #
That last line is line 5937 of the log file.
That’s good. The code correctly detected that this commit was touching a dead port and brought it back to life. In the background, this was flipping a flag in the database from INACTIVE to ACTIVE.
But wait, there’s more, around line 7558 of this 14994 line file:
...
port = editors/ved, port_id = '4397', category_id='1', needs_refresh='7'
This port is deleted: not refreshing.
This port is deleted: not saving.
...
The code thinks editors/ved is deleted, and therefore does not refresh the database with the values it could extract from the Makefile.
What?
This is a left-hand right-hand situation where the code is reading a list of ports from the database at the start of the processing, modifying the database (setting the deleted port to be undeleted), and then not updating its local copy of that port.
It can be successfully argued that the code which does the deletion should be the setting that flag in the local copy. I think that is the way to go. However, for now, just for testing, I’m going to try this:
[dan@devgit-ingress01:~/modules] $ svn di
Index: verifyport.pm
===================================================================
--- verifyport.pm (revision 5736)
+++ verifyport.pm (working copy)
@@ -972,6 +972,8 @@
$extra = $extra // '';
print " inspecting: '$action' , '$filename' , '$filename_stripped' , '$revision' , '$category_name' , '$port_name' , '$extra'\n";
+ # instead of doing this through $element, perhaps do it through $port
+ # when then invokes $element and then sets $port->{status}.
if ($category_name eq $port->{category} && $port->{name} eq $port_name) {
print " ...found a file from that port\n";
if ($action eq $FreshPorts::Constants::ADD || $action eq $FreshPorts::Constants::MODIFY) {
@@ -983,6 +985,9 @@
$element->update_status();
print " ........ resurrection done!\n";
print " we will not examine any more files for this port!\n";
+
+ # For later, when we are updating ports, mark this port as active.
+ $port->{status} = $FreshPorts::Element::Active;
# we are looping through files for a single port, only resurrect once.
last;
}
[dan@devgit-ingress01:~/modules] $
Preparing the database for a test run
I’ll run this through the devgit database first. Grabbing the element id from the logs above, I ran this query:
freshports.devgit=# select *, element_pathname(id) from element where id = 48745;
id | name | parent_id | directory_file_flag | status | element_pathname
-------+------+-----------+---------------------+--------+-------------------------
48745 | ved | 2 | D | A | /ports/head/editors/ved
(1 row)
freshports.devgit=#
In order to ‘delete’, this port, I can run this query:
freshports.devgit=# begin;
BEGIN
freshports.devgit=# update element set status = 'D' where id = 48745;
UPDATE 1
freshports.devgit=# select *, element_pathname(id) from element where id = 48745;
id | name | parent_id | directory_file_flag | status | element_pathname
-------+------+-----------+---------------------+--------+-------------------------
48745 | ved | 2 | D | D | /ports/head/editors/ved
(1 row)
freshports.devgit=# commit;
COMMIT
freshports.devgit=#
Now when I view this port on devgit, I see:
editors/ved, now deleted (see the RIP tombstone)
The test run
This is how I rerun the commit. First, it is deleted from the database:
freshports.devgit=# begin;
BEGIN
freshports.devgit=# delete from commit_log where message_id = '2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3';
DELETE 1
freshports.devgit=# commit;
COMMIT
freshports.devgit=#
Then, it is injected into the processing stream:
[dan@devgit-ingress01:/var/db/freshports/message-queues/archive/2021_07/2021_07_01] $ ls *2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3.xml
[dan@devgit-ingress01:/var/db/freshports/message-queues/archive/2021_07/2021_07_01] $ sudo mv -i 2021.07.01.21.11.52.000000.2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3.xml ~ingress/message-queues/incoming/
[dan@devgit-ingress01:/var/db/freshports/message-queues/archive/2021_07/2021_07_01] $
A few seconds later, we have these files:
[dan@devgit-ingress01:/var/db/freshports/message-queues/archive/2021_07/2021_07_01] $ cd ~freshports/message-queues/recent/
[dan@devgit-ingress01:/var/db/freshports/message-queues/recent] $ ls -l *2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3*
-rw-r--r-- 1 freshports freshports 891981 Jul 4 18:06 2021.07.01.21.11.52.000000.2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3.log
-rw-rw-r-- 1 freshports freshports 11993 Jul 1 21:15 2021.07.01.21.11.52.000000.2a40da7011b3545ea4d9c3ec48eed369cc3b1aa3.xml
[dan@devgit-ingress01:/var/db/freshports/message-queues/recent] $
The editors/ved port now looks like this:
editors/ved with the correct version: 2021.06.07
That’s editors/ved with the correct version: 2021.06.07
Success!
The most time consuming part of this exercise was tracing through the code to ensure that the values used by one part of the code matched the other part (left-hand vs right-hand mentioned above).
The logs again
This time the log was about 15400 lines and contained this at about line 8527:
port = editors/ved, port_id = '4397', category_id='1', needs_refresh='7'
into RefreshFromFiles()
working with repo='ports'
working on CommitBranch='main'
with svn_revision=''
this port *** EITHER *** does not need a refresh *** OR *** we were told not to fetch
RepoType='git'
Repository='ports'
CommitBranch='main'
calling FreshPorts::Branches::GetPathToRepoForBranch
CommitBranch: 'main
REPODIR: '/var/db/freshports/ports-jail/var/db/repos/ports'
REPODIR_CHROOT: '/var/db/repos/ports'
checking to make sure /var/db/freshports/ports-jail/var/db/repos/ports/editors/ved/Makefile exists
Phew. It's here. Moving on....
makecommand = /usr/local/bin/sudo /usr/sbin/chroot -u freshports /var/db/freshports/ports-jail /make-port.sh /var/db/repos/ports editors/ved 2>/tmp/FreshPorts.editors.ved.make-error.2021.7.4.18.6.22.59948
Result = 0
trying to get master sites. Errors will be in '/tmp/FreshPorts.editors.ved.make-mastersites-error.2021.7.4.18.6.23.59948'
'/usr/local/bin/sudo /usr/sbin/chroot -u freshports /var/db/freshports/ports-jail /make-master-sites-all.sh /var/db/repos/ports editors/ved 2>/tmp/FreshPorts.editors.ved.make-mastersites-error.2021.7.4.18
trying to get showconfig. Errors will be in '/tmp/FreshPorts.editors.ved.showconfig.2021.7.4.18.6.23.59948'
'/usr/local/bin/sudo /usr/sbin/chroot -u freshports /var/db/freshports/ports-jail /make-showconfig.sh /var/db/repos/ports editors/ved 2>/tmp/FreshPorts.editors.ved.showconfig.2021.7.4.18.6.23.59948'
$result='0'
$showconfig='===> The following configuration options are available for ved-2021.06.07:
DOCS=on: Build and/or install documentation
===> Use 'make config' to modify these settings'
portname = 'ved'
packagename = 'ved'
...
This time the logs show that the port was being refreshed and the database updated.
Updating the code
While reviewing the ports code, I saw:
[dan@devgit-ingress01:~/modules] $ grep sub port.pm
sub _initialize {
sub _GetValuesFromRow {
sub new {
sub save {
sub savePortTableOnly {
sub _save {
sub FetchByID {
sub FetchByPartialPathName {
sub _FetchElementIDByPartialPathName {
sub _ExtractValuesFromMakefile {
sub _Validate {
sub _GetDescrAndHomePage($) {
sub _GetFileContentsFromJail($) {
sub _GetRealPath($) {
sub RefreshFromFiles($;$;$;$;$;$) {
sub GetNeedsRefreshForNewPort {
sub IsActive {
sub IsDeleted {
sub SetActive {
sub SetDeleted {
sub IsValidDate($) {
sub upate_generate_plist {
sub update_package_flavors {
sub update_depends {
sub depends_type_long {
sub update_depends_helper {
sub _addMissingPORTSDIR {
sub CreatePortOnBranch {
[dan@devgit-ingress01:~/modules] $
See the SetDeleted function there on line 18? That’s much better than what I initially had in my test code.
Instead of this:
+ # For later, when we are updating ports, mark this port as active.
+ $port->{status} = $FreshPorts::Element::Active;
I now have this:
+ # For later, when we are updating ports, mark this port as active.
+ $port->SetActive();
It works and uses the proper function too. However, it does not undelete the port. It just sets a value in the local copy.
More work is required there. I think a $port->Undelete() function is called for.
The patch
This is the tested (on exactly one commit) patch:
[dan@devgit-ingress01:~/modules] $ svn di
Index: port.pm
===================================================================
--- port.pm (revision 5713)
+++ port.pm (working copy)
@@ -199,7 +199,7 @@
my $CommitBranch = shift;
my $FullSave = shift;
- print "into FreshPorts::Port::save\n";
+ print "into FreshPorts::Port::_save\n";
#
# to save, element_id and category_id must be valid
@@ -352,6 +352,26 @@
return $this->{id};
}
+sub Undelete {
+ my $this = shift;
+ my $dbh = shift;
+
+ print "into FreshPorts::Port::Undelete\n";
+ my $element = FreshPorts::Element->new($dbh);
+
+ $element->{id} = $this->{element_id};
+ $element->{status} = $FreshPorts::Element::Active;
+
+ $element->update_status();
+
+ # For later, when we are updating ports, mark this port as active.
+ $this->SetActive();
+
+ print "leaving FreshPorts::Port::Undelete\n";
+
+ return $this->{id};
+}
+
sub FetchByID {
my $this = shift;
Index: verifyport.pm
===================================================================
--- verifyport.pm (revision 5736)
+++ verifyport.pm (working copy)
@@ -952,7 +952,7 @@
#
print "# # # # Resurrecting deleted ports # # # #\n\n";
while (my ($portname, $port) = each %Ports) {
- if ($port->{status} eq $FreshPorts::Element::Deleted) {
+ if ($port->IsDeleted()) {
print "found a deleted port: port='$port->{name}', port_id='$port->{id}', element_id='$port->{element_id}'\n";
print "now looking for files which were modified...\n";
@@ -972,6 +972,8 @@
$extra = $extra // '';
print " inspecting: '$action' , '$filename' , '$filename_stripped' , '$revision' , '$category_name' , '$port_name' , '$extra'\n";
+ # instead of doing this through $element, perhaps do it through $port
+ # when then invokes $element and then sets $port->{status}.
if ($category_name eq $port->{category} && $port->{name} eq $port_name) {
print " ...found a file from that port\n";
if ($action eq $FreshPorts::Constants::ADD || $action eq $FreshPorts::Constants::MODIFY) {
@@ -978,11 +980,11 @@
print " ...hmmm, we are modifying a file for a port which is deleted...";
print " ........ I will resurrect that port for you\n";
FreshPorts::Utilities::ReportError('notice', "Port $category_name/$port_name needs to be Resurrected", 0);
- $element->{id} = $port->{element_id};
- $element->{status} = $FreshPorts::Element::Active;
- $element->update_status();
+ $port->Undelete($dbh);
+
print " ........ resurrection done!\n";
print " we will not examine any more files for this port!\n";
+
# we are looping through files for a single port, only resurrect once.
last;
}
[dan@devgit-ingress01:~/modules] $
