Moving devgit.freshports.org from GitHub to git.FreeBSD.org

The doc repo has moved from svn to git. This changeover occurred on 2020-12-09.

The last svn commit was: 54737

The first git commit was: 3be01a475855e7511ad755b2defd2e0da5d58bbe

To date, devgit.freshports.org has been using https://github.com/freebsd/freebsd-doc/ for processing commits.

Today’s work will convert from that GitHub repo to https://cgit.freebsd.org/doc/ (actually, https://git.freebsd.org/).

What changes are required

The following changes are required:

  1. A new working copy of the git.FreeBSD.org/doc repo
  2. A marker pointing the last commit processed
  3. Configuration file changes to point to that repo

EDIT: 2021-01-01 – NOTE: all the configuration files are also maintained via Ansible – the manual steps are not necessarily required. I think I should next document the changes to Ansible when changing database names / servers.

The rest of the post documents those changes.

A new working copy of the git.FreeBSD.org/doc repo

Let’s clone that repo:

$ cd /var/db/ingress/repos
[ingress@devgit-ingress01 ~]$ cd repos
[ingress@devgit-ingress01 ~/repos]$ ls -l
total 18
drwxr-xr-x  26 ingress  ingress  45 Dec 17 03:03 freebsd
drwxr-xr-x  23 ingress  ingress  26 Dec  8 07:12 freebsd-docĀ“
drwxr-xr-x  69 ingress  ingress  84 Dec 17 02:18 freebsd-ports
drwxr-xr-x  69 ingress  ingress  84 Nov 22 19:28 freebsd-ports-quarterly
-rw-r--r--   1 ingress  ingress  41 Dec 17 23:51 latest.freebsd
-rw-r--r--   1 ingress  ingress  41 Dec 17 23:51 latest.freebsd-doc
-rw-r--r--   1 ingress  ingress  41 Dec 17 23:51 latest.freebsd-ports
-rw-r--r--   1 ingress  ingress  41 Dec 17 23:51 latest.freebsd-ports-quarterly
[ingress@devgit-ingress01 ~/repos]$ git clone https://git.FreeBSD.org/doc.git
Cloning into 'doc'...
remote: Enumerating objects: 449334, done.
remote: Counting objects: 100% (449334/449334), done.
remote: Compressing objects: 100% (120607/120607), done.
remote: Total 449334 (delta 314272), reused 448476 (delta 313566), pack-reused 0
Receiving objects: 100% (449334/449334), 245.50 MiB | 11.60 MiB/s, done.
Resolving deltas: 100% (314272/314272), done.
Updating files: 100% (11346/11346), done.
[ingress@devgit-ingress01 ~/repos]$ 

Done.

A marker pointing the last commit processed

In the directory listing above, you can see files starting with latest. which mirror the list of repos, which are directories. We need to create a new file, named latest.doc, which contains the hash of the last commit processed by FreshPorts.

To find that value, I did this:

  1. browse to https://cgit.freebsd.org/doc/
  2. click on main
  3. scroll down to “Mark the repository as being converted to Git.”
  4. Take that hash value

You won’t be able to reproduce that once there are enough commits and that particular commit scrolls off the page. You will always be able to find the first commit (3be01a475855e7511ad755b2defd2e0da5d58bbe) and see parent listed as 89d0233560e4ba181d73143fc25248b407120e09

Let’s put that into a file:

[ingress@devgit-ingress01 ~/repos]$ echo 89d0233560e4ba181d73143fc25248b407120e09 > latest.doc

What do we have now:

[ingress@devgit-ingress01 ~/repos]$ ls -l
total 21
drwxr-xr-x  23 ingress  ingress  27 Dec 17 23:55 doc
drwxr-xr-x  26 ingress  ingress  45 Dec 17 03:03 freebsd
drwxr-xr-x  23 ingress  ingress  26 Dec  8 07:12 freebsd-doc
drwxr-xr-x  69 ingress  ingress  84 Dec 17 02:18 freebsd-ports
drwxr-xr-x  69 ingress  ingress  84 Nov 22 19:28 freebsd-ports-quarterly
-rw-r--r--   1 ingress  ingress  41 Dec 18 00:06 latest.doc
-rw-r--r--   1 ingress  ingress  41 Dec 18 00:06 latest.freebsd
-rw-r--r--   1 ingress  ingress  41 Dec 18 00:06 latest.freebsd-doc
-rw-r--r--   1 ingress  ingress  41 Dec 18 00:06 latest.freebsd-ports
-rw-r--r--   1 ingress  ingress  41 Dec 18 00:06 latest.freebsd-ports-quarterly
[ingress@devgit-ingress01 ~/repos]$ 

Next, configuration file changes.

Configuration file changes to point to that repo

What files need changing? Just one I think.

[dan@devgit-ingress01:~] $ cd /usr/local/etc/freshports
[dan@devgit-ingress01:/usr/local/etc/freshports] $ sudo grep freebsd-doc *
config.sh:     doc) dir='freebsd-doc';;

When I make this change, I want to disable the git processing so as not to change a configuration setting in the middle of processing.

$ sudo sysrc -f /etc/periodic.conf fp_check_for_git_commits_enable="NO"
fp_check_for_git_commits_enable: YES -> NO

Let’s verify nothing is running git processing:

 ps auwwx | grep ingress
ingress     60714  0.0  0.0 10676  2188  -  SCJ  00:14   0:00.00 sleep 3
ingress_svn 60727  0.0  0.0 10676  2188  -  SCJ  00:14   0:00.00 sleep 3
ingress_svn 98912  0.0  0.0 11004  2424  -  IsJ  Sat23   0:00.15 daemon: ingress_svn[98913] (daemon)
ingress_svn 98913  0.0  0.0 11868  2968  -  SJ   Sat23   0:32.78 /bin/sh /usr/local/libexec/freshports-service/ingress_svn.sh
ingress     98921  0.0  0.0 11004  2424  -  IsJ  Sat23   0:00.44 daemon: ingress[98923] (daemon)
ingress     98923  0.0  0.0 11844  2952  -  SJ   Sat23   0:21.70 /bin/sh /usr/local/libexec/freshports-service/ingress.sh
dan         60729  0.0  0.0 11384  2764  2  S+J  00:14   0:00.00 grep ingress

That’s normal. ingress_svn is the daemon checking for incoming svn commits.

ingress is the daemon which looks for incoming git commits, but that’s the daemon which processes the XML files. The periodic.conf setting we adjusted is for the creation of those XML files. That is what we are pausing now.

Next, I updated config.sh. This is what we have now:

[dan@devgit-ingress01:/usr/local/etc/freshports] $ sudo grep doc *
config.pm:# Values as found at https://www.postgresql.org/docs/current/static/libpq-ssl.html
config.pm:$FreshPorts::Config::Repo_DOC             = 'doc';
config.pm:$FreshPorts::Config::DB_Root_Prefix_DOC             = '/doc';
config.sh:# see https://www.postgresql.org/docs/12/libpq-envars.html
config.sh:     doc) dir='doc';;

That last line is the updated valud. The others refer to repo names and internal pathnames within the FreshPorts database.

What’s next?

Turn on git commit processing and watch the logs.

$ sudo sysrc -f /etc/periodic.conf fp_check_for_git_commits_enable="YES"
fp_check_for_git_commits_enable: NO -> YES
$ 

Now I wait until 7:24, because the script runs every 3 minutes.

BOOM! The latest commit is now in: 482d8311b8a1e25a66ee49af4bc7efadd8be22aa

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Leave a Comment

Scroll to Top