Processing commits on branches with git – part 2

Following on from last weekend’s work, today I’m learning more about git, and the difference between Plumbing and Porcelain commands.

The original intent was to use git tag -l freshports/$refname to detect if a tag exists. The problem there, it always returns 0.

[dan@mydev:/var/db/ingress/repos/ports] $ git tag -l | head
10-eol
4-eol
5-eol
6-eol
7-eol
8-eol
9-eol
pkg-install-eol
pre-xorg-7
release/10.0.0
[dan@mydev:/var/db/ingress/repos/ports] $ 


[dan@mydev:/var/db/ingress/repos/ports] $ git tag -l 10-eol
10-eol
[dan@mydev:/var/db/ingress/repos/ports] $ echo $?
0
[dan@mydev:/var/db/ingress/repos/ports] $ git tag -l 10-eollkjadslkjdf
[dan@mydev:/var/db/ingress/repos/ports] $ echo $?
0
[dan@mydev:/var/db/ingress/repos/ports] $ git tag -l | grep 10-eollkjadslkjdf
[dan@mydev:/var/db/ingress/repos/ports] $ 

From the above, whether the tag exists or not, the code can’t tell, well, without looking at the output.

Let’s use this instead:

[dan@mydev:/var/db/ingress/repos/ports] $ git rev-parse --verify refs/tags/10-eol
1aa5d44ff272774be87b1afa6a71fa6cbd62c8af
[dan@mydev:/var/db/ingress/repos/ports] $ echo $?
0
[dan@mydev:/var/db/ingress/repos/ports] $ git rev-parse --verify refs/tags/10-eollkjadslkjdf
fatal: Needed a single revision
[dan@mydev:/var/db/ingress/repos/ports] $ echo $
[dan@mydev:/var/db/ingress/repos/ports] $ 

We can also use the -q to suppress output.

Next goal

Let’s try this on the devgit.freshports.org server next.

This is the current code:

#!/bin/sh

# process the new commits
# based upon https://github.com/FreshPorts/git_proc_commit/issues/3
# An idea from https://github.com/sarcasticadmin

if [ ! -f /usr/local/etc/freshports/config.sh ]
then
	echo "/usr/local/etc/freshports/config.sh.sh not found by $0"
	exit 1
fi

# this can be a space separated list of repositories to check
# e.g. "doc ports src"
repos=$1

. /usr/local/etc/freshports/config.sh

LOGGERTAG='git-delta.sh'

logfile "has started. Will check these repos: '${repos}'"

# what remote are we using on this repo?
REMOTE='origin'

# where we do dump the XML files which we create?
XML="${INGRESS_MSGDIR}/incoming"

logfile "XML dir is $XML"

for repo in ${repos}
do
   logfile "Now processing repo: ${repo}"

   # convert the repo label to a physical directory on disk
   dir=$(convert_repo_label_to_directory ${repo})

   # empty means error
   if [  "${dir}" == "" ]; then
      logfile "FATAL error, repo='${repo}' is unknown: cannot translate it to a directory name"
      continue
   fi

   # where is the repo directory?
   # This is the directory which contains the repos.
   REPODIR="${INGRESS_PORTS_DIR_BASE}/${dir}"
   LATEST_FILE="${INGRESS_PORTS_DIR_BASE}/latest.${dir}"

   if [ -d ${REPODIR} ]; then
      logfile "REPODIR='${REPODIR}' exists"
   else
      logfile "FATAL error, REPODIR='${REPODIR}' is not a directory"
      continue
   fi

   logfile "Repodir is $REPODIR"
   # on with the work

   cd ${REPODIR}

   # Bring local branch up-to-date with the local remote
   logfile "Running: ${GIT} fetch:"
   ${GIT} fetch
   logfile "Done."

   NAME_OF_HEAD="main"
   NAME_OF_REMOTE="origin"
   MAIN_BRANCH="$NAME_OF_REMOTE/$NAME_OF_HEAD"

   git for-each-ref --format '%(objecttype) %(refname)' \
      | sed -n 's/^commit refs\/remotes\///p' \
      | while read -r refname
   do
      echo looking at $refname
      # for now, when testing, only this branch please
      if [ "$refname" != "origin/2021Q2" ] && [ "$refname" != "$MAIN_BRANCH" ]
      then
         continue
      fi

      echo "working on '$refname'"

      if ! git rev-parse -q --verify freshports/$refname
      then
         if [ "$refname" == "$MAIN_BRANCH" ]
         then
            echo "FATAL - '$MAIN_BRANCH' must have tag 'freshports/$refname' set manually  - special case the main branch because the best merge base is the most recent commit"
            exit
         fi
         echo "'git tag -l freshports/$refname'" found nothing
         echo "Let's find the first commit in this branch"
         first_ref=$(git merge-base $NAME_OF_REMOTE/$NAME_OF_HEAD $refname)
         echo "First ref is '$first_ref'"
         # get the first commit of that branch and create a tag.
         echo taging that now:
         git tag -m "first known commit of $refname" -f freshports/$refname $first_ref
      fi

      # get list of commits, if only to document them here
      logfile "Running: ${GIT} rev-list freshports/$refname..$refname"
      commits=$(${GIT} rev-list freshports/$refname..$refname)
      logfile "Done."

      if [ -z "$commits" ]
      then
         logfile "No commits were found"
      else
        logfile "The commits found are:"
        for commit in $commits
        do
           logfile "$commit"
        done
      fi

      logfile "${SCRIPTDIR}/git-to-freshports-xml.py --repo ${repo} --path ${REPODIR} ---commit-range ${xfreshports}/${refname}..$refname --spooling ${INGRESS_SPOOLINGDIR} --output ${XML}"
      
      
               ${SCRIPTDIR}/git-to-freshports-xml.py --repo ${repo} --path ${REPODIR} ---commit-range ${freshports/$refname}..$refname --spooling ${INGRESS_SPOOLINGDIR} --output ${XML}

      new_latest=$(${GIT}  rev-parse HEAD)
      
      echo new_latest = $new_latest

      # echo $new_latest > ${LATEST_FILE}
      # Store the last known commit that we just processed.
      git tag -m "last known commit of $refname" -f freshports/$refname $refname

   done
done

logfile "Ending"