I was scanning the error logs this morning, looking for anything unusual. I found it:
[Mon Oct 22 12:31:10 2007] [error] PHP Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 512 bytes) in display_commit.php on line 91
I have seen this before, but could not recall where. I also could not recall how I fixed it. I started tailing both the /var/log/messages file, which display cache hit and failures. I also tailed the website log. Both commands looked something like this:
tail -F /var/log/messages | grep "Cache: FAILED"
tail -F /usr/websites/log/freshports.org-error.log | grep "PHP Fatal error"
The first line looks for cache log events which failed to find something in the cache. If nothing was found in the cache, the system will create the entry, add it to the cache, and return the resulting HTML to the user. The cache in question is cached HTML so that each page is not created on the fly each time it is requested.
This information gave me an idea of what was failing, but the real test came when I looked to see if cache entries existed for those entries that initially had no cache entries. The HTML creation was failed, so any item with a cache entry might be the item that was failing. I found the failing item this way:
$ ls -l `grep " 12:37" /var/log/messages | grep FAILED | awk '{print $11}'`
ls: ports/audio/xmp/Makefile.PageSize100.PageNum1.html: No such file or directory
ls: ports/polish/gnugadu2/Makefile.PageSize100.PageNum1.html: No such file or directory
-rw-rw-r-- 1 www www 15150 Oct 22 12:37 ports/devel/linux-kmod-compat/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 11697 Oct 22 12:37 ports/devel/p5-ExtUtils-Constant/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 12248 Oct 22 12:37 ports/dns/dnsjava/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 12077 Oct 22 12:37 ports/dns/p5-Net-LibIDN/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 12359 Oct 22 12:37 ports/emulators/xpinmame/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 21150 Oct 22 12:37 ports/games/pvpgn/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 13881 Oct 22 12:37 ports/games/xminehunter/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 8255 Oct 22 12:37 ports/lang/gnat-doc-info/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 13282 Oct 22 12:37 ports/security/p5-Digest-JHash/Detail.PageSize100.PageNum1.html
-rw-rw-r-- 1 www www 6869 Oct 22 12:37 ports/security/pinentry-curses/Detail.PageSize100.PageNum1.html
Clearly, it is polish/gnugadu2/Makefile which is failing. I verified this by browsing to that web page. Sure enough, it generated the error message.
I decided to give PHP more memory by altering this line from /usr/local/etc/php.ini:
memory_limit = 40M
It was at 32M. I bumped it to 36M. Then to 40M, then to 42M. After each change, I would gracefully restart Apache:
# apachectl graceful /usr/local/sbin/apachectl graceful: httpd gracefully restarted
But this isn’t an ideal situation. I should change the way the commits are processed for display. One day…
