The problem causing the slow load times was reverse DNS look-ups throughout the server. I had set /etc/resolv.conf to use Cablevisions DNS and they were not responding properly so I changed it to use OpenDNS’s 4.2.2.x addresses. Everything seems happier now.