
The IT staff looked at what was happening with the website on Saturday, when several of you reported connection issues. The site was busy. There were about a million hits that day. That's not the number of visitors; each visit generates multiple hits. There were about 20,000 requests for the home page. Of those, 2,200 received an abnormal response code. That's 11% of the total requests for the main page, which is at least an order of magnitude above what would generally be regarded as acceptable. The busiest times when "502 errors" (service temporarily overloaded) were the most common were between 1810 and 1910 UTC and between 2330 UTC and 0200 UTC Sunday, becoming particularly intense after 0030 UTC. We have found no evidence of a denial of service attack; as far as we have been able to determine it was just heavy demand. Site performance is continuing to be problematic. Sales and Marketing staff fielded a half-dozen phone calls this afternoon from people who had been unable to complete an online order. Of course, we apologized and took their orders by phone. Some of you have asked whether the problem could be addressed by adding more resources at Rackspace, temporarily or otherwise. Unfortunately, it's not that simple. The database architecture makes the performance of a single server the critical element in how quickly requests can be fulfilled. That can't be scaled up. Our current cloud-based hosting system is not as efficient in this regard as dedicated hardware would be, which is what led Jon Bloom to propose bringing the servers in-house as part of the 2011 capital plan. However, since then Rackspace has offered new services that might better meet our needs. Jon is studying the current alternatives being offered by Rackspace and in the next week will compare the best of those with his earlier recommendation to bring the servers in-house. Dave Sumner, K1ZZ
participants (1)
-
Sumner, Dave, K1ZZ