Barry:
Thank you (and Mike) for the update.
I wouldn’t agree with the characterization of CentOS 7 or Apache 2.4 as being “Bleeding Edge”. CentOS 7 was introduced on 7/7/2014 and Apache 2.4 was released 02 2/21/2012—hardly new versions. (CentOS 6 was released in 2011.)
I’d also note that there are at least four Apache vulnerabilities that have had updates released since the last time the arrl.org server was updated. Two of those issues are rated as moderate and one as important.
One significant reason for moving to current software releases is performance. This includes being better able to handle newer attack vectors. While an older version might not be vulnerable to a certain attack, the newer version will typically handle that attack more gracefully.
From a security standpoint, I would strongly recommend turning off version reporting by all of our servers. As an example, here’s the information contained in the response header from arrl.org:
Server Apache/2.2.3 (CentOS)
X-Powered-By PHP/5.1.6
While the security vulnerabilities of Apache 2.2.3 may have been patched through April 2014, the 2.2.3 in the response header is going to catch the attention of potential hackers that are then going to begin beating on the server. They may find another vulnerability in that process or effectively make the server unreachable due to the bombardment of their attempts.
BTW, PHP 5.1.6 was released on 8/24/2006. The current version is 5.6.4. There are at least 25 PHP vulnerabilities, many for Denial Of Service attacks, that have been reported since the arrl.org server was updated in April 2014
Easy instructions for making the servers more stealth can be found at websites such as http://ask.xmodulo.com/turn-off-server-signature-apache-web-server.html
Please send me the raw log file from the arrl.org server (and any other server that was involved in the arrl.org server being unreachable) from Monday.
Thanks & 73,
Doug
K4AC
From: Shelley, Barry, N1VXY [mailto:bshelley@arrl.org]
Sent: Wednesday, February 18, 2015 11:52 AM
To: Rehman, Doug, K4AC; arrl-odv
Cc: Keane, Michael, K1MK
Subject: RE: [arrl-odv:23969] Re: arrl.org is Down
Doug:
I apologize for not responding to your post yesterday, but I was out of the office for most of the afternoon. I wanted to provide you additional information after discussing the situation with Michael Keane, K1MK.
You are correct; the website is indeed running on CentOS 5.10 / Apache 2.2.3. CentOS is, in the simplest of terms, the Linux operating system. The reason that we are still on CentOS 5.10 is that we need to do a full-scale risk reduction exercise to ensure that the Fathom website doesn't break upon upgrade to a more recent version of CentOS. And if it does break in the process, how to fix it.
However, the End-of-Life (EOL) for Maintenance Updates of CentOS 5.10 (and its Apache 2.2.3) is coming Mach 31, 2017 so we must plan on upgrading before then. The current security plan which was attached to the Report of the CFO as Appendix 2 includes all the tasks we need to do in 2015. This upgrade is not a priority for 2015. To do it this year would require us to eliminate/defer some of the steps in the current security plan or increase resources (inside or outside). Neither step is required immediately.
That said CentOS is being employed because it provides us with a degree of stability in our production environments. Upgrading individual pieces of CentOS would defeat the purpose of employing it and would likely end up being a maintenance and update nightmare. Having packages in CentoS that are on the "Legacy Branch" is more advantageous from an operations perspective than the alternative of being on the "Bleeding Edge" and having applications continuously breaking.
And please understand, the packages in CentOS are in fact being updated with bug and security fixes from upstream, but without applying the sort of upgrades that add or subtract features and change the version number. This is a primary value-added in using the CentOS distribution: having a product team behind CentOS to provide the engineering judgment about which upgrades can safely be applied and which should not be. While CentOS 5 does employ an Apache server that has a feature set and version number that were frozen seven years ago-- it is most definitely NOT a seven year old version of Apache. The version of Apache 2.2.3 on the website was most recently updated with fixes from upstream that were released in April 2014.
I realize this was a long answer but I thought it was important to bring all the details to light.
73,
Barry J. Shelley, N1VXY
Chief Financial Officer
ARRL, Inc.
The National Association for Amateur Radio
(860) 594-0212
From: arrl-odv [mailto:arrl-odv-bounces@reflector.arrl.org] On Behalf Of Rehman, Doug, K4AC
Sent: Monday, February 16, 2015 1:49 PM
To: arrl-odv
Subject: [arrl-odv:23969] Re: arrl.org is Down
It is rather concerning that the 503 page is indicating our server is running “Apache/2.2.3” which was released in July 2007. The current release of Apache 2.2 (the webserver software) is 2.2.29. Worse yet, 2.2 is a legacy branch of the software; the current software branch is 2.4 with the current release being 2.4.12.
I have brought up on this reflector and in person at Board Meetings for more than 8 months that a cornerstone of IT security is running up to date software. We’ve already been hacked twice before, at least one of which was likely due to substantially outdated software. Yet we still have a server running on seven year old software???????????????????????????????????????????????????????????
Doug
K4AC