I will remind my fellow Board members of my proposal for creating an IT Strategic Planning Committee. Had this Committee already existed and been properly constituted, this breach could have been averted. This is the second time in the exceedingly short time I have been on the Board that a system has been found with outdated software. (The email reflectors were running a years old version of the software.) Having anything in a production IT environment that is not running the most current software is absolutely unacceptable. This is basic computer security 101 and we have failed twice this year that I know of. I ask the members of the Executive Committee to take up my proposal at the upcoming meeting and put it out to the Board for an electronic vote. We cannot wait for the next crisis—we need to start getting ahead of problems instead of trying to play clean up. Doug K4AC (and Valencia College Professor in the Digital Forensics and Cyber Security degree program…) From: arrl-odv [mailto:arrl-odv-bounces@reflector.arrl.org] On Behalf Of Shelley, Barry, N1VXY Sent: Friday, October 3, 2014 6:24 PM To: Stratton, John, N5AUS Cc: arrl-odv Subject: [arrl-odv:23378] Re: CONFIDENTIAL -- Security Breach - Outside Help? John: We've already begun developing a list of possible actions/recommendations that we want to consider. That is on the list. Our first priority is insuring we've eliminated the threat and any lingering vulnerabilities. Then we'll tackle the list going forward. 73, Barry, N1VXY Sent from my Verizon Wireless 4G LTE DROID "Stratton, John, N5AUS" <jrs@hamradio.us.com> wrote: Barry Thank you for keeping the Board up to date. Have we considered bringing in outside security experts to review our implementation of security protocols and to make recommendations for improvements or additional methods to harden our data structure and servers? 73 ----------------------------------------------------- John Robert Stratton N5AUS Office telephone: 512-445-6262 Cell: 512-426-2028 PO Box 2232 Austin, Texas 78768-2232 ----------------------------------------------------- On 10/3/14 4:19 PM, Shelley, Barry, N1VXY wrote: ARRL Board of Directors: This is a confidential notification to the Board that we have identified an unauthorized breach of the ARRL computer systems. We are currently working through all the appropriate steps as described below. In addition, we are in the process of making the necessary notification to law enforcement agencies. Situation: Early yesterday, we became aware of files on one of our many servers that were “unusual” and the IT Dept., in their investigation, determined that there had been a breach of one of our oldest servers (P1K), a proxy server for the web site. While uncertain of the exact vulnerability, the recently identified “Shellshock” vulnerability for older Linux systems may have been a factor in this attack. We were able to track IP address activity to Eastern Europe and identify the activities from those sites. We immediately shut down access to P1K from outside the building. This has impacted some web services such as the On-line DXCC application and legacy connections to LoTW. On-going Activities: In these situations the general steps to follow are: 1. Lock systems down. 2. Identify what, if anything was accessed. 3. Notify law enforcement. 4. Decisions on notification. 5. Take corrective actions. At this moment we still have prohibited access to the P1K server from outside the building. We are in the process of bringing up another server to support those functions that were supported by the old server but with all the current patches, updates applied. Because of the age of the old server, we were not able to apply all the current updates, etc. Projections for the new proxy server coming on-line are early next week. We will give priority to the applications with a higher level of activity like LoTW and On-line DXCC. Please note, it is only the legacy applications that use the P1K proxy to access the LoTW server. In addition, IT is going through all the code for the database servers and programs, eliminating any potential vulnerabilities. The IT Department has been reviewing the log files in detail and, to this point it would appear that the database the hackers had most activity with was the On-line DXCC database. They did have access, however, to names, addresses and e-mail addresses from our main date base. Current web site passwords are encrypted but passwords used on the former website (prior to 2010) were not encrypted and archival copies of those passwords stored on HQ servers may have been compromised. A limited test of old passwords indicates that possibly 10% of the old passwords may still be in use. We have not seen any unusual activity on the LoTW server. Because of the configuration, the hackers were not able to get information from the ARRL web site itself. Hackers could have accessed name, address, phone numbers and e-mail addresses. We do not keep credit card or social security information in our databases so this is not information that could be accessed. We are still working on reviewing all the logs and activity in the various databases to identify what information, if any, might have been breached and transferred. This effort will be on-going. We are in process of notifying the appropriate law enforcement agencies, including Federal, but quite honestly, I don’t expect much attention from them. More important would be our obligation to notify individuals of a possible breach of their information. State laws vary so it will be complicated. I’ve been given the name of a law firm with an excellent reputation in these matters and plan to contact them as soon as possible. A quick reading of Connecticut statutes doesn’t indicate a requirement to notify in this situation because of the nature of the information potentially compromised, but there are 49 other states with different regulations and we have members in all of them. I’m sure there will be many questions but please understand that the situation is very fluid. I will provide you with updates as we move forward. _______________________________________________ arrl-odv mailing list arrl-odv@reflector.arrl.org http://reflector.arrl.org/mailman/listinfo/arrl-odv