Jim, Greetings from the airport in Ho Chi Minh City where I am sitting with Rod Stafford awaiting our flight to Hong Kong. I agree with you that information was too slow to be made available. I became aware of the problem on Tuesday morning here, but of course it was Monday night in Connecticut and we do not staff for 24/7 IT coverage. Of course it took a while for an assessment of the situation on Tuesday and IT's first priority was to fix it, but we should have had an explanation posted sooner. I have made that point to staff. LOTW is to some extent a victim of its own success. It was designed originally to provide a faster, cheaper, and less labor-intensive alternative to paper QSLs. It has accomplished that and more, at least with respect to the awards programs it supports. But while we certainly don't want to have the system unavailable for days at a time it was not designed to be a real-time confirmation system. One of the key design objectives was to preserve the integrity of the DXCC program. Achieving that required more complexity than, say, Club Log. They're about to call our flight so let me close with one thought. Many of our members are passionate about their particular interest within Amateur Radio. Sometimes that leads them to lose perspective, especially if other aspects of their lives are not going as well as they would like. One of the reasons people have hobbies is to be able to retreat from other frustrations in life, which makes problems such as this to become magnified. But it's when our members stop being passionate that we should really worry. 73, Dave K1ZZ arrl-odv-bounces@reflector.arrl.org on behalf of Jim Weaver K8JE Sent: Fri 11/9/2012 8:34 PM To: Shelley, Barry, N1VXY; arrl-odv Subject: [arrl-odv:21204] Re: LOTW Update Barry, Thanks for the update and further explanation. Without getting into the "whys" of "hows" of the actual situation, I have one question regarding its management. My question is why did we not send out a broadcast notice that described the situation and what was being done to rectify it as soon as the problem was recognized? Doing this would most certainly have headed off much of the ". . . misinformation being tossed around in cyberspace . . ." As the old adage says, "An ounce of prevention is worth a pound of cure." FYI, after the situation had been described to Bernie, as I recall, I sent a message to the Great Lakes Division to let members know what had happened, and more importantly, what had not happened (i.e., that the misinformation was not correct). A number of members replied. While a number of these were displeased over what they viewed as the failure by IT to perform a basic function, they were as pleased that finally someone told them what was going on. Conversely, several of them expressed considerable displeasure that by its inaction, HQ "once again" demonstrated (to paraphrase) "it is a self-serving operation that neither respects nor appreciates the members." My major concern is that just about the time "we" seem to recover from a problem that became inflated by insufficient communication (and "our" status is nearly repaired with the members), "we" drop the ball once again and undermine "our" prior good work by failing to acknowledge a situation exists or in delaying the acknowledgement until after the ARRL-naysayers have had free reign to establish the tone. Jim Jim Weaver, K8JE Director, Great Lakes Division 5065 Bethany Rd. Mason, OH 45040 Tel. 513-459-1661; e-mail K8JE@arrl.org ARRL: The reason Amateur Radio Is Members: The reason ARRL is ________________________________ From: arrl-odv-bounces@reflector.arrl.org [mailto:arrl-odv-bounces@reflector.arrl.org] On Behalf Of Shelley, Barry, N1VXY Sent: 09 November, 2012 2:35 PM To: arrl-odv Subject: [arrl-odv:21203] LOTW Update While the misinformation being tossed around in cyberspace regarding LOTW is almost too wild to answer, let me try and respond to some of the more relevant issues. Currently, LOTW is up and running as of 7:30 am EST with a processing queue of approximately 48 hours. What happened, despite what others may think, involved human error in not recognizing that the space allocated to the LOTW database was filling up. There were monitors on LOTW admin screens within the system but no one recognized the impending condition. This has been rectified and automated alarms have been added to the database. If we had recognized it earlier, expanding the storage to currently available space, it would have been done with no need to take the system down. And as has already been noted, there was no data lost. The system does, given the size of the database, take time to march through the recovery process and that's what was happening over the past 3 days. In the 2012 Plan (Capital Expenditure section) we recognized the need for more storage and got approval to add an additional 2.8 terabytes at a cost of $25,000 to allocate to various HQ functions including LOTW. This was in place. And to respond to a couple of other questions, the system is built on a commercial database product from SAP. Of course there is a lot of custom programming for the actual LOTW functions as there is no product waiting to be pulled off the shelf for this application. In addition, there is unique, custom programming to connect the LOTW software to the SAP database. The Chairman has assured me that this issue will be discussed further with the Administration and Finance Committee at their meeting Saturday, November 17th. 73, Barry J. Shelley, N1VXY Chief Financial Officer ARRL, Inc. The National Association for Amateur Radio (860) 594-0212 www.arrl.org