[arrl-odv:21274] LOTW bug fix
This is being posted and distributed this afternoon. In this circumstance I believe an apology from the CEO is appropriate. Dave ARRL's Logbook of The World: Bug Fix is on the Way The past few weeks have been a frustrating time for many users of the ARRL's Logbook of The World (LoTW) QSO confirmation system. The problems began with a 3-1/2 day outage during the first week of November - a particularly busy time of the year for log uploads. When the system was brought back on line a large queue of logs awaiting processing developed, and processing times lengthened well past normal expectations. Another, much more subtle problem cropped up at about the same: some logs began disappearing from the queue, apparently at random. Users had been advised at the time the log was uploaded that it was "queued for processing" and so they were understandably upset when the log was not processed, even after the several-day delay that most logs were experiencing. Because of its random nature it took the ARRL IT staff a while to figure out what was happening. When LoTW was designed more than a decade ago - long before the present IT staff was here - an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs. When a log is uploaded it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay combined with the dramatic increase in the number of submitted logs, the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten. Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use even though this may result in a few more logs being lost until the fix is in place. We apologize for the inconvenience that users have experienced, and especially for being unable to explain what was happening until now. We want to emphasize that no data from processed logs has been lost. That data is secure and backed up. If you have had a log disappear after it was "queued for processing" the solution is to upload the log file again, preferably after the bug fix is in place. We will announce when that occurs. 73, David Sumner, K1ZZ Chief Executive Officer, ARRL November 28, 2012
Dave, Excellent. Jim Weaver, K8JE Director, Great Lakes Division 5065 Bethany Rd. Mason, OH 45040 Tel. 513-459-1661; e-mail K8JE@arrl.org ARRL: The reason Amateur Radio Is Members: The reason ARRL is _____ From: arrl-odv-bounces@reflector.arrl.org [mailto:arrl-odv-bounces@reflector.arrl.org] On Behalf Of Sumner, Dave, K1ZZ Sent: 28 November, 2012 1:29 PM To: arrl-odv Subject: [arrl-odv:21274] LOTW bug fix This is being posted and distributed this afternoon. In this circumstance I believe an apology from the CEO is appropriate. Dave ARRL's Logbook of The World: Bug Fix is on the Way The past few weeks have been a frustrating time for many users of the ARRL's Logbook of The World (LoTW) QSO confirmation system. The problems began with a 3-1/2 day outage during the first week of November - a particularly busy time of the year for log uploads. When the system was brought back on line a large queue of logs awaiting processing developed, and processing times lengthened well past normal expectations. Another, much more subtle problem cropped up at about the same: some logs began disappearing from the queue, apparently at random. Users had been advised at the time the log was uploaded that it was "queued for processing" and so they were understandably upset when the log was not processed, even after the several-day delay that most logs were experiencing. Because of its random nature it took the ARRL IT staff a while to figure out what was happening. When LoTW was designed more than a decade ago - long before the present IT staff was here - an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs. When a log is uploaded it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay combined with the dramatic increase in the number of submitted logs, the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten. Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use even though this may result in a few more logs being lost until the fix is in place. We apologize for the inconvenience that users have experienced, and especially for being unable to explain what was happening until now. We want to emphasize that no data from processed logs has been lost. That data is secure and backed up. If you have had a log disappear after it was "queued for processing" the solution is to upload the log file again, preferably after the bug fix is in place. We will announce when that occurs. 73, David Sumner, K1ZZ Chief Executive Officer, ARRL November 28, 2012
Dave A nice explanation and a smart move to "lift the veil". It should prove to be a beneficial decision. Thanks. 73 *-----------------------------------------------------* ** John Robert Stratton N5AUS Office telephone: 512-445-6262 Cell: 512-426-2028 PO Box 2232 Austin, Texas 78768-2232 *-----------------------------------------------------* On 11/28/12 12:40 PM, Jim Weaver K8JE wrote:
Dave, Excellent. */Jim Weaver, K8JE/* /Director, Great Lakes Division/ /5065 Bethany Rd./ /Mason, OH 45040/ /Tel. 513-459-1661; e-mail K8JE@arrl.org <mailto:K8JE@arrl.org>/ ARRL: The reason Amateur Radio Is Members: The reason ARRL is
------------------------------------------------------------------------ *From:* arrl-odv-bounces@reflector.arrl.org [mailto:arrl-odv-bounces@reflector.arrl.org] *On Behalf Of *Sumner, Dave, K1ZZ *Sent:* 28 November, 2012 1:29 PM *To:* arrl-odv *Subject:* [arrl-odv:21274] LOTW bug fix
This is being posted and distributed this afternoon.
In this circumstance I believe an apology from the CEO is appropriate.
Dave
*ARRL's Logbook of The World: Bug Fix is on the Way*
**
The past few weeks have been a frustrating time for many users of the ARRL's Logbook of The World (LoTW) QSO confirmation system. The problems began with a 3-1/2 day outage during the first week of November -- a particularly busy time of the year for log uploads. When the system was brought back on line a large queue of logs awaiting processing developed, and processing times lengthened well past normal expectations.
Another, much more subtle problem cropped up at about the same: some logs began disappearing from the queue, apparently at random. Users had been advised at the time the log was uploaded that it was "queued for processing" and so they were understandably upset when the log was not processed, even after the several-day delay that most logs were experiencing.
Because of its random nature it took the ARRL IT staff a while to figure out what was happening. When LoTW was designed more than a decade ago -- long before the present IT staff was here -- an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs.
When a log is uploaded it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay combined with the dramatic increase in the number of submitted logs, the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten.
Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use even though this may result in a few more logs being lost until the fix is in place.
We apologize for the inconvenience that users have experienced, and especially for being unable to explain what was happening until now. We want to emphasize that no data from processed logs has been lost. That data is secure and backed up. If you have had a log disappear after it was "queued for processing" the solution is to upload the log file again, preferably after the bug fix is in place. We will announce when that occurs.
73,
David Sumner, K1ZZ
Chief Executive Officer, ARRL
November 28, 2012
_______________________________________________ arrl-odv mailing list arrl-odv@reflector.arrl.org http://reflector.arrl.org/mailman/listinfo/arrl-odv
TNX Dave! This will help the situation from my perspective. I think little updates like this help ameliorate touchy situations with the members. Good show on the explanation! 73 David A. Norris, K5UZ Director Delta Division Sent from my iPhone On Nov 28, 2012, at 12:29 PM, "Sumner, Dave, K1ZZ" <dsumner@arrl.org> wrote:
This is being posted and distributed this afternoon.
In this circumstance I believe an apology from the CEO is appropriate.
Dave
ARRL’s Logbook of The World: Bug Fix is on the Way
The past few weeks have been a frustrating time for many users of the ARRL’s Logbook of The World (LoTW) QSO confirmation system. The problems began with a 3-1/2 day outage during the first week of November – a particularly busy time of the year for log uploads. When the system was brought back on line a large queue of logs awaiting processing developed, and processing times lengthened well past normal expectations.
Another, much more subtle problem cropped up at about the same: some logs began disappearing from the queue, apparently at random. Users had been advised at the time the log was uploaded that it was “queued for processing” and so they were understandably upset when the log was not processed, even after the several-day delay that most logs were experiencing.
Because of its random nature it took the ARRL IT staff a while to figure out what was happening. When LoTW was designed more than a decade ago – long before the present IT staff was here – an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs. When a log is uploaded it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay combined with the dramatic increase in the number of submitted logs, the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten.
Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use even though this may result in a few more logs being lost until the fix is in place.
We apologize for the inconvenience that users have experienced, and especially for being unable to explain what was happening until now. We want to emphasize that no data from processed logs has been lost. That data is secure and backed up. If you have had a log disappear after it was “queued for processing” the solution is to upload the log file again, preferably after the bug fix is in place. We will announce when that occurs.
73, David Sumner, K1ZZ Chief Executive Officer, ARRL November 28, 2012
_______________________________________________ arrl-odv mailing list arrl-odv@reflector.arrl.org http://reflector.arrl.org/mailman/listinfo/arrl-odv
Dave, Thank you. This note will really help. Dale WA8EFK Vice Dir. Great Lakes On Nov 28, 2012 1:29 PM, "Sumner, Dave, K1ZZ" <dsumner@arrl.org> wrote:
This is being posted and distributed this afternoon.****
** **
In this circumstance I believe an apology from the CEO is appropriate.****
** **
Dave****
** **
** **
** **
** **
*ARRL’s Logbook of The World: Bug Fix is on the Way*
* *
The past few weeks have been a frustrating time for many users of the ARRL’s Logbook of The World (LoTW) QSO confirmation system. The problems began with a 3-1/2 day outage during the first week of November – a particularly busy time of the year for log uploads. When the system was brought back on line a large queue of logs awaiting processing developed, and processing times lengthened well past normal expectations.****
** **
Another, much more subtle problem cropped up at about the same: some logs began disappearing from the queue, apparently at random. Users had been advised at the time the log was uploaded that it was “queued for processing” and so they were understandably upset when the log was not processed, even after the several-day delay that most logs were experiencing.****
** **
Because of its random nature it took the ARRL IT staff a while to figure out what was happening. When LoTW was designed more than a decade ago – long before the present IT staff was here – an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs.****
When a log is uploaded it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay combined with the dramatic increase in the number of submitted logs, the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten.****
** **
Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use even though this may result in a few more logs being lost until the fix is in place.****
** **
We apologize for the inconvenience that users have experienced, and especially for being unable to explain what was happening until now. We want to emphasize that no data from processed logs has been lost. That data is secure and backed up. If you have had a log disappear after it was “queued for processing” the solution is to upload the log file again, preferably after the bug fix is in place. We will announce when that occurs. ****
** **
73,****
David Sumner, K1ZZ****
Chief Executive Officer, ARRL****
November 28, 2012****
** **
_______________________________________________ arrl-odv mailing list arrl-odv@reflector.arrl.org http://reflector.arrl.org/mailman/listinfo/arrl-odv
Dave and all, I concur with the other posts regarding the communication to members and others regarding the issues with LOTW as well as keeping them informed about the progress of fixing the system to help eliminate these type of issues. Barry and I have been trading e-mails regarding the communication and I hope we, collectively, can continue a mindset of communication with our constituents (think customers) for future issues. I look forward to our discussions in New Orleans in a few weeks where we can carry on this discussion. 73 Jim K9JF James E. Fenstermaker K9JF Director, Northwestern Division American Radio Relay League (206) 930-9372
participants (6)
-
Dale Williams -
David Norris -
jfenster＠pacifier.com -
Jim Weaver K8JE -
JRS -
Sumner, Dave, K1ZZ