This is a bit scary.. "unique and unknown process, only known to Michael, who is not responding to email" Mark, HDX On Tue, Dec 1, 2020 at 2:20 PM Minster, David NA2AA (CEO) <dminster@arrl.org> wrote:
As you’ve seen & heard, LoTW is struggling under the weight of the CQWW logs -and- people pounding on the system to see if their logs were processed, re-uploading the same file, and checking to see what affect their weekend effort had on their DXCC totals.
Here’s the headlines:
- We are using MaxDB (MySQL on steroids) for the database - Great for hobby use, not for enterprise use - We are not at the latest version - The database is configured for a maximum of 256 concurrent processes – which is THE bottleneck - Although that parameter could be increased (it was once 1024), downing and restarting the database is a unique and unknown process, only known to Michael, who is not responding to email - The database is at 94% utilization, which is a bad thing and is also likely causing paging to occur, slowing down the database - We’ve had new servers (faster with more storage) since this summer, but LoTW was never migrated - The system is written as a single queue single contact process that checks each line THEN checks it against EVERY award that user is configured for – which is unnecessarily SLOW - It is not likely we would be able, as architected, to gain any benefit from forcing into the cloud – we could get onto bigger machines, but we already have them! - Project X is exactly what our LoTW programmer (Doug) believes needs to happen: 3 tier design (database, logs & awards engines, UI) - Once the system is architected for Cloud, the UI and Log Processing can each run efficiently using independent servers, while the awards engines can auto-scale additional servers as needed - Project X will be sent to ODV ahead of the January board meeting as a proposed project with next steps
I have asked Greg Widin what the best course of action is in updating members.
David _______________________________________________ arrl-odv mailing list arrl-odv@reflector.arrl.org https://reflector.arrl.org/mailman/listinfo/arrl-odv