As you've seen & heard, LoTW is struggling under the weight of the CQWW logs -and- people pounding on the system to see if their logs were processed, re-uploading the same file, and checking to see what affect their weekend effort had on their DXCC totals. Here's the headlines: * We are using MaxDB (MySQL on steroids) for the database - Great for hobby use, not for enterprise use * We are not at the latest version * The database is configured for a maximum of 256 concurrent processes - which is THE bottleneck * Although that parameter could be increased (it was once 1024), downing and restarting the database is a unique and unknown process, only known to Michael, who is not responding to email * The database is at 94% utilization, which is a bad thing and is also likely causing paging to occur, slowing down the database * We've had new servers (faster with more storage) since this summer, but LoTW was never migrated * The system is written as a single queue single contact process that checks each line THEN checks it against EVERY award that user is configured for - which is unnecessarily SLOW * It is not likely we would be able, as architected, to gain any benefit from forcing into the cloud - we could get onto bigger machines, but we already have them! * Project X is exactly what our LoTW programmer (Doug) believes needs to happen: 3 tier design (database, logs & awards engines, UI) * Once the system is architected for Cloud, the UI and Log Processing can each run efficiently using independent servers, while the awards engines can auto-scale additional servers as needed * Project X will be sent to ODV ahead of the January board meeting as a proposed project with next steps I have asked Greg Widin what the best course of action is in updating members. David