AMDave
11-19-2012, 01:09 AM
Continued server problems.
We're continuing to have issues due to a database problem early last week and a botched attempt to fix it. The problem is that the result and host tables in the database have grown large enough, and hosts have gotten fast enough that the lookup of result in process for a host and the enumeration of new results to send don't finish before the web connection times out either on the server or the client side. This resulted in hosts being assigned large number or results to compute without the transaction that tells them about these results being completed. The host. think it received no results would then contact the server for more results, which it would again not receive.This isn't a hardware problem. The database currently fits in memory and the processors are fast. We've just crossed a threshold where each host computes fast enough that host queues and the result table have become large enough to cause this problem. To solve it, we've put per host limits on results in process back in place. But hosts that are having this problem will probably continue to have it until the average number of results per host has fallen to a workable level. That could take weeks.For a more permanent fix, we plan do more work in each result by quadrupling the size of the workunits. But that fix will probably take months to implement and test.
More... (http://setiathome.berkeley.edu/forum_thread.php?id=70080)
We're continuing to have issues due to a database problem early last week and a botched attempt to fix it. The problem is that the result and host tables in the database have grown large enough, and hosts have gotten fast enough that the lookup of result in process for a host and the enumeration of new results to send don't finish before the web connection times out either on the server or the client side. This resulted in hosts being assigned large number or results to compute without the transaction that tells them about these results being completed. The host. think it received no results would then contact the server for more results, which it would again not receive.This isn't a hardware problem. The database currently fits in memory and the processors are fast. We've just crossed a threshold where each host computes fast enough that host queues and the result table have become large enough to cause this problem. To solve it, we've put per host limits on results in process back in place. But hosts that are having this problem will probably continue to have it until the average number of results per host has fallen to a workable level. That could take weeks.For a more permanent fix, we plan do more work in each result by quadrupling the size of the workunits. But that fix will probably take months to implement and test.
More... (http://setiathome.berkeley.edu/forum_thread.php?id=70080)