PDA

View Full Version : bok's stats



vaughan
06-13-2008, 10:23 AM
Has anybody managed to get bok's forum signature stats to work over here? I couldn't get the old statstool to work due to some weird cookies issue and now the new statstool doesn't seem to have a forum linky. Not sure what I'm doing wrong but the old sig looked horrendous as it had every project selected so I deleted it from my forum signature.

Beerknurd
06-13-2008, 10:25 AM
If you have some stats that aren't showing up... you may have to get new numbers for them be visiting each project in Bok's stats, hovering over your name, and getting the unique identifier for your stats... I had to do this to get my Eon stats to show up... Whatpulse has been flaky and Bok is aware of this issue... other than that the stats tool seems to be working great...

Bender10
06-13-2008, 10:41 AM
I'm still using the 'Old' link to the original stats signature. That seems to be working fine, with a few quirks on some projects. I haven't looked at the new statstool link yet.

The other day, the 'look' of my 'old' stats sig changed. Must have been from Bok doing an update??

Beerknurd
06-13-2008, 10:44 AM
I am using the new one and it works great... I just had to change some colors and update the Eon username in my stats sig...

AMDave
06-13-2008, 11:11 AM
What Beerknurd is trying to say is "Yes." :icon_razz:

The url is about the same:

http://stats.free-dc.org/tag.php?name=AMDavehttp://stats.free-dc.org/tag.php?name=AMDave

Beerknurd
06-13-2008, 11:28 AM
Good lord Dave... Why must you show all your stats... :lol:

AMDave
06-13-2008, 11:35 AM
...just to remind myself how totally awesome (e'hem addicted) I am.

Nflight
06-13-2008, 11:50 AM
...just to remind myself how totally awesome (e'hem addicted) I am.

@AMDave: I say self amusement and enough to startle the rest of us in thinking we should think he is special... uh hum !

ON the Stats solution I have found everything to work out ok, but used Beerknurd as the lead to resolve the EoN issue, the whatpulse error should soon be fixed, I hope.

AMDave
06-13-2008, 12:15 PM
@AMDave: I say self amusement and enough to startle the rest of us in thinking we should think he is special... uh hum !
Shazbutt!
I have been sprung. :sad5:
Back to posing in the the mirror again. :5eek:

I'm thinking it may finally be time to get a web cam so I can admire myself, in-picture, while I work. :biggrin:

Beerknurd
06-13-2008, 12:39 PM
I am actually having difficulty getting my stats to stick on my sig page... There are quite a few stats on that page that won't update...

For Example DPAD

It will not let me save my information. I put in my ID number and hit update, when the page refreshes, it is still blank...

Hopefully Bok will see this post... Is this problem happening to anyone else??

Nflight
06-13-2008, 12:52 PM
As you can see above my stats signature in DPAD are fine,,, :sad5:

Beerknurd
06-13-2008, 01:05 PM
What does your DPAD identifier look like on the stats sig page?? Paste exactly what you have in the box...

Nflight
06-13-2008, 01:10 PM
[AMD Users]Nflight

AMDave
06-13-2008, 01:24 PM
There is much wisdom in the words of the Nflight.
To heed him, will bring you great fortitude.

Bok
06-13-2008, 01:29 PM
Beer, you would put in [AMD Users]Beerknurd I imagine you are putting in the '+' but that is just HTML formatting as spaces are not supported in a url.. Hope that helps Bok

Beerknurd
06-13-2008, 02:16 PM
Ok so [AMD Users]Beerknurd works...

But...

It still will not save the info... If I put the info in and click update, my signature will update to show the info, but the stats sig page will not save the data... All the info I just entered is gone... It is doing that for DPAD and Depspid... It just won't save the info I put in the blank...


***EDIT***

DPAD saved, but TSC(Old), Depspid, and PI Segment will not save...

Bok
06-13-2008, 02:22 PM
hmm, someone else reported that too... :( I'll have to take a look at the update script. Something must be wrong. Unfortunately I'm on a customer site and we are doing a go-live conversion this afternoon/evening, so I'll probably be here till midnight or later..

Beerknurd
06-13-2008, 02:31 PM
No hurry, just making sure it wasn't a user error... :)

Sometimes I have PEBKAC issues... lol

http://faroutshirts.com/images/PEBKAC_tn.jpg

Beerknurd
06-13-2008, 02:36 PM
I tried a few more times and TSC(Old) finally stuck... I will keep trying Depspid and Pi Segment...

***EDIT***

Depspid is the only project now that is not saving the username info... At least that I can tell...

Now the only issue is to get the checkmark saved... lol

I put a check mark on the project I want to show and hit update, my signature will reflect the change. But the checkmark is gone. If I hit update again without re-checking the box, it will remove the project from my signature...

Bok
06-13-2008, 03:01 PM
hmm, wait a moment, are you using http://stats.free-dc.org/stats.php?page=statstool or http://stats4.free-dc.org/stats.php?page=statstool ?? I fixed some things like that a few days but probably only on stats, not stats4..

Beerknurd
06-13-2008, 03:08 PM
It wasn't stats4... I went to it again to make sure, and now Depspid shows up, but I am still having the check mark issue from above...

Weird...

Beerknurd
06-16-2008, 03:39 PM
Bok,

Can you get the Dimes stats to show all the clients like in the old stats??

Bok
06-16-2008, 04:09 PM
yes, once I figure out what is wrong with them. That's one of the tasks I intend to look at this afternoon.

I'm still storing all the info, just that the code is only for BOINC right now, have to alter it slightly for dimes too.

Bok

Beerknurd
06-16-2008, 04:22 PM
Ok cool... No hurry... :)

Bok
06-16-2008, 06:42 PM
ask and ye shall receive :)

done...

Bok

Beerknurd
06-16-2008, 07:43 PM
YAY!!!!! Thanks Bok!!!!!!

:wav:

Beerknurd
06-16-2008, 11:52 PM
Bok,

Still having problems with Whatpulse on the new stats??

Bok
06-17-2008, 12:31 AM
I haven't had a chance to look at them yet

Beerknurd
06-17-2008, 12:45 AM
Oh ok... Cause that day we talked they updated.. I thought maybe you fixed them... :icon_redface:

Beerknurd
06-18-2008, 01:12 PM
Has DCR gone away?? There is a checkbox for it on the statstool configuration page but no link that I can find in the new stats? And it is not on our signatures anymore.

PoorBoy
06-18-2008, 02:58 PM
Since Bok went to the new Stat's Pages they've been kinda Bonked ... :icon_lol: ... He'll get it straightened out eventually I'm sure, but for now I have to rely on other Sites because Bok's don't tell me much of anything most of the time ... :icon_neutral:

Bok
06-18-2008, 03:44 PM
DCR has gone away for now.

Poorboy, without specifics I have no way of responding. They work fine for me for all but some of the more obscure projects (whatPulse, Bewes).

What exactly do you believe is not working? I'm always interested in any feedback on faults.

Bok

PoorBoy
06-18-2008, 04:09 PM
Most of the Time when I try to check the daily stats for Cosmology theres nothing showing, just 0 for the Participants & Teams. The Overall Totals are there but the Daily isn't. I just checked them again and that's what I saw, 0 for the day so far, seems like at times the Daily Stats are there & then they will be gone ... :)

Beerknurd
06-18-2008, 04:40 PM
There won't be any stats for BeWeS... The project is down currently...

Bok
06-18-2008, 05:11 PM
Well, daily is exactly that, I roll them over to zero at approx 2am MST, so they will be zero until there is an update available, which I check for every 2 hours. Some projects only update their xml files once per day, though I usually lobby for 6 hourly updates.

http://stats.free-dc.org/stats.php?page=proj&proj=cos

shows that they updated fairly recently.

PoorBoy
06-18-2008, 07:12 PM
The old Stats Pages didn't have that 0 Period, there was always something in the Daily Column, now it seems half the day theres just 0 for the Daily. If it's working the way it should then okay, I just liked the old pages better I guess ... :)

liuqyn
06-18-2008, 07:34 PM
they do tend to stay at 0 most of the morning, maybe a tweak is needed?

Bok
06-18-2008, 07:35 PM
Correct, I never had a rollover in the old stats, so the 'today' was actually incorrect most of the time :)

I guess I could just remove the today tables from the proj page.

PoorBoy
06-18-2008, 07:44 PM
I guess I could just remove the today tables from the proj page.

:shocked: ... But, But, But how are we going to get our Daily Stats Fix then ... :icon_lol:

Steve Lux
06-19-2008, 01:47 PM
I've turned off my stats signature. Doesn't work, can't fix it. E-mailed 3 weeks ago for my stats tool password from the website - no response. I still use the FreeDC stats pages though.

Bok
06-19-2008, 02:00 PM
you can always ask me here :) Perhaps the email from the forum went into spam ?

I'll reset it and PM you the pw here.. what is your username ? I don't see it on a search..

Bok
06-20-2008, 04:20 PM
implemented on proj.php page so that if the data has not been updated today, it will display yesterdays scores instead.. Only on stats4 for now..

http://stats4.free-dc.org/stats.php?page=proj&proj=rev

dnet is always a good example..

http://stats4.free-dc.org/stats.php?page=proj&proj=ogr25

Still a slight flaw, in that the update_date is actually set once I finish running a project through into the temp tables, but I have to wait until the overall_boinc_scores are finished before transitioning these to the live tables, so there may be a window where it does not work.

I have a plan in place to drastically reduce the update times anyway and it will correct this..

Bok

gatekeeper53
06-20-2008, 10:18 PM
BOK,
Is there a chance of changing the milestones in SOB so that there is something to shoot for after a week or so? Mine are somewhat close since I hit most of them in 2 days of running it http://stats4.free-dc.org/stats.php?page=user&proj=sob&name=14152 . I might suggest 50g, 100g, 500g, 1t, 5t, 10t, 50t, 100t, 500t, 1p, and 5p. That would give all but the biggest pharmers something to look forward to for quite sometime. It's just a thought and it's not anything that needs done until sometime that you might be bored out of your mind. lol

Bok
06-20-2008, 10:22 PM
sure, they are configurable.. I'll alter them.

Actually, you should have got one for the 50T milestone.. not sure why. ugh, I see why. SoB is a little different because of the way I display the scores. Your actual score held in the db is 54012753252464. I didn't have enough zeroes in my milestone table for it..

Hopefully I have it done now, so you should see a milestone when you hit 60T (&70,80,90,100,200,300 etc)..

My head hurts looking at the numbers but I think 60000000000000 is 60T

Bok

gatekeeper53
06-20-2008, 10:50 PM
I think there's something about being a government bureaucrat that is required to think in terms with that many zeros. lol Thanks

Steve Lux
06-23-2008, 03:38 AM
So, I create a new userID and password and logged into the FreeDC forums and I am welcomed by my user name - all is good with the world, yes? As I don't see a direct link to the Statstool configuration page, I click on the Stats link, and then click on the Statstool Configuration link and am eventually told that I am not logged in to the forum. I even tried going from the stats4... links posted in the forums - again to be told that I'm not logged into the forums.

After several attempts of logging in and being welcomed by my userID and failing to get to the statstool configuration page, because I'm not logged in, I decided to take the hint and give it a rest.

Bok
06-23-2008, 02:17 PM
When you log into the forum, you have to make sure you tick the 'Remember Me' box, otherwise vbulletin does not save the cookie correctly. (it's a vbulletin oddity)

If it still fails, then the problem is in the cookies on your machine and they need to be deleted first.

Most likely though it'll be the first thing.

Bok

Steve Lux
06-24-2008, 12:43 AM
Thanks Bok. Selecting the "Remember me?" box does it. Now, apparently I'm supposed to have some sort of member number to add to the hyperlink to activate the signature. I've tried looking anywhere I can find my name and in the statstool page, I even did a search in the member list (I'm not there either). Just for giggles I looked in your page source code and my ID info seems to be encrypted.

Bok
06-24-2008, 12:59 AM
http://stats.free-dc.org/tag.php?name=Steve+Lux

no member number at all, it's just your username..

Your userpage will be http://stats.free-dc.org/stats.php?page=usermain&name=Steve+Lux

and your tag should show up as above url is http://stats.free-dc.org/tag.php?name=Steve+Lux

Because there is a blank space, you need to use html formatting to replace the blank with a +

Bok

p.s. Your F@H userid should be Steve_Lux13129 (user + teamid concatenated)

Your PI Segment id is actually 9ba3eb88-7148-4e6d-a6dc-04500c860b05

vaughan
02-26-2009, 08:02 AM
Anybody know what's up with bok's stats today?

When I follow my bookmark it takes me to the Index page. So far so good. Now if I try to check eg the NPLB stats it goes to stats5 not stats. I tried a Control-F5 in Firefox 3.0.6 to refresh the page but it gives the same result. At the top of the page is a message box - a grey ribbon with some text that I couldn't read - I tried a refresh but the text didn't reappear.

The stats show all Zeroes for the Today fields. Also noticed that when you mouse over a user or team name that it gets a BOLD font.

Nflight
02-26-2009, 08:15 AM
There must be a Cats Furball caught in the operation. I too am having problems updating my stats tool configuration. :icon_mad: Lets see what Bok has to say?

AMDave
02-26-2009, 01:05 PM
I raised a bug report.
It is being traced.

Bok
02-26-2009, 01:18 PM
oopsy.

fixed.

I inadvertently altered a file making all base pages in the links go to the dev site...start of some database caching operations which are going into effect soon..

Fixed. Let me know if statstool works again.

Bok :icon_redface:

Nflight
02-26-2009, 08:47 PM
Stats Tool is fixed thanks Bok! :blob3:

Dirk Broer
04-11-2013, 12:44 AM
Being Dutch I can't help wondering what happened to Genefer (almost sounds like Jenever doesn't it?)
(Should be between PPS Sieve and GCW)
http://stats.free-dc.org/pgridtag.php?id=72063&theme=26
Jennifer, passed out after too much Jenever

Dirk Broer
04-30-2013, 12:02 AM
Meanwhile FreeDC has stepped up the badge support: only WCG and OProject miss as compared to my signature as provided by http://signature.statseb.fr

Dirk Broer
06-05-2013, 12:14 PM
Another SSD has failed:cussing: at FreeDC, makes you wonder whether SSD are up to this particular task.:BangHead:

AMDave
06-05-2013, 02:14 PM
Bok is working on it. Apparently there is a spare. Hopefully it is in working order.
I'm pretty disappointed with these SSDs completely failing to live up to their manufacturers published quality standards.
It has sworn me off them for this type of IO duty.
It has also sworn me off the manufacturers they were from.
Based on real evidence, I can't trust them or their product.
I was looking to Bok's experience with these before taking the plunge and upgrading all of the machines here, but I'll be looking elsewhere for a better product.

AMDave
06-09-2013, 02:56 AM
Free-DC stats are back up. Enjoy.

Dirk Broer
06-09-2013, 11:01 PM
Free-DC stats are back up. Enjoy.

better, stronger, faster...
must have costed about 6 million whatshallwecallits...

Dirk Broer
06-11-2013, 07:26 PM
The new FreeDC stats:
As a plus: Almost all my project badges -except WCG and OProject (in my signature between WCG and Enigma) which are still missing
As a minus: One badge per project -even PrimeGrid and Yoyo, where you can have more.

AMDave
06-12-2013, 11:55 AM
OProject not points based. Bok asked admin for extract.
WCG badges under development, I believe.

http://www.free-dc.org/forum/showthread.php?8810-New-additions&p=166309#post166309

Dirk Broer
06-12-2013, 01:00 PM
Hi AMDave,

Thanks for asking Bok!

http://stats.free-dc.org/badgesbanner.php?cpid=76d20230028d9ceee3a1aa52fcf1 4dd4
My present badges according to FreeDC...an ever changing landscape at the moment

This looks sunnier:
http://stats.free-dc.org/mosttag.php?cpid=76d20230028d9ceee3a1aa52fcf14dd4&theme=26

Dirk Broer
06-14-2013, 12:15 PM
Still problems at FreeDC it seems.
Apart from the [happily appearing and then sadly missing again] badges there's now a non-updating score...

vaughan
06-14-2013, 03:00 PM
I thought something was wrong so I'm glad its not only me that found the stats are stuck. :)

Dirk Broer
06-14-2013, 04:08 PM
Just as we are about to break the 1,000,000,000 barrier at Moo!
(and I just completed my 25,000,000 points contribution to that)

AMDave
06-14-2013, 11:50 PM
It should be fixed now.
http://www.free-dc.org/forum/showthread.php?39517-Faults-after-failure-of-SSD-and-rewrite-of-stats&p=166447#post166447

Dirk Broer
06-15-2013, 12:01 AM
Scores are alright again, badges still aren't (see below/above -depending on your settings-).
Tried to register at FreeDC -so I can do my barking at the right tree-, but still waiting for the confirmation mail.

AMDave
06-15-2013, 01:54 AM
An update script had been stopped and was restarted.
Don't forget that Bok reduced the number of update cycles per day.
So it will take a bit longer than you are used to for a full cycle to bring it all up to date again.

There is at least one bug with the badges that I know of and I've made note of that to Bok.
Roger has spotted yet more of them so Bok will be adding those too.
He's pretty quick on his fixes so it won't be far off.
But it's Friday night there so I hope he's out getting quality time with the family.

Dirk Broer
06-16-2013, 09:05 AM
All looks alright on the FreeDC development server: http://stats6.free-dc.org/stats.php?page=userbycpid&cpid=76d20230028d9ceee3a1aa52fcf14dd4
(Still not received a confirmation mail to post straight on their forum)
Edit: But it looks even better now in production!

Dirk Broer
09-02-2013, 12:47 AM
Another SSD failure?

vaughan
02-09-2014, 12:37 AM
I'm unable to reach bok's stats at free-dc today.
Anyone concur?

AMDave
02-09-2014, 01:44 AM
ok for me from here atm.

vaughan
02-09-2014, 02:41 AM
OK now maybe temp glitch

vaughan
08-27-2014, 09:49 AM
Is stats at Free-DC unavailable for anybody else?

Dirk Broer
08-27-2014, 09:56 AM
Is stats at Free-DC unavailable for anybody else?

yes, unfortunately:-( But it's back up again now:biggrin:

Dirk Broer
11-25-2014, 09:05 PM
Is stats at Free-DC unavailable for anybody else?

Is stats at Free-DC available for anybody today? It just went up again here!

P.S. Our own site is quick as shit at minus twenty-two too....

AMDave
11-29-2014, 03:27 AM
...P.S. Our own site is quick as shit at minus twenty-two too....
I have noticed a remarkable drop in latency across the board to all US based hosts.
Something significant happened.
I don't know what, yet.

Nflight
11-29-2014, 09:01 AM
The other day I was speaking to Gopher about the issue of disappearance of my stats. Since then yes I agree with Dave something happened, response times are super fast. The distance physically from myself to Bok is about 300 miles, that is if he keeps the main arsenal in his basement? And I am right off the backbone sort of, so latency is never a problem yet in the last couple days a whoosh of speed has occurred even that I can see in Free-DC stats emerging quicker. I thought maybe I got my upgrade made a difference but then I found out that my upgrade came 2 months ago after asking my local rep. I noticed it I just wanted to mention that, Thanks AMDave :blob3:

Dirk Broer
11-29-2014, 11:53 AM
You know you can ask Bok (http://www.free-dc.org/forum/showthread.php?39594-Request-WCG-badges-in-this-thread) to incorporate the WCG badges into your FreeDC signature?
As well as your OProject (http://www.free-dc.org/forum/showthread.php?39636-Request-OProject-badges-here) badges -if you have any?

A competing product for Nflight is to be found via http://signature.statseb.fr/index.py?cpid=8f15950e07e24a5c0c73016a77c952e9

Mine looks like this:
http://signature.statseb.fr/sig-1455.png

vaughan
10-30-2015, 12:16 PM
bok's stats seem to be having problems today

Nflight
10-30-2015, 05:09 PM
At one point I was the only one in the whole Team, and at another point all points for the last two days went missing. Its been a wild and frantic rustle of nightmarish holding to my patience that it will all settle down and I will be back where i should be, yet it still isn't finished ! :blob3: :blob3:

Dirk Broer
10-30-2015, 05:15 PM
It must all have happened while I either slept or delivered parcels: everything seems alright now -at least on one of BOK's stats servers.

vaughan
12-10-2015, 10:20 AM
No bok's stats from Free-DC for me at the moment.

Nflight
12-10-2015, 12:14 PM
BOk's Stats are back now, yea earlier they were out and that was not good thing to wake up too! :sad5:

Nflight
12-10-2015, 12:18 PM
Now just as I spoke moments ago they are gone again, Bok must be working on it.

Nflight
02-23-2016, 02:02 PM
Bok's Stats are out again, seems I awoke to nothingness. :(

Dirk Broer
02-23-2016, 07:24 PM
Bok's Stats are out again, seems I awoke to nothingness. :(

Another Bitcoin Utopia sub-project that need never stop.....and it seems like a very violent crash: the complete Free-DC.org is down:(

vaughan
02-23-2016, 07:39 PM
Bok's stats at Free-DC seems to be down today.

Nflight
02-23-2016, 08:11 PM
I spoke to Bok he is in Dallas Texas on Business he is to return late this evening.

AMDave
02-23-2016, 09:13 PM
No ping.
"It's dead, Jim."

Dirk Broer
02-23-2016, 10:24 PM
No ping. "It's dead, Jim."

http://i26.photobucket.com/albums/c109/TheGlaswegian/ST3.jpghttps://s-media-cache-ak0.pinimg.com/736x/e4/b9/22/e4b9228564af1b1558192653607c2988.jpghttps://memecrunch.com/meme/2A5SF/oh-no-he-s-dead-jim/image.png?w=542&c=1

On a more serious note: this looks to be like a major crash that Free-DC suffered.:-(

Nflight
02-24-2016, 02:36 PM
From Phil - FYI, My up Address had changed on me. Incredible how these things always seem to occur when I'm not at home!! :blob3:

vaughan
02-25-2016, 03:41 AM
I'm so glad it wasn't a major hardware catastrophe again.
Get so darned used to the hourly stats fix :)

Nflight
04-06-2016, 02:02 PM
Well Jim; That is a deep subject matter! - - Since meeting Phil aka Bok in February in person in my home town, we have kept in contact pretty seriously since then. I noticed something on Saturday and since then everything is just not working itself out to function normally.
This also is a good time to explain that all his equipment except for the SSD Hard Drives are older type servers, he has some single cpu boards and dual core boards running the whole gamut of operations. I am addicted to the hourly stats fix too Vaughan, So much I have figured out how to place it on every device I own just so that if I put one down I can always reach my scores somewhere in my travels. I had hoped by this time to have been funded for my project, everything comes in its own timeline, patience will win out!

Nflight
05-30-2016, 10:00 PM
Oopsy seems the cantankerous Stats system is down again, and its a holiday no less. I lost all my stats on my CPID. :( Time to jump off a bridge I suppose.

Dirk Broer
05-30-2016, 10:22 PM
:hello:Nflight on FreeDC (http://stats.free-dc.org/stats.php?page=userbycpid&cpid=8f15950e07e24a5c0c73016a77c952e9)
Nflight on BOINCstats (http://boincstats.com/en/stats/-1/user/detail/4089):icon_cool:

Four Nflight's on Signature for BOINC users (http://signature.statseb.fr/index.py?search_name=nflight&search_exact=on)
(the other three may be found on FreeDC and BOINC stats as well)

One of your other three CPIDs has your Numberfields credit, another has old non-BOINC or retired projects (XtremLab, NanoHive@home, HashClash, Zivis Superordenador Ciudadano and SciLINC) and there's one with some Seti, Einstein and Enigma credit.

vaughan
06-19-2016, 06:44 AM
Is it just me or does anyone else find that Bok's stats site is not as smooth and responsive as it used to be? For me, I get a delay until it updates when I go Ctrl-F5 and occasional pauses as I move between sections eg click on "CPID page"

Dirk Broer
06-19-2016, 09:08 AM
Time to re-open Bitcoin Utopia's Campaign #12, I guess... Bok should build a new server from the money that is fast, can take a lot of IO, and is fault tolerant.
Something like RAID 60 (Raid 6 + Raid 10, a RAID 60 array--also known as dual drive failure protection--is built from -at least- eight disk drives, configured as two or more RAID 6 arrays, and stripes stored data and two sets of parity data across all disk drives in both RAID 6 arrays.

NeoGen
06-23-2016, 06:05 AM
I love assembling, upgrading and maintaining my own desktops and servers, probably as much as a mechanic loves to constantly improve and upgrade his cars and add better parts to the engine, and etc.

But it comes at quite a cost depending on the size of the operation you are running... I have no idea how is bok running the stats these days but the prices of cloud services and the added convenience of not having to do maintenance of any kind are very attractive even to me these days.
I have wondered how would it be to run Amazon EC2 nodes for Distributed Computing instead of all these Frankenstein machines I have at home, and even looked up some prices for Amazon EC2, but if I read their tables correctly it would still be at a much much higher cost than me running and maintaining these noisy space heaters here so I'm not switching to Cloud Computing just yet.

All this to say (and circling back around to bok's servers again), I wonder if bok is running his stats already in a cloud service (which then wouldn't need to worry about RAID arrays and other fault tolerance issues), and if not then wouldn't it be more cost effective given the size (and global scale) of his operation?



EDIT: Oops! Duh me!! I didn't properly read earlier months posts where Nflight explains how Bok's servers are the good old kind that I like. Nevermind the first part of my question if he is running it on cloud services. :rolleyes:

NeoGen
06-23-2016, 06:27 AM
Come to think of it... if Bok were to build some specs for a really good future proof server (with AMD Zen Processor! :icon_mrgreen:) and would list them out publicly...

Wouldn't it be neat if come around Christmas (or bok's birthday, or some other date just because), that people from the whole DC community including ourselves, Free-DC, Team Ninja, The Knights Who Say Ni!, Sicituradastra (just to name a few off the top of my head), would pitch in with the parts themselves, to help him build it and take the stats to the next level? (Much like a wedding registry works.)

I mean, it's just a thought here but for the incredible stats service he provides for the whole world, and that I myself consult frequently, I would gladly throw some new SSD's or RAM sticks his way! :)

Dirk Broer
06-23-2016, 07:18 AM
I would gladly throw some new SSD's or RAM sticks his way! :)

64 GB Registered ECC DDR4 sticks even?

NeoGen
06-23-2016, 11:53 PM
64 GB Registered ECC DDR4 sticks even?

Well... maybe not the full 64Gb, at current prices that would hurt my budget quite a bit. :icon_rolleyes:
But say for example he was looking for 4x16Gb, if it didn't require to be a full set I would happily send a 1x 16Gb. (hoping others would send the remaining 3 for the set?)

The most expensive part I ever purchased (so far) is an Adaptec 81605ZQ (http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP3NJ0663) Raid Adapter... it is the heart of my current VM server (which right now is offline for hardware maintenance), and it was nearly $1000 USD.

AMDave
06-24-2016, 10:48 AM
wait a week or so for the financial markets to settle after the brexit vote.
we might be able to afford more than we could yesterday :)

vaughan
06-24-2016, 12:01 PM
...
I have wondered how would it be to run Amazon EC2 nodes for Distributed Computing...

It can be done :)

I have 1 core running "free for a year" on a Xeon with AWS. Check my SRBase computers ID:3983 its an E5-2670 v2

The cost is prohibitive to actually pay for any of these cores.

Dirk Broer
06-24-2016, 01:25 PM
wait a week or so for the financial markets to settle after the brexit vote.
we might be able to afford more than we could yesterday :)

That's giving the brexiteers too much credit

Nflight
06-24-2016, 01:39 PM
Come to think of it... if Bok were to build some specs for a really good future proof server (with AMD Zen Processor! :icon_mrgreen:) and would list them out publicly...

Wouldn't it be neat if come around Christmas (or bok's birthday, or some other date just because), that people from the whole DC community including ourselves, Free-DC, Team Ninja, The Knights Who Say Ni!, Sicituradastra (just to name a few off the top of my head), would pitch in with the parts themselves, to help him build it and take the stats to the next level? (Much like a wedding registry works.))

NeoGen => IF you develop this idea and we can wrangle the who Team area that you mentioned, this could work. It is just June and Christmas as about 6 months away. We can do this! => I can ask Bok about what the current hardware list he is currently using. Then ask for a better array list of what he would like using AMD products, and we can then go from there asking the DC community for assistance. I like the idea as I know how addicted I am to those stats too! Any comments? Please say something because I like this idea...

AMDave
06-24-2016, 02:19 PM
That's giving the brexiteers too much credit
It's not the brexiteers you need to watch. It's the investment market and the gold price. Or just the gold price really.
Our market dropped a cool 60bn off the ASX between the result and our close of market today. (http://www.smh.com.au/business/markets/brexit-complacent-markets-sink-asx-pound-20160624-gpr5vl.html)
The prediction is that Monday is likely to be a bloodbath with harsh consequences for the superannuation industry when the markets re-open.
We will see a re-balance across currencies as the markets adjust - and possibly an immediate boost in tourism to the UK. Holding any UK airline shares? No? Me neither sadly.

Back to the point. the markets will re-adjust themselves.
There's a scare in progress but I'd expect it will all go back to pretty much where it was yesterday in a little while as long as no-one really important says anything really stupid when everyone realises that the consequences are actually 1 to 2 years away.
But in the mean time there will be purchasing opportunities. :)

Dirk Broer
06-24-2016, 07:40 PM
I can ask Bok about what the current hardware list he is currently using. Then ask for a better array list of what he would like using AMD products, and we can then go from there asking the DC community for assistance.

Nothing less than the lastest AMD Zen-Opteron for Bok! Preferably in a dual-mobo, crammed to the max with DDR4 RAM and a big, fat hardwarematic RAID-controller (http://www.newegg.com/Product/Product.aspx?Item=9SIA67S34A6410)

NeoGen
06-25-2016, 06:58 PM
NeoGen => IF you develop this idea and we can wrangle the who Team area that you mentioned, this could work. It is just June and Christmas as about 6 months away. We can do this! => I can ask Bok about what the current hardware list he is currently using. Then ask for a better array list of what he would like using AMD products, and we can then go from there asking the DC community for assistance. I like the idea as I know how addicted I am to those stats too! Any comments? Please say something because I like this idea...
Cool, I like the idea too but didn't think it would gain traction. I never even worked a wedding registry site before, didn't have that for my own wedding. lol. Those are supposed to be automatic and take off the items people buy I think. But a tech website you list out all the parts and as people send them in someone managing the list (bok? or someone on his behalf?) would scratch them off the list so he doesn't get duplicates?

Something like a wish list may work? I keep my own little wish lists at NewEgg for example. That's my main spot for tech purchasing, every now and then I will also go to Amazon and others, but 90% of my shopping is done there. You can do a "Public wish list" on NewEgg, Amazon, or other websites that everyone can see just by distributing the link but it would still have to be managed manually, removing items from the list as bok gets them.

Dirk Broer
06-26-2016, 08:59 AM
Bok should -ideally- be sponsored by the likes of Tyan or SuperMicro for his Mobos, by Intel or AMD for his CPUs and by RAM and SSD/HDD manufacturers for putting their stuff to work.
And, as I have written before, Bitcoin Utopia should have a permanent campaign supporting the Stats sites.
Next best thing is crowdfunding.

Nflight
07-01-2016, 07:13 PM
Boy am I confused what are 'Squarepants' alternative Fun Stats? In the stats boxes that enclose Boks, Vaios, Gibsons, and trigggls?

Dirk Broer
07-01-2016, 10:18 PM
Boy am I confused what are 'Squarepants' alternative Fun Stats? In the stats boxes that enclose Boks, Vaios, Gibsons, and trigggls?

http://vignette3.wikia.nocookie.net/spongebob/images/4/4e/SpongeBob-Squarepants.png/revision/latest?cb=20121014113425I ain't got a clue, and I am having 1,368 of them....

AMDave
07-02-2016, 11:14 PM
Boy am I confused what are 'Squarepants' alternative Fun Stats? In the stats boxes that enclose Boks, Vaios, Gibsons, and trigggls?
Here we go. The guilty party has owned up:
http://www.free-dc.org/forum/showthread.php?53062-SquarePants&highlight=squarepants

Nflight
07-17-2016, 01:18 PM
Holy Cow the WHEELS FELL OFF. :XXarcade::eusa_wall::BangHead::bs::2gunsfiring_v1 ::5eek::cussing::qleft7::qleft3::qright5:Maybe he dropped a piston, cracked the head, timing is off, lost the condenser, it got a short, or maybe the muffler needs re-threading?

Nflight
07-22-2016, 09:04 PM
A continual discussion between myself and NeoGen has been ongoing for now some 2 weeks. I finally got around to asking Bok about enhancements to his system. I have his reply here let me post what he stated:

Bok - Did you see the email I sent from NeoGen and Myself discussing your predicament? Any thoughts on that?

His Reply is below
I did, just swamped with work. Been putting in 60-70hr weeks. like I've said before though, the main bottleneck is really the fact that the boinc stats and nonboinc stats are on the same server. I think if I seperate them to two different servers it will speed things up a lot, just can't easily do that when I have limited servers to test on. I have one other Dell T7500 which has been running the non boinc stats in parallel, but without having a second one just running the boinc stats I can;t just switch them off on the main live one. That's where another T7500 and some more SSD's would probably do the trick. Of course faster processors, faster memory and all that would speed it up a bit, but it's not worth it in my opinion. After doing this and probably while I did it there are a few changes in scripting / DB stuff that might speed it up. triggers for the milestones would probably help. They take quite some time to generate oddly enough and I've never found a better way to do them, even some other mysql experts I've talked to haven't suggested a better way.
Can't do any of that testing on the live server unfortunately.

Bok

NeoGen
07-23-2016, 02:01 AM
Aww man... I just sent the reply back to your PM before hitting "What's new" and seeing this thread. lol

Anyway, to cut a long story short, I batted around a number of crazy ideas with Clint and Phil (really... one of them was beyond crazy:icon_lol:) and have gotten to the conclusion that we need to hear what the Master himself is requesting instead of trying to create bigger, badder, crazier things.

So... I say we (not just us AMD Users but the DC community as a whole) should find a barebones Dell T7500, dual-processor equipped, and stack it with as much RAM as it can take (192GB according to specs) plus some SSDs, and have it all shipped over to Phil, and throw in some more extra RAM sticks for his other T7500 just for good measure.

There's a number of small detail questions I still have but the main idea is exactly that. What do you guys think?

Nflight
07-23-2016, 03:11 PM
NeoGen ==> Dell T7500's are old pieces of equipment. Not only are we attempting to speed up the process but, we are also looking to enhance his capabilities, less old stuff more new stuff. Do more with less attitude, all over situation not just functions completed in an old system I am talking more system access quicker with less energy required. Its not buy an old system and add a few more RAM boosters, we need to find something that takes lots less energy and allows multiple accesses of the data since more and more people are signing up to run BOINC, as you may well be aware the notice I have found is we are up to about 3.8 million users at this time.

Let's recap Dell T5700 is an old machine, lets find a replacement in the AMD FinFet future motherboard arrangement that gives Bok the extra oomph without the excessive energy draw. Let's make the improvement noticeable not just in access times but also in energy used to run these puppies.

Nflight

NeoGen
07-25-2016, 04:20 AM
I hate to be the bearer of bad news but... :-(

I have looked around for alternatives but currently AMD does not have good offers in terms of CPU performance/power ratio, or even raw performance head to head against Intel. The next real lineup upgrade cycle will be at the end of this year when the Zen micro-architecture (on 14nm) debuts. So far all that's been floating around about it have been snippets of unconfirmed information and code names like Summit Ridge (https://www.techpowerup.com/223086/amd-confirms-key-summit-ridge-specs), Raven Ridge (http://www.kitguru.net/components/cpu/anton-shilov/amd-raven-ridge-mainstream-apu-with-zen-cores-due-in-2017/), Zeppelin (http://www.fudzilla.com/news/processors/41164-amd-2017-opteron-comes-in-three-sockets), and others, with very little real information and a lot of rumors.

I believe the release dates are scheduled for Q4 2016 for desktop/mobile and 1H 2017 for server components. The next generation of CPUs will also need new motherboards with new chipsets that have not been released yet, the "Promontory" AM4 for desktops, and the SP3 and SP4r2 for the server segment.

In looking up the performance of the dual Xeon x5675 that is equipped in Phil's Dell T7500 I found that it still beats almost all current AMD configurations in synthetic benchmarks, according to CPU Benchmark.net
http://www.cpubenchmark.net/multi_cpu.html (Dual Xeon x5675 is ranked 81st with 12,955 points, only 3 AMD configurations listed produce higher scores than it)

Also the thing that gets me annoyed the most at AMD is the fact they have never supported PCI-Express 3.0 in either the desktop, mobile or server segments. Any modern RAID adapter, PCIe SSD or possibly even GPU that is thrown at an AMD motherboard will not run at full potential. And by 2017 when the new AMD Zen products come out with full PCIe 3.0 support, guess what's going to happen (http://www.kitguru.net/components/graphic-cards/anton-shilov/pci-express-4-0-with-16gts-data-rates-and-new-connector-to-be-finalized-by-2017/). :BangHead:

So... should we look for an offering from the blue team or just wait for 2017 to come around when the new lineup is available and then make up a plan?

Dirk Broer
07-25-2016, 09:13 AM
Also the thing that gets me annoyed the most at AMD is the fact they have never supported PCI-Express 3.0 in either the desktop, mobile or server segments.

Desktop:
PCIe 3.0 support was offered with the -pulled- Sabertooth 990FX GEN3/R2.0 (https://www.asus.com/Motherboards/SABERTOOTH_990FXGEN3_R20/).
PCIe 3.0 support is offered with FM2+ boards -but needs FM2+ APUs for it -if you do not use a discrete PCIe3.0 card.

Mobile:
PCIe 3.0 support is supplied with AMD laptop graphic cards (http://www.amd.com/en-us/products/graphics/notebook/8900m) of the latest generation (see specs).

Server
PCIe 3.0 support is also supplied with the Opteron A1100 (http://www.extremetech.com/extreme/221282-amds-first-arm-based-processor-the-opteron-a1100-is-finally-here), the new ARM server CPU from AMD.

CPU-World gives this comparison between the Xeon X5675 and the Opteron 6328 (http://www.cpu-world.com/Compare/726/AMD_Opteron_6300_series_6328_vs_Intel_Xeon_X5675.h tml) -roughly the same age and price.

One might argue that the CPUBenchmark is a bit skewed, showing so much Intel-based systems and hardly any AMD ones.
Could it be that the benchmark is compiled using an Intel compiler that favors Intel CPUs, giving them the best path while sending CPUs of other vendors into the worst one (http://www.agner.org/optimize/blog/read.php?i=49)?
Or are there hardly any AMD servers around -and people willing to send their benchmarks for them?

Nflight
07-25-2016, 01:52 PM
Thanks Dirk and NeoGen but, as I mentioned in my previous post And I quote myself - "lets find a replacement in the AMD FinFet future motherboard arrangement that gives Bok the extra oomph without the excessive energy draw." - Looks like the ladder part of 2016 as we may peer into the world of what will be available and when we can muster the finances to arrange this transference of his fantastic arrangement. Patience Team, Patience !! !! !!

Dirk Broer
07-25-2016, 03:59 PM
I agree with NeoGen that if Bok needs new snappy hardware NOW, he'd probably have to buy Intel. The Zen Opterons may be out of reach, both time-wise and financially.
Two big fat Xeons in a dual Socket 2011-3 mobo might do, with big fat RAID controllers to provide RAID-60 support, and loaded with 1 TB DDR4 ECC RAM
-but a bit less (and lots cheaper) might do as well: one Xeon in old Socket 2011, RAID-6 controller and loads of DDR3 ECC RAM.
But for that price you might be able to buy a dual or quad Socket G34 board with High-Efficiency (HE) 8-core 65 Watt TDP Opterons too.

NeoGen
07-26-2016, 06:06 AM
Ah... I missed the PCIe 3.0 introduction on the APU side. Thanks Dirk. I honestly stopped looking around for it a couple of years ago I think as I was so discouraged to never see motherboards supporting it. I see that it was first introduced sometime in 2014 with "Kaveri" APUs and all others after that.

That Sabertooth motherboard is the one I have currently on my desktop and I was very proud of myself when I got it, but later I read it wasn't true PCIe 3.0, but rather a bit of a hack with some extra chips to make it near PCIe 3, and that might have probably been why no other manufacturer done it, and why it was discontinued so quickly. I'm now the unfortunate owner of a discontinued motherboard. :-( (But so far it's been working ok... knock on wood)

I remember hearing about the Opteron A1100 but then nothing concrete came out to the market, it seemed like a paper launch. The only application I found on a quick search was the Lemaker Cello (http://www.lenovator.com/product/103.html), which is still on pre-order as of this day. (Says on the website it should have shipped Q2 2016)

AMDave
09-20-2016, 08:01 AM
Getting a fair few timeouts when loading the F-DC pages.
Getting a little bit concerned.

Nflight
09-20-2016, 01:06 PM
AMDave I hane had no problems here, maybe your country is not permitting this web site? The pages load quickly every time, no glitches! :blob3:

Dirk Broer
09-20-2016, 01:24 PM
They load slowly here -as usual.

Nflight
09-24-2016, 07:51 PM
Several columns and rows are not propagating correctly, its concerning since it is over 4 days of watching figures shift and waver. He probably isn't home to change what needs fixed again. :blob3:

Dirk Broer
09-26-2016, 08:13 PM
Several columns and rows are not propagating correctly, its concerning since it is over 4 days of watching figures shift and waver. He probably isn't home to change what needs fixed again. :blob3:

6 days now and counting... :(

But: Bok is on to it (http://www.free-dc.org/forum/showthread.php?33304-Stats-Issues&p=183095#post183095)

vaughan
09-27-2016, 01:13 PM
Yeah they are really bad at the moment. I wonder what hardware has failed this time?

Dirk Broer
09-27-2016, 01:37 PM
If it is a RAID array, I'd really like to have a chat with someone who claims RAID arrays are better than independent disks.
So far in my life I've seen nothing but trouble with RAID arrays -mostly because people can not/will not spend enough money to make them fault-tolerant.
And as most RAID arrays feature disks of the same age, they have the uncanny tendency to fail just when changing another, earlier failed disk...

vaughan
09-29-2016, 12:45 PM
...
And as most RAID arrays feature disks of the same age, they have the uncanny tendency to fail just when changing another, earlier failed disk...
Semi related to this, I bought 6 identical Dell Optiplex 990 PCs from a former employer about 18 months ago. They were all Intel I7-2600k CPUs with 16GB DDR3 RAM and WD Blue 1TB HDDs. I have had 4 of the 6 machines die from HDD failures in the last month. I have already got replacement drives on standby for when the remaining 2 machines fail. The odd thing is that the disk failures all coincided with re-boots after installing Windows 7 updates!

Dirk Broer
09-29-2016, 03:39 PM
The odd thing is that the disk failures all coincided with re-boots after installing Windows 7 updates!

Better install Linux, just to be sure....

Brucifer
09-29-2016, 04:33 PM
yeah.... after all the win10 cram the free upgrade down your throat shitty little games it's hard to put any trust in MS anymore regarding win7. :-( Two choices for the crunching world really, either Linux or win10-pro. At least with pro you can "defer" your updates so you can get things to a point where your work units, specially long term work units are done and uploaded so you don't lose them due to an "accident". Can't remember what the max deferral is, I think I read something somewhere like up to forty days???? At any rate I'm really sorry to hear about your multiple hard disk failures. :-(

Dirk Broer
10-02-2016, 01:25 AM
The odd thing is that the disk failures all coincided with re-boots after installing Windows 7 updates!

Already tried to re-format them?

vaughan
10-03-2016, 11:27 AM
Already tried to re-format them?

No; just replaced them.

Dirk Broer
10-03-2016, 02:50 PM
Big chance that they could be useful again, once formatted.
You could change those systems to run the OS (Windows 10 or Linux) from SSD,
while BOINC -with all its IO- runs from the HDD.

NeoGen
10-05-2016, 07:46 AM
RAID arrays have given me a great amount of "perceived safety" on all my machines ever since I started creating them, but much like the TSA "perceived safety" does not equal absolute safety.
I've had a few dumb errors happening to me over time that cost me my data several times like:

- "soft" RAID on low end motherboard... worked great until the day I tried partitioning my glorious array and split the free space to spin off a second logical drive. The RAID driver didn't handle drive partitioning correctly causing the file allocation tables to be completely corrupted. Maybe there might have been hope of data recovery with proper tools but about 15 years ago I didn't know what I know today, so... all was lost. :-(

- RAID-5 array corrupted due to system crash... again no hope for recovery :-(

- RAID arrays corrupted due to minor power outages.... :-( This actually happened to me a bunch of times until I stopped being stubborn and started purchasing UPS'es to protect my equipment a little better.

So in the end, no, RAID arrays alone are not a miracle saver, but with the proper support infrastructure around it (Power protection + reliable RAID drivers + regular backups) they will save you in the event of occasional drive failures and keep the system going with no downtime.

Dirk Broer
10-05-2016, 11:07 AM
At some point in time I did 2nd line application support next door to the 1st line hardware support. The receptionist of the building once brought a PC that did not function anymore but, he proudly said "everything is -by means of the RAID-1 array- automatically written to the second disk, they are identical, you only have to switch the active drive".
No such luck: the cheapskate had one physical disk partitioned into two logical ones, and the disk had crashed in the worst possible way, making "cathsin-cathsin-cathsin" noises when powered up. May have been "soft" RAID as well...

vaughan
10-06-2016, 09:34 AM
No need for raid on the machine as it is purely a cruncher.

Dirk Broer
10-06-2016, 12:05 PM
But splitting tasks between normal disks is also an option -OS on a SSD, downloads on a HDD, documents on another HDD, Music and video on yet another one.

Nflight
10-06-2016, 01:26 PM
Raid explained the best methodology I can find out there: http://www.thegeekstuff.com/2010/08/raid-levels-tutorial & http://www.pcmag.com/article2/0,2817,2370235,00.asp

In the numerous years I have had my system and all its memory moved from system to system, my tech has never lost anything for me. Yes I had a drive completely fail, but I had my system in a mirror setup and the other drive was fully operational and all assets were recovered. Yes to support Vaughan's statement we only crunching but some of us have other material on our systems which require retrieval if it ever fails.

Dirk Broer
10-07-2016, 12:13 AM
Much too basic!
RAID 0 (Shit, but fast. Suicide for dummies and gamers. One drive gone, everything gone. Don't try this at home. The more disks you use in RAID 0, the faster it gets -reading and writing) .
RAID 1 (Mirroring only, does not enhance performance one bit and it is a lousy way of backing up. The more disks you use in RAID 1, the slower it gets).
RAID 5 (RAID for cheap skates: it will allow one disk failure, but all-to-often a second disk fails just when replacing the first. The more disks you use in RAID 5, the better -faster in reading- it gets).
RAID 6 (RAID for less cheap skates: it will allow two disks failures, but all-to-often a third also disk fails just when replacing the first two. The more disks you use in RAID 6, the better it gets).

You can also try to combine raid levels.
RAID 10 (= RAID 1 and RAID 0 combined= Mirrored Shit).
Having two -or more- RAID 5 arrays running under RAID 0 gets you RAID 50.
But you should go, IMHO, for RAID 60 which is also known as dual drive failure protection and is built from -at least- eight disk drives configured as two -or more- RAID 6 arrays, and stripes stored data and two sets of parity data across all disk drives in both -or more- RAID 6 arrays.

Performance-wise, why not use 8 arrays of 8 disks each? Okay, that's 64 disks, but each 8-disk array may lose up to two disks before it gets critical.

vaughan
10-08-2016, 08:28 AM
RAID 60 sounds expensive. Why not use a Cloud backup company such as Backblaze, Crashplan, Spideroak or Carbonite? There are others but I haven't tried them.

Dirk Broer
10-08-2016, 09:13 AM
Backup should never be a purpose of RAID, not even when using RAID 1.
RAID should be used to ensure uptime, read- and/or write speeds.
RAID 60 can already be done with two minimal RAID 6 arrays (4 disks each) running under RAID 0.
A site as FreeDC, especially the stats pages, could benefit from a RAID 60 setup.

NeoGen
10-11-2016, 12:03 AM
Make it a RAID 60 of 1Tb SSDs with a powerful RAID adapter and you got yourself one heck of an I/O workhorse. :icon_mrgreen:

8x 1Tb standard SSDs drives (2.5'' form factor) will cost over $2000 minimum... it's really expensive but the responsiveness and read/write data bandwidth would be amazing! Something that could only be surpassed by dedicated storage cluster servers or NVMe dedicated arrays.

Dirk Broer
10-11-2016, 09:29 AM
8x 1 TB or
16x 512 GB or
32x 256 GB, why not
64x 120 GB drives?

AMDave
10-11-2016, 10:47 AM
Sounds to me like your aspirations may be greater than your needs ;)

To dig up an old quote that directly lends to an old principle that I was taught a very long time ago and I still think is fun:
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." (Computer Networks, 4th ed., p. 91., Andrew S. Tanenbaum)
https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum

Very little we do as individuals outside of the commercial environment is important enough to warrant "live" on-site and off-site mirror backup.
Some consideration around 2 metrics will illustrate this for most of us with a grasp of risk analysis and Disaster recovery: RTO and RPO
https://en.wikipedia.org/wiki/Recovery_point_objective
https://en.wikipedia.org/wiki/Recovery_time_objective

Once you figured that out you should realize that 1 extra large internal HDD inside a workstation / server + 1 large external NAS (on-site) + 1 large external NAS (offsite) handles your issue for a fraction of the price.
The large HDD keeps the immediate backups available.
The 2 externals NASs are rotated to keep a regularly updated copy of the internal HDD; frequently refreshed, portable and in a risk-reduced location
(ie the on-site NAS is just as screwed as the internal HDD if your house or office falls into a sink hole or gets burned down by a neighbor kid's charming little experiment with matches or flattened by the cyclone tearing across the coastline)

If you want fast AND reliable then split the 2 functions.
RAID-0 SSD for primary disk is awesome. No argument.
Local HDD(s) for backup is very useful esp for RTO and RPO. No argument.
It is easily enhanced by on and offsite external NAS backups in rotation for assurance.

A local NAS serves as a part of the grab-and-go kit (along with a charged laptop) in case of emergency evacuation.
A remote NAS serves as a recovery point if the local site is destroyed.
In the mean time the local backup HDD(s) give you as much recovery as you can fit (add more disks as required)
Occasionally swap the local and off-site NAS copies to keep the RPO low.

A lot of NAS devices have RAID options and mirror should be used for protection when it is available.
When a 1TB or 3TB disk fails be prepared for a very long image catch-up when you plug in the replacement.
That process can take a day or 2 going on current experience.
But remember that is happening on your backup set not in your live environment - so minimal interruption unless it happens while you are trying to recover in the case of a real DR.

IMHO Easy. Fast. Cheap. Reliable. Private. Secure. No 3rd party reliance, contracts, access restrictions, costs.

As far as the myriad RAID configurations available - "Just because you can doesn't mean you should" ("Lest We Forget", Lt. Col. Carlos A. Keasler)

re: SSD mirror RAID Given the design limitations of SSDs, putting a new SSD into a mirrored SSD RAID configuration to replace another that has failed will shorten the life of the new SSD through the huge number of writes that will be processed to catch it up
(although it is still faster than catching up a HDD you are compromising speed for a false sense of reliability)

re: NVMe is fast. NVMe is coming down in price, but currently still at approx. 4 x $/GB compared to SSD.
We will probably see it half and maybe even match price in the next year.
But the rational outlined above still stands whether NVMe or SSD is considered.
ITMT still in the server-level realm unless you have money to burn - in which case, could you buy me one too? for testing of course. I'm sure I'll get it back to you oh sometime ;)
Like SSDs - still best used for high read rates and very low write rates.

I'd hate to miss this opportunity to remind everyone:
Your last backup is only as good as your last restore.
An untested backup is NOT a reliable backup.
Having observed organisations large and small and small business owners and individuals suffer from failed restores from backups, I have always tested my backups right after I make them.
Some of the early ones did not work well and highlighted issues and false-faith in common backup software and led me to improve my own backup and restore procedures to the point where I have not had a restore fail for many years now.
Please do not waste any time doing backups that you don't know will work when you suddenly really need them. That's just pointless.
When you blast the summer dust bunnies out of your PC this Autumn, do yourself a favor and make a fresh backup set and store it in a dry box at a different address.
You cannot go back in time and re-take all those family photos, re-scan the tax invoices you threw out last year. (well not yet anyway. maybe it will happen in our lifetime. maybe not. how confident are you?)

ed - coincidentally, our newegg partner is running a clearance on storage devices. clik on the newegg promo logo at the top of the forum. or > here (http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100006670%204810&IsNodeId=1&name=Hard%20Drives&SpeTabStoreType=98) <

Dirk Broer
10-11-2016, 01:06 PM
I was purely thinking in FreeDC's problems. Bok needs:

uptime
fast reads
fast writes
to be able to change a failed disk without having to take the site down

He would be served by a 64-disk RAID 60 solution, IMHO.

AMDave
10-11-2016, 01:19 PM
Sure. As long as you are talking about HDDs.

Mirror RAID for consumer SSDs is also a really over-confident idea and can result in complete loss when they start failing together.
eg: It looks like I'll soon be re-building a $16K SSD raid array on a business machine. Fortunately I saw an indicator their SSD mirror RAID was going bad - so I installed a 3TB internal disk and built in a backup process, just like I outlined above. I am anticipating total failure of the RAID volume sooner than we can replace all of the drives in the array.

SSD mirrror RAID is unreliable over the long term.
If one goes bad you need to be replacing the lot of them.
In which case, why mirror raid SSDs?? Experience shows it to be a waste of time and money relative to the write-rate.
You want them for speed, so RAID-0 & do some backups to another more reliable media.

For Bok, there is an additional couple of lines of script that could resolve the current symptom & allow SSD replacement to occur without downtime in the current architecture by detecting the issue and pausing the web-side DB flip-flop.
It's something I thought of since we discussed the implementation of the current architecture, but thought he would never need.
Little did we know just how crap these consumer SSDs would actually turn out to be for this kind of usage.

Now if there was tons of RAM available ... maybe copy it to RAM disk and back that up to HDD and skip the SSD altogether.

Dirk Broer
10-11-2016, 03:49 PM
We could try to open a new instance of Bitcoin Utopia's campaign #12, for 32 GB registered ECC (no doubt) RAM modules in Bok's box (if his box takes such biggies).

vaughan
03-17-2017, 01:22 PM
Is bok's stats at Free-DC down again? I am unable to reach it.

Nflight
03-17-2017, 08:52 PM
There was a glitch earlier today where the stats were not updating but, that is past and everything looks good here! It is now 4:50 PM EST and everything is working fine except my green beer is empty already damn Leprechauns... :blob3:

Dirk Broer
03-21-2017, 08:32 PM
There was a glitch earlier today where the stats were not updating but, that is past and everything looks good here! It is now 4:50 PM EST and everything is working fine except my green beer is empty already damn Leprechauns... :blob3:

It's glitchy time again...stats at least work again, but signatures are off.

Dirk Broer
06-27-2017, 11:41 AM
Bok is down at the moment, hope it is short like yesterday..

NeoGen
06-28-2017, 03:39 AM
The stats are showing up for me right now. Might have been a short glitch indeed.

Dirk Broer
06-28-2017, 09:44 AM
Short? more than 12 hours..

vaughan
07-29-2017, 06:38 AM
My computer says bok's stats are down. Can anyone confirm?

AMDave
07-29-2017, 07:49 AM
Yep. Confirmed.

NeoGen
07-29-2017, 08:36 AM
Confirmed by me too. Seems most of the page loads, but it hangs on "Loading Stats data"

Nflight
08-10-2017, 07:20 PM
Boks' stats are down at this time, August 10, 2017 3:20 PM EST

Dirk Broer
08-11-2017, 12:51 AM
Boks' stats are down at this time, August 10, 2017 3:20 PM EST

August 11, 2017 02:50 AM CET too, earlier free-dc could still be reached but stats didn't show. I managed to post on their forum about it an hour or so ago.

vaughan
08-18-2017, 02:00 PM
bok's stats are down for me.

Dirk Broer
08-18-2017, 04:27 PM
The complete FreeDC domain is unreachable, maintenance I hope...

Nflight
08-18-2017, 05:24 PM
Last night late Bok had to fly to Dallas Texas, explained it was unexpected for his work. Which means he will not be available till he returns to fix this situation. :-(

vaughan
08-19-2017, 01:40 AM
Oh OK thanks for update Nfight.

I've moved my GPUs from Primegrid over to Seti to help our Seti Wow effort.

Soon my CPUs need to swap from Sophie Germain to Generalized Cullen-Woodall LLR'ing. Its the next competition over at Primegrid for the 3 day Solar Eclipse Challenge starting August 20 at 18:00 UTC.

Dirk Broer
09-02-2017, 01:48 AM
Bok's FreeDC is unreachable ATM, I go to sleep, almost 04:00 now...

Nflight
09-02-2017, 04:00 AM
NOw Midnite at my house, Bok's Stats are still not available :sad5:

Dirk Broer
09-02-2017, 01:46 PM
There I am, (sub)clanking like a madman, but nothing to show for it..

Dirk Broer
09-06-2017, 09:16 AM
Bok's FreeDC is unreachable ATM, I go to sleep, almost 04:00 now...

Just woke up to find FreeDC down again, hopefully just to implement changes/replace hardware...

vaughan
09-06-2017, 10:54 AM
Aw shucks we broke the stats again :)

Dirk Broer
09-24-2017, 02:59 PM
This morning at first no sub-project scores, then later no stats at all and now even no free-dc at all...
I do hope it is maintenance.

Ototero
12-16-2017, 10:26 AM
I think the NPLB stats have gone berserk. Suddenly I've found no primes, and MiHost is top ranked.
Hope it gets sorted soon.

Edit, I see that AMDave is the admin.

AMDave
12-18-2017, 08:18 PM
Hi Ototero.
Apologies for the NPLB blip. Problem with the server motherboard giving up the ghost.
It took some repair from back up once we got it running again as a couple of tables were corrupted.
Should all be ok now.
If you spot anything wrong there please report it in the NPLB "Server outrages" thread:
http://www.mersenneforum.org/showthread.php?t=13840&page=48

Ototero
12-18-2017, 11:00 PM
Thanks Dave,
I spotted the stats looking good today (GMT).

vaughan
12-19-2017, 09:39 PM
Still running NPLB hey Ototero? I haven't run that for a long time for a couple of reasons:
1. Really confusing trying to find out what application to install and then what tasks to crunch. Whole site needs a revamp.
2. No BOINC cobblestones.

Ototero
12-20-2017, 11:08 PM
Hi Vaughan, I've been running this for about 5 years non stop. Once I had the instructions, it was simple.
And.......I don't do boinc.
There aren't many non boinc apps out there.
I had RC5, OGR and 8 NPLB running on my Ryzan 1600.
Really sweet.

Nflight
02-19-2018, 03:06 AM
Looks like Free-DC is down again, my addiction is waning. I am going to jump soon! :blob3:

vaughan
02-19-2018, 04:03 AM
Confirmed ... IsUp.Me reports: It's not just you! stats.free-dc.org looks down from here.

Nflight
02-22-2018, 01:38 AM
Twice in one week, oh my :blob3:

vaughan
02-22-2018, 03:27 AM
Happening right now.

vaughan
05-02-2018, 05:36 AM
Is stats.free-dc.org down?

It's not just you! stats.free-dc.org looks down from here

vaughan
07-23-2018, 10:32 PM
bok's stats appear to be down for me today

NeoGen
07-23-2018, 11:49 PM
Yup, down here for me too. :(

Dirk Broer
07-25-2018, 01:25 AM
Bok has added DHEP to his stats

NeoGen
07-30-2018, 03:01 AM
Nice, he added the BOINC ones no less. That's pretty cool. And the site looks different from what I last remember, there must have been a software update since the last time I was on it.

AMDave
08-01-2018, 11:02 AM
Prior development work, mostly done by Bok and Gopher, which was waiting for time to apply finishing touches.
It has been available via the dev-URLs for quite some time and has been my default view on and off.
Kudos to Bok for the upgrades that I can see since the last version I participated in.
It is very satisfying to see it go main-stream at long last.

:icon_thumright:

vaughan
08-02-2018, 08:05 AM
Great work by all. We, the crunchers, appreciate the huge effort it takes to create and maintain the stats pages.

Dirk Broer
08-11-2018, 11:10 PM
I hope it is another upgrade to Bok's site that's preventing me from checking my stats...

vaughan
08-12-2018, 02:25 AM
Drat and its mid race at Primegrid too.

NeoGen
08-12-2018, 10:03 PM
Still down for me too :(

vaughan
08-31-2018, 12:12 PM
Looks like bok's stats are down.

Nflight
08-31-2018, 04:29 PM
Yes they are, he is my neck of the woods this weekend, he will be back I think Sunday night, sorry... :blob3:

Dirk Broer
10-20-2018, 12:21 PM
I can't reach Bok's stats today

vaughan
10-20-2018, 01:05 PM
They're down for me today too.

vaughan
10-21-2018, 12:13 AM
On the subject of stats, why do our statistics pages show kaspersky-labs during what seems to be a long delay before displaying the stats pages?

Dirk Broer
10-21-2018, 08:23 AM
I only see a black page where you get Kaspersky, then stats load.
Bok is still out, must be a severe crash...

vaughan
10-21-2018, 10:16 PM
I'm suffering from bok's stats withdrawal symptoms.

Dirk Broer
10-22-2018, 12:32 AM
I'm suffering from bok's stats withdrawal symptoms.

Me too, I just don't know where to check my subproject stats...

Dirk Broer
10-22-2018, 12:00 PM
Yes they are, he is my neck of the woods this weekend, he will be back I think Sunday night, sorry... :blob3:

Can you check your neck of the woods to see if Bok is around? His site is still down (the complete FreeDC, not just the stats).

Nflight
10-22-2018, 02:13 PM
It could be Wednesday till our Stats withdrawal is refreshed and renewed. God, I hate not having my Stats Fetish around,

Dirk Broer
10-22-2018, 03:09 PM
It could be Wednesday till our Stats withdrawal is refreshed and renewed. God, I hate not having my Stats Fetish around,

Sounds like a failure of a complete RAID array

Nflight
10-22-2018, 06:26 PM
Bok is in Minneapolis MN today, returning via airplane to his home state so by 6 or 6:30 PM EST tonight the whole system should be up and running maybe, really a maybe if he has not got a quick fix.

Dirk Broer
11-03-2018, 11:05 AM
Bok is in Minneapolis MN today, returning via airplane to his home state so by 6 or 6:30 PM EST tonight the whole system should be up and running maybe, really a maybe if he has not got a quick fix.

Is he anywhere near his servers today? He's needed....

vaughan
11-04-2018, 02:26 AM
bok's stats seem to be having more outages this last few weeks. They're generally very reliable. I wonder if it is a hardware issue?

vaughan
11-09-2018, 09:22 PM
bok's stats have gone off-line again

Dirk Broer
11-09-2018, 10:54 PM
bok's stats have gone off-line again

Every time it happens -the last four times at least- trouble starts friday night (UTC) it seems...
When was Michael of PrimeGrid experiencing DDOS attacks from the GridCoin clients? Just a hunch...

vaughan
11-13-2018, 05:35 AM
And the site is down again.

Dirk Broer
11-15-2018, 06:21 PM
LHC@Home Dev and Albert@Home missing from Bok's stats at the moment

Edit: all is restored now, but what will the future bring when a really project stops?

vaughan
11-17-2018, 11:37 PM
Drat the stat's down again.

NeoGen
11-18-2018, 10:27 PM
Every time it happens -the last four times at least- trouble starts friday night (UTC) it seems...
When was Michael of PrimeGrid experiencing DDOS attacks from the GridCoin clients? Just a hunch...
I could see a correlation indeed... unlike businesses that work monday-friday 8am-5pm, distributed computing is a hobby, which means it would have it's higher peaks after business hours and on weekends, as people are not usually checking their stats and doing things on boinc during the week while at work.
So the after hours and weekends are probably when all DC and related websites get their highest traffic and amount of requests, which means the stress factor for the hardware components increases greatly and any possible bad hardware will be at a risk of failure.

About the DDOS attacks I don't know, I haven't read about that one, but I'd conjecture that hackers also have day jobs so they mostly attack after hours and weekends too :icon_lol:

Dirk Broer
11-19-2018, 01:25 PM
The DDOS attacks on PrimeGrid were carried out by GridCoin client software, scraping stats from the PrimeGrid website. They/it can likewise be active at FreeDC -if these are bots the developers of the GridCoin client are to blaim of course.

Dirk Broer
11-29-2018, 06:43 PM
FreeDC is having troubles today

vaughan
11-29-2018, 09:15 PM
It is Friday here so I guess it must be the day that bok's stats go AWOL.

vaughan
01-25-2019, 05:16 AM
bok's stats are down for me today.

Dirk Broer
01-25-2019, 08:06 AM
bok's stats are down for me today.

Must have been temporary, they're up for me here

Dirk Broer
09-01-2019, 08:06 PM
Server was hit by a power surge this morning. Bok has been trying to repair the databases all day. Only one table on one of the databases to go, but it's giving him issues. He had to bring the site down to stop the connections coming in which are holding things.

NeoGen
09-02-2019, 01:39 AM
Ouch... power surges and outages suck for database servers, more so than for regular computers. I hope he can restore the table data without any issues.

Dirk Broer
09-02-2019, 07:13 AM
Looks like a complete recovery without issues, from here..

Dirk Broer
03-04-2020, 01:50 AM
FreeDC is having troubles today

Just as now, tonight. I go to bed, it is 03:50 here.

vaughan
03-04-2020, 04:34 AM
Yes it is still unreachable as at 4:30pm Australian Eastern Daylight Savings Time (UTC +11 = 5:30am March 4th)

Dirk Broer
08-15-2020, 08:34 AM
Looks like a new crash at Bok's side of things....

vaughan
08-15-2020, 12:20 PM
Yes unreachable for me tonight (Saturday)

Dirk Broer
08-15-2020, 02:27 PM
FreeDC is up again!

vaughan
08-16-2020, 02:47 AM
Phew! Glad they're back. I find them useful to keep progress on my WCG stats. In about 30,000 years I'll catch Jason :)

Jason1478963
08-25-2020, 02:32 AM
be prepared for the launch of GPU work at WCG if you want to catch me. :)

vaughan
08-25-2020, 11:50 AM
be prepared for the launch of GPU work at WCG if you want to catch me. :)

When is this starting?

Dirk Broer
08-25-2020, 01:25 PM
be prepared for the launch of GPU work at WCG if you want to catch me. :)

What GPU(s), which OS?

Jason1478963
09-09-2020, 12:27 AM
I thought we would have seen a beta already for GPU at wcg. I may have missed it, but it has been a while since WCG has setup a GPU project. Lets hope they can get it to work with this research.

vaughan
03-18-2021, 07:41 AM
Looks as if bok's stats at free-dc are down at the moment.

Dirk Broer
03-29-2021, 10:34 AM
Looks as if bok's stats at free-dc are down at the moment.

Today again, I do hope it is the ISP that's having troubles, not BOK's hardware.

Dirk Broer
09-06-2021, 11:03 PM
Bok on FreeDC
"Appreciate the few donations in the last couple of days. A lot of change going on right now. My job is in bit of flux and my health is not the best (though getting better slowly with some lifestyle changes). You guys have certainly helped in the past but that main last time dried up over a year ago. I'm going to continue paying for this hobby though end of year but without some major help or changes before that I'll be winding down."

As soon as I win the lottery Bok gets a dual-socket EPYC server on my cost, I pledge herewith.

vaughan
09-07-2021, 01:50 AM
I sent a USD300 donation to bok to assist. Just need another 25 DC'ers to match this and his electricity bill is covered for a year.

Dirk Broer
09-10-2021, 11:04 PM
Bok on FreeDC
As soon as I win the lottery Bok gets a dual-socket EPYC server on my cost, I pledge herewith.

An all-time low at the lottery: zilch
Four more lotteries to go till the end of the year...

vaughan
09-13-2021, 07:40 AM
I'm unable to reach stats3 just now.

Dirk Broer
09-14-2021, 12:37 AM
Main server url (without '3' or '6') has no problems

Dirk Broer
12-06-2021, 03:49 PM
Looks as if bok's stats at free-dc are down at the moment.

Looks like it again

vaughan
12-08-2021, 05:49 AM
Looks down for me.

Dirk Broer
12-08-2021, 09:40 AM
I get a 'ERR_CONNECTION_REFUSED' at first, but it is now 'ERR_CONNECTION_TIMED_OUT' again.

vaughan
12-20-2021, 09:05 PM
stats3.free-dc.org seems to be down

Dirk Broer
12-20-2021, 11:21 PM
Firewall problems: https://www.facebook.com/groups/AMDUsersTeam

vaughan
01-27-2022, 10:28 PM
bok's stats are down for me today.

Dirk Broer
01-28-2022, 12:47 AM
bok's stats are down for me today.

They should be up again, there was some weird Apache error: Anyone able to reach FreeDC? (https://www.facebook.com/groups/AMDUsersTeam)

Dirk Broer
08-04-2022, 11:20 AM
FreeDC seems down at the moment

Dirk Broer
10-18-2023, 01:39 AM
Can't reach Bok's stats site, nor any other FreeDC url, ATM.

vaughan
10-18-2023, 12:12 PM
They work for me.

Dirk Broer
12-18-2023, 08:09 AM
Can't reach Bok's stats site, nor any other FreeDC url, ATM.
Since last night, around 24:00 UTC -or even earlier

Dirk Broer
04-16-2024, 10:17 AM
Since last night, around 24:00 UTC -or even earlier

Today again, since last night, around 24:00 UTC -or even earlier

Dirk Broer
04-17-2024, 01:33 AM
Today again, since last night, around 24:00 UTC -or even earlier

FreeDC is up again, only to see BOINCStats going down today