Building a 64 thread cruncher

[Dirk Broer]

Is this ever gonna stop? A 64 thread cruncher! Is this possible, other than with the latest hardware?

No, it can still be done with Socket G34


Buy a quad Socket G34 board and four 16-core Opteron 6262 HE "Interlagos" of 85 Watt (= 340 Watt to dispose of)
Buy a quad Socket G34 board and four 16-core Opteron 6366 HE "Abu Dhabi" of 85 Watt

There are more possible 16-core Opterons, but they have an even higher TDP. Compare that with the new platform and see why I do my recommendation.

Socket SP3


Buy any SP3 board and a EPYC 7551P of 180 Watt TDP. [recommended]
Buy any SP3 dual board and two EPYC 7281 of 170 Watt TDP each.
Buy any SP3 dual board and two EPYC 7301 of 170 Watt TDP each.
Buy any SP3 dual board and two EPYC 7351 of 170 Watt TDP each.


The dual SP3 boards can of course also be used to build a 96 or 128 thread cruncher....right at this very moment that is.
When Starship comes out the 96 thread cruncher can already be build on a single socket SP3 board...

[Nflight]

I want one or maybe two !

[Dirk Broer]

I want one or maybe two !

Perhaps worth noting is that Windows can't handle this many threads well. On the Scottish Boinc Team forum I came across a topic discussing the limits of Windows when building many-thread crunchers, where some 'heavies' from Seti.USA reported their findings. They've made a '640k' barrier again...Now it is the implementation of NUMA that is at fault.


There are 64 NUMA memory pipes available. If you run 64 threads or less then each thread will run at 100% load. If you enable 88 threads then Windows will start "sharing" the memory pipes. So the 24 threads (above the 64) would start sharing the memory pipes with 24 of the 1st 64 and therefore would only be running at 50% efficiency because 1/2 the time they are running and 1/2 the time they are waiting for memory.

You would wind up with 40 threads running at full load and 48 threads running at 50% load ... ie 64 threads. Since the memory channels would have to be continuously loaded/unloaded it would actually be less efficient than just running 64 threads.

So to overcome this limitation you would set your program up to launch 44 threads at Node 0 (CPU0) and then 44 threads to Node 1 (CPU1). Then all 88 threads would be running at 100% loading.

We proved the technique works as I had describe in my post to Pete. We launched one BOINC client to node 0 and a 2nd BOINC client to node 1. On our 72 thread machines all threads were running at 100% load.

Note that Linux is not affected by this limit.

[Nflight]

I am looking at COmbo's by NewEgg (https://www.newegg.com/Product/ComboBundleDetails.aspx?ItemList=Combo.3616294) = But really the idea comes from reviewing all the AMD combo's (https://promotions.newegg.com/nepro/16-0744/index.html) they have put together!

[Dirk Broer]

I am looking at COmbo's by NewEgg (https://www.newegg.com/Product/ComboBundleDetails.aspx?ItemList=Combo.3616294) = But really the idea comes from reviewing all the AMD combo's (https://promotions.newegg.com/nepro/16-0744/index.html) they have put together!

Was that with the cynicism mode on? It is all Intel they're throwing at me...

[Nflight]

Actually no, They were all leads to AMD processors, really Dirk it must be just you :icon_razz: It is a link to = Combo: 3616294

[Dirk Broer]

Actually no, They were all leads to AMD processors, really Dirk it must be just you :icon_razz: It is a link to = Combo: 3616294

? Clicking 'All the AMD combo's' I get on a page https://promotions.newegg.com/nepro/16-0744/index.html with a AMD motherboard configurator that won't work (https://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100006519%2050136046%2040000022&IsNodeId=1&SubCategory=22&Manufactory=136046&SpeTabStoreType=1), here's why (https://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100006654%2050136046&IsNodeId=1&Manufactory=136046&name=Motherboards&SpeTabStoreType=1): no AMD.

The AMD cpu configurator will let me choose between two out-of-stock FX CPUs (https://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100006519%2050136046%2040000343&IsNodeId=1&Manufactory=136046&page=1&bop=And&SpeTabStoreType=1&SrchInDesc=amd&Page=1) and by now also an A6-7600K FM2+ APU

How you have managed to get the combo you selected is beyond me. A CPU you can't select, combined with a mobo that's not there...
I get it: you went to the preconfigured combo's. I hate preconfigured, I build myself.

[Dirk Broer]

A new option is available: Buy a 32-core/64-thread Threadripper 2990WX, a TR4 mobo, a M.2 SSD and an unholy lot of DDR4 RAM (to support 64 threads in BOINC you need at least 128GB in my opinion, and I'd rather have 256GB), and you're in big business!
The Threadripper costs around $1,500; the Mobo a little over $300; the M.2 SSD Samsung 970 Evo 500GB is around $160; eight 16GB sticks for a total of 128GB sets you back around $1,500 again, so for a little under $3,500 you'll be able to blow all your previous CPU records. I do trust you had a Radeon or GeForce video card from a previous system, to keep things cheap...

[Brucifer]

Lol, to keep things cheap huh? :-) Well one thing that won't be cheap is the monthly power bill... :-( But then maybe Vaughan would go for it???? The computer that is, not someone elses power bill. hahaha

Of course I guess it would depend on how you load it up with work.

[Dirk Broer]

Lol, to keep things cheap huh? :-) Well one thing that won't be cheap is the monthly power bill... :-( But then maybe Vaughan would go for it???? The computer that is, not someone elses power bill. hahaha

Of course I guess it would depend on how you load it up with work.

Power bill? Depends. I could replace my whole farm with one 32-core/64-thread behemoth, using just one PSU, One SSD, one GPU and one set of RAM and come out better -in terms of active threads- than my present setup.

But when economics come into play you'd be better off buying a dual EPYC board, and hope that a lower Wattage TDP EPYC2 indeed has 64-cores/128-threads per CPU.

[vaughan]

AMD Ryzen Threadripper 2990WX (Base: 3.00GHz, Boost: 4.20GHz / 80MB Cache / TR4 / 32 Core / 250 Watt / Without Heatsink/Fan) is showing as Pre-Order status at one of my suppliers for the princely sum of AUD2,679. Ouch!

I think I'll wait on that.

I got a quote for an EPYC as described by Dirk but it was $7,500 so that was really a big ouch too.

[Brucifer]

ooooh. wow! :icon_eek:

[Nflight]

Lots of time to wait here too

[Dirk Broer]

AMD Ryzen Threadripper 2990WX (Base: 3.00GHz, Boost: 4.20GHz / 80MB Cache / TR4 / 32 Core / 250 Watt / Without Heatsink/Fan) is showing as Pre-Order status at one of my suppliers for the princely sum of AUD2,679. Ouch!

I think I'll wait on that.

I got a quote for an EPYC as described by Dirk but it was $7,500 so that was really a big ouch too.

The most expensive 32-core EPYC is 4,600 Euro's here (AMD Epyc 7601, Tray version. Boxed it is 4,400 Euro's)

Hardware prices can go down south very quickly: the 16-Core/32-Thread Threadripper 1950X 'only' costs 580 Euro's here now...

[plonk420]

don't forget a line conditioner and ethernet protection, too! (or maybe just wifi) just lost 2 switches and maybe some ports on an AP, but thankfully no computers (tho a NIC on our pfSense may have gotten zapped) in a thunderstorm. and our house wasn't even hit (just the one beside us).

oh, also an SSD seemed to get impressively corrupted, too

[NeoGen]

AMD Ryzen Threadripper 2990WX (Base: 3.00GHz, Boost: 4.20GHz / 80MB Cache / TR4 / 32 Core / 250 Watt / Without Heatsink/Fan) is showing as Pre-Order status at one of my suppliers for the princely sum of AUD2,679. Ouch!

I think I'll wait on that.

I got a quote for an EPYC as described by Dirk but it was $7,500 so that was really a big ouch too.
I want to see when one of those 2990WX gets put on water cooling and overclocked, how far do you think 32 cores can be pushed? :icon_mrgreen:

EDIT: Just googled out of curiosity and the Threadripper does indeed put out more heat than some small personal space heaters :icon_lol:
https://soleilheaters.com/ceramic-heaters/personal-ceramic-heater-mh-01

[plonk420]

Perhaps worth noting is that Windows can't handle this many threads well. On the Scottish Boinc Team forum I came across a topic discussing the limits of Windows when building many-thread crunchers, where some 'heavies' from Seti.USA reported their findings. They've made a '640k' barrier again...Now it is the implementation of NUMA that is at fault.

Note that Linux is not affected by this limit.

yeah, here's some benches https://www.phoronix.com/scan.php?page=article&item=2990wx-linux-windows&num=1


I want to see when one of those 2990WX gets put on water cooling and overclocked, how far do you think 32 cores can be pushed? :icon_mrgreen:

EDIT: Just googled out of curiosity and the Threadripper does indeed put out more heat than some small personal space heaters :icon_lol:
https://soleilheaters.com/ceramic-heaters/personal-ceramic-heater-mh-01

here's some LTT silliness, but i think the GPU was 250 watt TDP, also


https://www.youtube.com/watch?v=1WLIm4XLPAE

[NeoGen]

I want to see when one of those 2990WX gets put on water cooling and overclocked, how far do you think 32 cores can be pushed? :icon_mrgreen:
Of course somebody would do it... and with Liquid Nitrogen no less! :icon_mrgreen:
https://www.pcgamer.com/overclocker-cranks-all-32-cores-of-amds-threadripper-2990wx-to-nearly-6ghz/

32 cores x 5.955GHz each = ~190GHz combined computing power in 1 CPU! :shocked:

[Dirk Broer]

yeah, here's some benches https://www.phoronix.com/scan.php?page=article&item=2990wx-linux-windows&num=1
here's some LTT silliness, but i think the GPU was 250 watt TDP, also

https://www.youtube.com/watch?v=1WLIm4XLPAE&t=10s

A bit more class, Linus
https://www.techpowerup.com/img/17-03-29/ea87fb213d8c.jpg