Building a 24 thread cruncher

[Dirk Broer]

As every Tom, Dick and Harry can build his/her 16 thread cruncher nowadays by simply buying an AM4 mobo and a Ryzen 7, let's try for the next level:
The 24 thread cruncher.

Goodbye to Socket 940, as there are no mobo's with 12 sockets for ye olde Socket 940.

Socket F:


Buy a quad Socket F mobo and four six-core Opteron 8400 EE- or HE-series "Istanbul" of 60-79 Watt TDP.


Socket G34:


Buy a dual Socket G34 mobo and two twelve-core Opteron 6100 HE-series "Magny-Cours" of 85 watt TDP.
Buy a dual Socket G34 mobo and two twelve-core Opteron 6230 HE-series "Interlagos" of 85 watt TDP.
Buy a dual Socket G34 mobo and two twelve-core Opteron 6300-series "Abu Dhabi" of 99-115Watt TDP.

I've read reports of SETI.USA members that prefer the old K10 core over the Bulldozer for performance reasons, so if you want to give Socket G34 a try, go for the Opteron 6100 HE-series.

No Socket AM4 here -yet. This might change with Ryzen v2 or Ryzen v3, as Starship is rumoured to have 48 cores.

Socket TR4


Take any TR4 mobo and buy a 12-core Ryzen Threadripper 1920X. [recommended]

Advantages of Threadripper against the Bulldozer-based Dual Opterons is the amount of PCIe lanes and the fact that a single CPU system requires less space. Disadvantage is the fact that the quad-channel RAM dictates that you have to buy the RAM in quantities of at least four sticks at a time. This option also gives you the opportunity to make your cruncher a 32-thread cruncher in time though.

No Socket SP3 either, there are no 12-core EPYCs planned yet.

[Dirk Broer]

Going the Threadripper road here costs you -at least- a mobo, the $339.99 Asrock X399 Taichi as most economical choice gets my recommendation here.
A few years ago I never thought I would ever recommend Asrock, but they have some rock-solid boards nowadays, and never shrink away from being the only supplier in a given niece (e.g. Socket 478 with DDR3)
You need a Threadripper too, and that 12 core is gonna cost you some $799.99
I'd buy -at least- 96GB of ram in a 24-thread sytem. A 128 GB G.Skill quad channel set sets you back some $1,170.99 though...and that's the cheapest option at Newegg.

$339.99 + $799.99 + $1,170.99 is some hefty $2,311.....

What does socket G34 offer?

Don't ask me why, but Newegg has a hard to beat offer: an OEM Dell DW8Y5 Dell PowerEdge C6145 Server Quad Socket G34 for $113.90 (https://www.newegg.com/Product/Product.aspx?Item=9SIAA7W65K1281). No need to fill them all at the same time. I'd say...
Two Opteron 6338Ps for this mobo will cost you $392.37 each at Newegg and two sets of 48GB of Crucial Registered ECC DDR3 RAM will cost you $489.00 each,
-> grand total $1877,00 for the G34 that has only half its sockets filled. Not too bad. Hard to make a choice here, especially when you already have some G34 hardware lying around.

[Jason1478963]

I was able to get my g34 server blades for about 150 each used with 2 6174's and 32gb of ram :P

[Dirk Broer]

I'd buy more RAM, if I were you. Pity you can't carry over the RAM of Socket F unto Socket G34...

[SuicideCabbage]

Don't forget about server pulls, I have been playing with the idea of a 48-core for a month or so, probably going to pull the trigger before years end. I have seen single CPUs under $7 and sets of four slightly faster ones for $30-40. My current plan, though always subject to change, is 48-core in a quad G34 board, speed somewhere in the 2.1-2.4Ghz range. Used board and CPUs ~$100-150 haven't priced out RAM yet as I am fearful to do so, but server pulls may be dirt cheap as well.

I have always been an avid supporter of used GPUs, thanks to our friendly neighbourhood miners and "1337 gamers" dumping top end cards dirt cheap when the next gen comes out. Though just recently I stepped into the realm of used storage, picked up a set of four 500GB VelociRaptors for my cold-storage server at less than the price of similar sized "normal" drives used or otherwise. The raptors were $13 each shipped, zero SMART errors as advertised, and a destructive run of badblocks tossed 0/0/0 after four passes. This is what inspired me to turn all my builds to used when possible, will still be getting a top-end threadripper, but for non-flagship rigs I will be going used until it bites me.

As far as GPU, if you run Milkyway you almost have to go used these days, all the R series can't hold a candle to the 6000-8000's double-precision. Current king of the hill is the Radeon RX Vega 64 Liquid with an IMPRESSIVE 13.7 TFLOP single, but a sad 859 GFLOP double and costs around $700. HD 7970 puts out 4 TFLOP single 1 TFLOP double, and runs you $150 currently. That's top of the line R serious though, have a look through older R lines and lower ends of all lines, pathetic output. For example an RX 470 will run you about the same as a 7970, but 3.7 TFLOP single, 239 GFLOP double. R9 390 5 TFLOP single, 640 GFLOP double, and runs about $200.

[Jason1478963]

The ram seems excessive already with Linux and the projects I run with it.

[Jason1478963]

I like the idea of the used GPU as its nice to find a used with a nice water blocks. :)

[Dirk Broer]

The ram seems excessive already with Linux and the projects I run with it.

I run several projects that sometimes needs as much as 3GB for a WU.
A 24 thread cruncher runs 24 WUs at a time, so 96GB of RAM should be safe/enough.
If you have several Virtual Box WUs running -they eat RAM like it is nothing-, 32GB will not be enough.

Did you never encounter a 'waiting for memory' message in the BOINC Manager?

[Jason1478963]

I'll keep it in mind but I haven't had any issues at WCG lately with 1GB per core. This system is mostly run in the winter months and a few races here and there do the high number of BTU(around 9000)it puts out when running all the blades. If I remember right the memory banks are full and should be matches serer memory making that a bit more painful. I think i did slightly better on the Ryzen builds but I may have fallen short of the 3GB per core. I think it may be in my best interest to have a mix of projects and app_config files when it comes to memory hog projects. My gut feeling is most computers aren't that loaded with ram and reduce the computers that can work on projects like that. If i recall WCG tries to keep the apps as computer friendly as possible.

[Dirk Broer]

9000 BTU, It should come with its own windmill and solar panels....
I have two servers myself I daren't run -but mainly because of the noise.
One is a 3U dual Xeon III 550, the other a 1U single Pentium III-1000 in a dual mobo.
I tested them once, they can be booted with a Knoppix live CD.

[Dirk Broer]

As every Tom, Dick and Harry can build his/her 16 thread cruncher nowadays by simply buying an AM4 mobo and a Ryzen 7, let's try for the next level:
The 24 thread cruncher.

Goodbye to Socket 940, as there are no mobo's with 12 sockets for ye olde Socket 940.

Socket F:


Buy a quad Socket F mobo and four six-core Opteron 8400 EE- or HE-series "Istanbul" of 60-79 Watt TDP.


Socket G34:


Buy a dual Socket G34 mobo and two twelve-core Opteron 6100 HE-series "Magny-Cours" of 85 watt TDP.
Buy a dual Socket G34 mobo and two twelve-core Opteron 6230 HE-series "Interlagos" of 85 watt TDP.
Buy a dual Socket G34 mobo and two twelve-core Opteron 6300-series "Abu Dhabi" of 99-115Watt TDP.

I've read reports of SETI.USA members that prefer the old K10 core over the Bulldozer for performance reasons, so if you want to give Socket G34 a try, go for the Opteron 6100 HE-series.

No Socket AM4 here -yet. This might change with Ryzen v2 or Ryzen v3, as Starship is rumoured to have 48 cores.

Socket TR4


Take any TR4 mobo and buy a 12-core Ryzen Threadripper 1920X. [recommended]

Advantages of Threadripper against the Bulldozer-based Dual Opterons is the amount of PCIe lanes and the fact that a single CPU system requires less space. Disadvantage is the fact that the quad-channel RAM dictates that you have to buy the RAM in quantities of at least four sticks at a time. This option also gives you the opportunity to make your cruncher a 32-thread cruncher in time though.

No Socket SP3 either, there are no 12-core EPYCs planned yet.

Within a few months you can build yourself a 24-thread cruncher using a AM4 board with a Ryzen 9 3900 (65 Watt TDP) or 3900X (105 Watt TDP) CPU...

[plonk420]

meh, i don't believe those rumors. would be nice, but i think they need to work on clock speed and cross-CCX latency. and i think clock speed will be one hell of a challenge on a new process... one that pretty much nobody has used before, or at least at these clock speeds (mobile barely counts) and voltages.

[Dirk Broer]

meh, i don't believe those rumors. would be nice, but i think they need to work on clock speed and cross-CCX latency. and i think clock speed will be one hell of a challenge on a new process... one that pretty much nobody has used before, or at least at these clock speeds (mobile barely counts) and voltages.

Well, rumours/rumors have it that the fastest Ryzen 7's and 9's will get up to and beyond 5 GHz....nothing known about their cross-CCX latency yet though.
But also nice to know that the lowest 3000-series Ryzen 3, the Ryzen 3 3300, will have 6 cores/12 threads -able to do 3.2 Ghz (base) up to 4.0 GHz (turbo) at 50 Watt TDP and all that at $99.99....

Makes you wonder what the Athlon 300GE will be capable off....

[Dirk Broer]

The new recommendation is to buy a recent Socket AM4 mobo -you might already own one, you just may have to update your BIOS- and a 65 Watt TDP Ryzen 9 3900, as soon as they become available for Joe Sixpack -us, that is. If the A320 board you bought before had a too small BIOS chip (32MB instead of 64MB, IIRC) you may need to buy a new AM4 board. Check the cpu support info of the board of your choice. At least MSI (the MAX boards) and Asrock (R3.0 and R4.0) have A320 boards that will support the R9 3900. The CPU support page of my own Asrock A320M Pro4 does mention the 105 Watt TDP R9 3900X, but makes no mention of the 65 Watt TDP R9 3900, strange.

[plonk420]

well, now with the Ryzen 1600AF for $85 back in stock, get two of those, two cheap mobos (buy with an AmEx or Mastercard to extend warranty by a year... and then pay it off immediately, of course). cheaper than the threadripper (those $250+ (new) boards are oooofff)

[Dirk Broer]

But you no longer need a Threadripper to build a 24-thread cruncher. A simple AM4 board with a Ryzen 9 3900 (65 Watt tdp) or 3900X (105 Watt tdp) is enough,
needing just one PSU, one set RAM, one mobo, one graphics card. Pitty the Ryzen 9 3900 seems to be an OEM model...

[plonk420]

2 1600 AFs (maybe with a PSU splitter ... or cheapish PSUs) is cheaper than a 3900X, tho!

oh, speaking of PSUs, have you seen either the LTT PSU Tier List or a g.sheet The Power Supply Guide? kinda interesting...

https://docs.google.com/spreadsheets/d/1_GMev0EwK37J3zZL98zIqF-OSBuHlFEHmrc_SPuYsjs

https://linustechtips.com/main/topic/1116640-psu-tier-list-40/

amused to see my beloved Seasonics S12IIs in Tier D lol

[Dirk Broer]

Problem is that, even while purchasing two Ryzen 5 1600 systems may be actually cheaper, running two systems 24/7/365 is more expensive than running one.
Two times the Wattage for both CPU and GPU adds up, and two times the rest of a system does too.

[plonk420]

well, i'm trying to confirm whether it can run headless, but the power consumption should be somewhere around 80-100 watts x 2 vs 190-210 without GPUs

[Dirk Broer]

2 1600 AFs (maybe with a PSU splitter ... or cheapish PSUs) is cheaper than a 3900X, tho!

oh, speaking of PSUs, have you seen either the LTT PSU Tier List or a g.sheet The Power Supply Guide? kinda interesting...

https://docs.google.com/spreadsheets/d/1_GMev0EwK37J3zZL98zIqF-OSBuHlFEHmrc_SPuYsjs

https://linustechtips.com/main/topic/1116640-psu-tier-list-40/

amused to see my beloved Seasonics S12IIs in Tier D lol

Funny thing here is that some of the better rated PSUs are sold by the likes of Cooler Master and Corsair. I am not saying that these are bad products, but who actually makes them?
Both Cooler Master and Corsair have no factories with their name on it. They just order, to their specifications, at other manufactures (like e.g. SeaSonic)

[vaughan]

Interesting Dirk. From my limited experience of having only ever purchased one Seasonic p/s it failed within 2 years of 24/7 usage. I vowed never to buy a Seasonic branded power supply again.

My go to p/s is Corsair and I have had zero failures in 10+ years of 24/7 run-time in many many computers. What usually happens is that the CPU gets so old that it is too expensive from an electricity usage point of view to run a low core computer; better to replace with a new multiple core machine.

[Dirk Broer]

As Anandtech wrote in 2010 (https://www.anandtech.com/show/4011/corsair-ax750-80plus-gold): " In recent times they [Corsair] have used CWT, Seasonic and Flextronics as their PSU ODMs. A few weeks ago [plus ten years] Corsair presented their newest PSUs, the AX series. The goal is to provide performance, quality and high efficiency, this is apparently the best power supply Corsair can offer at the moment. Today we'll look at the AX750; is the 80Plus Gold certificate justified? And what other useful features does it provide? The AX750 and 850W models are based on a Seasonic design (X-400FL and/or X-760) with some modifications in the details."

The RM series seems to be made by CWT (https://www.guru3d.com/articles-pages/corsair-rm850x-(2018)-power-supply-review,8.html)
The AX series seems to be made by Seasonic, as written above, the ODM for the AXi series is Flextronics (https://www.reddit.com/r/buildapc/comments/48s2m5/discussion_the_corsair_ax760i_psu_and_why_you/) though

[Dirk Broer]

Great news for those into more threads: the two cheapest CPUs per thread offered are here now the 6-core/12-thread Ryzen 5 1600 and -surprise!- the 12-core/24 thread Ryzen Threadripper 1920X.
Both are priced so that the price per thread falls below 9 Euro's on purchase, the Threadripper 1920X costing just over 200 Euro's, the Ryzen 5 1600 just over 100.

Meanwhile the price for the 4-core/8-thread Ryzen 5 1500X (the one with the biggest L3 cache per core) has gone through the roof -to the moon even: 2200 Euro's...
Nice response from AMD: the new budget model Ryzen 3 3100 has as much cores/threads/cache as the 1500X at a mere $99, and supports PCIe 4.0

[plonk420]

https://i.redd.it/piwt9kgitsx41.jpg

[Dirk Broer]

If you go for an AM5 consumer system with 24 threads you also, as with the AM5 16 thread cruncher, have the choice between three CPUs but both prices and Tdp's are a bit more extreme though:



Model
Ryzen 9 7900
Ryzen 9 7900X
Ryzen 9 7900X3D


Normal Speed
3700 MHz
4700 MHz
4400 MHz


Turbo Speed
5400 MHz
5600 MHz
5600 MHz


L3 Cache
64 MB
64 MB
128 MB


Tdp
65 Watt
170 Watt
120 Watt


Price
$ 450
$ 410
$ 600



If all you want is to be able to run 24 threads, and do it as cheap as possible, you now buy a 65 Watt Tdp Ryzen 9 7900, put it in that same ASRock A620M Pro RS mobo (at least some passive VRM cooling),
and stick in as much RAM as you can afford -it will take up to 128 GB DDR5.
You don't need to bother with a GPU, as all three mentioned CPUs have the same IGP -a 128 shader core RDNA 2 Radeon, that runs between 400 and 2200 MHz.
Running costs-wise I'd suggest a M.2 NVMe SSD of the PCI Express 3.0 x4 type, as they need far less Watts to read/write than the PCI Express 4.0 x4 variety -PCI Express 5.0 x4 won't run at all at those speeds in an A620 board.
In case you are wondering how much this is supposed to help you cut costs: PCIe 4.0 x4 SSDs will draw as much as 7 Watt reading/writing, while the most cost-effective PCIe 3.0 x4 ones will do 0.07 Watt, doing the same.

If it's performance you want, you'll take that Ryzen 9 7900X3D. That huge L3 cache will sure help at e.g. SRBase, albeit at the cost of a bigger power bill at the end of the year.
You might be put off by the 170 Watt Tdp of the Ryzen 9 7900X -and rightly so- but in practice it will eat 'only' twice as much Watts from the wall as it X-less brother.

And when money is no concern, you still might as well go all the way: buy that X670E ATX board -or even an EATX- for better cooling/overclocking options and enjoy all the benefits of PCI Express 5.0 x4!
When you're spending so much money though, you might also want to consider a Ryzen 9 7950X or 7950X3D for a 32 thread cruncher (no 65 Watt Tdp 16-core Ryzen 9)....or wait for the next generation of Threadrippers -or go EPYC.