Benchmarking Help

[drezha]

I'm trying to benchmark a wide range of CPU's for some work I'm doing for some research and wondered if anyone fancied helping me out by running the following benchmark test?

Either method will help me out. :)

Single Core Test
First way - download and run the single core model - this allows me to see how each individual core performs on it's own. Easiest way to do this is to download the ZIP file I created here (http://chrissalter.me.uk/Files/FDS.zip). This downloads the 64 Bit and 32 bit program and the test file with batch files. Just run the batch file for the operating system you have (64 or 32 bit). The test will jump to 100% CPU on one core and take about 5-10 minutes to run. When run, can you send me the bench1.out file and a screen shot of your computer performance from Control Panel (or just send the details of the PC - need Windows OS, amount of RAM, CPU name (exact) and clock speed) to ChrisSalter@hoarelea.com. You can then delete the folder (nothing is installed).

Current time to beat is a mates i7 @ 4.9GHz (!) - completed the test in 144 seconds.

Multi Core Test
Secondly, if you want to take part in the multicore test, it's a bit more involved and will require the installation of an additional program. This test will use all cores on you machine and will take 30-120 minutes so only undertake this one if you don't mind running your PC at 100% on all cores for this amount of time.

If you know how to run MPI programs and can already, then feel free to do it that way by using the program here. Alternatively, if you have no idea what that means, you need to download Pyrosim.

Download and install Pyrosim from here (http://www.thunderheadeng.com/pyrosim/download/)
When it starts up, it'll ask for a license - use the code
4F95 D4D9 7A94 4872 4509 04 to have 30 days free trial.
Download the Zip file as above

http://i.imgur.com/86B6Z.png
Click the button highlighted in red and select the file scale1.fds
Click the arrow on the button highlighted in blue and select the Run FDS parallell button and wait for the model to finish.
Like the single core test, I need the scale1.out file and the PC specs.

I'm sorely interested to see what AMD chips do on it as everyone seems to have Intel these days and I have an inkling that the 8 core AMD chip might actually outperform the flagship Intel based on the method of programming for the mutlicore functionality.

If you want to view the current results, you can view the Google Spreadsheet here (https://docs.google.com/spreadsheet/ccc?key=0Ag1i-n-25DvndFFnYnM2dWo4d3NQdlJyRXhqNWlkRGc&usp=sharing).

[AMDave]

I don't know anything about that software. Looks interesting though.
Honestly I'd use the Phoronix Test Suite (http://www.phoronix-test-suite.com/) for the immense range of tests.
And compare results with the massive database of results on OpenBenchmarking.org (http://openbenchmarking.org/)
Comparing i7's with Llano's -> http://openbenchmarking.org/result/1112293-AR-1112252AR57

I'll give your app a run on the Win7 Phenom II gamer-box tomorrow+++ and see what falls out :)

+++ make that in a couple of days. It's crunching hard right now. I got carried away for a moment :P

[drezha]

Thanks Dave.

The issue that arises is that the software is an odd one - sometimes it runs better single core than multicore and therefore a faster clock speed would benefit.

[NeoGen]

Hey Chris

Has no one been sending in results the last 2 days? I was hoping to see that spreadsheet grow :)

I am about to leave work and tonight will get those benchmarks on my machines, and just for fun I will even throw it on the MSI Windpad tablet! (With its "powerful" AMD Z-01 1GHz dual core processor) :icon_lol: I might have to leave the multicore test running on it overnight though, something tells me it won't finish between 30 and 120 minutes. :icon_razz:

[Dirk Broer]

Hi Drezha/Chris,

Most of my AMD systems are running different Ubuntu flavours at the moment, does your benchmarking object to that?

[Ramjet]

Hello Chris, I just sent test results for an AMD 8150 and a 1090T, I can't wait to see where I stand against the rest! :icon_mrgreen:

[Ramjet]

Click the button highlighted in red and select the file scale1.fds
Click the arrow on the button highlighted in blue and select the Run FDS parallell button and wait for the model to finish.
Like the single core test, I need the scale1.out file and the PC specs.

H..E..L..P...I decided to run the multicore test, but I do not see the file " scale1.fds" anywhere in that whole directory, and yes I did change the default to view all files. I downloaded the newest version of the 64 bit Pyrosim.
Anybody?........................

[NeoGen]

Hi Ramjet

The scale1.fds file is located on that first zip file for the single core benchmark test.

Chris, I'll be uploading the results sometime this afternoon. I have been running the tests with different settings to see the difference.

[Ramjet]

Hi Ramjet

The scale1.fds file is located on that first zip file for the single core benchmark test.

Thanks NeoGen,

I found that but wasn't sure you could use the same file for single core and multi-core. Now I'm fighting with the dreaded Win7 permissions settings to get it to run. :eusa_wall: I'll get it eventually, just don't hold your breath in the mean time.

[drezha]

Hi Drezha/Chris,

Most of my AMD systems are running different Ubuntu flavours at the moment, does your benchmarking object to that?

Dirk - The ZIP files are for Windows only, I'll create a shell script and download for the linux one this evening. Multi core would be more of a hassle to run (requires the installation of additional software and some messing about - the hassle I got trying to run Multicore on linux was silly! However, think I've cracked that now.)

Neogen - I've been slowly adding - I've recently moved house and they're taking their time installing my internet. Should be complete by Thursday ಠ_ಠ but in the mean time I've been adding the results at slow periods during work. In comparison to your Windjet tablet, I'm trying to get the code compiled on my Raspberry Pi and get some benchmarks ;) - it's possible (I've seen another guy do it but without internet, I can't download the packages I need!)

Ramjet - No idea why the Windows 7 permissions is coming up, we manage to run it fine here in a corporate environment. Sometimes Windows can be a right PITA!

[drezha]

Hi Drezha/Chris,

Most of my AMD systems are running different Ubuntu flavours at the moment, does your benchmarking object to that?

Dirk - Uploaded the Linux version here. (https://docs.google.com/file/d/0Bw1i-n-25DvnbGQ3UXVYdDR5dnc/edit?usp=sharing) It assumes you're running a 64 bit distro.

The shell script should run the single core and then move onto the multicore.
Shell script is:



#!/bin/sh

# FDS Benchmarking - Scripts for Linux

# Make program executable
chmod +x ./fds5_linux_64
chmod +x ./fds5_mpi_linux_64

# Run FDS Single Core Benchmark
fds5_linux_64 bench1.fds

# Multicore Benchmark

# Install MPI Runtime
sudo apt-get install lam-runtime

# Start MPI
lamboot

# Run
mpirun -n 8 fds5_mpi_linux_64 Scale1.fds



If you don't want to run multicore, just delete the Multicore lines. You might want to play around with the -n flag and change it to match the number of cores you have on the machine. I also can't guarantee that the lam-runtime will work - it seemed to in an old test I ran on linux but I think it's a bit dated now.

Linux is an area I need to play around with. I'm hoping to get a cheap second hand PC system to play with and try it as all the rigs at work are Windows and I don't want to change my OSX setup on my Mac at home - downside is I'm looking for old AMD workstations of multicore machines with RAM and I can get some cheap looking rack servers but nowhere to store/run one!

EDIT:
Just realised that I hadn't set the script to CHMOD the MPI one so it should be as script shown above.

[drezha]

I've been adding some more figures.

I imagine the Xeon X5660 is getting the same kind of speeds as the i7's due to the additional onboard cache (12M compared to 8M) as well as the additional 2 cores (though I don't think the cores play so much of issue). I think that one of the current i7's could outstrip a current Xeon.

[Dirk Broer]

Better compare the current i7's with the top segment of the Xeon line:
the X5690 (3.46 Ghz, 6 cores, 12 threads); the X7560 (2.26 Ghz, 8 cores, 16 threads) or the top Opteron: the 6386 SE (2.8 Ghz, 16 cores and dito threads)

But then we have to have people with such CPUs who want them benchmarked... I will post my Llano's soon

[vaughan]

I just ran it and sent results for my Lenovo notebook.

[drezha]

Thanks Dirk and vaughan.

Dirk, that's the problem I've come across. It's one thing to say run the benchmark but you need the people with them to be willing to run them.

[Dirk Broer]

Started the single core using ./fds5_linux_64 bench1.fds
Started the multi core using ./fds5_mpi_linux_64 scale1.fds ('mpirun' was not recognised as a command, may have to reboot first. Will do when it is finished)
Reboot did not help, helas. Did do the lamboot before trying to invoke the mpirun command.

[drezha]

Dirk, thanks for trying it. It looks like (from the attached text you sent me), that unfortunately it ran the test on a single core



./fds5_mpi_linux_64 scale1.fds
Process 0 of 0 is running on ASUS-F1A75
Mesh 1 is assigned to Process 0
Mesh 2 is assigned to Process 0
Mesh 3 is assigned to Process 0
Mesh 4 is assigned to Process 0
Mesh 5 is assigned to Process 0
Mesh 6 is assigned to Process 0
Mesh 7 is assigned to Process 0
Mesh 8 is assigned to Process 0


I would have expected that to say something like Process 0, Process 1 or Process 2 for three threads.

I'm afraid running under Linux isn't something I've really tried. It's something I'm aiming for but I currently don't have a machine I can test it all on. I briefly tried using a live USB late last year but it didn't give particularly good benchmarks. I'm hoping to get a server machine cheap on eBay I can play with at home.

I mean my best bet would be to use the Amazon EC2 instances as that would save me money in comparison to buying something - however, they can't guarantee that I'll get the same CPU each time so it makes benchmarking a bit hard!

[Dirk Broer]

Hi Drezha,

Unfortunately the mpirun was not recognised as a command, so the parameters that went with it neither.
Guess that there needs to be some linking done in order to let the lamboot take effect, or that the multi-core run needs to be done in another directory/map.
Shall send the single core runs for my A8-3820 and A8-3870K soon.

[drezha]

I'll be getting an OVH dedicated server for a month to have a play with linux and see what I can come up with. That'll let me play around with both linux and windows benchmarks (I have a Windows 2012 Server code) as it's guaranteed settings.

Would let me get some additional points in a DC project of choice for a month :P

[drezha]

Might have found the issue. I missed a p from the mpi command.

Should be



mpirun -np 8 fds5_mpi_linux_64 Scale1.fds

[Dirk Broer]

I'm afraid there is more to it than that. The Lib64 map that has been created under the FDS map in the Downloads folder may have it's contents at least linked to another map and the command line
mpirun -np 8 fds5_mpi_linux_64 Scale1.fds must at least be changed to mpirun -np 8 fds5_mpi_linux_64 scale1.fds (it will just not run with a capital S)

[Dirk Broer]

Might have found the issue. I missed a p from the mpi command.

Should be



mpirun -np 8 fds5_mpi_linux_64 Scale1.fds


Well, I'll be dripped in dogs**t! It's running right now on three cores. Did install OpenMP in the meantime, by the way. I'll send the results and will let it try to run on four cores as well.
(as I suspect that a A6-3500 is actually a A6-3600 with a disabled or defect core. Sometimes BOINC is running four WUs at a time on the A6-3500...not counting GPU or nci tasks)
Triple-core run output sent, now running processes 0,1,2&3 on the 'triple'-core....

[drezha]

Thanks for trying again :)

Not sure how it worked but it did first time when I tried it. On the i3 server I'm running for the testing (and a server for myself to do some game server testing with), I got noticeably faster speeds running linux than I did on Windows. (61% slower on the single core and 83% on the multi core. That's quite a difference. I now need to do some more investigating because that's a fairly impressive difference.*

I did notice mind that the Windows install of the i3 didn't reach full clock speed during the single core run. The CPU frequency was being kept low.


*One was a Debian CLI server install and the other is Windows 2012.

[drezha]

For those that are interested, I deleted the Google Doc and moved all the data to Github (as I wanted to combine it with some other data on the program).

Details can be found here.

https://github.com/drezha/FDS_Resources