16-core workstation
As of 04/03/2009, I finished building a 16-core workstation for my wife. What was interesting about this little project is that the machine’s cost ($1550) turned out to be so much lower than made-to-order machines (eg, from Dell, HP, etc.). So, I decided to share my experiences here. In this case, the main benefit of building-my-own was that 8000-series Opteron CPUs were much cheaper on the used market. This reduces the total parts price by about $1600 ($400 times four CPUs).
The finished product
If you look carefully you may notice that, at this point, the wrong RAM slot combination was being used. This has been corrected.
Specifications
Tyan S4980 Thunder n3600QE extended ATX quad socket F motherboard (Nvidia NFP3600 chipset), with 2X gigabit ethernet and one PCI express x16 slot
4x AMD Opteron Barcelona 8350 2Ghz quad-core CPUs with 4x 512KB L2 cache, shared 2MB L3 cache
8x 2GB = 16GB DDR2 667mhz registered ECC RAM (model Transcend TS256MQR72V6U)
Coolermaster Cosmos 1000 tower case
750W Xigmatek MC NRP-MC751 ATX12V power supply. (UPDATE 9/2009: I no longer recommend this power supply. I had to replace its fan after it started making annoying clicking noises.)
4x Xigmatek HDT-SD964 92mm Rifle CPU Cooler
Parts list and cost
All parts (except CPUs and hard disks) were bought new from newegg for a cost of $1100 plus shipping. If you want to use all four sockets, you need 8000 series Opterons; these are very expensive new (over $500 each) but can be found for about $100 each on ebay used. That brought the total cost to just over $1500. Not bad for 16 cores.
Peformance
My colleague James Swaine ran some performance experiments on this machine which were published at OOPSLA 2010 in the paper Back to the Futures: Incremental Parallelization of Existing Sequential Runtime Systems.
Conclusion
So far, this system is running great with CentOS Linux 5.3 and OpenMPI. It is silent, stable, and surprisingly power efficient. It draws only 1.5 amps of current at idle which translates into 165 watts. When fully loaded, current jumps up to 3.2 amps. These power numbers include only one 7200rpm hard disk and no graphics card. One thing that this machine does not handle well is graphics. The onboard XGI volari video is a joke, so I installed a basic NVidia quadro card in the PCI express slot.