[NCLUG] Anyone have Experience with an SGI Altix UV 10 machine

[Bob Proulx]

Kevin Olson wrote:
> I have an opportunity to acquire an SGI Altix UV 10 machine. It is about 6
> years old, so it is pushing the architecture back a bit. It has 4 x Xeon
> x7550 CPUs,with 512GB of memory.

Wow!  That looks like a fun machine!  Each X7550 has 8 cores with 2
threads per core.  Even now I would guess that it would have quite a
competitive single thread benchmark.  It also rates at 130 watts too
so would have a high total power consumption too.  I might take a swag
at 500+ watts.  If you meter it I would love to hear the actual power
consumption of it.  But I would guess you would be able to compile the
entire Linux kernel source on it in less than a minute.

> My understanding is that the memory is "globally shared", but it is
> a "deep hierarchy" so for performance reasons one really wants to
> keep to a single CPU for most activities.

If I guess correctly that is NUMA.  In which case Linux has been NUMA
aware for a long time.

> My questions are:
>   * I would like to install CentOS 6.x or 7.x on it. Currently it has
> SLES11 SP4 installed. Are there any barriers to such an installation? There
> were notes on the web about RHEL being an available option for it years
> ago, but I do not know if that was enabled due to proprietary drivers of
> any sort.

I think any current distro would be okay.  The press release I found
for this machine
(http://www.sgi.com/company_info/newsroom/press_releases/2010/march/altix_uv10.html)
says "The system's open x86 architecture leverages the latest quad-,
six- or eight-core Intel Xeon 7500 series processors, allowing for
completely unmodified Novell SUSE or Red Hat Linux operating systems."
Therefore I would guess that you would be okay installing any current
version of any distro.  However RHEL / CentOS might be best if there
is a funky driver needed that they have ported into their tree that
isn't elsewhere.

>   * Are there any tips on how to allow CentOS to perform best given the
> memory architecture?

Linux has been NUMA aware for a long time.  Performance varies with
the workload of course.  However the Linux kernel will try to schedule
processes close to the memory.  But it will depend upon the size of
the workload.

I recommend running benchmarks both with and without HT enabled.  I
found that disabling HT improved performance on my workloads.

> Does anyone here have experience with such a machine? Or advice on whether
> the attempt is foolhardy?

I have worked with large HP server systems which, since they were
prototype machines, I will just say had more cpus but a similar amount
to memory.  In my case I ran Debian on them.  They ran the OS fine.
It was an incredible amount of fun.  Here are some thoughts...

How loud is the machine?  Where are you going to run it?  I doubt this
is a system you would want in your living room.  It is probably very
loud.  I have an HP J5000 "endtable" system that when powered up in
the house everyone in the house definitely knows it.  Creates a rumble
like a Star Wars Death Star background track.

How much power does it consume?  How much will that cost you to run
it?  A good rule of thumb for cost, somewhat concervative, appears to
be $1/year per watt.  If it burns 500 watts then it will cost almost
$500 per year in power consumption.  If your house has electric heat
in the winter then this is simply a trade of one watt for another.
But in the summer it is extra heat when you don't want it.

Everything is on a long grey scale from one end to the other.  If you
are getting this for free then it could be fun to power up only
sometimes when you needed it and run it and then shut it down at other
times.  If it is significantly pricey however then you would not get a
return on your investment.  Everything is relative.  Only you can
decide.  It is a solution looking for a problem.  It would be fun.
And foolhardy too.  But that is okay if you know it.  If you do it
then you should write it up and give a presentation about it to NCLUG.
You would not be the first with a personal mainframe.

  http://ibmsystemsmag.com/mainframe/stoprun/Stop-Run/Connor-Krukosky/
  http://imgur.com/a/5uWit
  https://www.youtube.com/watch?v=45X4VP8CGtk

Bob