Tuesday, July 17, 2012

Cray XMT

Cray XMT (Xtreme Multithreaded Technology) is a new hardware designed with graph algorithms in mind. Those of you who attended our GraphLab workshop may have seen the poster by YarcData, a Cray company about their data analtyics contest.

Today I got some more details from Venkat Krishnamurthy about the XMT platform. XMT contains special purpose CPUs. Each CPU runs 128 threads in parallel (up to 4096 cpus). The machine has one huge shared memory infrastructure (with up to 512TB of data). The machine is specially design for non-uniform memory access, no CPU cache. This allows for very efficient graph analytics.

Some of the problems we are straggling to solve in distributed graphlab are answered using this architecture.   For example:

  • Balanced graph partitioning across machines - is not needed.
  • Dynamic and asynchronous methods are supported.
  • Tolerates long global latencies while continuing the computation locally.
Here are some impressive numbers from their presentation:

The drawback of the XMT machine it is price - I hear it is not cheap. For further details contact YarcData.

Additional material I got from Venkat:

Here's more information from the Cray web site

Also here's tons of information from Sandia's own MTGL site with several detailed presentations on XMT and the MTGL library on several key graph algorithms

and a presentation by David Mizell of the engineering team

YarcData is building a semantic database on top of the Cray XMT machine.

No comments:

Post a Comment