Tuesday, July 17, 2012

Cray XMT

Cray XMT (Xtreme Multithreaded Technology) is a new hardware designed with graph algorithms in mind. Those of you who attended our GraphLab workshop may have seen the poster by YarcData, a Cray company about their data analtyics contest.

Today I got some more details from Venkat Krishnamurthy about the XMT platform. XMT contains special purpose CPUs. Each CPU runs 128 threads in parallel (up to 4096 cpus). The machine has one huge shared memory infrastructure (with up to 512TB of data). The machine is specially design for non-uniform memory access, no CPU cache. This allows for very efficient graph analytics.

Some of the problems we are straggling to solve in distributed graphlab are answered using this architecture.   For example:

  • Balanced graph partitioning across machines - is not needed.
  • Dynamic and asynchronous methods are supported.
  • Tolerates long global latencies while continuing the computation locally.
Here are some impressive numbers from their presentation:

The drawback of the XMT machine it is price - I hear it is not cheap. For further details contact YarcData.

Additional material I got from Venkat:

Here's more information from the Cray web site

Also here's tons of information from Sandia's own MTGL site with several detailed presentations on XMT and the MTGL library on several key graph algorithms

and a presentation by David Mizell of the engineering team

YarcData is building a semantic database on top of the Cray XMT machine.

Monday, July 16, 2012

A new dog on the block: GraphChi

Anyone who attended our GraphLab workshop could not have missed the new GraphChi release.
It is a new project by Aapo Kyorla from CMU, with some surprisingly great performance results.
The basic idea is that computation is limited to a weak machine (mac mini) with a large hard drive or SSD drive. Instead of loading the full problem into memory, the problem is read in parts from disk.

It is hard to believe what kind of results Aapo extracts from a mac mini. For example:

ApplicationInput graphGraph sizeComparisonGraphChi on Mac Mini (SSD)Ref
Pagerank - 3 iterationstwitter-20101.5B edgesSpark, 50 machines, 8.1 min13 min1
Pagerank - 100 iterationsuk-union3.8B edgesStanford GPS (Pregel), 30 machines, 144 min581 min2
Web-graph Belief Propagation (1 iter.)yahoo-web6.7B edgesPegasus, 100 machines, 22 min27 min3
Matrix factorization (ALS), 10 itersNetflix99M edgesGraphLab, 8-core machine, 4.7 min9.8 min4
Triangle countingtwitter-20101.5B edgesHadoop, 1636 machines, 423 mins55 min5

Namely, one mac mini does the work of 1636 Hadoop machines, x8 faster!!!

If you don't believe it, you are welcome to download GraphChi. The software is open source with Apache license. It is written in C++ but now I hear Aapo is created a Java version as well.

Additional reading: MIT Tech Review.

First Graphlab video tutorial by Nick - BigDataR Linux

I just got this cool video tutorial from Nick Kolegraff, BigDataR Linux distributor:
It explains how to run the collaborative filtering package, GraphLab v1 (multicore) on Amazon EC2. Thanks Nick for this great effort!

I will next help Nick to produce some newer tutorials of GraphLab v2 and GraphChi.

Sunday, July 15, 2012

Large Scale Machine Learning and Other Animals

This week we are celebrating 100,000 page views!
Thanks to YOU - my amazing collection of readers!
Next (ambitious) milestone: is 1,000,000 page views.. :-)

Some more good news to report: we won the 3rd place in ACM KDD CUP track 2 - 2012 (out of 192 groups). We had a great team for the chinese academy of science:

Xingxing Wang
Shijie Lin
Dongying Kong
Liheng Xu
Qiang Yan
Siwei Lai
Liang Wu
Guibo Zhu
Heng Gao
Yang Wu
Danny Bickson
Yuanfeng Du
Neng Gong
Chengchun Shu
Shuang Wang
Fei Tan
Jun Zhao
Yuanchun Zhou
Kang Liu

I had a VERY minor part at the team success. Most of the credits are for the great chinese guys.
We are now submitting a paper called: "Click-Through Prediction for Sponsored Search
Advertising with Hybrid Models". As soon it is ready I will post it here.

You may recall that this is the 2nd year we get to a high place in this competition.

Tuesday, July 10, 2012

The GraphLab workshop is over!!

Thanks so much to the 320+ people attended - and to the great speakers - we have an amazing user community!

I can not think of a better way to demonstrate the buzz and excitement about GraphLab than to list some the twits we had yesterday. Here is a link to Carlos' lecture.

Monday, July 2, 2012

Amazing ML visualization software

I stumbled upon this by viewing Nirod's Priell mailing archives. MLDemos is a beautiful
ML visualization software which can be very useful when trying to understand the different  ML methods.

I installed MLDemos easily using the MAC .dmg file. Here are some images I created
when playing with the software. There are two classes: red and white points.
Here is classification using kernelized SVM:

And the same problem solved using linear SVM:
And using sigmoids:

The disks are the support vectors.

The apparent drawback is that visualization is done using two dimensions. This software does not replace the need of studying ML methods, but gives a very nice visualization that
helps understand the different methods.