I got the following interesting paper from my collaborator Aapo Kyrola:
Pankaj Gupta, Ashish Goel, Jimmy Lin, Aneesh Sharma, Dong Wang, and Reza Zadeh. WTF: The Who to Follow Service at Twitter. Proceedings of the 22th International World Wide Web Conference (WWW 2013), May 2013, Rio de Janeiro, Brazil.
It details Twitter "Who to Follow" recommendation service. In a nutshell Twitter uses two algorithms: an egocentric random walk (personalized pagerank) and SALSA which is a bipartite random walk (similar to HITS algorithm).
In terms of infrastructure they use Twitter Cassovary graph processing system, on top of a single multicore machine, which is rather surprising considering Twitter graph size. Anyway this shows that proper efficient implementation on a single multicore machine can scale to very large models.
No comments:
Post a Comment