Thursday, February 24, 2011

The GraphLab machine learning framework on Amazon EC2 - part 2 - testing

This page has moved.

For linear solver example applications see: http://graphlab.org/gabp.html

For matrix factorization example applications see http://graphlab.org/pmf.html

For clustering example applications see http://graphlab.org/clustering.html
Posted by at
Labels:

14 comments:

  1. DanMarch 5, 2011 at 1:46 PM

    Is the gabp demo called something else? In OSX just built:


    bash-3.2$ ls
    boost_1_46_0 graphlabapi
    boost_1_46_0.zip graphlabapi_v1_535.tar.gz
    bash-3.2$ ls graphlabapi/debug/demoapps/
    CMakeFiles Makefile coem image_denoise svm
    CTestTestfile.cmake cmake_install.cmake demo pagerank

    ReplyDelete
  2. Danny BicksonMarch 5, 2011 at 2:06 PM

    I am sending you soon instructions about how to add gabp to the code. (It will be simple - I promise..)

    ReplyDelete
  3. Danny BicksonMarch 5, 2011 at 2:46 PM

    Please use this link: http://graphlabapi.googlecode.com/files/graphlabapi_v1_878.tar.gz
    and repeat the instructions. This tar file has both pmf and gabp. You may need to install itpp for pmf. Let me know if this works for you.

    ReplyDelete
  4. DanMarch 5, 2011 at 4:14 PM

    Great, it's rebuilding now.

    Where do I look for SVD-like matrix decomposition functionality?

    ReplyDelete
  5. Danny BicksonMarch 5, 2011 at 4:21 PM

    It is found in the directory demoapps/pmf
    A detailed tutorial is found on: http://www.graphlab.ml.cmu.edu/pmf.html

    You will need to install the last library: itpp.
    Detailed instructions are found on my blog on: http://bickson.blogspot.com/search/label/itpp
    Note, that I think you need only to install step 6. (Since MAC is supposed to come with Atlas, so lapack installation is not needed). Let me know if you have any issues.

    ReplyDelete
  6. Danny BicksonMarch 5, 2011 at 4:21 PM

    p.s.
    After you install itpp, you will need to ./configure and make again.

    ReplyDelete
  7. DanJune 22, 2011 at 4:28 AM

    OK, I'm back here with a working pmf and some test data (TV ratings; also NetFlix; also some smaller experimental set derrived from wikipedia descriptions of movies in archive.org). Will try movielens from http://www.graphlab.ml.cmu.edu/pmf.html first ... ... ok that worked (https://gist.github.com/1039631 ) ... we're in business finally :)

    Is it reasonable to expect http://www.graphlab.ml.cmu.edu/matlab.html to work with Octave too? Hmm not sure that page is exactly what I'm after, which is -for now- just to have Octave access to the contents of the binary file that pmf gives me. RTFM time!

    ReplyDelete
  8. Danny BicksonJune 22, 2011 at 4:41 AM

    HI Dan
    Always glad to hear your feedback! Currently we have 3 ways for reading the PMF output format. 1) Matlab 2) simple binary format 3) python script. On default the output is based on itpp format, but if you use the --binaryoutput=true flag the output will be a simple binary array. Here is its format:
    The flag you should use is --binaryoutput=true
    it writes the following format:

    //OUTPUT: SAVE FACTORS U,V,T to a binary file

    // FORMAT: M N K D (4 x ints = user, movies, time bins, feature width (dimension))
    // MATRIX U ( M x D doubles)
    // MATRIX V ( N x D doubles)
    // MATRIX K ( K x D doubles - optional, only for tensor)
    // TOTAL FILE SIZE: 4 ints + (M+N+K)*D - for tensor
    // 4 ints + (M+N)*D - for matrix

    There is also python script written by Timmy, I will send it in a separate email.

    ReplyDelete
  9. DanJune 22, 2011 at 5:32 AM

    Thanks. itload() in Octave is working now. I was expecting to use it like this initially "[Movie, User] = itload('movielens20.out')" but I see now with 'whos' it chooses the variable names for me.

    ReplyDelete
  10. Danny BicksonJune 22, 2011 at 5:38 AM

    Did you change anything in the script, or do you use it as in Matlab?

    ReplyDelete
  11. DanJune 22, 2011 at 7:33 AM

    it works untouched. The only thing changed was realising how to invoke the function.

    ReplyDelete
  12. John TooheyDecember 4, 2011 at 9:39 AM

    Hi,
    I've started the configured AMI instance, and ran make in this demo's directory. However, the executable complains about invalid syntax on the command line invocation.

    I've previously tried updating the image, as you mentioned in the previous post, but that fails with missing libraries. I just assumed that the prebuilt image would work as is, but is there more that I need to do?

    Thanks
    John

    ReplyDelete
  13. John TooheyDecember 4, 2011 at 10:30 AM

    Hey,
    Got it to work with this command line :-
    ./gabp --algorithm 0 --data mat3x3

    ReplyDelete
  14. Danny BicksonDecember 4, 2011 at 11:12 AM

    Hi John,
    I apologize for the trouble. Those instructions where a bit too old.. I have updated the blog post to point to the newer instructions. Let us know if you have any questions! You are welcome to join our user mailing list:
    http://groups.google.com/group/graphlab-kdd where you can ask any question.

    best,

    DB

    ReplyDelete

Subscribe to: Post Comments (Atom)