Monday, February 28, 2011

The GraphLab large scale machine learning framework - installation on MAC OSX 10.6

GraphLab is an open source large scale parallel machine learning framework.

I was asked by Dan, an avid reader of this blog, to supply installation instructions for MAC OSX Snow Leopard 10.6. Installation is quite simple.

1) Install cmake using the following link: cmake 2.8.4

2) Download graphlab code and install GraphLab where XXX is the latest version you can find in here
wget http://graphlab.org/release/graphlabapi_v1_XXXX.tar.gz
tar xvzf graphlabapi_v1_XXXX.tar.gz
cd graphlabapi
./configure --bootstrap 
cd debug #or equivalently cd release
make -j 4
cd tests
./runtests.sh

Anyone who tries it out - update me if it went smoothly!

NOTE: We now support Eigen linear algebra package. It can be installed using the command:
./configure --bootstrap --eigen
However, on MAC OS, gcc45 is required. See explanation here.

NOTE2: The current GraphLab MAC setup default to gcc-4.2. If you like to use your default compiler, you can comment the first few lines of the file CMakeLists.txt in the root graphlab folder, namely the lines:
if(APPLE)
  set(CMAKE_C_COMPILER "gcc-4.2")
  set(CMAKE_CXX_COMPILER "c++-4.2")
endif(APPLE)

17 comments:

  1. Said avid reader here. Boost on OSX seems to be less than smooth - the bootstrap script failed, trying to install via HomeBrew now instead....

    ReplyDelete
  2. Send me the error log, I will ask my MAC OS guys here... :-)

    ReplyDelete
  3. The ./configure script finds the version Brew installed (it gets symlinked into /usr/local/lib/libboost*). So that looks fine. I then tried to be too clever and gave a direct path to cmake instead of invoking make, but I see now that you intended just that.

    Anyhow, all seems to have built fine.

    demoapp/demo ran,

    Completed in 0.885903 seconds

    === REPORT FOR core ===
    [Numeric]
    ncpus: 2
    [Other]
    affinities: false
    compile_flags: -g -ggdb -O0 -Wall -Winit-self
    engine: async
    scheduler: fifo
    schedyield: true
    scope: edge

    My poor little laptop's not up to much, but maybe I can make a start here...

    ReplyDelete
  4. Cool!
    I will send you ASAP instructions about how to compile the matrix factorization code (if I am not wrong it is not included in the framework main code - but after you setup the framework adding an application is faily easy).

    - Danny

    ReplyDelete
  5. Ok, it's time for me to revisit this! Do you have any new recommendations for OSX, or shall I just go back over this + mails?

    ReplyDelete
  6. I tried again with most graphlabapi_v1_1215. Boost auto-installed ok, but we get stuck this time with Kyoto Cabinet - https://gist.github.com/1035498
    (...searching around a bit, I'm not alone in hitting such problems in OSX.)

    ReplyDelete
  7. Hi Dan!
    We will happily look into this ASAP. Would you mind providing us some additional details by filling out this form: https://spreadsheets.google.com/spreadsheet/viewform?formkey=dGxEbHNYb1NNVG9SQ0lJQTNPajhGSWc6MQ

    I have also sent your question to our google user group: http://groups.google.com/group/graphlab-kdd
    where we answer user questions.

    Best,

    DB

    ReplyDelete
  8. Hey Danny, this process worked for me on OSX 10.6.8 (64-bit) using MacPorts:

    {{{

    ## install utils and deps available from MacPorts
    sudo port install cmake
    # current boost port version is 1.47.0_0 which breaks graphlabapi build
    #sudo port install boost +debug
    sudo port install google-perftools
    sudo port install mercurial

    ## no port for gfortran; download from HPC
    cd ~/Downloads
    wget http://prdownloads.sourceforge.net/hpc/gfortran-snwleo-intel-bin.tar.gz?download
    gzip -cd gfortran-snwleo-intel-bin.tar.gz | sudo tar xv -C /

    ## when building itpp, we include '-framework vecLib' to pull in optimized BLAS/LAPACK libs from OSX
    wget http://sourceforge.net/projects/itpp/files/itpp/4.2.0/itpp-4.2.tar.gz
    gzip -cd itpp-4.2.tar.gz | tar x
    cd itpp-4.2
    export CPPFLAGS="-fPIC"
    export CFLAGS="$CPPFLAGS"
    export CXXFLAGS="$CPPFLAGS"
    export LDFLAGS="-lblas -llapack -framework vecLib"
    ./autogen.sh
    ./configure --without-fft \
    && make \
    && sudo make install

    ## fix permissions on installed itpp resources
    sudo chmod 755 /usr/local/include/itpp
    sudo chmod -h 755 /usr/local/lib/libitpp*

    ## clone and build graphlab
    hg clone https://code.google.com/p/graphlabapi/
    cd graphlabapi
    export CFLAGS="$(itpp-config --cflags)"
    export CXXFLAGS="$CFLAGS"
    export LDFLAGS="$(itpp-config --libs) -L/opt/local/lib"
    ./configure --bootstrap
    # hit enter for boost dep installation
    # hit enter for Kyoto Cabinet dep installation
    cd debug
    make -j 4

    }}}

    ReplyDelete
  9. Thanks Andy!!
    I would also suggest joining our google group:
    http://groups.google.com/group/graphlab-kdd
    I will post your instructions there, so other people may use it.

    ReplyDelete
  10. Hi Danny,

    My compilation on OS X fails with the following error:


    Scanning dependencies of target anytests_loader
    [ 80%] Building CXX object tests/CMakeFiles/anytests_loader.dir/anytests_loader.o
    Linking CXX executable anytests
    ld: library not found for -litpp_debug
    collect2: ld returned 1 exit status
    make[2]: *** [tests/anytests] Error 1
    make[1]: *** [tests/CMakeFiles/anytests.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....

    ---------------------------------------------

    I am using Andy's instructions posted above.
    Platform: OS X Lion (64-bit)

    ReplyDelete
  11. Sorry about that - please take again from mercurial. Problem should be fixed.

    ReplyDelete
  12. Hi Danny,

    Thanks for the response. I managed to fix that error earlier by using the --enable-debug option when running ./configure for itpp. But now, I am getting a different compiler error that I am not able to figure out:


    Scanning dependencies of target itdiff
    [ 86%] Building CXX object demoapps/pmf/CMakeFiles/itdiff.dir/itdiff.o
    Linking CXX executable randomtest.cxxtest
    In file included from /home/nihar/graphlabapi/demoapps/pmf/itdiff.cpp:23:0:
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp: In function ‘itpp::ivec randi(int, int, int)’:
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:850:23: error: ‘graphlab::random’ has not been declared
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:850:39: error: expected primary-expression before ‘int’
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:850:39: error: expected ‘;’ before ‘int’
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp: In function ‘int randi(int, int)’:
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:854:20: error: ‘graphlab::random’ has not been declared
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:854:36: error: expected primary-expression before ‘int’
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:854:36: error: expected ‘;’ before ‘int’
    /home/nihar/graphlabapi/demoapps/pmf/mathlayer.hpp:854:39: error: expected unqualified-id before ‘>’ token
    Linking CXX static library libpgibbs_pic.a
    [ 86%] Built target randomtest.cxxtest
    Scanning dependencies of target pmf
    [ 87%] Building CXX object demoapps/pmf/CMakeFiles/pmf.dir/pmf.o
    [ 87%] Built target pgibbs_pic
    [ 87%] Building CXX object demoapps/pgibbs/CMakeFiles/pgibbs.dir/util.o
    Scanning dependencies of target demo
    [ 88%] Building CXX object demoapps/demo/CMakeFiles/demo.dir/demo.o
    make[2]: *** [demoapps/pmf/CMakeFiles/itdiff.dir/itdiff.o] Error 1
    make[1]: *** [demoapps/pmf/CMakeFiles/itdiff.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....

    ------------------------------------

    Please let me know what this is about whenever you get the chance. I have faced this error both when compiling on Ubuntu 64-bit Oneiric and OS X Lion 64-bit.

    ReplyDelete
  13. Sorry about that. Please try again!

    ReplyDelete
  14. Hi Danny,

    Could you help me with this compilation error? I am working on OSX 10.6.6 and I followed Andy's instructions. I had a problem with the itpp compilation initially that I was able to solve since in OSX the GNU libtool is actually called glibtool. Until that point, ok, but then, in the last step that Andy mentioned:

    cd debug
    make -j 4

    I had this error


    [ 76%] Building CXX object tests/CMakeFiles/graph_test.cxxtest.dir/graph_test.cxx.o
    [ 77%] Building CXX object tests/CMakeFiles/graphlab_test.cxxtest.dir/graphlab_test.cxx.o
    Linking CXX executable anytests_loader
    ld: library not found for -ltcmalloc
    collect2: ld returned 1 exit status
    make[2]: *** [tests/anytests_loader] Error 1
    make[1]: *** [tests/CMakeFiles/anytests_loader.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....
    Linking CXX executable anytests
    ld: library not found for -ltcmalloc
    collect2: ld returned 1 exit status
    make[2]: *** [tests/anytests] Error 1
    make[1]: *** [tests/CMakeFiles/anytests.dir/all] Error 2
    Linking CXX executable graph_test.cxxtest
    ld: library not found for -ltcmalloc
    collect2: ld returned 1 exit status
    make[2]: *** [tests/graph_test.cxxtest] Error 1
    make[1]: *** [tests/CMakeFiles/graph_test.cxxtest.dir/all] Error 2
    Linking CXX executable graphlab_test.cxxtest
    ld: library not found for -ltcmalloc
    collect2: ld returned 1 exit status
    make[2]: *** [tests/graphlab_test.cxxtest] Error 1
    make[1]: *** [tests/CMakeFiles/graphlab_test.cxxtest.dir/all] Error 2
    make: *** [all] Error 2

    Thanks,

    Denis

    ReplyDelete
  15. The --bootstrap option didn't work for me (indicated boost lib was missing), but it built and tested fine without the option. I'm pretty sure my BOOST_ROOT is set correctly, and I tried with boost_1_48_0 and boost_1_46_1.

    ReplyDelete
    Replies
    1. Hi,
      Can you please send me (via graphlab user mailing list) the full configure output, make output, and also gcc --version, g++ --version, cmake --version. Did you try to install version 2 or version 1? In version 2 the --boostrap option is not required to setup.

      Delete