Mahout - SVD matrix factorization - reading output

Converting Mahout's SVD Distributed Matrix Factorization Solver Output Format into CSV format

Purpose

The code below, shows how to convert a matrix from Mahout's SVD output format

into a matrix format.

This code is based on code by Danny Leshem, ContextIn.

Command line arguments:

args[0] - path to svd output file
args[1] - path to output csv file

Compilation:
Copy the java code below into an java file named SVD2CSV.java
Add to the project path both Mahout and Hadoop jars.


#

import java.io.BufferedWriter;
import java.io.FileWriter;
import java.util.Iterator;

import org.apache.mahout.math.SequentialAccessSparseVector;
import org.apache.mahout.math.Vector;
import org.apache.mahout.math.VectorWritable;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.SequenceFile;


public class SVD2CSV {
   
   
    public static int Cardinality;
   
    /**
     *
     * @param args[0] - input csv file
     * @param args[1] - cardinality (length of vector)
     * @param args[2] - output file for svd
     */
    public static void main(String[] args){
   
try {
    final Configuration conf = new Configuration();
    final FileSystem fs = FileSystem.get(conf);
    final SequenceFile.Reader reader = new SequenceFile.Reader(fs, new Path(args[0]), conf);
    BufferedWriter br = new BufferedWriter(new FileWriter(args[1]));
    IntWritable key = new IntWritable();
    VectorWritable vec = new VectorWritable();
         
          while (reader.next(key, vec)) {
              //System.out.println("key " + key);
              SequentialAccessSparseVector vect = (SequentialAccessSparseVector)vec.get();
               System.out.println("key " + key + " value: " + vect);
              Iterator<Vector.Element> iter = vect.iterateNonZero();

               while(iter.hasNext()){
                  Vector.Element element = iter.next();
                  br.write(key + "," + element.index() + "," + vect.getQuick(element.index())+"\n");
            }
          }
     
          reader.close();
          br.close();

       
    } catch(Exception ex){
        ex.printStackTrace();
    }
    }
}

When parsing the output please look here about a discussion regarding validity of computed results.

Further reading: Yahoo! KDD Cup 2011 - large scale matrix factorization.