I will start with an example, but I believe that the rule holds generally across many domains. I just got back from the KDD cup workshop in Beijing. The first presentation by Tianqi Chen from SJTU who won the first place in track 1. Here is a summary of the results obtained by his team: Note, that the basic method, item based collaborative filtering got a score of 34.0 %
While a combination of an additional 7 methods got a score of 42.7 %. Now divide
34.0 / 42.7 = 0.796. Namely, the basic method has 79.6% accuracy of the most sophisticated method! And of course it is way simpler.
Now let's jump to the results of the second track. A team from the NUTA had the following result:
Anyway, I can continue on and on, checking other contest results like Netflix and KDD CUP 2011. The bottom line is that typically the simpler methods gives an accuracy of 80% or more.
From talking to many companies who are working on collaborative filtering, 95% of them are happy if they have the basic models working right.
Here is some quote from Netlix technical blog (bold marking is mine):
If you followed the Prize competition, you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later. This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line. We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment.In other words, some of the fancy methods that justify publishing a research track paper, or winning an epsilon accuracy contest, are simply not worth the effort of practical deployment.