In part 1 of this two-parter, we discussed the absolute basics of Recommender Systems - what they are, why they're used, and some basic vocabulary. This post will talk about some of the important decisions and problems which must be addressed when implementing a Recommender System.
Technologies
The basic tool sets for Recommender Systems are pretty much the same as those for other web applications and machine learning applications, with the exception of a few specialty offerings tailored specifically for such systems. Here is an overview of a few of the options available today.
Mahout and Hadoop
The default for data science these days is to look into the Apache Hadoop ecosystem. As luck would have it, the Apache Mahout library provides Collaborative Filtering algorithms right out of the box. While pretty much any machine learning library can be used to provide collaborative filtering (and content-based filtering, for that matter), Mahout is unique in that it can be developed in a traditional real-time single-server architecture and later ported to Hadoop using its MapReduce-compliant algorithm implementations. This setup would be my benchmark for considering any other technology stacks.
Note that bringing in Hadoop entails a whole new set of architectural challenges, such as learning to maintain a Hadoop cluster (although there are many nice Hadoop-as-a-Service offerings available these days), finding developers who know Hadoop, accounting for the fact that Hadoop processing is not real-time, and munging the data back into your application. That said, you may find that you really don't need Hadoop based on the size of your data set...but it's nice to have the option for once you hit it big.
Prediction.io
Prediction.io is an interesting player. It is an open-source machine learning server solution which exposes REST APIs for making recommendations. Prediction.io boasts compatibility with a variety of programming languages and interoperability with Mahout and Hadoop, as well as a UI for configuring the server and managing data. I haven't had enough experience to say for sure whether the value-adds of prediction.io outweigh the cognitive and computational overhead, but it certainly has its share of pundits.
Google Prediction API
Google's Prediction API is pretty much exactly what you would expect - a pretty straightforward offering with an opaque backend, all of the assurances of Google's infrastructure, and all of the not-free that goes along with it.
Other Machine Learning Libraries
A Recommender System can be put together with any machine learning library and a small amount of reading up on Recommender System algorithms (as always, start on Wikipedia and work your way out). A few of the ones that I know about include Weka for the JVM and Python's scikit-learn library.
Basic Architecture
The basic architecture of a Recommender System isn't too difficult. As feedback comes into the system via site visits, item views, clicks, and transactions, the data is appropriately routed and persisted via the usual mechanisms. In some cases, the data may also be stored in HDFS or a secondary NoSQL store such as a key-value store.
As the user navigates the site, the web application appropriately requests and displays item recommendations to the user. The request is handled by the Recommender System, which may query a variety of subsystems for their recommendations (in the case of "hybridized" recommendations). These subsystems could include multiple collaborative filters, multiple content-based filters, loading of batch-calculated recommendations (such as from Hadoop), and prescriptive rules-engine based recommendations (such as a list of "Featured Recommendations").
The logic for combining these recommendations is something of an art form, and may mature to include factors such as weighting of various types of recommendations and the number of times that particular items have previously been shown to the user.
In particular, the important aspects of the architecture are to ensure that additional data stores and additional algorithms can be added with minimal pain. Furthermore, it can be wise to plan to incorporate AB testing infrastructure at the outset. Which reminds me...
Testing
Testing of recommendations is somewhat unique in that the results are non-deterministic. In general, there are two ways to determine the quality of the Recommender System's recommendations. The "easy" way is to have human domain experts examine results for quality. As algorithms are changed or augmented, the human experts can determine whether results for particular users "make sense" or not. It is helpful to have these tests incorporate real data, since it is not particularly meaningful to tune a system to provide excellent recommendations for unrealistic situations.
In the optimal case, recommendation quality would be evaluated by measuring production metrics such as click-through-rate, views, purchase frequency, and average purchase amount. In this case, AB testing can be very useful for incrementally tuning algorithms to maximize desirable outcomes.
Security
While an exhaustive treatment of the security requirements for Recommender Systems is well outside the scope of the post, suffice it to say that Recommender Systems are subject to all of the same concerns of typical web applications in addition to having several exploitable venues in terms of the recommendations themselves. Users can game the system by creating falsified accounts intended to artificially boost the frequency of recommendations for a particular item (or to reduce the number of recommendations). For a great discussion of the types of attacks possible against Recommender Systems and many of the other subjects presented in this post, I would recommend the aptly-named Recommender Systems: An Introduction by Jannach et al.
Quality of Results
Ah, the crux of the matter. A Recommender System is only as good as the quality of its results. But what is a quality result? The short answer is "anything that increases purchase amounts or user engagement". The longer answer is that it is a keen balancing act of providing accurate, relevant results (precision) without leaving out any potential winners (recall). To some real degree, choosing appropriate algorithms and results is an art in the hands of the data scientist.
Additionally, there are several other issues which must be overcome by the system's algorithms. New systems have the "cold start" problem, which is essentially the issue of overall initial data sparsity. New users will have skimpy histories and content-based results may be more appropriate; however, collaborative results have greater potential for quality because they encompass all of the complex nuance of human preference. New items will have relatively few user ratings or purchases and thus may be unfairly left out of recommendation results. Some users may have strange profiles which are unlike any other users. Finally, if algorithms are too tight, results will be "predictable" and users will feel as though the recommendations never suggest anything interesting that they wouldn't have found on their own (a quality known as serendipity).
At the end of the day, the highest-quality recommendations are the result of iterative testing and tuning of a variety of types of algorithms whose results are subsequently hybridized and combined by sophisticated rules engines. Specific methods, again, are well beyond the scope of this post, but some example terms to get you started searching are: memory-based algorithms, model-based algorithms, Slope One, item-to-item collaborative filtering, user-based recommendations, and item-based recommendations. I would also recommend looking through a library such as Mahout for examples of the algorithms that they provide.
That's It!
I feel like I barely scratched the surface of this intensely interesting topic, but I hope that it was enough to give you a feel for the domain and pique your interest to continue looking into Recommender Systems. The really cool thing about Recommender Systems is that you can get one up-and-running in a matter of minutes (no seriously, there are tons of tutorials like this), and then spend years tuning the architecture and algorithms...the old "minute to learn, lifetime to master" idea.
There are plenty of libraries, algorithms, and issues that I didn't know about or didn't have time to go into. Let me know your thoughts, and then get out there and start making the world's applications provide better experiences - billions of people are looking forward to it!
