What is the goal of a recommender system?

In a paper I was skimming this week I read something to the effect of "ultimately, the goal of a recommender system is to correctly rank items for a user". I'm not going to link the paper for several reasons, the biggest of which is that I can't remember which paper it was. For some reason, this statement really bugged me. Just like it isn't true to say that the goal of a recommender system is to perfectly predict future user ratings, or to maximize clickthrough, it isn't true to say that the goal is to perfectly rank items.

Let me start by trying to understand what the authors were trying to say with their statement. I think the authors were trying to focus their work on the recommendation task in recommender systems. While the name "recommender system" might make you think that recommendation is the only task, for years researchers had focused on algorithms for the rating prediction task. As time passed however, we became quite good at the rating prediction task. With the help of several influential papers, the research community has been moving away from rating prediction based algorithms. In this light, I think the author was trying to frame their work in the ranking/recommending task, and explain why they will not discuss rating prediction. As a research goal: "solve the ranking problem" is fine, as a goal for the whole field I find it not acceptable.

Here are a few of the problems I have with the statement:

• Ranking is not Recommending: The ranking problem: given two items correctly predict which item a user might prefer. The recommendation problem: for a given user and a given size "n" create a list (ordered) of n items that is best for the user. While these problems are related, they are not the same. A lot of research has gone into factors like novelty, diversity and popularity of recommendation lists that show that a good recommendation list is more than just a ranked order list of the best items for a user. A recommendation succeeds or fails as a whole, and as far as I am aware a recommendation must be optimized as a whole as well. Maybe next week I'll try to review an interesting paper on this subject.

• There is no such think as perfect: A keen reader may have noticed in my definition of the recommendation task I used the phrase "best for the user". This is a lot less formal than "correctly predict which item a user might prefer". The problem here is that to my knowledge we have not came up with a truly comprehensive definition of what a "best" recommendation could be. A lot of this leads back to the novelty and diversity work I talked about earlier. This work has a proud record of comparing two algorithms and showing that the one that a given metric prefers is worse for users than one that has more diversity (or any number of other properties).

• Different deployments call for different algorithms: Every system is unique, and attracts different types of users. In one systems a lot of users may look at rating predictions before making purchase or consumption predictions. In another there are no ratings and users only get personalized recommendations based on past use. Each of these systems are recommender systems, but they each call for a different trade off between capabilities in algorithms. Claiming that there is one true goal in this domain becomes somewhat difficult.

Ultimately, the goal of a recommender is not to predict ratings, order items, or form recommendations. The goal of a recommender system is to help users. We have rating, ranking, and recommendation tasks as a simplification of this goal that helps us focus, derive metrics, and advance the state of the art in recommender algorithms. These tasks are important, but we should never mistake them for the true goal of a recommender system

I spent a some time this week trying to find my own statement for the goal of a recommender system. What I came up with is "Assist users in accessing and understanding large digital collections in domains subject to significant personal taste." I originally had toyed with goals related to helping users find better items, consume more good items, or find desired items faster. Ultimately I decided to avoid these value judgments and stick to the slightly more vague "assist". Likewise, since some users like to use recommendations as a way to browse, I included access and understand as tasks. Finally, to separate recommender systems from general information retrieval, I constrain the definitions to domains subject to significant personal taste, such as movies, music, and books. I'm not sure that this is the one true definition, but I think its a step in the right direction. Feel free to contact me by twitter or email if you think I missed something in my definition, I would love to discuss your ideas.