JuliaCon 2022 (Times are UTC)

This talk demonstrates Recommendation.jl, a Julia package for building recommender systems, with a special emphasis on its design principle and evaluation framework. While the package was first presented at JuliaCon 2019 to collect early feedback from the community, this talk highlights how the implementation has evolved afterwards and gives a preview of upcoming "v1.0.0" major release, accompanied by a proceeding paper.

An underlying question for the audiences throughout the talk is: How should "good" recommender systems be? On one hand, improving the accuracy of recommendation with sophisticated algorithms is indeed desired. However, at the same time, recommendation is not always the same as machine learning problems, and non-accuracy aspects of the systems are equally or even more important in practice; we particularly discuss the importance of decoupling data from business logic and validating data/model quality based on a diverse set of decision criteria.

First of all, a core of recommendation engine largely relies on simple math and matrix computation against sparse user-item data, where we can take full advantage of numerical computing methods. Thus, Julia is a great choice to efficiently and effectively implement an end-to-end recommendation pipeline that typically consists of multiple sub-tasks as follows:

1. preprocessing user-item data;
2. building a recommendation model;
3. evaluating a ranked list of recommended contents;
4. post-processing the recommendation.

Here, Recommendation.jl provides a unified abstraction layer, namely DataAccessor, which represents user-item interactions in an accessible form. Since data for recommender systems is readily standardizable as a collection of user, item, and contextual features, the common interface helps us to follow the separation of concerns principle and ensure the easiness and reliability of data manipulation. To be more precise, raw data is always converted into a DataAccessor instance at the data preprocessing phase (Phase#1) with proper validation (e.g., data type check, missing value handling), and hence the subsequent steps can simply access the data (or metadata) through the instance without worrying about unexpected input.

Moreover, when it comes to generating recommendations at later phases (Phase#2-4), Recommendation.jl enables developers to optimize recommenders against not only standard accuracy metrics (e.g., recall, precision) but non-accuracy measures such as novelty, diversity, and serendipity. Even though the idea of diverse or serendipitous recommendation is not new in the literature, the topic has rapidly gained traction as society realizes the importance of fairness in intelligent systems. In this talk, we dive deep into the concept of these non-accuracy metrics and their implementation in Julia.

Last but not least, there are a couple of new recommendation models recently added to the package, including matrix factorization with Bayesian personalized ranking loss and factorization machines. We plan to provide comprehensive benchmark results for supported recommender-metric pairs to undergo trade-off discussion. Furthermore, we compare Recommendation.jl with other publicly available recommendation toolkit like LensKit (Python), MyMediaLite (C#), and LibRec (Java).