High sparsity and the problem of overspecialization are challenges faced
by collaborative filtering (CF) algorithms in recommender systems. In
this paper, we design an approach that efficiently tackles the above
problems. We address the first issue of high sparsity in CF by modifying
the popular parallel seeding technique proposed by Bahmani et al. for
use in a spherical setting, for the clustering of users. Experimental
evaluations on highly sparse real world datasets demonstrate the better
performance of our algorithm than existing Spherical K-Means algorithms.
Contrary to the common belief that users are only interested in items
similar to those of their previous liking, it has been well established
that including serendipitous recommendations improves their
satisfaction. Thus, to tackle the second problem of overspecialization,
we effectuate serendipity in movie recommender systems with an
end-to-end algorithm, Serendipitous Clustering for Collaborative
Filtering (SC-CF) that considers diversity, unexpectedness and
relevance. SC-CF takes advantage of carefully generated user profiles to
then assign users to a ``serendipitous cluster.'' The overall clustering
process leverages these user profiles and improves recommendation
quality through serendipity. A series of experiments on Serendipity
2018 (part of Movielens) dataset built on real user feedback has shown
that SC-CF outperforms the existing popular recommendation methods like
K-Means and deep learning based CF.