Neurocomputing 207 (2016) 501–510

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

Group Bayesian personalized ranking with rich interactions for one-class collaborative filtering$, $$ Weike Pan a,b,n, Li Chen b a b

College of Computer Science and Software Engineering, Shenzhen University, China Department of Computer Science, Hong Kong Baptist University, Hong Kong

art ic l e i nf o

a b s t r a c t

Article history: Received 29 July 2015 Received in revised form 5 May 2016 Accepted 9 May 2016 Communicated by Peng Cui Available online 16 May 2016

Both researchers and practitioners in the field of collaborative filtering have shown keen interest to user behaviors of the “one-class” feedback form such as transactions in e-commerce and “likes” in social networks. This recommendation problem is termed as one-class collaborative filtering (OCCF). In most of the previous work, a pairwise preference assumption called Bayesian personalized ranking (BPR) was empirically proved to be able to exploit such one-class data well. In one of the most recent work, an upgraded model called group preference based BPR (GBPR) leverages the group preference and obtains better performance. In this paper, we go one step beyond GBPR, and propose a new and generic assumption, i.e., group Bayesian personalized ranking with rich interactions (GBPR þ ). In our GBPR þ , we adopt a set of items instead of one single item as used in GBPR, which is expected to introduce rich interactions. GBPR is a special case of our GPBR þ when the item set contains only one single item. We study the empirical performance of our GBPR þ with several state-of-the-art methods on four real-world datasets, and find that our GPBR þ can generate more accurate recommendations. & 2016 Elsevier B.V. All rights reserved.

Keywords: One-class collaborative filtering Implicit feedback Group pairwise preference Item set

1. Introduction In industry, recommender system as a critical engine in various online entertainments [10] and shopping services [24] has caught much attention and contributed significantly in revenue growth in recent years. Lots of internet, electronic and telecom giants embed recommendation technologies in their existing systems, e.g., YouTube1 and Amazon,2 in order to increase user engagement and earn more revenues from sales of product or advertisement. In academia, most research studies [2,6] on recommendation are biased towards the numerical rating prediction problem associated with the Netflix competition,3 partially due to the public availability ☆ This work is an extension of our previous work [35]. Compared with our previous work [35], we have added the following new contents in this paper, (1) we have developed an extension of GBPR (i.e., GBPR þ ) in Section 4; (2) we have included new experimental results (i.e., five additional state-of-the-art baselines in Table 4, more detailed results in Figs 5 and 6, and new results in Fig. 7) and associated analyses in Section 5; (3) we have added more related work and discussions in Sections 1 and 2; and (4) we have made many improvements on the Abstract, Introduction, illustrations in Figs 2 and 3, and result presentation. ☆☆ Some of this work was done while Weike Pan was a post-doctoral research fellow in the Department of Computer Science, Hong Kong Baptist University. n Corresponding author. E-mail addresses: [email protected] (W. Pan), [email protected] (L. Chen). 1 http://www.youtube.com/. 2 http://www.amazon.com/. 3 http://www.netflixprize.com/.

http://dx.doi.org/10.1016/j.neucom.2016.05.019 0925-2312/& 2016 Elsevier B.V. All rights reserved.

of the data. The Netflix contest can be categorized as a “multi-class” recommendation problem, where the inputs are categorical scores or classes. For example, the input data of Netflix are numerical ratings with “1” for bad, “2” for fair, “3” for good, “4” for excellent, and “5” for perfect, for which various algorithms have been proposed in order to predict the users’ preference scores accurately. Such categorical data contain both positive feedback of “4” and “5” and negative feedback of “1” and “2”. For such data, some regression or ranking loss functions were designed to fit the scores or to preserve the ordering. Matrix-factorization based models have been shown to be the most effective solutions [20,21,39,48], which makes use of these loss functions. There are also some work that exploits both multi-class ratings and additional information [33,55], such as social connections, interests and behaviors [16,25,38,53], content [1,3], taxonomy [29] and context [44,49], and adopts more than one recommendation techniques [14,17,26]. However, in most applications, user behavior data are in “oneclass” form rather than in multi-class form, e.g., “like” in Facebook, “bought” in Amazon, and “click” in Google Advertisement. Such data are called implicit [28,41] or one-class [32] feedback. We illustrate the problem in Fig. 1, where some users have expressed positive feedback on some movies. This kind of one-class collaborative filtering problem is different from that of the 5-star rating prediction problem encountered in the Netflix competition, because the former only contains positive feedback compared to both positive feedback and negative feedback used in the

502

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

Fig. 1. An illustrative example of the one-class collaborative filtering (OCCF) problem. The observed data only contains positive feedback (e.g., “like”) from users on items.

competition. Furthermore, the goal of OCCF is to rank items rather than predict ratings. For solving the one-class collaborative filtering problem, previous matrix-factorization based algorithms can be roughly summarized into two categories, (1) pointwise regression methods and (2) pairwise ranking methods. The former methods learn a latent representation of both users and items by minimizing a pointwise square loss [13,32] in order to fit the absolute rating scores, while the latter methods take pairs of items as basic units and maximize the likelihood of pairwise preference over observed items and unobserved items [41]. Empirically, the pairwise ranking method [41] achieves much better performance than pointwise methods [13,32], and has been successfully adopted in many applications, e.g., tag recommendation [43], news recommendation [50], and shopping products recommendation [19]. Notice that there are also some work that generalize pairwise ranking to listwise ranking [45,46], though it is of low efficiency. In this paper, we study the two fundamental assumptions made in the seminal work of pairwise ranking method [41], i.e., (1) individual pairwise preference over two items and (2) independence between two users, and point out that they may not always hold in real applications. As a response, we introduce group pairwise preference instead of individual pairwise preference in [41], in order to inject rich interactions among users and thus relax the individual and independence assumptions. We then propose a new and improved assumption called group Bayesian personalized ranking with rich interactions (GBPR þ ) and design an efficient algorithm correspondingly. Notice that GBPR þ is extended from our previous conference work GBPR [35]. Empirically, we find that our new assumption can make use of the one-class data more effectively and achieves better recommendation performance on all the four real-world datasets in our experiments. We organize the rest of the paper as follows. We discuss some closely related work on one-class collaborative filtering in Section 2. We present the background and limitations of the existing pairwise preference learning methods in Section 3. We formally propose our assumption and algorithm in Section 4, and then study its empirical performance in Section 5. Finally, we give some concluding remarks in Section 6.

2. Related work In this section, we briefly discuss some closely related work on one-class collaborative filtering in two branches, including

(1) pointwise methods with absolute preference assumptions, and (2) pairwise methods with relative preference assumptions. Pointwise methods with absolute preference assumptions: Pointwise methods take implicit feedback as absolute preference scores. For example, an observed user–item pair, (u,i), is interpreted as that user u likes item i with a high absolute score, e.g., weighted matrix factorization (WMF) [31,32], pure singular value decomposition (PureSVD) [9], sparse linear models (SLIM) [30], factored item similarity model with RMSE (FISMrmse) [18] and implicit matrix factorization (IMF) [13] are typical approaches for solving this recommendation problem. WMF [32] uses different sampling strategies for unobserved user–item pairs and takes them as negative feedback to augment the observed positive feedback, so that existing matrix factorization methods can be applied. PureSVD [9] bypasses the sampling step in WMF [32] and takes all unobserved user–item pairs as negative feedback, which are then combined with observed pairs and fed to the mathematical tool of singular value decomposition (SVD) [5,11]. SLIM [30] and FISMrmse [18] learn similarities between items instead of calculate them via some predefined measurement such as cosine similarity, where the former is associated with ℓ1 regularization and the latter with factored latent features. IMF [13] introduces confidence weights on implicit feedback, which can then be approximated by two latent feature matrices. However, the limitation of WMF [32], PureSVD [9], SLIM [30] and FISMrmse [18] is that taking unobserved user–item pairs as negative feedback may introduce errors. As for IMF [13], it requires auxiliary knowledge of confidence for each observed feedback, which may not be available in real applications. There are also some non-negative matrix factorization (NMF) [23] based pointwise methods for one-class collaborative filtering, e.g., NMF with a low density assumption [47], NMF with side information of social trust and item content [51], etc. Pairwise methods with relative preference assumptions: Pairwise methods take implicit feedback as relative preference rather than absolute ones, e.g., a user u is assumed to prefer an item i to an item j if the user–item pair (u,i) is observed, and (u,j) is not observed [41]. The proposed algorithm, Bayesian personalized ranking (BPR) [41], is the first method with such pairwise preference assumption for addressing the one-class collaborative filtering problem. Due to the great success of pairwise methods in various oneclass collaborative filtering problems, some new algorithms have been proposed to combine BPR with some auxiliary data, such as BPR with temporal information [42], BPR with user-side social connections [54], and BPR with item-side taxonomy [19], etc. There are also some work that (1) extend BPR from two dimensions to three dimensions [43], from one user–item matrix to multiple ones [22,37], (2) associate some sophisticated sampling strategies with BPR [40,52], or (3) revise the loss function [27,45,46] in BPR. However, the limitation of pairwise methods can be attributed to the two fundamental assumptions made in BPR, namely individual pairwise assumption over two items and independence assumption between two users. Most follow-up work do not refine the fundamental assumptions, but just directly adopt the BPR criterion in their own applications. A recent algorithm [34] generalizes BPR via proposing a new assumption that an individual user is likely to prefer a set of observed items to a set of unobserved items. Compared with the aforementioned work, our proposed group Bayesian personalized ranking with rich interactions (GBPR þ ) is a novel algorithm in one-class collaborative filtering. In particular, GBPR þ inherits the merit of pairwise methods, and improves the two fundamental assumptions in BPR via introducing group pairwise preference. Notice that GBPR þ is a more generic version of our previous conference work GBPR [35], and it becomes equivalent to GBPR when the item set is of one single item. We summarize GBPR þ and the aforementioned related work in Table 1.

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

Table 1 Summary of GBPR þ and other methods for one-class collaborative filtering w.r.t. different preference assumptions. Preference assumption

Typical work

Absolute Relative (individual) Relative (group)

WMF [32], PureSVD [9], etc. BPR [41], etc. GBPR [35] and GBPR þ (proposed in this paper)

503

where i A I tr u means that the user–item pair (u,i) is observed, and jA I tr ⧹I tr u means that the user–item pair (u,j) is not observed. 2. Assumption of independence among users: It assumes that the joint likelihood of pairwise preference of two users, u and w, can be decomposed as BPRðu; wÞ ¼ BPRðuÞBPRðwÞ, which means that the likelihood of pairwise preference of user u is independent of that of user w. With this assumption, the overall likelihood among the users can be represented as follows [41]: BPR ¼ ∏ BPRðuÞ:

3. Background

u A U tr

In this section, we first give the problem definition, and then introduce the likelihood of pairwise preference and the two fundamental assumptions made in BPR [41]. 3.1. Problem definition We use U tr ¼ fugnu ¼ 1 and I tr ¼ figm i ¼ 1 to denote the sets of users and items, respectively. For each user u A U tr , we have a set of tr on which user u has expressed positive feedback, items I tr u DI e.g., “like” (see Fig. 1 for an illustration). Our goal is then to recommend each user u a personalized ranking list of items from I tr ⧹I tr u . As mentioned before, this problem has been steadily receiving more attentions, and is called one-class collaborative filtering [32] or collaborative ranking with implicit feedback [41].

However, the above two assumptions may not always hold in real applications. First, a user u may potentially prefer an item j to an item i, though the user u has expressed positive feedback on item i instead of on item j. For example, in Fig. 1, John may like Prince of Egypt more than Forrest Gump though we have only observed positive feedback on the latter yet. Notice that the assumption of individual pairwise preference over two items (as made in BPR [41]) assumes that John will prefer an observed movie to an unobserved movie. Second, two users, u and w, may be correlated, and their joint likelihood may thus not be decomposed into two independent likelihoods, e.g., John and Rebecca in Fig. 1.

4. Our solution

3.2. Likelihood of pairwise preference

4.1. Group pairwise preference

In order to represent a user u's relative preference on two items i and j, Rendle et al. [41] use a binary random variable δððu; iÞ g ð u; jÞÞ to denote whether user u prefers item i to item j or not. The function δðzÞ is a binary indicator with δðzÞ ¼ 1 if the equation z is true, and δðzÞ ¼ 0 otherwise. This representation is called a user's pairwise preference and has dominated in one-class collaborative filtering tasks in recent studies [19,41,54]. For a typical user u, in order to calculate the overall likelihood of pairwise preference (LPP) among all items, Bernoulli distribution over the binary random variable δððu; iÞ g ðu; jÞÞ is used in [41],

As a response to the possible violations of the two fundamental assumptions made in BPR [41], we propose a new assumption and introduce rich interactions among users via group preference. In the following, we first describe two definitions before introducing our new assumption. Definition (Individual preference). The individual preference is a preference score of a user on an item. For example, the individual preference of user u on item i is denoted as r^ ui . Definition (Group preference). The group preference is an overall preference score of a group of users on an item. For example, the group preference of users from group PG on item i can 1 ^ be estimated from individual preference, r^ Gi ¼ j Gj w A G r wi . Notice that our primary goal is to recommend items for a single user, not for a group of users [4]. We assume that the group preference of group G D U tr i on item i is more likely to be stronger than the individual preference of user u on item j, if the user–item pair ðu; iÞ is observed and the user– item pair ðu; jÞ is not observed. The group pairwise preference can then be written conceptually,

LPPðuÞ ¼ ∏ Prðr^ ui 4 r^ uj Þδððu;iÞ g ðu;jÞÞ i;j A I tr

½1  Prðr^ ui 4 r^ uj Þ½1  δððu;iÞ g ðu;jÞÞ ¼

∏

Prðr^ ui 4 r^ uj Þ

ðu;iÞ g ðu;jÞ



∏

Prðr^ ui 4 r^ uj Þ

ðu;iÞ g ðu;jÞ

¼

∏

∏

½1  Prðr^ ui 4 r^ uj Þ

ðu;iÞ⪯ðu;jÞ

∏

½1  Prðr^ ui 4 r^ uj Þ

ðu;iÞ ! ðu;jÞ

Prðr^ ui 4 r^ uj Þ½1  Prðr^ uj 4 r^ ui Þ

ðu;iÞ g ðu;jÞ

ðG; iÞ g ðu; jÞ; where ðu; iÞ g ðu; jÞ denotes that user u prefers item i to item j, and ðu; iÞ⪯ðu; jÞ denotes that user u does not prefer item i to item j. Notice that the right side of the math symbol “ E” in the above equation is obtained by dropping cases “ðu; iÞ ¼ ðu; jÞ”.

where G DU tr i and u A G. Our assumption can be interpreted from two aspects,

 For items, it is more likely to be true if user u can find some

3.3. Bayesian personalized ranking The two fundamental assumptions adopted by the method Bayesian personalized ranking (BPR) [41] are: 1. Assumption of individual pairwise preference over two items: It assumes that a user u prefers an item i to an item j, ðu; iÞ g ðu; jÞ, if the user–item pair ðu; iÞ is observed and ðu; jÞ is not observed. With this assumption, LPPðuÞ can be simplified to BPRðuÞ [41], BPRðuÞ ¼ ∏

∏

tr tr i A I tr u j A I ⧹I u

Prðr^ ui 4 r^ uj Þ½1  Prðr^ uj 4 r^ ui Þ;

ð1Þ



other users’ support on his pairwise preference on item i and item j. This is reflected by the replacement of the individual pairwise relationship ðu; iÞ g ðu; jÞ with a new one ðG; iÞ g ðu; jÞ that involves the group preference. For users, it is natural to introduce interactions and collaborations among users who are all with positive feedback on a specific item, since that implies common interests of those users. This is reflected in the group of like-minded users, G D U tr i , who share the same positive feedback to item i.

Notice that we do not introduce interactions among users who do not have positive feedback on a certain item, because the space of

504

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

unobserved items is usually large and we may not infer some correlation of those users' interest. To explicitly study the unified effect of group preference and individual preference, we combine them linearly, ðG; iÞ þ ðu; iÞ g ðu; jÞ

or

r^ Gui 4 r^ uj

ð2Þ

where r^ Gui ¼ ρr^ Gi þ ð1  ρÞr^ ui is the fused preference of group preference r^ Gi and individual preference r^ ui . Notice that 0 r ρ r 1 is a tradeoff parameter used to fuse the two preference, which can be determined via empirically testing a validation set. So far, we have reached the assumption of group Bayesian personalized ranking (GBPR) [35]. We put the individual and group preference used in the paper in Table 2. In order to further introduce rich interactions, we propose to use a set of unobserved items S instead of one single item j, r^ Gui 4 r^ uj ;

j A S; tr

specifically, for any two users u and w who have the same positive feedback to an item i, the corresponding user groups Gðu; iÞ D U tr i and Gðw; iÞ D U tr i are likely to be overlapped, i.e., Gðu; iÞ \ Gðw; iÞ a∅. Then, we have the following overall likelihood for all users and all items: GBPR þ  ∏

GBPR þ ¼ ∏

With the above group pairwise preference, we have a new criterion called GBPR þ for user u, GBPR þ ðuÞ ¼ ∏

∏

∏ Prðr^ Gui 4 r^ uj Þ½1  Prðr^ uj 4 r^ Gui Þ;

tr tr j A S i A I tr u S D I ⧹I u

where G D U tr i is a user group, item i is observed by user u, and items in S are not observed. For any two users, u and w, the joint likelihood can then be approximated via multiplication, GBPR þ ðu; wÞ  GBPR þ ðuÞGBPR þ ðwÞ, since the user correlations have been introduced via the user group G already. More Table 2 Preference notations used in the paper. Preference

Description

r^ ui ¼ U u V Ti þ bi r^ uj ¼ U u V Tj þ bj

Individual preference of user u on item i

1 P r^ Gi ¼ r^ j Gj w A G wi r^ Gui ¼ ρr^ Gi þ ð1  ρÞr^ ui r^ uij ¼ r^ ui  r^ uj r^ Gui;uj ¼ r^ Gui  r^ uj

Group preference of users from group G on item i

Individual preference of user u on item j

Fused preference of r^ Gi and r^ ui Pairwise preference of user u Pairwise preference of user u and group G

∏ Prðr^ Gui 4 r^ uj Þ½1  Prðr^ uj 4 r^ Gui Þ;

ð4Þ

∏

∏

∏ σ 2 ðr^ Gui  r^ uj Þ:

ð5Þ

tr tr j A S u A U tr i A I tr u S D I ⧹I u

ð3Þ

4.2. Group Bayesian personalized ranking with rich interactions

∏

where G D U tr i . Following [41], we use σ ðr^ Gui  r^ uj Þ ¼ 1 þ expð 1r^ þ r^ Þ to approxGui uj imate the probability Prðr^ Gui 4 r^ uj Þ, and have Prðr^ Gui 4 r^ uj Þ½1  Prðr^ uj 4 r^ Gui Þ ¼ σ ðr^ Gui  r^ uj Þ½1  σ ðr^ uj  r^ Gui Þ ¼ σ 2 ðr^ Gui  r^ uj Þ. The overall likelihood is as follows:

Finally, we reach the objective function of our GBPR þ ,

⧹I tr u

where S DI is a randomly sampled subset of unobserved items. Using a set of items can not only introduce more opportunities for parameter learning but also inject rich interactions, which is also supported by our empirical studies. And for this reason, we call our assumption group Bayesian personalized ranking with rich interactions (GBPR þ ), which is illustrated via a toy example in Fig. 2.

∏

tr tr j A S u A U tr i A I tr u S D I ⧹I u

min  Θ

1 1 lnGBPR þ þ RðΘÞ; 2j Sj 2

ð6Þ

set of where Θ ¼ fU u A R1d ; V i A R1d ; bi A R; u A U tr ; iA I tr gPis a P model parameters to be learned, ln GBPR þ ¼ u A U tr i A I tru P P þ ^ ^ S D I tr ⧹I tr j A S 2lnσ ðr Gui  r uj Þ is the log-likelihood of GBPR , Pu P P P RðΘÞ ¼ u A U tr i A I tru S D I tr ⧹I tru ½αu w A G J U w J 2 þ αv J V i J 2 þ αv P P 2 2 2 j A S J V j J þ β v J bi J þ β v j A S J bj J  is the regularization term used to avoid overfitting, and G D U tr i is a group of like-minded users who share the same positive feedback to item i. Notice that the difference between our GBPR þ and GBPR [35] is the item set S, which is designed to bring in more interactions. We show the graphical model of GBPR þ in Fig. 3, where the individual preference is generated via r^ ui ¼ U u V Ti þ bi , r^ uj ¼ U u V Tj þ bj , and group preference r^ Gi ¼ U G V Ti þ bi with P U G ¼ w A G U w =j Gj . The main difference between GBPR þ and BPR [41] is the first term in Eq. (6), which introduces rich interactions among users via the user group G and item set S, and as a consequence relax the two fundamental assumptions made in BPR [41]. We show the relationship between GBPR þ , GBPR [35] and BPR [41] as follows: j Sj ¼ 1

j Gj ¼ 1

GBPR þ ⟶ GBPR ⟶ BPR;

ð7Þ þ

from which we can see that GBPR is a generalization of GBPR and BPR with group preference and rich interactions. We can reach GBPR and BPR from GBPR þ with special values of j Gj and j Sj , but not vice versa. Once we have learned the model parameters Θ, we can predict the preference of user u on item j via r^ uj ¼ U u V Tj þ bj , which can then be used to generate a personalized ranking list for user u via picking up the top-k items with largest preference scores.

Fig. 2. Illustration of group pairwise preference over an observed item and a set of unobserved items. We assume that the group preference (shown in oval) of John, Jacky and Rebecca on movies Forrest Gump is stronger than the individual preference of John on movie Prince of Egypt, A Beautiful Mind and Jurassic Park, since there is positive feedback on movie Forrest Gump from all those three like-minded guys. Notice that we use a set of unobserved items in GBPR þ instead of one unobserved item as that in GBPR [35].

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

505

Fig. 3. Graphical model of group Bayesian personalized ranking with rich interactions (GBPR þ ). Notice that GBPR [35] is a special case of GBPR þ with j S j ¼ 1.

4.3. Learning the GBPR þ We follow the widely used stochastic gradient descent (SGD) algorithm to optimize the objective function in Eq. (6). We go one step beyond SGD, and randomly sample a subset of like-minded users to construct the user group G, and a subset of unobserved items S. For each randomly sampled record, it consists of a user u, an item i, a user group G, and an item set S. The tentative objective function P can then be written as f ðG; u; i; SÞ ¼  j S1 j j A S lnσ ðr^ Gui  r^ uj Þ þ α2u P P βv βv P 2 2 2 2 2 αv αv w A G J U w J þ 2 J V i J þ 2 j A S J V j J þ 2 J bi J þ 2 j A S J bj J P P 2 2 α α 1 u v ^ ¼ j S j j A S ln½1 þ expð  r Gui;uj Þ þ 2 w A G J U w J þ 2 J V i J þ βv βv P 2 2 2 αv P ^ ^ ^ j A S J V j J þ 2 J bi J þ 2 j A S J bj J , where r Gui;uj ¼ r Gui  r uj is 2 the difference between the fused preference r^ Gui and individual preference r^ uj . We then have the gradients of the user-specific parameters w.r. t. the tentative objective function f ðG; u; i; SÞ,   8 P ∂f ðG; u; i; SÞ V i > >  V ð1  ρ ÞV þ ρ þ αu U w ; if u ¼ w > i j > < j A S ∂r^ Gui;uj j Gj ∇U w ¼ P ∂f ðG; u; i; SÞ V i > > > þ αu U w =j Gj ; ρ if u a w; > : j A S ∂r^ Gui;uj j Gj ð8Þ and the gradients of the item-specific parameters, ∇V i ¼

X∂f ðG; u; i; SÞ ½ð1  ρÞU u þ ρU G  þ αv V i ; ∂r^ Gui;uj jAS

∂f ðG; u; i; SÞ ð  U u Þ þ αv V j ; ∂r^ Gui;uj X∂f ðG; u; i; SÞ 1 þ β v bi ; ∇bi ¼ ∂r^ Gui;uj jAS ∇V j ¼

∇bj ¼

∂f ðG; u; i; SÞ ð  1Þ þ β v bj ; ∂r^ Gui;uj expð  r^

∂f ðG; u; i; SÞ ; ∂θ

of iterations and n is the number of users. We can see that introducing interactions in GBPR þ does not increase the time complexity much, because j Gj and j Sj are usually small, e.g., j Gj ; j Sj r5 in our experiments. For predicting a user's preference on an item, the time complexity is O(d), which is the same as that of BPR. Thus, GBPR þ can be comparable to the seminal work BPR [41] and our previous conference work GBPR [35] in terms of efficiency. In the experiments, we mainly assess whether GBPR þ would be more accurate than BPR by introducing rich interactions.

jA S; 5. Experimental results 5.1. Datasets

j A S; Þ

¼  j S1 j 1 þ expð Gui;uj , w A G, and j A S. With the above where ∂f∂ðG;u;i;SÞ r^ Gui;uj Þ r^ Gui;uj gradients, we can update the model parameters as follows:

θ ¼ θγ

Fig. 4. The algorithm of group Bayesian personalized ranking with rich interactions (GBPR þ ).

ð9Þ

where θ can be U w , V i , V j , bi or bj, w A G, jA S, and γ 40 is the learning rate. Notice that when S ¼ fjg and G ¼ fug, GBPR þ is reduced to BPR [41], which does not explicitly incorporate interactions among like-minded users. The complete steps to learn the model parameters are depicted in Fig. 4. The time complexity of the update rule in Eq. (9) is Oðj Gj j Sj dÞ, where j Gj and j Sj are the sizes of user group and item set, respectively, and d is the number of latent features. The total time complexity is then OðTnj Gj j Sj dÞ, where T is the number

We use four real-world datasets in our empirical studies, including MovieLens100K4, MovieLens1M, UserTag [32] and a subset of Netflix. MovieLens100K contains 100,000 ratings assigned by 943 users on 1682 movies, MovieLens1M contains 1,000,209 ratings assigned by 6040 users on 3952 movies, and UserTag contains 246,436 user–tag pairs from 3000 users and 2000 tags. We randomly sample 5000 users from the user pool and 5000 items from the item pool of the Netflix dataset, and obtain 282,474 ratings by those 5000 users on those 5000 items. We call this subset of Netflix dataset NF5K5K. We use “item” to denote movie (for MovieLens100K, MovieLens1M and NF5K5K) or tag (for UserTag). For MovieLens100K, MovieLens1M and NF5K5K, 4

http://grouplens.org/datasets/movielens/.

506

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

 Random Walk (denoted as RandomWalk) first calculates the

Table 3 Description of the datasets used in the experiments. Dataset

User–item pairs

MovieLens100K

Training Test Training Test Training Test Training Test

MovieLens1M UserTag NF5K5K

27,688 27,687 287,641 287,640 123,218 123,218 77,936 77,936

we take a pre-processing step [47], which only keeps the ratings larger than 3 as the observed positive feedback (to simulate the one-class feedback). For all the four datasets, we randomly sample half of the observed user–item pairs as training data, and the rest as test data; we then randomly take 1 user–item pair for each user from the training data to construct a validation set. We repeat the above procedure for three times, so we have three copies of training data and test data. The final datasets used in the experiments are shown in Table 3, which is also publicly available.5 The experimental results are averaged over the performance on those three copies of test data. 5.2. Evaluation metrics

P

 Rec@k ¼ j U1 j

P

te

k u A U te j I u

te

 F1@k ¼ j U1 j

P

te

k u A U te j I u

k u A U te 2j I u

 NDCG@k ¼ j U1 j

P

te

 1call@k ¼ j U1 j te

 AUC ¼ j U1 j

P

\ I te u j =k; te \ I te u j =j I u j ; te \ I te u j =ðj I u j þ kÞ;

1 u A U te Pminðk;j I te jÞ u ℓ ¼ 1

P uAU

te

Pk 1 log ðℓ þ 1Þ

ℓ¼1

ℓ

te

2δðiu A I u Þ  1 ; log ðℓ þ 1Þ

δðj I ku \ I teu j Z 1Þ; and

P

δ

1 ^ ^ u A U te j PðuÞj ði;jÞ A PðuÞ ðr ui 4 r uj Þ, tr tr fði; jÞj i A I te = I te u ; j2 u [ I u g, where I u denotes te



 



Because users usually only check a few top-ranked items [8], we use top-k evaluation metrics to study the recommendation performance, including top-k results of precision, recall, F1, NDCG [15] and 1-call [7]. Since BPR [41] optimizes the AUC [12] criterion, we also include it in our evaluation. For each evaluation metric, we first calculate the performance of each user from the test data, and then obtain the averaged performance over all users. Specifically, assuming the observed items of each user u in the test user set U te is denoted as I te u , and the items ranked among top k positions by a learned model are I ku ¼ fiℓu j 1 r ℓ rkg, we then have the evaluation formulas as follows:

 Pre@k ¼ j U1 j



where PðuÞ ¼ the set of observed

items by user u in the training data. 5.3. Baselines and parameter settings We use several popular baseline algorithms in our experiments, including PopRank, RandomWalk, MemoryBasedCF, PureSVD [9], wiZAN [51], SLIM [30], BPR [41] and GBPR [35]. We describe the baselines below:

 PopRank is the basic algorithm in one-class collaborative filtering, which ranks the items according to their popularity in the training data.



probability of walking from an active user to any other user, and then estimates the active user's preference on a certain item via a weighted average of all reachable users' preference on that item. Memory-based Collaborative Filtering (denoted as MemoryBasedCF) contains two steps of neighborhood searching via Jaccard index,6 and preference prediction via aggregation of the most similar neighboring users' preference. We use 20 neighboring users in the experiments. PureSVD [9] is a recent method based on the well-known mathematical tool of singular value decomposition [5,11], which factorizes the constructed zero-one matrix (one for an observed feedback and zero for an unobserved entry) into three latent matrices. The value of each user–item pair can then be estimated via latent-matrix multiplication, which can further be used to rank the items for each user for personalized recommendation. Weighting and Imputation based Zeros As Negative (denoted as wiZAN) [51] is a recent method for one-class collaborative filtering, which is based on non-negative matrix factorization. Sparse Linear Models (SLIM) [30] is a state-of-the-art method for one-class collaborative filtering, which takes unobserved user–item pairs as negative feedback and introduces both ℓ1 and ℓ2 regularization terms in order to learn a sparse item–item similarity matrix. We adopt the SLIM source code7 to learn a sparse item–item similarity matrix, which is then used to generate a personalized ranking list of items for each user. BPR [41] is a seminal work for this problem and is also a strong baseline, which is shown to be significantly better than two wellknown pointwise methods, i.e., IMF [13] and WMF [32]. Lots of follow-up work directly adopted the pairwise assumption in BPR, and witnessed promising results in various applications [19,43]. GBPR [35] is a generic pairwise preference learning method with group preference, which absorbs BPR [41] as a special case when the user group is replaced by a single user.

In this paper, we extend GBPR [35] via introducing rich interactions through item sets and design a novel algorithm, i.e., GBPR þ . For fair comparison, BPR, GBPR and GBPR þ are all implemented using Java in the same code framework as shown in Fig. 4. For BPR and GBPR, the tradeoff parameters are searched as αu ¼ αv ¼ β v A f0:001; 0:01; 0:1g and ρ A f0:2; 0:4; 0:6; 0:8; 1g, and the iteration number is chosen from T A f1000; 10000; 100000g. The NDCG@5 performance on the validation data is used to select the best parameters αu, αv, βv and ρ, and the best iteration number T for both BPR and GBPR. The learning rate in BPR and GBPR is fixed as γ ¼ 0:01. The number of latent dimensions in PureSVD, BPR, GBPR and GBPR þ is fixed as d¼ 20. The initialization value of U u , V i , bi in BPR, GBPR and GBPR þ are set the same as in [36]. For GBPR þ , we use the same values of αu ; αv ; βv , ρ, T and γ as that for GBPR. For the user group G in GBPR and G; S in GBPR þ , we first fix the size as j Gj ¼ j Sj ¼ 3, and then change them with different values of f1; 2; 3; 4; 5g in order to study the effect of different levels of interactions as introduced in GBPR and GBPR þ . For wiZAN [51], we search the tradeoff parameter on regularization terms in f0:001; 0:01; 0:1g, the global weight on unobserved (user, item) pairs in f0:01; 0:1; 0:5; 1g, and the global imputation value in f0:01; 0:1; 0:5; 1g, the iteration number in f100; 1000; 10000g, and fix the number of latent dimensions as d¼20. For the tradeoff parameters on ℓ1 and ℓ2 regularization terms in SLIM [30], we follow [30] and search the best parameters from ℓ1 ; ℓ2 A f0:5; 1; 2; 3; 5g using the NDCG@5 performance on the validation set. 6

5

http://www.cse.ust.hk/weikep/GBPR/.

7

http://en.wikipedia.org/wiki/Jaccard_index. http://glaros.dtc.umn.edu/gkhome/slim/overview.

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

507

Table 4 Recommendation performance of GBPR [35], GBPR þ and other methods on MovieLens100K (ML100K), MovieLens1M (ML1M), UserTag and NF5K5K. Dataset

Method

Prec@5

Rec@5

F1@5

NDCG@5

1call@5

AUC

ML100K

PopRank RandomWalk MemoryBasedCF PureSVD wiZAN SLIM BPR GBPR (ρ ¼ 1) GBPR þ (ρ ¼ 1)

0.27247 0.0094 0.2963 7 0.0067 0.346770.0022 0.34747 0.0006 0.3623 7 0.0066 0.37387 0.0034 0.37097 0.0066 0.4051 70.0038 0.4084 7 0.0047

0.05497 0.0028 0.0603 7 0.0010 0.08647 0.0022 0.0886 70.0007 0.0959 70.0031 0.0914 70.0020 0.0950 70.0014 0.10467 0.0016 0.1065 7 0.0012

0.0821 70.0036 0.0896 70.0015 0.11997 0.0021 0.1226 7 0.0011 0.13177 0.0038 0.1279 70.0021 0.1308 70.0026 0.14457 0.0015 0.1463 7 0.0021

0.2915 70.0072 0.3166 70.0041 0.3650 70.0067 0.3636 70.0020 0.3787 7 0.0065 0.3952 70.0010 0.3885 70.0107 0.42017 0.0031 0.4263 7 0.0048

0.6520 7 0.0201 0.68007 0.0089 0.7887 7 0.0118 0.8039 7 0.0063 0.8166 7 0.0124 0.80717 0.0142 0.81567 0.0015 0.84147 0.0058 0.8425 7 0.0092

0.8526 7 0.0006 0.86337 0.0006 0.87187 0.0005 0.8288 7 0.0068 0.8980 7 0.0020 0.86767 0.0005 0.90337 0.0007 0.91407 0.0008 0.91487 0.0008

ML1M

PopRank RandomWalk MemoryBasedCF PureSVD wiZAN SLIM BPR GBPR (ρ ¼ 0:6) GBPR þ (ρ ¼ 0:6)

0.2822 7 0.0019 0.2958 7 0.0021 0.37277 0.0013 0.4064 7 0.0017 0.4283 7 0.0067 0.45667 0.0005 0.44107 0.0008 0.4494 70.0020 0.4634 7 0.0017

0.0407 70.0004 0.0432 7 0.0005 0.0618 70.0008 0.0632 7 0.0004 0.07207 0.0012 0.07507 0.0007 0.07447 0.0003 0.07817 0.0009 0.0800 70.0010

0.0634 7 0.0003 0.06737 0.0003 0.0947 70.0007 0.0985 70.0006 0.11037 0.0017 0.11527 0.0008 0.11357 0.0003 0.11887 0.0010 0.12187 0.0011

0.29357 0.0010 0.30747 0.0039 0.3875 70.0013 0.42077 0.0022 0.44247 0.0067 0.4755 7 0.0011 0.4540 70.0009 0.4636 70.0014 0.4784 7 0.0022

0.66767 0.0006 0.6853 70.0023 0.8090 7 0.0021 0.81337 0.0034 0.84247 0.0016 0.8555 7 0.0030 0.8496 70.0047 0.86707 0.0022 0.87227 0.0021

0.87717 0.0002 0.8863 7 0.0003 0.8926 7 0.0004 0.8712 70.0011 0.91997 0.0014 0.9088 7 0.0006 0.93397 0.0004 0.93547 0.0005 0.9362 70.0005

UserTag

PopRank RandomWalk MemoryBasedCF PureSVD wiZAN SLIM BPR GBPR (ρ ¼ 0:8) GBPR þ (ρ ¼ 0:8)

0.26477 0.0012 0.2692 7 0.0028 0.252470.0016 0.26747 0.0030 0.2819 70.0022 0.27307 0.0016 0.2969 7 0.0025 0.30117 0.0008 0.30217 0.0014

0.0405 7 0.0003 0.04137 0.0005 0.0402 7 0.0009 0.0395 70.0008 0.0459 7 0.0006 0.0404 7 0.0007 0.04767 0.0008 0.04917 0.0014 0.0494 7 0.0018

0.06407 0.0004 0.0653 7 0.0006 0.0623 7 0.0005 0.06317 0.0010 0.07167 0.0005 0.06417 0.0009 0.0740 7 0.0006 0.0766 7 0.0012 0.0766 7 0.0015

0.27307 0.0014 0.27627 0.0025 0.26187 0.0019 0.2755 7 0.0028 0.2905 70.0033 0.28497 0.0025 0.30727 0.0017 0.3104 70.0009 0.31237 0.0018

0.52217 0.0062 0.5317 70.0059 0.5719 7 0.0014 0.5672 70.0046 0.60717 0.0031 0.5643 70.0028 0.61727 0.0008 0.6226 7 0.0019 0.61887 0.0034

0.68107 0.0016 0.7063 7 0.0016 0.71507 0.0021 0.72757 0.0019 0.77807 0.0019 0.72017 0.0017 0.77117 0.0013 0.7892 7 0.0011 0.7913 70.0018

NF5K5K

PopRank RandomWalk MemoryBasedCF PureSVD wiZAN SLIM BPR GBPR (ρ ¼ 0:8) GBPR þ (ρ ¼ 0:8)

0.1728 7 0.0012 0.1762 70.0022 0.1899 7 0.0017 0.2096 7 0.0036 0.2313 70.0037 0.245670.0030 0.23187 0.0006 0.24117 0.0027 0.2515 70.0003

0.0563 70.0013 0.05787 0.0016 0.0724 70.0009 0.08017 0.0021 0.09617 0.0041 0.0934 70.0023 0.0945 70.0012 0.09797 0.0013 0.1023 7 0.0014

0.06837 0.0001 0.0698 7 0.0010 0.08167 0.0008 0.09127 0.0018 0.10617 0.0032 0.1067 70.0020 0.10467 0.0002 0.1095 70.0013 0.11377 0.0007

0.1794 7 0.0004 0.1846 70.0019 0.2067 70.0015 0.23107 0.0050 0.2536 70.0045 0.27017 0.0035 0.2508 70.0006 0.26117 0.0025 0.27387 0.0027

0.44727 0.0059 0.4534 7 0.0085 0.5091 70.0047 0.5449 7 0.0063 0.5826 7 0.0076 0.5832 7 0.0058 0.5683 70.0016 0.58447 0.0015 0.59477 0.0007

0.91477 0.0015 0.91697 0.0017 0.91167 0.0017 0.7656 7 0.0038 0.84997 0.0031 0.9096 7 0.0019 0.92687 0.0019 0.9321 70.0014 0.9309 7 0.0012

0.12

0.45 2

3

4

0.076 0.074

5

1

2

0.3 0.29 2

3

0.11

5

4

5

0.085 0.08

0.045 1

2

3

0.25

4

5

0.235 2

3

|G|

4

5

2

3

4

5

1

2

3

|G|

4

5

4

5

1

2

3

4

5

3

|G|

2

0.46 0.45

1

2

3

4

5

4

5

0.3 1

2

3

4

5

0.9

5

0.26 0.255 2

3

|G|

2

0.84

4

5

1

2

3

4

5

4

5

0.936 0.934 0.932

5

1

2

3

|G| 0.8

0.62 0.6 1

2

3

4

0.78 0.76

5

1

2

3

4

5

|G| 0.935

0.59 0.58 0.57 0.56

4

0.938

0.64

0.58

3

|G|

0.6

1

1

|G|

0.265

0.25

4

|G|

0.32

0.28

3

0.86

|G|

0.105 2

1

0.88

0.47

0.27

1

0.8

0.91 0.905

|G|

0.34

0.11

0.1

3

AUC

1−call@5 2

|G|

0.115

0.095 0.09

1

|G|

F1@5

Rec@5

0.24

0.35

|G|

0.075 0.07

0.1

1

1

|G|

0.245

5

|G|

0.05

0.04

4

0.115

0.055

|G|

0.23

4

F1@5

Rec@5

0.31

1

3

3

0.48

|G|

0.32

0.28

2

AUC

0.125

F1@5

0.08

1

1

|G|

0.078

Rec@5

Pre@5

0.12

5

0.46

|G|

Pre@5

4

0.47

0.44

Pre@5

3

|G|

0.82

AUC

2

0.92 0.915

0.84

AUC

1

1−call@5

0.09

5

1−call@5

4

0.86

0.4

1−call@5

3

|G|

NDCG@5

2

0.13

NDCG@5

1

0.45

0.14

NDCG@5

0.36

0.1

NDCG@5

0.38

0.15

F1@5

0.11

0.4

Rec@5

Pre@5

0.42

1

2

3

4

5

0.93 0.925 0.92

1

2

3

|G|

4

5

Fig. 5. Recommendation performance of GBPR with different sizes of user group (from top row to bottom row: MovieLens100K, MovieLens1M, UserTag, and NF5K5K).

5.4. Summary of experimental results 5.4.1. Main results The recommendation performance of GBPR þ and other baselines are shown in Table 4, from which we can have the following observations:

1. PopRank performs worse than the basic memory-based algorithm (i.e., MemoryBasedCF) and all other model-based baselines in most cases, which shows that a non-personalized recommendation technique is not competitive in exploiting one-class feedback;

1

2

3

4

5

1

2

4

3

4

0.04

1

2

2

3

3

4

5

4

5

2

|G|=|S|

3

1

2

3

0.085 0.08 0.075 0.07 0.065

1

2

4

3

4

5

5

4

5

|G|=|S|

2

3

0.8

5

1

2

1

2

3

4

5

0.32 0.3 1

2

3

4

5

0.28 0.27 0.26 0.25 0.24

1

2

|G|=|S|

3

5

1

2

0.88 0.87 0.86 0.85 0.84

4

5

0.66 0.64 0.62 0.6 0.58

5

1

2

3

4

5

4

5

0.936 0.934 0.932

5

1

2

3

|G|=|S| 0.8

1

2

3

4

0.78 0.76 0.74

5

1

2

3

4

5

|G|=|S| 0.935

0.62 0.6 0.58 0.56

4

0.938

|G|=|S|

4

3

|G|=|S|

|G|=|S|

0.34

0.28

4

AUC

0.5 0.49 0.48 0.47 0.46 0.45

3

0.92 0.915 0.91 0.905 0.9

|G|=|S|

|G|=|S|

0.105 1

4

|G|=|S|

0.11

0.1

3

AUC

1−call@5 2

0.82

|G|=|S|

0.115

1

1

|G|=|S|

F1@5

Rec@5

0.24

5

0.115

|G|=|S| 0.11 0.105 0.1 0.095 0.09

4

|G|=|S|

0.045

5

0.25

1

3

0.12

0.11

5

0.05

0.26

Pre@5

3

F1@5

Pre@5

Rec@5 2

2

|G|=|S|

0.055

1

1

|G|=|S|

|G|=|S|

0.23

5

0.125

|G|=|S| 0.34 0.32 0.3 0.28 0.26

4

F1@5

0.082 0.08 0.078 0.076 0.074

Rec@5

Pre@5

0.48 0.47 0.46 0.45 0.44

3

|G|=|S|

0.84

AUC

2

0.86

AUC

1

0.35

1−call@5

5

0.4

1−call@5

4

0.45

1−call@5

3

|G|=|S|

NDCG@5

2

NDCG@5

1

0.16 0.15 0.14 0.13 0.12

NDCG@5

0.11 0.105 0.1 0.095 0.09

NDCG@5

0.44 0.42 0.4 0.38 0.36

F1@5

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

Rec@5

Pre@5

508

1

2

|G|=|S|

3

4

5

0.93 0.925 0.92

|G|=|S|

1

2

3

4

5

|G|=|S|

Fig. 6. Recommendation performance of GBPR þ with different sizes of user group and item set (from top row to bottom row: MovieLens100K, MovieLens1M, UserTag, and NF5K5K).

Performance

0.8

1

BPR,T=900000 GBPR,T=300000 + GBPR ,T=100000

0.6 0.4 0.2

1

Performance

0.8

0.4

0 Pre@5

Rec@5

F1@5

NDCG@5 1−call@5

AUC

Pre@5

Rec@5

F1@5

NDCG@5 1−call@5

Evaluation Metric

Evaluation Metric

MovieLens100K

MovieLens1M

1

BPR,T=900000 GBPR,T=300000 + GBPR ,T=100000

0.4 0.2

AUC

BPR,T=900000 GBPR,T=300000 GBPR +,T=100000

0.8

0.6

0

0.6

0.2

Performance

0

BPR,T=900000 GBPR,T=300000 + GBPR ,T=100000

0.8 Performance

1

0.6 0.4 0.2

Pre@5

Rec@5

F1@5

NDCG@5 1−call@5

AUC

0

Pre@5

Evaluation Metric

5.4.2. Impact of group and set size To have a deep understanding of the effect of group pairwise preference in GBPR, we adjust the group size as j Gj A f1; 2; 3; 4; 5g

NDCG@5 1−call@5

AUC

NF5K5K

Fig. 7. Recommendation performance of BPR, GBPR and GBPR

We can thus see that the assumption that combines pairwise preference and group preference in GBPR and GBPR þ is indeed more effective than that of individual pairwise preference assumed in BPR [41].

F1@5

Evaluation Metric

UserTag

2. BPR is better than the pointwise baselines (i.e., PureSVD, wiZAN and SLIM) and all other baselines in most cases though it is slightly worse than wiZAN and SLIM in some cases, which demonstrates the effectiveness of pairwise preference assumptions; and 3. GBPR and GBPR þ further improve BPR on all evaluation metrics on all the four datasets, which shows the effect of the injected rich interactions among users via group pairwise preference from user groups and item sets.

Rec@5

þ

with different iteration numbers.

and show the results of Pre@5, Rec@5, F1@5, NDCG@5, 1call@5 and AUC in Fig. 5. From Fig. 5, we can see that using a relatively larger user group (e.g., j Gj ¼ 3 or 4) improves the recommendation performance on all four datasets. This can be explained by the effect of introducing the user group G for modeling the pairwise preference in Eq. (2) and learning the model parameters in Eq. (9). To further investigate the effect of the injected interactions via user groups and item sets, we conduct more empirical studies with different sizes of user group and item set, i.e., j Gj ¼ j Sj A f1; 2; 3; 4; 5g. We report the results in Fig. 6, from which we can have a similar observation to that of Fig. 5, i.e., a relatively larger user group and item set is helpful for the recommendation performance. The recommendation results and observations in Figs. 5 and 6 show that the injected interactions via user groups and/or item sets in GBPR and GBPR þ are helpful in exploiting one-class

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

feedback as compared with the major baseline BPR. Notice that when j Gj ¼ j Sj ¼ 1, GBPR þ reduces to BPR. 5.4.3. Improvement from rich interactions Considering the close relationship among BPR, GBPR and GBPR þ as shown in Eq. (7), and the learning steps 1  4 in Fig. 4, a natural question may be raised, i.e., whether the improvement of GBPR and GBPR þ is from more parameter learning opportunities as compared with that of BPR when j Gj ; j Sj 4 1. In order to investigate the improvement from rich interactions via user group G and item set S, we conduct further empirical studies with equal learning opportunities for the model parameters of those three algorithms. In particular, we fix j Gj ¼ j Sj ¼ 3 and use T¼ 900,000, T¼300,000 and T¼ 100,000 for BPR, GBPR and GBPR þ , respectively. In this case, the learning or update opportunities of the model parameters in those three algorithms are similar. The results on all six evaluation metrics and four datasets are shown in Fig. 7. We can see that the overall performance ordering is BPR o GBPR o GBPR þ . Although the results of GBPR and GBPR þ seem close to that of BPR, GBPR þ is always better than BPR, in particular of the top-5 recommendation evaluation metrics. The improvement of GBPR þ over GBPR is from the more sufficient interactions between one user group and some unobserved items in a certain iteration as shown in Fig. 4, instead of one user group and one unobserved item in that of GBPR. The results in Fig. 7 clearly show the effect of the injected rich interactions via user groups and item sets. Hence, the answer to the above question is negative, and using a relatively large user group and/or item set is indeed a promising way to help improve the recommendation accuracy.

6. Conclusions and future work In this paper, we study the one-class collaborative filtering (OCCF) problem and design a novel algorithm called group Bayesian personalized ranking with rich interactions (GBPR þ ). GBPR þ extends GBPR [35] by introducing rich interactions through item sets instead of single items. GBPR þ is comparable to GBPR [35] and BPR [41] in terms of time complexity. Experimental results on the four real-world datasets show that GBPR þ can recommend items more accurately than several state-of-the-art baselines using various evaluation metrics. For future work, we are interested in extending GBPR þ and GBPR by (1) optimizing the user group construction process, such as incorporating time, location, taxonomy and other possible context and social information to refine the like-minded user groups, (2) learning the quality or confidence value of one-class feedback and/or items, (3) adaptively changing the group size for different segments of users and items during the learning process, (4) introducing some advanced sampling strategies for unobserved user–item pairs so as to further improve the efficiency and efficacy of the algorithm, and (5) embedding GBPR þ in some hybrid recommendation framework for real-world deployment.

Acknowledgment We thank the support of Hong Kong RGC under the project ECS/ HKBU211912, Natural Science Foundation of China, Nos. 61272365 and 61502307, and Natural Science Foundation of Guangdong Province, Nos. 2014A030310268 and 2016A030313038. We are also thankful to Prof. Zhong Ming and Mr. Zhuode Liu for their support and assistance on empirical studies, our colleague Mr. George Basker for his help on linguistic quality improvement, and the handling editor and reviewers for their thoughtful and expert comments.

509

References [1] Ryan P. Adams, George E. Dahl, Iain Murray, Incorporating side information into probabilistic matrix factorization using Gaussian processes, in: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, UAI '10, 2010, pp. 1–9. [2] Gediminas Adomavicius, Alexander Tuzhilin, Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions, IEEE Trans. Knowl. Data Eng. 17 (6) (2005) 734–749. [3] Deepak Agarwal, Bee-Chung Chen, Regression-based latent factor models, in: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '09, 2009, pp. 19–28. [4] Sihem Amer-Yahia, Senjuti Basu Roy, Ashish Chawlat, Gautam Das, Cong Yu, Group recommendation: semantics and efficiency, Proc. VLDB Endow. 2 (August (1)) (2009) 754–765. [5] Michael W. Berry, Svdpack: A Fortran-77 Software Library for the Sparse Singular Value Decomposition, Technical Report, Knoxville, TN, USA, 1992. [6] Jesús Bobadilla, Fernando Ortega, Antonio Hernando, Abraham GutiéRrez, Recommender systems survey, Knowl.-Based Syst. 46 (2013) 109–132. [7] Harr Chen, David R. Karger, Less is more: probabilistic models for retrieving fewer relevant documents, in: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, 2006, pp. 429–436. [8] Li Chen, Pearl Pu, Users' eye gaze pattern in organization-based recommender interfaces, in: Proceedings of the 16th International Conference on Intelligent User Interfaces, IUI '11, 2011, pp. 311–314. [9] Paolo Cremonesi, Yehuda Koren, Roberto Turrin, Performance of recommender algorithms on top-n recommendation tasks, in: Proceedings of the 4th ACM Conference on Recommender Systems, RecSys '10, 2010, pp. 39–46. [10] James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, Dasarathi Sampath, The youtube video recommendation system, in: Proceedings of the 4th ACM Conference on Recommender Systems, RecSys '10, 2010, pp. 293–296. [11] Gene H. Golub, Charles F. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, USA, 1996. [12] Alan Herschtal, Bhavani Raskutti, Optimising area under the roc curve using gradient descent, in: Proceedings of the 21st International Conference on Machine Learning, ICML '04, 2004, pp. 49–57. [13] Yifan Hu, Yehuda Koren, Chris Volinsky, Collaborative filtering for implicit feedback datasets, in: Proceedings of the 8th IEEE International Conference on Data Mining, ICDM '08, 2008, pp. 263–272. [14] Michael Jahrer, Andreas Töscher, Collaborative filtering ensemble for ranking, in: Proceedings of KDD Cup 2011 Competition, 2012, pp. 153–167. [15] Kalervo Järvelin, Jaana Kekäläinen, Cumulated gain-based evaluation of ir techniques, ACM Trans. Inf. Syst. 20 (October (4)) (2002) 422–446. [16] Meng Jiang, Peng Cui, Xumin Chen, Fei Wang, Wenwu Zhu, Shiqiang Yang, Social recommendation with cross-domain transferable knowledge, IEEE Trans. Knowl. Data Eng. 27 (11) (2015) 3084–3097. [17] Shuhui Jiang, Xueming Qian, Jialie Shen, Yun Fu, Tao Mei, Author topic modelbased collaborative filtering for personalized POI recommendations, IEEE Trans. Multimed. 17 (6) (2015) 907–918. [18] Santosh Kabbur, Xia Ning, George Karypis, Fism: factored item similarity models for top-n recommender systems, in: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '13, 2013, pp. 659–667. [19] Bhargav Kanagal, Amr Ahmed, Sandeep Pandey, Vanja Josifovski, Jeff Yuan, Lluis Garcia-Pueyo, Supercharging recommender systems using taxonomies for learning user purchase behavior, Proc. VLDB Endow. 5 (June (10)) (2012) 956–967. [20] Yehuda Koren, Factorization meets the neighborhood: a multifaceted collaborative filtering model, in: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '08, 2008, pp. 426–434. [21] Yehuda Koren, Joe Sill, Ordrec: an ordinal model for predicting personalized item rating distributions, in: Proceedings of the 5th ACM Conference on Recommender Systems, RecSys '11, 2011, pp. 117–124. [22] Artus Krohn-Grimberghe, Lucas Drumond, Christoph Freudenthaler, Lars Schmidt-Thieme, Multi-relational matrix factorization using Bayesian personalized ranking for social network data, in: Proceedings of the 5th ACM International Conference on Web Search and Data Mining, WSDM '12, 2012, pp. 173–182. [23] Daniel D. Lee, H. Sebastian Seung, Algorithms for non-negative matrix factorization, in: Annual Conference on Neural Information Processing Systems 13, NIPS '01, MIT Press, Cambridge, Massachusetts, USA, 2001, pp. 556–562. [24] Greg Linden, Brent Smith, Jeremy York, Amazon.com recommendations: itemto-item collaborative filtering, IEEE Internet Comput. 7 (January (1)) (2003) 76–80. [25] Nathan N. Liu, Luheng He, Min Zhao, Social temporal collaborative ranking for context aware movie recommendation, ACM Trans. Intell. Syst. Technol. 4 (February (1)) (2013) 15:1–15:26. [26] Hao Ma, Irwin King, Michael R. Lyu, Learning to recommend with explicit and implicit social relations, ACM Trans. Intell. Syst. Technol. 2 (May (3)) (2011) 29:1–29:19.

510

W. Pan, L. Chen / Neurocomputing 207 (2016) 501–510

[27] Xudong Mao, Qing Li, Haoran Xie, Yanghui Rao, Popularity tendency analysis of ranking-oriented collaborative filtering from the perspective of loss function, in: Proceedings of the 19th International Conference on Database Systems for Advanced Applications, DASFAA '14, 2014, pp. 451–465. [28] David M. Nichols, Implicit rating and filtering, in: Proceedings of the 5th DELOS Workshop on Filtering and Collaborative Filtering, November 1997. [29] Athanasios N. Nikolakopoulos, Marianna A. Kouneli, John D. Garofalakis, Hierarchical itemspace rank: exploiting hierarchy to alleviate sparsity in ranking-based recommendation, Neurocomputing 163 (2015) 126–136. [30] Xia Ning, George Karypis, SLIM: sparse linear methods for top-n recommender systems, in: Proceedings of the 11th IEEE International Conference on Data Mining, ICDM '11, 2011, pp. 497–506. [31] Rong Pan, Martin Scholz, Mind the gaps: weighting the unknown in largescale one-class collaborative filtering, in: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '09, 2009, pp. 667–676. [32] Rong Pan, Yunhong Zhou, Bin Cao, Nathan Nan Liu, Rajan M. Lukose, Martin Scholz, Qiang Yang, One-class collaborative filtering, in: Proceedings of the 8th IEEE International Conference on Data Mining, ICDM '08, 2008, pp. 502–511. [33] Weike Pan, A survey of transfer learning for collaborative recommendation with auxiliary data, Neurocomputing 177 (2016) 447–453. [34] Weike Pan, Li Chen, Cofiset: collaborative filtering via learning pairwise preferences over item-sets, in: Proceedings of SIAM International Conference on Data Mining, SDM '13, 2013, pp. 180–188. [35] Weike Pan, Li Chen, Gbpr: group preference based Bayesian personalized ranking for one-class collaborative filtering, in: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, IJCAI '13, 2013, pp. 2691–2697. [36] Weike Pan, Shanchuan Xia, Zhuode Liu, Xiaogang Peng, Zhong Ming, Mixed factorization for collaborative recommendation with heterogeneous explicit feedbacks, Inf. Sci. 332 (2016) 84–93. [37] Weike Pan, Hao Zhong, Congfu Xu, Zhong Ming, Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks, Knowl.-Based Syst. 73 (2015) 173–180. [38] Xueming Qian, He Feng, Guoshuai Zhao, Tao Mei, Personalized recommendation combining user interest and social circle, IEEE Trans. Knowl. Data Eng. 26 (7) (2014) 1763–1777. [39] Steffen Rendle, Factorization machines with libfm, ACM Trans. Intell. Syst. Technol. 3 (May (3)) (2012) 57:1–57:22. [40] Steffen Rendle, Christoph Freudenthaler, Improving pairwise learning for item recommendation from implicit feedback, in: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, WSDM '14, 2014, pp. 273–282. [41] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme, Bpr: Bayesian personalized ranking from implicit feedback, in: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI '09, 2009, pp. 452–461. [42] Steffen Rendle, Christoph Freudenthaler, Lars Schmidt-Thieme, Factorizing personalized Markov chains for next-basket recommendation, in: Proceedings of the 19th International Conference on World Wide Web, WWW '10, 2010, pp. 811–820. [43] Steffen Rendle, Lars Schmidt-Thieme, Pairwise interaction tensor factorization for personalized tag recommendation, in: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining, WSDM '10, 2010, pp. 81–90. [44] Junge Shen, Cheng Deng, Xinbo Gao, Attraction recommendation: towards personalized tourism via collective intelligence, Neurocomputing 173 (2016) 789–798. [45] Yue Shi, Alexandros Karatzoglou, Linas Baltrunas, Martha Larson, Alan Hanjalic, Nuria Oliver, Tfmap: optimizing map for top-n context-aware recommendation, in: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '12, 2012, pp. 155–164. [46] Yue Shi, Alexandros Karatzoglou, Linas Baltrunas, Martha Larson, Nuria Oliver, Alan Hanjalic, Climf: learning to maximize reciprocal rank with collaborative less-is-more filtering, in: Proceedings of the 6th ACM Conference on Recommender Systems, RecSys '12, 2012, pp. 139–146.

[47] Vikas Sindhwani, Serhat Selcuk Bucak, Jianying Hu, Aleksandra Mojsilovic, One-class matrix completion with low-density factorizations, in: Proceedings of the 10th IEEE International Conference on Data Mining, ICDM '10, 2010, pp. 1055–1060. [48] Markus Weimer, Alexandros Karatzoglou, Quoc V. Le, Alex J. Smola, Cofirank – maximum margin matrix factorization for collaborative ranking, in: Annual Conference on Neural Information Processing Systems 19, NIPS '07, 2007. [49] Zhenxing Xu, Ling Chen, Gencai Chen, Topic based context-aware travel recommendation method exploiting geotagged photos, Neurocomputing 155 (2015) 99–107. [50] Shuang-Hong Yang, Bo Long, Alex J. Smola, Hongyuan Zha, Zhaohui Zheng, Collaborative competitive filtering: learning recommender using context of user choice, in: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '11, 2011, pp. 295–304. [51] Yuan Yao, Hanghang Tong, Guo Yan, Feng Xu, Xiang Zhang, Boleslaw Szymanski, Jian Lu, Dual-regularized one-class collaborative filtering, in: Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, CIKM '14, 2014, pp. 759–768. [52] Weinan Zhang, Tianqi Chen, Jun Wang, Yong Yu, Optimizing top-n collaborative filtering via dynamic negative item sampling, in: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, 2013, pp. 785–788. [53] Guoshuai Zhao, Xueming Qian, Xing Xie, User-service rating prediction by exploring social users' rating behaviors, IEEE Trans. Multimed. 18 (3) (2016) 496–506. [54] Tong Zhao, Julian McAuley, Irwin King, Leveraging social connections to improve personalized ranking for collaborative filtering, in: Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, CIKM '14, 2014. [55] Yu Zheng, Methodologies for cross-domain data fusion: an overview, IEEE Trans. Big Data 1 (1) (2015) 16–33.

Weike Pan is a lecturer (research oriented) in the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. He received the Ph.D. degree in Computer Science and Engineering from the Hong Kong University of Science and Technology, Kowloon, Hong Kong, China, in 2012. He was a post-doctoral research fellow at Hong Kong Baptist University and a senior engineer at Baidu Inc. He was also the information officer of ACM Transactions on Intelligent Systems and Technology (ACM TIST). His research interests include transfer learning, recommender systems, and statistical machine learning.

Li Chen is an assistant professor at Hong Kong Baptist University. She obtained her Ph.D. degree in human computer interaction at Swiss Federal Institute of Technology in Lausanne (EPFL), and Bachelor and Master degrees in computer science at Peking University, China. Her research interests are mainly in the areas of human–computer interaction, usercentered development of recommender systems and ecommerce decision supports. She is now an ACM senior member, and an editorial board member of User Modeling and User-Adapted Interaction Journal (UMUAI). She has also been serving in a number of journals and conferences as guest editor, coorganizer, (senior) PC member and reviewer.