Pyspark ml recommendation - all recommendations

Question

Hy

I am new to Spark and I am trying to use ML recommendations.

My code

df = sqlContext.createDataFrame(
[(0, 0, 4.0), (0, 1, 2.0), (1, 1, 3.0), (1, 2, 4.0), (2, 1, 1.0), (2, 2, 5.0)],
["user", "item", "rating"])

als = ALS(rank=10, maxIter=5)

model = als.fit(df)

model.userFactors.orderBy("id").collect()

How can I get 2 recommendations for all users for all films?

thank you for your time.

Answer 1

This is not possible directly with ml.recommendation.ALSModel. You can use the methodtransform

users = df.select("user").distinct()
items = df.select("item").distinct()

model.transform(users.join(items))

and then filter the results, but it is extremely inefficient. As far as I can tell, it would be better to just use mllib.recommendation.ALShere:

from pyspark.mllib.recommendation import ALS, Rating

model = ALS.train(df.rdd.map(lambda r: Rating(*r)), 10, 5)
model.recommendProductsForUsers(2)