All geek questions in one place

Application Engine: How does memory work? (using Mahout with the application engine)

I am trying to use Mahout with an application engine. There are several problems that have arisen, but the main problem now: how can I make a recommendation if the model is too large to store in memory?

I use memcache, but by default the layout of Mahout is not serialized. I serialize FastIDSet instead, but then, when I need to make a recommendation, I have to get FastIDSet from memcache, use it as a parameter for the recommender, and then make a recommendation.

That would be nice, but keeping both in memory gives me an error in the heap area, although my database is not too large (19 MB).

I cannot do this in chunks because Mahout expects initialization with a complete data model.

Does anyone have any experience with this issue?

Thank you and good afternoon.

+4
source share
2 answers

I realized I had to set the scaling in appengine-web.xml.

Still watching the recommender speed if I need to rebuild the model too quickly from quickIDSET.

+1
source

, , . Mahout, SimilarityAnalysis.cooccurrence, - Apache 2. Universal Recommendender, Spark Mahout

BTW , Mahout :-)

0

Source: https://habr.com/ru/post/1623770/

More articles:


All Articles