The best way to determine which key exists in the data warehouse

Question

I have several hundred keys, all from the same model that I previously calculated:

candidate_keys = [db.Key(...), db.Key(...), db.Key(...), ...]

Some of these keys relate to the actual objects in the data warehouse, and some do not. I want to determine which keys correspond to objects.

There is no need to know the data inside the objects, only they exist.

One solution would be to use db.get ():

keys_with_entities = set()
for entity in db.get(candidate_keys):
  if entity:
    keys_with_entities.add(entity.key())

However, this procedure will retrieve all entity data from the store, which is not necessary and expensive.

The second idea is to use Query with a filter INon key_name, manually selecting 30 in pieces to match the requirements of a pseudo-filter IN. However, filtering INdoes not allow key-only requests.

Is there a better way?

Answer 1

IN App Engine; . IN 30 30 , 30 !

- , , , . , : . :

foo = AnEntity(...)
foo.put()
presence = PresenceEntity(key_name='x', parent=foo)
presence.put()
...
def exists(keys):
  test_keys = [db.Key.from_path('PresenceEntity', 'x', parent=x) for x in keys)
  return [x is not None for x in db.get(test_keys)]

Answer 2

, , - keys_only=True, .

for key in candidate_keys:
  if MyModel.all(keys_only=True).filter('__key__ =', key).count():
    keys_with_entities.add(key)

, , Data Received from API.

Answer 3

( ):

IN.

class MyModel(db.Model):
  """Some model"""
  # ... all the old stuff
  the_key = db.StringProperty(required=True) # just a duplicate of the key_name

#... meanwhile back in the example

for key_batch in batches_of_30(candidate_keys):
  key_names = [x.name() for x in key_batch]
  found_keys = MyModel.all(keys_only=True).filter('the_key IN', key_names)
  keys_with_entities.update(found_keys)

, , , IN , IN. 160-200 , .