There is another way to do the same. But this method may be less effective than @Styvane's answer. This should work if there are smaller elements.
[ {$group: {_id: null, senders: {$addToSet: '$invitedBy'}, everyone: {$addToSet: '$_id'}}}, {$project: {_id: 0, res: {$setDifference: ['$everyone', '$senders']}}} ]
If other information is needed, it can also be obtained (sorry, but here it gets dirty)
[ {$group: {_id: null, senders: {$addToSet: '$invitedBy'}, everyone: {$addToSet: {id: '$_id', name: '$name'}}}}, {$project: {_id: 0, everyone: 1, res: {$setDifference: ['$everyone.id', '$senders']}}}, {$unwind: '$everyone'}, {$unwind: '$res'}, {$project: {flag: {$eq: ['$everyone.id', '$res']}, everyone: 1}}, {$match: {flag: true}} ]
source share