One option is to find a specific item and click with an offset:
var elm = element(by.css('.material-dialog-container'));
browser.actions()
.mouseMove(elm, -20, -20)
.click()
.perform();
If the offset is calculated from the upper left corner of the element.