No email notifications for new comments

I didn't receive an email when a user posted a comment. That is minimally required in my opinion for this product to work for us.

Is that a setting somewhere? I can't find it.