Mailbox integration - replying by mail creates new issue instead of comment

I'm using youtrack's docker image, version 2018.2.44329

I've enabled email notifications and mailbox integration. Now I want that replies to issues via email are assigned as a comment, but they always end up as a new issue. 

I've been trying this as an admin user, so the permissions should be set. 

I don't see any option on how to change the behavior, can someone please guide me on how to make this work?

Thanks!