Follow-up for https://gitlab.com/gitlab-org/gitlab/-/issues/347407

In previous MR I added a permission verification when user requests
the list of participants. But it increased the number of db queries.

I found that notes association does not preload project object and
that generates additional queries.

Changelog: fixed