SSH Key for different projects

I have a little problem that I have 2 private (X-Project , Y-Project ) projects , I deployed an SSH key for X-Project with write access enabled.. It works fine for accessing (pulling, pushing X-Project) with that SSH Key , The problem is that I'm also have the same rights for Y-Project despite the fact that i didn't deploy that key to the project and when I go to it's SSH Keys I find that there's no keys deployed and also i see the option to enable the key i'm using for X-Project..

I'm not sure if this is a bug or there's some thing I have to do which i didn't ...