Unable to access mountpoint: Permission denied

I get this error from BuildBox (rev bfdab0a6) running via the BuildGrid buildbox plugin.

fusermount3: failed to access mountpoint /home/jimmacarthur/fuse: Permission denied

If I delete /home/jimmacarthur/fuse and use mkdir to create it again, BuildBox will start working again, but after several calls I start getting this message again. I don't know any more causes than that, sorry. I can't think of anything other than buildbox which would interact with /home/jimmacarthur/fuse.