Avoid incorrect FEP exclusions

A distance check for excluded FEP pair distances was missing, which
meant that extra, invalid periodic image distances could be used
for exclusion calculations. This can only happen when a perturbed
molecule is close to a periodic boundary and the system is small
such that that the NBNxM grid is one cell wide along a dimension.

Additionally, the reaction-field contribution should only act
up to rcoulomb.

Fixes #4665
Fixes #4667