Are you unable to put up fencing? Frankly I'd ask my neighbors if it bothered them, if it's an issue I'd put up a fence asap; actually either way, I'd find a way to put of fencing so as not to have that worry of them either wandering too far off that they don't find their way home via stray, roadkill or whatever.