I've actually had this done to me before in a 3v3 game. The opposition had two oaks and we were generally at least matching them in play if not being more dominant in the field.
Then the spirit charging started. Now, in the back and forth of the match, they did knock down some of the entry portals infront of our citadel on Leviathan (which isn't all that well protected compared to some maps anyways) so that made it a bit easier for the spirits. Still, it completely turned the tide of the game. At least one of us had to guard the citadel at all times, and even then the spirits weren't that easy to kill since they fly right up into the citadel. Yes we killed them off over and over, but they most certainly did a hefty amount of damage to the citadel nonetheless. We did have at least one or more health regen upgrades bought at the citadel as well - it helps but it definitely didn't stop the wittling away. Almost as importantly, since at least one of our team had to be on spirit duty, it gave the opposition a big advantage out in the rest of the field as they were now 3v2 or 3v1 most of the time. It was an effective strategy and I'm fairly certain that it won them the game.
Now, I'm not going to call it a cheat or anything, because it obviously isn't, but I can't say that I particularily think it's an upstanding tactic either. At worst I'd call it an exploit, since it's clearly allowed by the game's mechanics. I do have to wonder if this sort of tactic, particularly with Oak's flying spirits was intended by design or just something unforseen by the devs when they coded the overall game mechanics. If it isn't, I'm not sure what a good proposal for a fix would be off hand. If you limit minion movement away from the general (or even into unseen fogged areas), you do hamper them in a lot of other ways that make them useful. This is mainly an Oak thing too, as I play a lot of minion build Erebus and I can't see nightwalkers being nearly as effective at this tactic.
-dolynick