Its a potential risk, guilds usually have sub groups within them though. The core team, then little groups that have joined from other games/guilds, RL friends and finally the odd single player.
Would it damage the server? Not sure, would guilds just split and make an alliance, you bet. But...