It depends how the game is going. I like plaing QoT with minions, but I won't usually get the idols until later. But if I need to quickly power up, I'll the best possible idols I can afford, preferably one of each. Good clerics are the most important. It depends on your build ofc, but if you get the class spesific minions and buff them up, you're pretty well set. At some point though, getting all the minions is well worth it, if you're also getting items and skills to boost them. The more, the more the boosts stack.