- Allow people to sign up for the Beta and invite a randomly selected percentage of applicants.  Preorders should be let in later on in the testing process, when balance and bug fixes, not concept / gameplay are the biggest issues to be discussed and worked on.  Having a pre-order only Beta ensured that the group of testers was fairly small and also generally biased.  There are positives and negatives to a pre-order beta as well as an invite-Beta, but I feel that the positives of a varied testing group outweigh the potential negatives of having people who haven't already bought the game and who therefore aren't necessarily as motivated.

- The testing community needs to be more open to criticism of the game during the testing process.  This is a direct result of the first point; since the majority of the people here were loyal Stardock (and to a lesser degree GPG) fans, they tended to not be particularly receptive to criticism of the game or radical gameplay suggestions.

- The developers should have been more willing to make significant changes in gameplay and concept as suggested by testers.  This means that the time frame for testing and development probably could have been a bit longer, but more than anything it means the developers ought to have viewed testers as people involved in the development process, not merely as "taste-testers" or "bug-finders"  What I mean by "taste-testers" is the developers give you a game and they are only interested in your reaction to what they give you; they want to hear whether you liked it or not, and how much, but dont' really care about your suggestions on specifically how to make it "taste better."  I felt like this was to a large degree the case with this Beta.  The fact that Generals have changed very little over the course of development, despite a general (no pun intended) outcry from the testers for significant changes, is indicative of this problem.

- The developers can't be entirely blamed for this (shit happens) but the faulty network code really made it difficult to do significant testing and give good feedback for a large part of the testing process.  For the next game, the developers should really try to get the network code to a better point before putting the game in the hands of testers.  I get the feeling that many people were turned off to the game before they ever got to really try it out very much, thanks to the faulty network code. 

- The developers ought to have been in greater dialogue with the community.  Frogboy posts a fair amount, but he's just one guy, and he's from Stardock.  I don't think I've seen anybody actually from GPG, on the actual development team, posting on these forums and bouncing around ideas with the testers.  This is what I really like to see from developers in Beta testing.  Having more than just a couple community representatives from the publisher of the game trading comments (and occasionally insults) with the testers would really help to foster a sense that the developers really want to integrate with the community, not just say "here's the game we're making, take a look and tell us if you like it or not."  I doubt anybody here was in the Mythos Beta, but that's what I see as the ideal.  Travis Baldree, the lead designer of Mythos, consistently created threads on the forums and actually responded to poster comments, besides also talking in IRC.  Other members of the development crew were similarly hands-on, especially Volbard and Max Schaeffer.  Moreover, besides simply talking with testers, Travis and crew constantly took tester feedback into consideration, and were willing to make large, sweeping changes to the game as it currently stood to improve it in light of that feedback.  You never got the sense from them that "this is how things are, this is how we've decided they will be, and that's how they're going to stay."

I may sound overly critical, but I'm just trying to help the developers out in the future.  This isn't the worst Beta I've been in.  I don't see this game going on to fail less than a year after it's release, like one game I tested (Hellgate: London).  This game is actually going to get released unlike Mythos (which was looking really great before Flagship failed).  I want the best for Demigod and future game from GPG / Stardock.  This Beta went okay, but it could have gone a lot better, and the game would have been much better because of it.

I personally think it was a great Beta process.

The whole pre-order only access, however bad and negative it my seem on paper, the fact is it has produced some very good feedback and there was very little stupid suggestions/post like there is in a lot of other betas.

I think the Devs communicated with us on a regular basis which was great, and patched/updated Demigod very frequently (despite naming dates for Beta 3 and missing them three times+) which is exactly what is needed in the beta. Had lots of chats with Frogboy on irc AND the forums, as well as the other devs (Yarlen, Cari, kyro etc. etc.). Communication was A LOT better than most betas I have tried.

Stardock also implemented quite a few of the things we requested. There were a lot of gameplay changing mechanics that were mentioned, but I am not suprised they didnt make it into the game. Changing the mechanics of the game is simply not pratical and easy to do, and I doubt they would have been able to implement all our ideas.

The whole generals idea of them being "RTS'ish" failed, not because Stardock didnt listen to us properly, but because it was just a very impratical idea at heart. It was never going to work, and I dont think it would have been possible to implement it well into the game. We did ask them to give us more "micro-able" minions and make them less spammy, and they did that last patch (minions come in pairs now) which was great, but the problem isnt really fixable imo. Im happy with generals as they are, but many people will be angry that they are not what it was advertised to be.

So far thats a pretty positive view of the process. However I havent mentioned how poor the connectivity was. Fair enough, you guys were building from scratch and making a peer-peer only system to reduce lag and this is what a beta is for....But there is still MORE lag then a lot of other games out there that dont use peer-peer, and connecting to games is difficult. We were told Beta 1 = Engine Test, Beta 2 = Connectivity, Beta 3 = Gameplay. What we got though was more Beta 1 = Engine Test, Beta 2 = Connectivity, Beta 3 = Connectivity and a bit of gameplay. Thats what really annoyed me. You guys were still focusing on connectivity for about 90% of beta 3 and only 10% gameplay and you stopped making gameplay updates after about 2-3 patches (Beta 3C was last one was it? It had like two-four small balance changes).

So overall, rough ride, but was a good beta.

Things done wrong: 

- Beta schedule was too tight for the clinical, gradual introduction of phased elements.  Rejecting balance input in Beta 1 or 2 as "Beta 3 is when we will talk about balance" only works if Beta 3 is, indeed, about balance...as it was Beta 3 was about balance for about 2 weeks and then oops!  Private Gamma! 

It seemed like the devs would work on something behind the scenes in preparation for the appropriately scheduled phase and then if it didn't work immediately when brought in there would be no time to fix it.  Unless you are looking at a 9-12 month beta cycle it's better just to put what you have out there at the beginning and make a controlled schedule on the dev side.  

- The devs offered very little negative transparency.  General dev posts are always good, and we got a lot of them.  What we didn't get as much was a dev response to specific posts along the lines of "O yes, good idea, but we tried it out and it doesn't work."  You had page after page of tester ranting that could have been instantly corked by someone briefly sketching where the devs had already been with regard to that particular subject.  

- The Gamma at the beginning was held out as a carrot for the "best" testers.  AFAIK no testers were invited into the Gamma.  It's a status thing, but it also keeps the community invested.  Reward your testers and do it publically.   

- At times the devs seemed to ignore the testers on the most significant balance issues.  We have been talking about broken stunlocking mechanics since beta 1.  They never went away and that meant people stayed focused on them rather than finding the next exploitable mechanic.  Anyone remember when flag locks were broken?  They got fixed and we moved on.    After six months of screaming we might have fixed stunlocking at release.  But what did we miss meantime?

- Per below the small and committed test group is a good thing, but tends to stifle criticism.  If the devs weigh in against a critic then they start to foster an Emperor's New Clothes culture.  It's ok to be provocative but don't underestimate your influence.

Things done right:

- The posts detailing a specific aspect of developement like UI or AI were awesome and insightful.  Having the developer responsible for that aspect talk about it was inspired.  Similarly, getting general overviews from Frogboy was always good. 

- Tester ideas are visible in many, many ways in the final product.  In this sense the developers were very responsive to beta feedback.  

- The developers were present in the forums and talked about their ideas and tester ideas.  Really nice to see that, and I think it kept some tight mechanics in and some bad ones out.

- The pre-order Beta gives you a small, mature, and committed group of testers.   

- Finally, you guys produced a good, good game

It's really difficult to rate a Beta numerically so I'm going to use a few different indicators:

Intended: 9/10

Actual: 7/10

Compared to Industry Standard: 9/10

Combined: 8.3/10

Edit: after seeing the list of proposed day zero patch changes I feel bad about alot of the frustration I expressed above.  Turns out you guys were listening the whole time.  I'll leave it up as an indicator of tester feeling toward the end of the Beta process, but I'm upping my scores

I was hoping that since Stardock is a private company we would get to see the beta to a relatively polished (not perfect) point *before* gold master.  Unfortunately, I was disappointed to find out that Demigod went gold before everything was fixed (meaning the developers had to crunch at the end to get things fixed, that may not even be fixed by release like the connection issues.

Personally, I am OK with the beta test being a "test" but I think a game should stay in beta until it's at least stable (connection, crashes, ability to get into/out of games).  Overall I'm OK with release having some balance issues as long as stability is ready but going gold without stability just seems like a risky move for the life of the game (since stability will get it bad launch reviews, therefor smaller community).

When Sins did internal gamma testing, we all signed NDA's and were told not to talk about it.  So if you did know someone had been invited, it's probably only be because they are about to get their ass sued off for breaking an NDA.

Just an FYI:  but the beta we're playing isn't even related to the game that's getting put on the CD.  (Well, it is but that's not the point).  Basically, what we're playing is based on a very old version of the code while they've been busy fixing and updating stuff left right and center.  Take a look, for example, at Sins:  Beta 4 and the gold version didn't even come close to playing the same.  And the Day 0 patch is less SD's way of 'crunching' fixes in at the last second and more a part of their anti-piracy routine.

Above and beyond all of this, I'll agree with those that want more communication.  But I also understand why it's so hard to achieve that level of communication.  Given the sheer amount of posting going on on the boards, you'd need to hire someone just to keep track of the boards... and then he has to get permission before saying anything.  Ouch.

• Unless absolutely necessary, please don't make people download the entire beta client over and over and over. That was probably the worst part of the beta for me. I know sometimes that's what has to happen but there was a couple times the entire thing had to be downloaded again a couple of days later.
• An occasional update on when we can expect the next version to go up would be appreciated - ignore the whiners who wouldn't understand if you miss any of those dates. Everyone else would like to know if a new update is close or a month off. Frogboy's updates earlier this year about the planned progress of the beta were great. Beta's won't follow an exact schedule but it's nice to hear from the devs that things are progressing, even if slower than expected!

I didn't really participate much in the "process" beyond updating and following the progress of the game. I don't think Beta's really the right time to make wholesale changes to the game (e.g. Pink Pony Demigod) unless you're willing to push release back to match. I get the feeling a decision was made that April would be the month and thus it's bugfixing and balance from that point forth. Which I'm fine with, but some of the people above seem less okay with.

Now I'll go back to waiting for June to roll round.

    During this Beta, Time was Wasted, too much time.

   In my eyes, very little testing has actually been done because much of the game was in a stale, stagnet limbo for nearly five months without much testing going on. Beta 1 was perhaps the most effective phase of Beta because testers would openly use Hamachi to connect to one another and continue testing without servers. Beta 2 introduced a big mistake on the developers part. They shut down the LAN function and forced players to mostly non-extistant, almost always broken multiplayer match matching system. Since no one could connect, legit testing against other players was non-existant for the most part.

    Many testers had to fight AIs by themselves, myself included. But even I couldn't continue testing beyond Janurary. GPG failed to clarify what they added to Beta 2C. They simply said "We changed far too much to create a change long". Well I really wish they did, because once Beta 2C I was out for the count. I could no longer run the game due to horrible performance issues. Ever since then I kept posting my problem, listing various facts about the problem, kept posting all my troubleshoot methods (such as drivers for starters), but frankly it felt like no one was listening to me.

   Eventually they gave me an Email Address to someone in GPG QA, but in the end he really couldn't figure out what was going on and just pretty much said it was my fault and not the game. This is not true, as the game still suffers poorly after doing a complete system Reformat. I could no longer take part in a Beta Test I paid for, and I was very angry and frustrated because no one seemed to care about my problem at all until the very end of the Beta.

   Frankly, I'm very displeased with GPG right now because of their resistance to Beta Tester feedback about the games short falls. The game is good, but that's it, right now it's just "good". So much work still needs to be done, and hopefully Demigod will eventually shape up to be something. But frankly, people are just paying for another phase of Beta, just in a prettier package.

   Welcome to Beta 4 people.