The memory limits are starting to bite

Sorry kyro I meant in general not specific for galciv2, what would the advice then be considering near future game development

Successfully updated karma reason!

Regarding memory fragmentation - this is something we are aware of and will be attempting to address in 1.7. There are various things that can be done, such as allocating in bulk and using the Low Fragmentation Heap, though ultimately there is no permanent solution except for but some form of garbage collection. Do not worry, the spirit of memory-pinching is still alive!

Large Address Aware only helps if the user has also enabled the USERVA option, so we will probably be providing instructions on that (for both XP and Vista). There is a tradeoff, your kernel isn't just wasting that virtual address space, and increasing the space for user mode may cause problems elsewhere - but not for most people.

Successfully updated karma reason!

GreenReaper: Won't x64 versions of Windows be able to use more memory with "large address aware" without any further settings?

Also, can I take it from your comment that the large address aware flag is set (or is going to be set in the next expansion)? That would certainly put my mind at ease, since the new GFX memory management stuff in Vista coupled with 1 GiB graphics cards and BIG maps in GC2 could potentially be a problem (if I'm reading you guys correctly) if the flag isn't set.

Successfully updated karma reason!

Regarding memory fragmentation - this is something we are aware of and will be attempting to address in 1.7. There are various things that can be done, such as allocating in bulk and using the Low Fragmentation Heap

Sounds promising.

though ultimately there is no permanent solution except for but some form of garbage collection.

I don't think garbage collection is the ultimate solution. Surely it helps to remedy this in most cases, however even with GC, there exist usage patterns which trigger some sort of fragmentation.
The only true fix (at the cost of some performance) would be to use double-pointers, i.e. the program doesn't hold a pointer to a (logical) memory location, but instead a pointer to a pointer to the memory location. That way a background thread can move things around. This is in some way similar to the MMU, with is done in hardware however.

In general however that's pure overkill though

On a sidenote, I amazed at the shortsightedness of the architecture designers. From 32 bit to 64 bit? History has shown us that all previous limits have been exceeded very quickly due to the exponential growth rate of technology development. Personally I would have opted for 128bit systems.

Do not worry, the spirit of memory-pinching is still alive!

Stop the build scripts! We cannot release it yet. There's still a critical patch to reduce memory consumption by two bytes! Thousands of gamers are awaiting the new version, but won't someone please think of the memory?

Successfully updated karma reason!

People really need to quit assuming that because an application or game consumes memory through the course of running that it has some sort of "leak". Because when people are so quick to yell "leak" it may distract developers from the report of a legitimate leak.

on the rare occasion i've reported bugs i stick to a phenomenological approach: which is to say, i describe what i see and how it is a problem for me. i don't presume a thing about the underlying process; i know it's way over my head. it's unfortunate we have a culture that engenders people to want to sound like they know what they're talking about at all times.

as far as the 64 bit issue more broadly goes... it just depresses me, the whole subject. i love gaming, but i hate Windows, but i hate Apple more. it's not some fancy technological argument i have. since the original iMacs, Apple has designed its products (both hardware casing and GUI) like its consumers were kindergartners. but now Windows is going that way too, and i hate it. i miss the days of the DOS prompt. i'm just whining, i guess.

Successfully updated karma reason!

on the rare occasion i've reported bugs i stick to a phenomenological approach: which is to say, i describe what i see and how it is a problem for me. i don't presume a thing about the underlying process; i know it's way over my head. it's unfortunate we have a culture that engenders people to want to sound like they know what they're talking about at all times.

Too bad that the tools available to most users have absolutely no relevance or just report plain wrong things.

Example: Window's own task manager and it's memory usage column for the running processes.

I'm a developer for over a decade and still don't know what that number actually represents. Try this: Start some program, write down it's memory usage. Minimize it's window and restore it. Compare memory usage.
-> Usually A HUGE difference

Example: calc.exe, part of Windows.
Startup: 6.1MB. After minimize + restore cycle: 2.3MB, after yet another cycle down to 2MB even. Makes no sense.

Yet people scream bloody leak if the mem usage column rises. But is it really a leak or just some memory pages not yet reclaimed by the operating system?

Another example would be Firefox. People have been crying over it's memory consumption and that it would leak memory and so on. I'm using Firefox too and usually I have long running instances, sometimes I don't close Firefox for days. I've never had problems with it taking too much memory.

I summary, most users have no clue, they got broken tools and interpret the numbers reported by these tools in the wrong way.

Real life analogy: Users cannot distinguish between a leaky bucket and a sieve, but try to measure the leakyness by the amount of light that shines through it. Guess Windows are very leaky

Successfully updated karma reason!

I summary, most users have no clue, they got broken tools and interpret the numbers reported by these tools in the wrong way.

Real life analogy: Users cannot distinguish between a leaky bucket and a sieve, but try to measure the leakyness by the amount of light that shines through it. Guess Windows are very leaky

nice analogies! the sad thing is, i'm a really smart guy. it's a real tough question. 'digital literacy' is part of the general education program in the college i work for. but i realize it's kind of an exercise in futility. sure, we can teach kids to understand computers in 2007. but in 2011, everything we taught will be out of date. only the students working in the field, and the rare few super-power-users, will keep themselves up to date.

shakespere and einstein won't really become out of date. but how do you teach people about something that that shifts radically every few years? i dunno, as a 'really smart guy' i resent that Microsoft makes cute little metaphors to describe what's really happening. but i realize most people would be put off by the level of detail i want.

...i just rented a movie at a friend's recommendation, Idiocracy (dir. Mike Judge, 2006). it was great, and this line of lamentation fits right in with the theme.

Successfully updated karma reason!

nice analogies! the sad thing is, i'm a really smart guy. it's a real tough question. 'digital literacy' is part of the general education program in the college i work for. but i realize it's kind of an exercise in futility. sure, we can teach kids to understand computers in 2007. but in 2011, everything we taught will be out of date. only the students working in the field, and the rare few super-power-users, will keep themselves up to date.

Really understanding? Or just being able to use the current reincarnation of Windows + Office? Two different things.

shakespere and einstein won't really become out of date. but how do you teach people about something that that shifts radically every few years? i dunno, as a 'really smart guy' i resent that Microsoft makes cute little metaphors to describe what's really happening. but i realize most people would be put off by the level of detail i want.

Does it really have shifted that fast? The basics are still roughly the same then they were 50 years ago. We still have CPUs, RAM, secondary storage (be it magnetic tapes or modern hard disks, the purpose is the same). Pick up an old book, everything there is still relevant if you abstract away from any specific hardware.

Explaining this all to others shouldn't be that hard. They need to be interested though, no point in trying to teach Bob the builder ^^.
After explaining the basic components you explain how to talk to them in a very low level, but abstracted from the current platform. Some theoretical assembler language on some theoretical architecture. Next give a brief overview of a high level language.
That has covered the hardware and bridged over to the software. Now explain what an operating system does, file systems, libraries and so on.
In a last step, explain the modern metaphors which are preventing a clear view on what's really going on a recent computer.

..i just rented a movie at a friend's recommendation, Idiocracy (dir. Mike Judge, 2006). it was great, and this line of lamentation fits right in with the theme.

Great movie.

"If you have one bucket with 5 gallons and another bucket with 2 gallons, how many buckets do you have?"

I think in such a world I would perish.

Successfully updated karma reason!

"If you have one bucket with 5 gallons and another bucket with 2 gallons, how many buckets do you have?"

I think in such a world I would perish.

Are we discounting alternate time lines and parallel universes?

Successfully updated karma reason!

Really understanding? Or just being able to use the current reincarnation of Windows + Office? Two different things.

a little bit of real understanding, and then on to production (mostly Adobe products rather than MS, actually - they do some digital film/image/music editing, sometimes learn DreamWeaver and occasionally we let them get away with PowerPoint).

Does it really have shifted that fast? The basics are still roughly the same then they were 50 years ago. We still have CPUs, RAM, secondary storage (be it magnetic tapes or modern hard disks, the purpose is the same).

true. and even past the level of components, a logic gate is still a logic gate - at least until quantum computing becomes a reality (if and when).

Successfully updated karma reason!

I don't think quantum computing will ever enter the realm of consumer products. And if so, then only in form of a co-processor or so.
The algorithms are just too different, a quantum computer is unsuited for everyday tasks. Only a very narrow range of algorithms can work on quantum computers.
And on top of that, it's not even certain yet that quantum computers are viable at all, there's still the possibility that a quantum computer won't have more power than a traditional computer.

Successfully updated karma reason!

You know, I think a big part of the problem is that people are used to things both coming easily and not needing to learn or think.

I've dealt with training a significant number of people on the basics of computer support over a number of years. There is always a point in time where you just tell someone that they should have the tools to figure out their own question themselves and it's time to start working on teaching them how to ask the right questions to draw on their knowledge. A typical success will say they don't know where to start. I respond with, "Well what do you know about X?" and keep working down until they do know where to start or we can correct the knowledge deficit. Probably 30-40% simply will not try...They want the spoon fed answer, not the ability to find the answer themselves. Mind, most of these are people drawing a salary while this is happening and they need to learn this to hold their job!

It explains so much about our society when you realize that such a large group of people never learn or think!

When I interview people now, I always ask at least one question that requires them to think to respond. I don't even care if they get the answer right, I just want to know that they can think about a problem...As expected this winnows a good, solid, one third of the candidates.

Successfully updated karma reason!

I'm not skeptical about quantum computers at all. It's not necessarily about the performance, it's about the power consumption and area usage. This could mean "flash" drives with more memory than today's hard drives, and mobile CPU's and video cards with 1/10th the battery drain. There's just several key technology breakthroughs researchers need to find. There's: a) how to get subatomic particles to "remember" states indefinitely (i.e. store bits),

how to create switches, analogous to today's transistors, c) how to create basic logic gate circuitry analogous to today's CMOS logic--not just that, but how to create them in a self-contained manner such that you can create blocks of logic, that all behave the same way regardless of what circuits they are interfacing to, d) how to do this in a practical, cost-effective manner.

This all sounds way off, but the reality is, our current integrated circuit technology is really reaching its limits. Transistors are getting down into the 32nm range, which is smaller than the wavelengths of the light which etches them. And when you get that small, transistors start to really burn a lot of power just sitting there. I remember when Intel was first researching 1GHz processors, and at the time that sounded all sci-fi and crazy. Well, we're at 3GHz now. Pretty soon you'll get 1GHz on your cell phone. We might be able to squeeze out a few more GHz, but we simply are not able to go 10GHz and above with our current family of technology.

Successfully updated karma reason!

Wouldn't it be possible to store rarely used objects in some kind of database without direct memory mapping?

Successfully updated karma reason!

Wouldn't it be possible to store rarely used objects in some kind of database without direct memory mapping?

That would have the same effect as a swapfile. Unless you can predict when you need some data and load it back before it's needed, you'll suffer from the low disk performance.
Which also raises the question of the complexity of the predictor.

Successfully updated karma reason!

I'm not skeptical about quantum computers at all. It's not necessarily about the performance, it's about the power consumption and area usage. This could mean "flash" drives with more memory than today's hard drives, and mobile CPU's and video cards with 1/10th the battery drain. There's just several key technology breakthroughs researchers need to find. There's: a) how to get subatomic particles to "remember" states indefinitely (i.e. store bits),

how to create switches, analogous to today's transistors, c) how to create basic logic gate circuitry analogous to today's CMOS logic--not just that, but how to create them in a self-contained manner such that you can create blocks of logic, that all behave the same way regardless of what circuits they are interfacing to, d) how to do this in a practical, cost-effective manner.

Isn't that more like Spintronics and optical computers? Quantum computers however are an entirely different concept. Wikipedia has some great articles about this.

Successfully updated karma reason!

I don't think quantum computing will ever enter the realm of consumer products. And if so, then only in form of a co-processor or so.
The algorithms are just too different, a quantum computer is unsuited for everyday tasks. Only a very narrow range of algorithms can work on quantum computers.
And on top of that, it's not even certain yet that quantum computers are viable at all, there's still the possibility that a quantum computer won't have more power than a traditional computer.

i'd tend to agree with you, but my conclusion stems more from the uncertainty principal than from an understanding of computation.

i think a more significant breakthrough will come when optical computers become commonplace and affordable. i have a fairly interesting link about some recent research along these lines, but it's it's in my yahoo mail, which is being wonkey at the moment so i'll have have to post it later.

Successfully updated karma reason!

Optical computers aren't the same thing as quantum computing--yet--and I'm more skeptical of optical computing. You have to fab optical components on Galium Arsenide wafers instead of silicon and the power consumption is way up there. Basically all your wires are lasers. You can imagine the power that takes. It would be nice if you could use optical components for just your long transmission lines (such as on PCB boards), but the technology to fuse together GaAs with silicon doesn't exist yet.

On the other hand, if you started dealing on the tens of photons level, now you're talking--but now you're getting back into quantum computing.

Successfully updated karma reason!

I'm more skeptical of optical computing

ahem, here's that link: WWW Link

gotta love skepticism

Successfully updated karma reason!

as far as the 64 bit issue more broadly goes... it just depresses me, the whole subject. i love gaming, but i hate Windows, but i hate Apple more. it's not some fancy technological argument i have. since the original iMacs, Apple has designed its products (both hardware casing and GUI) like its consumers were kindergartners. but now Windows is going that way too, and i hate it. i miss the days of the DOS prompt. i'm just whining, i guess.

If you hate microsoft AND apple you can always use a UNIX clone. If you REALLY like abusing yourself you could do a stage 1 install of Gentoo Linux (full OS compile) or get one of the DOS clones such as FreeDOS (i think is the correct one) And if its the chipset that is the problem (Intel/AMD or Apple) you can alwasy make the switch over to one of the other cores such as sparc or alpha cpus.

Successfully updated karma reason!

Perhaps future games can be engineered as multiple cooperative processes, each with its own memory space.

Successfully updated karma reason!

Microsoft missed an opportunity with Vista to encourage 64 bit.

64-bit driver support was always going to be an issue but MS put fuel on the fire by refusing to allow unsigned drivers to be loaded (yes, there is a walk-around but it is cumbersome)

Not being able to run 16-bit applications is also a concern.

Successfully updated karma reason!

The memory limits are starting to bite

Why you run out of memory