Tuesday, March 24, 2015
Wednesday, March 18, 2015
On Programming Languages and Complexity
It's been awhile since I ranted a bit on programming methods. I've had some thoughts on languages recently that I want to write down to help solidify. Bear with me or skip this if languages aren't your thing.
In college I was taught in Java, with a bit of C, and even less of other languages like Python, ML, Prolog. I and Tristan learned C++ to write our first game. Later I took more software engineering courses and learned some new patterns for complicated projects. I wrote a pile of libraries in C++ and got a pretty good handle on it. At Bunkspeed I continued with C++ but we pretty quickly replaced that with C#, which I worked with from version 1.1 until I left. World has been almost entirely C# since its inception in 2007. Now for work I'm working with C++ again, now for approaching 2 years.
So I have a pretty good comparison now of C++ and C#, both used for larger codebases. Bunkspeed's main product was pushing 600k, my current work is around 1M, and the World project is around 150k. I admit I do have better experience in C#; I try to keep that in mind.
I firmly believe that what you can do in one you can do in the other. The limitations of both can be circumvented to achieve the same ends.
C# has the reputation of being slow, and the problem of being single platform and having a harder time interacting with 3rd party libraries (which are usually native). The slowness problem I think is a red herring; optimizing is usually done at the algorithmic level, which is indifferent to language. The GC and memory management can be controlled if you desperately need to, but honestly you do not. .NET Native coming out soon should alleviate the rest of whatever performance issues exist. The single-platform issue is improving with Mono and recent C# changes announced by Microsoft. The third party library issue is more complicated and I don't know of a good solution. P/Invoke is not a good solution, nor is C++/CLI.
C++ has the reputation of being fast and powerful, but complex, low-level and imprecise. The low-level issue can be resolved by writing code within C++ to add high-level features. C++ code MUST manage memory, and so large C++ codebases MUST write something to track pointers and allocations. In the end these systems approximate garbage collection, and they work. By 'imprecise', I mean that there are often unanswerable questions, the answers for which are not apparent immediately in the code. The size of pointers and integers, the nature of strings, how 'out' parameters are passed, naming conventions, includes ordering, namespace management, smart-pointer management, etc. All of these things have to be handled in some way by the programmer. And they can be with great grace and power, but they must be handled.
Herein lies my point. C#'s failings are fixed by external structure. C++'s failings are fixed by internal structure.
The upper limit of manageable complexity in C++ is lower, because some of that manageable complexity is ubiquitous and cannot be ignored. When writing in a large C++ codebase, you cannot forget about the internal structures that handle C++'s limitations. Every last line needs additional consideration as you ponder string encodings and which of 6 ways to declare your pointers (smart, shared, dumb, reference, constness, etc...).
The upper limit of manageable complexity is higher in C# than in C++, because when the structure becomes more complicated, the code does not. The complexity is managed in a parallel way, a way that does not impact the programmer when trying to read and understand a lone function.
Maybe with time I'll be able to background the necessary considerations in C++ and be able to acheive similar productivity. Or maybe I'm just spoiled by the cleanliness of C# lambdas, extensions, and continuations. The only thing to do is to try and see if I can get there. Tsuyoku Naritai indeed.
In college I was taught in Java, with a bit of C, and even less of other languages like Python, ML, Prolog. I and Tristan learned C++ to write our first game. Later I took more software engineering courses and learned some new patterns for complicated projects. I wrote a pile of libraries in C++ and got a pretty good handle on it. At Bunkspeed I continued with C++ but we pretty quickly replaced that with C#, which I worked with from version 1.1 until I left. World has been almost entirely C# since its inception in 2007. Now for work I'm working with C++ again, now for approaching 2 years.
So I have a pretty good comparison now of C++ and C#, both used for larger codebases. Bunkspeed's main product was pushing 600k, my current work is around 1M, and the World project is around 150k. I admit I do have better experience in C#; I try to keep that in mind.
I firmly believe that what you can do in one you can do in the other. The limitations of both can be circumvented to achieve the same ends.
C# has the reputation of being slow, and the problem of being single platform and having a harder time interacting with 3rd party libraries (which are usually native). The slowness problem I think is a red herring; optimizing is usually done at the algorithmic level, which is indifferent to language. The GC and memory management can be controlled if you desperately need to, but honestly you do not. .NET Native coming out soon should alleviate the rest of whatever performance issues exist. The single-platform issue is improving with Mono and recent C# changes announced by Microsoft. The third party library issue is more complicated and I don't know of a good solution. P/Invoke is not a good solution, nor is C++/CLI.
C++ has the reputation of being fast and powerful, but complex, low-level and imprecise. The low-level issue can be resolved by writing code within C++ to add high-level features. C++ code MUST manage memory, and so large C++ codebases MUST write something to track pointers and allocations. In the end these systems approximate garbage collection, and they work. By 'imprecise', I mean that there are often unanswerable questions, the answers for which are not apparent immediately in the code. The size of pointers and integers, the nature of strings, how 'out' parameters are passed, naming conventions, includes ordering, namespace management, smart-pointer management, etc. All of these things have to be handled in some way by the programmer. And they can be with great grace and power, but they must be handled.
Herein lies my point. C#'s failings are fixed by external structure. C++'s failings are fixed by internal structure.
The upper limit of manageable complexity in C++ is lower, because some of that manageable complexity is ubiquitous and cannot be ignored. When writing in a large C++ codebase, you cannot forget about the internal structures that handle C++'s limitations. Every last line needs additional consideration as you ponder string encodings and which of 6 ways to declare your pointers (smart, shared, dumb, reference, constness, etc...).
The upper limit of manageable complexity is higher in C# than in C++, because when the structure becomes more complicated, the code does not. The complexity is managed in a parallel way, a way that does not impact the programmer when trying to read and understand a lone function.
Maybe with time I'll be able to background the necessary considerations in C++ and be able to acheive similar productivity. Or maybe I'm just spoiled by the cleanliness of C# lambdas, extensions, and continuations. The only thing to do is to try and see if I can get there. Tsuyoku Naritai indeed.
Tuesday, March 17, 2015
Mana Manipulation (and Videos!)
At the tail of the last post, I said that the incendence system and environmental magic were next. Guess what! It was next. It was complicated, but it is working and it is good :)
Let's break this down and make some new video clips.
So. Environmental Magic. This is a system designed to let the player change the terrain. We already had (somewhat stable) terrain changes, but the nature of those changes is quite a bit different from the new system. A few of the old-style changes included the bridge builder (featured at the end of the tutorial, and on Outpost), and a couple hacky commander skills that no one but I used. These will be gone soon. A second set of old-style changes are the elemental impacts system, which will be staying. It compliments the new system nicely, and will be used for non-mana manipulation skills. Any explosions or active flames and whatnot will still use the old impacts system, which works well.
The new system is built specifically to give the skills to the player. To that end it has to be limited and controlled enough to prevent the player from abusing the skills to absurd extent. Honestly letting the player arbitrarily change the terrain is a terrible idea if you want to control the gameplay. The fact we're trying this at all is a testament to pushing boundaries that oughtn't be attempted. The payoff of course, is that no one else has really succeeded in giving this style of control, so if we do succeed to any extent with it it's a mark in our favor.
So anyway, the limiters. The new system is pretty much entirely based on the concept of conservation of energy. The energy in this case is not potential, nor kinetic, nor chemical, it's just Mana. Every type of material has a mana density; how much mana they contain per unit volume. Mana, by the way, comes in 4 forms, as per classical elements. Mana is conserved. From this concept we defined 5 basic operations, which are pieced together to build the actual skills.
First is extract; this takes a chunk of material and converts it into a less mana-dense form, while preserving overall mana. The net effect is that new material volume is created, but it is less mana-dense.
Second is draw; this is very similar to extract, except instead of creating new volume, the excess mana is absorbed into the caster, stored and usable later. The net effect is that material is converted to weaker types in the local area.
Third is infuse; this is the opposite of draw; mana is pushed into materials, converting them to more mana-dense forms without change in volume.
Fourth is compress; which is the analogue of extract to draw for infuse. Compress reduces overall volume when converting to more mana-dense forms.
Last is shift; which just moves a chunk of mana from one location to another.
Each has some mana cost; small but important.
The nature of these is that mana is conserved or lost. This enforces that things do not get too crazy on the map; There's a limit to what can be generated; implied by the original amount of mana present. If we build a map to play on that consists entirely of sand (a very low-mana material), then that by itself completely removes any possibility of manipulation. There simply isn't enough mana to do anything with. The density also limits any volume changes. The total mana of a bridge's material has to be put into the bridge to construct it, and that mana (plus expenses) has to come from somewhere.
After that we added more limiters. Extractions and Infusions and Shifts have an overall power requirement, per material. Granite is hard to work with; extra skill is required to work with it. Likewise at the bottom of the earth-spectrum, sand is complicated. This limits the player in where he can manipulate the terrain, but in a graphically obvious way that can be played around.
Then after that we have to manage reach; how far the operations can work through the ground. I invited the concept of mana-pressure; how much pressure the manipulator can exert on the terrain. Pressure travels out through the materials using a conductivity calculation that depends on their mana density. Earth pressure reaches further through earth-aligned materials, for example. This conductivity is fairly unbalanced; so reach will fall along aligned conduits in the ground, potentially extending total range in the right circumstances. You could even generate explicit lines of high conductivity material to control how the mana flow is directed.
How about some visuals?
Here's the first version of the Mana Manipulation skill tree.
And here are short videos for the various new skills in action:
Mana Draw (Pulling mana from the grass and dirt, turning it into lesser rocks. The first outward pulse represents the mana pressure, that is, the effective range of the draw force)
Mana Infuse (Pushing mana into dry mud and mud, turning into mud and dirt, then running out of mana)
Mana Extract (Extracting Gravel from Shale, and Sand from Gravel, to produce a column of material at distance)
Mana Wall Construction (Drawing a line in the ground, then infuse/extract/shift to move it into a wall shape)
Mana Bridge Construction (1) (Extract/Shifting a bridge out of the ground)
Mana Bridge Construction (2) (Building a second bridge from less dense materials. You can see that sand is getting used at the end of the bridge, since the source area is running out of mana-dense materials. This bridge could not get much longer, and when physics is written, the sand would fall out of it)
Mana Shift-Smoothing (Using Shift to smooth out the surrounding area, making it traversible)
Hope you enjoyed!
Next time I'll talk about Incitement and Incendence. It is currently WIP, maybe halfway through implementation.
Let's break this down and make some new video clips.
So. Environmental Magic. This is a system designed to let the player change the terrain. We already had (somewhat stable) terrain changes, but the nature of those changes is quite a bit different from the new system. A few of the old-style changes included the bridge builder (featured at the end of the tutorial, and on Outpost), and a couple hacky commander skills that no one but I used. These will be gone soon. A second set of old-style changes are the elemental impacts system, which will be staying. It compliments the new system nicely, and will be used for non-mana manipulation skills. Any explosions or active flames and whatnot will still use the old impacts system, which works well.
The new system is built specifically to give the skills to the player. To that end it has to be limited and controlled enough to prevent the player from abusing the skills to absurd extent. Honestly letting the player arbitrarily change the terrain is a terrible idea if you want to control the gameplay. The fact we're trying this at all is a testament to pushing boundaries that oughtn't be attempted. The payoff of course, is that no one else has really succeeded in giving this style of control, so if we do succeed to any extent with it it's a mark in our favor.
So anyway, the limiters. The new system is pretty much entirely based on the concept of conservation of energy. The energy in this case is not potential, nor kinetic, nor chemical, it's just Mana. Every type of material has a mana density; how much mana they contain per unit volume. Mana, by the way, comes in 4 forms, as per classical elements. Mana is conserved. From this concept we defined 5 basic operations, which are pieced together to build the actual skills.
First is extract; this takes a chunk of material and converts it into a less mana-dense form, while preserving overall mana. The net effect is that new material volume is created, but it is less mana-dense.
Second is draw; this is very similar to extract, except instead of creating new volume, the excess mana is absorbed into the caster, stored and usable later. The net effect is that material is converted to weaker types in the local area.
Third is infuse; this is the opposite of draw; mana is pushed into materials, converting them to more mana-dense forms without change in volume.
Fourth is compress; which is the analogue of extract to draw for infuse. Compress reduces overall volume when converting to more mana-dense forms.
Last is shift; which just moves a chunk of mana from one location to another.
Each has some mana cost; small but important.
The nature of these is that mana is conserved or lost. This enforces that things do not get too crazy on the map; There's a limit to what can be generated; implied by the original amount of mana present. If we build a map to play on that consists entirely of sand (a very low-mana material), then that by itself completely removes any possibility of manipulation. There simply isn't enough mana to do anything with. The density also limits any volume changes. The total mana of a bridge's material has to be put into the bridge to construct it, and that mana (plus expenses) has to come from somewhere.
After that we added more limiters. Extractions and Infusions and Shifts have an overall power requirement, per material. Granite is hard to work with; extra skill is required to work with it. Likewise at the bottom of the earth-spectrum, sand is complicated. This limits the player in where he can manipulate the terrain, but in a graphically obvious way that can be played around.
Then after that we have to manage reach; how far the operations can work through the ground. I invited the concept of mana-pressure; how much pressure the manipulator can exert on the terrain. Pressure travels out through the materials using a conductivity calculation that depends on their mana density. Earth pressure reaches further through earth-aligned materials, for example. This conductivity is fairly unbalanced; so reach will fall along aligned conduits in the ground, potentially extending total range in the right circumstances. You could even generate explicit lines of high conductivity material to control how the mana flow is directed.
How about some visuals?
Here's the first version of the Mana Manipulation skill tree.
And here are short videos for the various new skills in action:
Mana Draw (Pulling mana from the grass and dirt, turning it into lesser rocks. The first outward pulse represents the mana pressure, that is, the effective range of the draw force)
Mana Infuse (Pushing mana into dry mud and mud, turning into mud and dirt, then running out of mana)
Mana Extract (Extracting Gravel from Shale, and Sand from Gravel, to produce a column of material at distance)
Mana Wall Construction (Drawing a line in the ground, then infuse/extract/shift to move it into a wall shape)
Mana Bridge Construction (1) (Extract/Shifting a bridge out of the ground)
Mana Bridge Construction (2) (Building a second bridge from less dense materials. You can see that sand is getting used at the end of the bridge, since the source area is running out of mana-dense materials. This bridge could not get much longer, and when physics is written, the sand would fall out of it)
Mana Shift-Smoothing (Using Shift to smooth out the surrounding area, making it traversible)
Hope you enjoyed!
Next time I'll talk about Incitement and Incendence. It is currently WIP, maybe halfway through implementation.
Subscribe to:
Posts (Atom)