Music! Music! Music! Smash!

Now, the introduction down there says that these here blag is for four topics: The World Project, which will inexorably get the most attention, Bunkspeed, Programming, and Music.

So let's get into that last one there. I've been playing unprofessionally for a long time. I composed a bit, though I don't hold much weight by most of my compositions. For the most part this aspect of my life fell by the wayside through college. The only exception was a single course in composition, in which I wrote a piano duet, and then later when I finally had a room for my parents' piano, which was gifted to me.

So I began playing again, just my own tinkering as usual. Rather than writing songs I seem to write basic melodies and chord progressions, and improvise around them. The basic pattern is always very simple, but they are fun to play, and Irene seems to like listening to it.

I've yet to really pin down how to describe the style of music I play. I may be too self-conscious about it to try and define it. So I'll leave that to others. On that note, here's the first decent recording I made. This one has some audio issues (especially near the beginning, it gets better as it gets louder), but stick with it and let me know what you think! This one is a medley of the 6 main rhythms I've been improvising upon.

Medley 1

I'll work on an embedded player eventually, but I'm not terribly experienced at that sort of thing.

Yet.

Anyway, I'll be making a point to keep recording. Let me know what you think!

Forward! For a game!

'The state of things' was a pretty dense description of what has been written, code wise. Let's get into the more recent work, actual progress on the actual game.

Rather than a list, let's just say what we have.

We have a zone that runs a scenario. Players can connect to the zone, and are then given several dialogs to select a team and commander type. AI players will be created to fill out any empty teams. The scenario specifies the team count, typically 2 but it will work with any number. When the scenario begins the commanders will move to a spawn point, summon their primary units, and they'll enter a combat state. If they enter proximity with an enemy unit they'll begin an attack sequence. The user can execute a couple different 'AI' actions, which tell the units where to go and how to act. This is all pretty simple stuff, but clearly necessary.

More time has been spent on the unit development. We've come up with a relatively detailed design for the primary unit types, how they develop, how they act, and what control the user has over them. Primaries develop with a simple class system; Each class has 10 ranks, each rank may grant new abilities and attributes. Some of these are allowed to be permanent improvements, which continue to take effect after the primary changes class. Some classes have prerequisites on other class ranks. This forms a fairly wide tree of possible primary unit types.

The intent for the primaries is that they are your group's customization. Selection of which class they are (and which they have been in the past) determines your ability selection. It also determines how you'll want to build formations and develop your commander. The commander's development methodology is not as well determined or implemented as of yet, but the plan is to use a fairly simple skill system. Commanders focus on formations, managing morale, and utilizing territory, rather than styles of attacks and defenses. This makes them a bit more complicated to design, but more interesting as well.

We started designing each primary class in the generic server editor, but quickly ran into a scalability problem. The editor was not designed for mass data entry. Even just the 4 introductory classes would require 50 states and 40 actions, almost all of which are identical but for small changes. Since we want there to be a wide variety of primary types, this had to be resolved.

So we started building a file to store all our data in, which we would parse, and then construct the gamedata from. This is remarkably easy with the World as it is. Next, since now I'm not the only one working on the system, we decided to make this file sharable and easily to collaborate on. So we uploaded it to google docs. Next, I decided that downloading the file, processing, and running it, was entirely too cumbersome, so we wrote some code to do all that automatically.

So now we have an online document, and the system automatically builds the classes from it. This is very cool. Trust me. We expanded the file to include action declarations, state descriptions, class data, and some commander data as well. 

So great! We've got some data. Creating a new class is wonderfully straightforward; add its abilities and states into the doc, make the class ranks, and rebuild. Takes about 10 minutes. Sure, there isn't a client to actually play with them yet, but that'll get here soon. 

About this time someone mentioned how cool it would be to have a simulator to test class balance. That sounded fun, so I wrote it. Continuing on from the previous kick, I decided that we'd use another google doc to store the sim inputs. And heck, why not just spit it back out into the spreadsheet as well? Oh, and since we're using a scheduler for all the operations, why not just fake the timing so that it runs at 100% CPU, instead of waiting for realtime to pass? Sure, why not?

So we have a single program now, that reads in the class data from online, builds the class data, actions, and states, then runs every single class rank against every other class at the same rank, 100 times, then spits the output back to a google spreadsheet. 100 fights takes about 7 seconds, so the current testing spreadsheet (about 70 comparisons, 100 iterations each) takes a couple minutes. But that's 7000 fights! You can watch the spreadsheet just fill in with data. Make a small tweak; rerun just one line, test the output, and repeat! It's addictive. I think I'll go make a new class now, just for fun.

Wait, maybe I should make the game playable.

Nah, the sim's good enough. We'll just play that. 

An Aside: Independent software development methodology

I said last time that I sounded lazy and that it was intentional, and that I'd talk about it later. This is later. Welcome to later.

The idea is that with a side project, motivation and managing time are the most important aspects of the project.

If you're not motivated internally (since there are no external motivations like money), you just won't do it. I ran into this twice, once after writing libraries post Twilight 1, and again in the first year or so of World. Work just slows down because it's boring. So first and foremost, you have to work on the part that is interesting, even if that means that other pieces don't get written. Yet.

Secondly, if you waste your time doing menial tasks, you never get to the interesting stuff, and the motivation detractor pulls you in again, and the project stops. Getting rid of the cruft is extremely important for a long-winded independent project. To that end I made a fateful decision to write this project in C#. This has saved me countless hours of tasks that should have fallen by the wayside in the last century. Writing a kick-ass reference counter does not qualify as interesting code. And if it does, then I'm supremely jealous. In any case, any costs of C# have been utterly overshadowed by the impractibility of not using it. 

So to counter the first one, I eventually picked a specific game, rather than being working on a toolset, and I started work on specific aspects of the game. This of course shows exactly which parts of the generic system just don't work. The kicker is, I had to throw stuff away or just ignore code I'd already written! Any professional programmer can tell you that this sucks but is still common. But in an indie project where such things can sap the entire energy out of a project, it's a bit more painful.

Now, working with Bunkspeed and specifically a guy named Jamie Briant, taught me about Agile development. Maybe I'm drinking the Kool-aid here, but being agile is essential. Now, Agile development has a lot of codified and formal methods associated with it. All that is cruft. The important thing is this: Don't do what you don't have to. 

So remember how I wrote this complicated proxy and server/client system for the gamedata, but never used it? It's rotted by now I'm sure. I didn't 'have' to do it, but I did, and it was a waste of time.

Aha! But there is a catch! Maintaining interest is of extreme importance! At the time, the network stuff was what I wanted to do (otherwise I wouldn't have written it, yes?). So, stopping yourself from doing what you want to is another good way to kill your project.

So which takes precedence? Being interested or being efficient? It's a good question. Ideally one that doesn't have to be answered, if you're lucky with your project. In my case, if I'd started the game instead of working on the generic solution (which I've already stated I should have), the decision would have never become relevant! 

I suppose the concluding advice for this aspect is to have a goal. Know what you are aiming for and keep it in sight. If you think you're 90% there, the last 10% always becomes the most interesting. So set a goal that's 10% out, then when you get there do it again. 

The State of Things

The World Project was started in January 2007. There's been plenty of time to get moving on things, even though the end goals were not known at the time. Despite this, I was working throughout this time, and I definitely gave more attention to work throughout the process. Basically, the World got worked on whenever I had the spare time, energy, and willpower. I haven't obsessed over it like I have previous projects, but the further it gets along, the clearer the resulting vision becomes, and the greater the drive to reach it. As always it feels like the last 10%, but anyone in software knows: The first 90% of a project takes the first 90% of the time, the last 10% takes the other 90% of the time.

So here's what I wrote:

• State Machines: Separates machine structure, machine layout (visual), and machine execution. Machines can be edited using a visual layout method. States and Transitions execute behaviors. Transitions utilize a TriggerListener, which invoked the transition via an event. Decision points can be created, and evaluators on transitions that exit from them. Sub-machines are somewhat supported, but I ended up doing these a different way within the World, so I scrapped the original methodology. Idealy we'd have nested machines, but until I need it, I won't write it. Writing the visual editor, generic methodology for creating behaviors/transitions/evaluators, and allowing extensibility, all took some time.
• Scheduler: The main game thread executes within a scheduler, from which any task may be invoked at any time. It's single threaded (simpler), but does the job. Priorities have been in the task system from day 1, but never used. I'll get there when I need it (noticing something, I bet, but don't mistake it for laziness).
  
• Socket wrapper: Wrapped a socket based communication system for ease of use. UDP or TCP. It's designed to take a set of possible objects and optimize for transmitting the limited set. Originally it used the C# BinaryFormatter. Once I built a better data protocol the transmission size shrunk by a factor of 20 or so. In any case, it makes data send and receive very easy. It also fires an event (for the receives) for each individual type of data received. Since the data types are known ahead of time it's easy. I like being clever with C# and generics.
• Network Scheduler: A second thread, similar to the scheduler, that handles incoming and outgoing (well, someday) traffic scheduling. Because of this and the scheduler, an idle zone sits at 0% CPU usage. It might not seem like a big deal, but it means that the code is structured well enough to know when it is idle and not waste resources. Coming from the typical client-side game process of update->draw->repeat ad nauseum, this is quite the change.
• Directory server: A simple client/server protocol that supports looking up the location of a zone server. It's really pretty easy given the above. Not even used yet, but I wanted to get right into it.
• Generic GameDatabase system: This was probably implemented much more complicatedly than necessary, but I had some incorrect ideas about the layering of this system. This is a gnarly bit of code that represents the storage and transmission mechanism of the game system data. This includes the mechanics, formulae, attributes, etc, even though this layer doesn't know what the data even is. Probably more complicated than necessary. It also supports client/server updates of the gamedata, but my hunch (2 years later) is that it was a waste of time. Oh well. It's neat code anyway.
  
• Lua scripting for embedded language: Lua can be bound into behaviors, transitions, and evaluators of state machines. The higher up system can also use lua to evaluate Effects and Actions. This was pretty much cake.
• GameSystem implementation of the GameDatabase: Specific types to implement the World project. I also wrote a system for editing the data, using some wacky Windows Forms code. It's mostly PropertyGrid based, so it's a bit... raw..., but functional if you happen to be the writer. There's some pretty crazy behind the scenes stuff in there though. For example, if you assign a State Invoke to a behavior, and that state (in the database) has parameters, it will automatically setup the parameter list for you, with the provided defaults. It's amazing how much a pain UI can be.
• Zone implementation of the GameSystem: Yet another nested layer! This is the sort of main execution code. It takes the GameSystem and implements an instance of it: A Zone and its constituent Entities. There's a lot of stuff here that I'll leave for later.
• Zone Client/Server Protocol: Uses the network stuff to keep a set of clients up to date and allow them to play in the server. The relevant data is all in client-space now, even though we don't have much of a client to view it.
  
• Custom modules: Zones can establish a plugin module to provide additional types of behaviors, triggers, evaluators, effects, and actions. It also provides a set of options to the server UI which happen to have become extremely useful.

Code wise that's relatively where we stand. Of course there's more detail than that implemented. It's not a small codebase, but I wouldn't call it massive. There are 2 major points I want to touch on, since they're at the core of the design. First is the event-driven nature of the system. Second is to address the 'but I haven't done that yet' part, since I think it's one of the strengths of my development method, despite sounding like laziness. The second I'll delay for another post entirely, the first I think I'll write now.

Event driven systems have a lot of benefits, and a lot of problems. First let me define that as I see it. Event driven systems work off of knowing when things happen, and reacting, rather than poking through the system to see if something needs to happen. It's reactive rather than proactive. Why do things this way?

Well, consider the cost of poking through the system analyzing potential occurances. What if nothing needs to happen? You just wasted time! You also forced the entire zone to cycle through memory! That's more time! You also might have missed something happening if the cycle takes too long! That's inaccurate! If the expected zone size involves crazy-large numbers of entities, you just can't afford to run naively.

Now consider the cost of figuring out when things will happen. First, you have to somehow know how to know when things'll happen. This varies dramatically from thing-to-thing. Right there you know this will be harder than the alternative. Each case is different. Let's consider a set of possibilities:

1. X seconds pass: This is easy! Just tell the scheduler to fire in X seconds. Yay!
2. An Entities is moving north: Wait, when do we care? Well, the entity needs to move! Continuously! When do we move it? Every millisecond? Every second? Well, I guess in some games you could get away with infrequent moves, but not many. Chess maybe. And the first? Well, then you're basically being proactive again, you're forcing anything that moves to be updated even when it doesn't really care. So instead, represent motion as a start point and a vector at a specific time. Then the location is a function rather than a constant. Physics complicates things, but as long the equations are solvable you can do this. Now, this limits the game, of course. No crazy space physics, nothing hyper-non-linear. But most games don't need those things, and if you did, you wouldn't write an event-driven server.
   
3. Two entities collide in the world: Typically collision is done after everything moves, then analyzing if things intersected. Well, we aren't explicitly moving. So how to do this? Well, we can intersect two lines, right? Just extrapolate the entity's location into the future and figure out the next thing it collides with. Use some aggressive space-time culling to remove unnecessary tests. And then, any time anything changes motion, you get to recalculate. All sorts of fun! This is the hairiest part of the system. If motion becomes very dense, this system may not work out as written! But we'll see. The key will be culling unneeded tests. You don't care about something that will collide with you in 3 hours and 12 seconds. You can just recalculate in 10 seconds, and ignore anything that is 'probably' more than 10 seconds away. Put a speed clamp on things and you can limit the space sampling needed pretty easily. It was tricky but this system works. Two objects moving at each other, if they care about collisions at all, will be woken up when they touch. And in between that time? 0 resources spent on collisions between them. Heck, if they were the only thing there the CPU would go idle.
   
4. Do something when someone else acts: Well, we can just hook an event on 'someone else' and then do 'something' when needed. This means we have a LOT of events throughout the zone and entities. keep in mind that state machines pretty much work off these events as well. This task is 100% natural for a state machine transition and behavior.

Since I want to be running very large, sparse zones, I decided to invest the necessary time to make the event driven system work. It's hard to know when things will happen, but the rewards have been pretty sweet. I can run zones with 100 or so entities fighting each other, using about 1% of my CPU. If I actually want to watch it, I use about 15%, due to the bad drawing algorithms I use right now. (There's some irony there, huh..)

So that's the code and what it can do. Ah, but remember 'It Must Be a Game'? Where's the game! Gra! We got started on that fairly recently, and it's not just me anymore, and I think we've done some pretty neat things in the last couple months. Next-next time.

On the World Map, Tier 2.

So the remaining piece of the gameplay is the top tier. This component has only undergone theoretical development, so it'll feel pretty rough. That's okay. First we're working on the skirmish, and when that's fun, we'll contemplate this level further.

The top tier gameplay has several important aspects, which I'll list in order of importance and likeliness to get implemented. First, it is where the player manages his commanders and their units. Second, it is where the player finds and joins skirmishes. Third, it is where the player interacts with other players, with the exception of grouped skirmishes and global chat channels. Fourth, it is another aspect of the gameplay; the player will create settlements, set up trade, set up resource harvesting, construct new items and buildings, learn new abilities, and train new kinds of units. Fifth, it provides impetus for skirmishes; Skirmish events may be created dynamically based on the environment. That is, the game (or a GM) may spawn a set of moving skirmishes that represent a force attacking a player settlement.

Perhaps more detail is in order.

Management: Commanders develop in terms of skills, learning new abilities, setting up new formations. Primary units may change class. Equipment can be shifted around. Primary units may be traded in and out of the active set for the commander. Most of these are not 'twitch'y changes, so we'll keep it disabled in skirmish mode. This way the skirmish stays focused on combat and the skirmish objectives. More options of management are provided as the player's faction's settlements expand and resources are harvested.

Skirmish selection: I think of this similarly to questing in WoW. You want something to do, you go find a skirmish to join. If you succeed your team is rewarded. Skirmishes come in several flavors. There may be permanent skirmishes located on the map, say for a type of dungeon exploration. There may be temporary skirmishes involving attacking an enemy camp. There could also be a user-generated skirmish ability, for, say, something like dueling another player or testing formation abilities in a controlled environment.

Faction, Social, Settlment, Resources: The intent is that players belong to several tiers of groups. The top level being their associated major faction, i.e. the Alliance. Secondly being a group of players, like a guild. As a guild players will share resources, and should want to develop guild-centric settlements, rather than solitary small settlements. Guilds may fight over resources, but we haven't worked out how we want to limit this. With user controlled settling and exploration, you run the risk that you cannot control the development of the world and keep it within realistic limits. My tendency is to limit this not with artificial methods, but rather through realistic ones? Why don't cities just spring up right next to each other (ignoring the modern world for a moment)? Because they would not have enough resources to both survive, or maybe because they'd blow each other up, depending on what they thought of each other. This aspect isn't exactly finished design.

Dynamic Skirmishes: Part of settling is the risk of being attacked. Why build defenses and walls if they won't be used? We want to provide the sense that, in the more dangerous areas, you will be attacked and you could theoretically lose resources. But players don't like resources being lost. They don't want to have their work undone. The current thought is that there are two levels of these NPC attacks. First, there may be a set of nested skirmishes that change as the enemy approaches. The first aspect may be a set of scouting missions, which open up further skirmishes. The second may be some hit-and-run tactical missions, or perhaps some assassinations. Deeper in could be some group-based larger army missions. Depending on the type of NPC group attack, this could all culminate in a raid-style boss encounter.

But what happens if the player isn't there to defend his stuff? Well, first, we want people to guild-up. Guilds as a whole can help defend each other's property, and this would also keep the focus on larger, fewer, guild settlements. Second, pacing. These attacks would be slow moving. The overall attack may take weeks of real-time. Third, automated defenses. Commanders of offline players are still in the game. While they are not used in a skirmish, they may be providing static defenses to the settlement. So if an NPC group reaches an attack phase, the attack will be 'simulated' against the present defenses of the town, including offline commanders. So mere presence will help defend, though in defending this way fewer (if any!) rewards are granted to the player.

We want this tier of gameplay to be able to stand on its own. We'd like players to be able to focus on one aspect of the game, if they desire to, and play it relatively exclusively. The top tier is also more natural for mobile or web-based gaming, so we'd like to manage a portal to the world map via the web.

Now that we've created years of work for ourselves, perhaps we should get started? Next time, where we stand now.

Finally, on Skirmish.

The World Project, from last post, is just an engine. A rather limited engine in terms of sophisticated capabilities, but it runs the simple stuff pretty well. Like making libraries from Twilight 1, though, a game must be made.

The plan is to make a persistent world Real time Strategy. Now, I can't think of any terribly successful games that have done this. I suspect there are reasons. We can come up with some if you like. But you realize by now that this just adds fuel to the fire. More reason to do it. Let's list a few:

1. RTS games are traditionally about rapid build up, economic balancing, and relatively simple combat, where losses are expected and necessary.
2. RTS games tend to be light on customization and freedom, both of which are typical in a persistent world.
3. RTS games tend to involve large quantities of units acting in formations of some sort. This requires unit-unit boundaries/collision of some sort. Large unit count works against a massive setup, as the computational requirements are impressive. Unit-unit collision is problematic in the massive case as well, but this is of lesser concern.
   

So what to do about these? Well, how bout we split the RTS gameplay into two distinct, but necessarily linked, playing fields? In turn:

1. Rapid build up will not work in a persistent game, or the increase of power over the lifetime of the player would be enormous. So, we split power gains into 2 types. One, a mid-combat gain, which is largely temporary. Second, an out-of-combat gain, similar to a typical levelling system.
2. Make customization part of the players units. Typically in RTS games units are selected for relative power in the current scenario, strategically. Change that from 'selected' to 'developed' or 'trained' and you have a longer term customization. Regarding freedom, this is a bit trickier. RTS Player vs Environment is almost universally scripted scenarios, or random fights over set maps. Freedom to travel and choose what the player wants to do would lean towards the latter. Build a world and a set of scenarios to accomplish.
3. Large unit count. Hmm. RTS players like armies. Typical approaches to managing army sized groups are to cluster them into 'group' units. This is contrary to a customization point, though. With a group it is harder to justify massive retraining. So how bout we make this a sort of 'elite skirmish group', where each individual is a powerful fighter in his own right, but the workings of the group as a whole are more important? A close-knit fighting group. Let's give them a leader too; a Commander. This gives the player a troupe to customize as a whole, and as indivuduals. This keeps the unit count to customizably sane levels. Say, varying from 4 to 20 as the player designs.

So now we have a sort of 2 tier game. The combat tier (which I call the Skirmish) is fought in a relatively traditional RTS manner. But there's no base development and economics. RTS are interesting because they require managing multiple aspects simultaneously. If we remove 2, we'll have a pretty boring game. So we either find something to replace these aspects, or increase the complexity and management of combat itself.

Economics we'll replace with unit resources and morale. Morale will become a per Commander value that fluctuates with the users actions, allows certain actions, and may be spent for others. Individual units may be granted abilities that require some local personal resource, typically mana, which may require some maintainence.

Base Development we'll retain in a temporary sense. Defenses can be constructed, or in some cases, inhabited. This makes terrain and location relevant even without a base development aspect. Let's throw some local, disposable units in there too. Now we've got a resource, extra units, that territory may grant (or revoke!).

With these aspects in place, we can build a set of types of scenarios. Scouting, ambushes, escorts, sieges, strikes. Since the combat phase is only a part of the game, and local power gains are small, let's keep these short. 10 to 15 minutes, maybe 30 for a particularly difficult scenario. Starting to sound a little more like a quest in an MMO.

That's the combat tier in a nutshell. The second tier is for managing units, guilds, cities, exploration, customizing, and construction. This tier is not played with your squad! It's for the player himself, so to speak. This way we avoid trying to explore towns and landscapes with an army in tow. Can you imagine Ironforge where every player was really 20 different individuals? It'd be madness. Oh, and really really slow.

So for our first incarnation of the top tier, we imagine a sort of world map. An abstract high level view of the world. Players travel about, either hunting for a scenario to engage in. But what's the purpose? It's it just a glorified scenario selector? Is it a pretty facade for a multiplayer game lobby?

More on the World soon.

On the World project.

You know that bit in the description regarding over-ambition and general lunacy? Well, this here should explain it nicely.

Everyone who plays games has ideas to improve them. Well, maybe not, but I do. And my friends do. And we talk about them. Many ideas, most of them utterly and completely terrible. I've also spent a good time (in undergraduate lectures, of cours), developing a setting for tabletop roleplaying. While I haven't run a game in years, I'm still enamored of the setting I built. The 2 Twilight games were built on this setting.

The conclusion to all this, is that I really want to make this setting a virtuality. Not a reality. That'd be complicated. A virtuality; a persistent environment. Er, an MMO, if you want to be uncouth about it. The concept of starting from nothing and building even a small part of the world is... enticing.

The obvious problem is that these games are hard, and expensive. Look at the state of the industry right now. Overworked and underpayed for late games, most of which fail. And the MMO market is worse than the typical games market in those regards.

You're beginning to see the connection to "over-ambition and general lunacy"? I hope so.

After NegativeAgain became fun, and I became more used to work and schedule wrangling, it was time to start the project. The goal is to make a persistent environment. The theory was to keep the system possible by limiting its scope. That is, the core system would only handle a small set of behaviors. This keeps the networking communication, server-side communication, and related programming time low. Complex behaviors need to be built up from the basic stuff. It's a pretty standard layered abstraction.

The system manages 6 data tables, attribute types, states, actions, entity types, event ids, and effects. Effects got added in as a method for containing complexity. These are effectively just functions that actions use to execute. Events and Attributes could be removed entirely, as they just provide a name. However, I prefer having a sort of 'strongly typed' system, so laying out the set of attributes and events explicitly makes a lot of sense to me. In the long run, attributes ended up with a bit more data anyway (default value, derivation formulae). The low level data system handles serialization, update, etc. It also has a wrapper that supports a server-client set up for the database. I never use this, though, so it's probably rotted.

The next layer up takes this data and executes it. I call this the Zone level. The zone is actually quite simple. It's just a set of entities. There are no other types of objects. If it isn't an entity, it doesn't exist. Sometimes this feels weird when you're working above this level, but it's clean and simple, once you're used to it.

On top of the zone lies the server communication management layer.

Also on top of the zone is the sort of game logic. It's implemented largely within the system, using states and types constructed either in code or within the system editor. Note that this means that the game management and communications layers are completely independent.

The important thing here is that all the guts are in their appropriate places. The gamedata layer (bottom) is the only layer that cares about the gamedata storage method, serialization, interfaces, management, etc. The upper level pretty much uses 2 interfaces, aside from initialization. The zone layer manages execution of said data. The server does server things, and the game manages game things.

So there's clearly a lot of things that I didn't mention. No zone-to-zone communication at all, for example. No instancing, no classes, no weapons, no items, etc. Everything is built within the system, that way everything works without updating the protocols. Items are just entities too. Classes are states. Instances? Well, that's just a new zone.

So now, to make a game that uses what we've got, and not what we've not. Next time.

Blog rules

• More links!
• Not allowed to go back and edit; perfectionism must be curbed. The Line Must Be Drawn.
  
• More... Englishisms.

Grar.

On redTOAST (A History)

Regular, Every Day TOAST, Or, Alternatively, Simply, TOAST.

It's really quite ridiculous, but that's a pretty good description of what we do anyway, so it fits.

redTOAST was started in 2001 or so as a collaborative project between 2 of us, who realized that we were no longer missing any critical knowledge to make a game. Now of course we knew very little, but we decided to just do it. And we did it. We made a game. It was even fun. It was a StarControl style game (that is, top down space-physics ship combat). The code was disgusting. The interface frightening. BUT. It. Worked.

If nothing else, TOAST has been a vehicle of learning. We called the first project Star Control: Negative. It made us better programmers.

We followed up with Negative Two, dropping the StarControl homage. Negative Two taught us exactly what happens to every single other project. We called it second-system-syndrome, but we like alliteration, so you'll have to forgive us. It the simple fact that, after having learned so much from Negative, we designed a much more complicated system for Two, and it imploded under its own impossibility. So we learned a lot there too.

I was in graduate school about this time, and my partner in crime ended up working elsewhere, out of range of easy collaboration. We buried Negative Two, consecrated the ground and let it fade into memory. Having learned a bit more about software architecture in graduate school, I decided to apply the combination of personal experience and lecture, and started a new project. I called this project Twilight in the City of Kaiur, a Real-time strategy/economics game set in a... well, the setting isn't important.

I obsessed over this project to an absurd degree. It was in my thoughts all the time. I bought a laptop so I could work everywhere. Step by step, line by line, I built this crazy system over the course of a year. It worked. The software did not implode under its own weight; well, not for a while at least. Knowledge only goes so far without experience. Working with this software, with the new methods learned (I found out 6 months later they're called Design Patterns), was an absolute dream. Stuff just worked. Well, the network never really did. And the game design was so miserable, it's really quite entertaining (in retrospect).

Twilight had to end though, it got too big a game and did become unmanagable. I spent the next year splitting twilight's codebase into a set of libraries, refining and improving as I went. 

This was a mistake. 

Now, I learned a lot and had some lovely code, but it was MIND NUMBING. I stopped because I was losing interest. It had to be a game. If there wasn't a goal product, there was no point. Therefore, to such ends a new project was started.

Two new friends joined in for this one. We called it the one-week project. We took the fun parts of the Negative (one) game, and rebuilt it for multiplayer, with the new libraries and code to back it. The 3 of us cranked for a week, and 2 weeks later we played a multiplayer session with 8 people at my birthday LAN. No, it wasn't perfect, but it was a only bloody week. The game was called Negative again, but rather than being '3', we called it 'ether', both as a pun on 'e' (2.718), and as a reference to some physics adjustments we introduced. The ships sort of surfed in space, rather than using free-space physics.

But apparently we're not good at working on group projects together. The e-team didn't reconvene to continue Negative Ether in any meaningful way.

I think around this point I began to write my thesis and get busier in graduate school. It follows, then, that I started work on Twilight 2. Did I repeat second-system-syndrome? Certainly. But I was at least aware of it this time. I focused on certain elements that I wanted to have experience with. I wrote a scripting engine from scratch. I did more focus on generic actions, states, user interface. The game was substantially more data driven, and it worked, even if it wasn't elegant. The economics aspect of the game worked, but it was only about one third the game content. So, naturally, the project had to die. It was too big, but more importantly...

I began working for Bunkspeed. Now, I love being the type of person who programs all day at work, then goes home, and continues as if nothing changed. For awhile, though, this wasn't achievable. This was my first actual job, since graduate school and related activities don't count. I mean, I spent all of graduate school on the previous paragraphs, so you know how serious I was about my 'formal' education. My actual education was quite successful, thanks to the free time afforded by being in graduate school. Something seems weird there. Must be me.

Where was I? OR right. Twilight 2 was too complicated in conjunction with work. So I reluctantly buried it too. It felt like my game programming history was a sequence of graveyards, but each grave had yielded a flowerbed of knowledge, so I never considered it time wasted.

6 months or so later I decided I wanted to do an 'easy' project. I must've gotten over my fetish for complexity. I began touting principles like KISS, YAGNI, bla bla. I also got tired of fighting multiplayer networking. So! A simple project. This project would be YAN: Yet Another Negative. "Negative Again". Built for simplicity. Graphically? Everything was particles, no shaders, no complex features. Just lots and lots of particles. Data complexity? No states or complicated data; just property bags of ship, weapon, projectile parameters.

It was simple and fun. I still play it at work. I still tweak the numbers in the files, even though I haven't recompiled in years. I stopped active work on it 2 years ago. It's the closest thing I have to complete, and it isn't.

So in short: made some games, played one.

On Bunkspeed

Bunkspeed, for those of you not aware of their presence, is a Carlsbad (San Diego area) based company specializing in rendering software. Our 3 primary products are HyperShot, HyperMove, and HyperDrive. Shot is based on an excessively fast and gorgeous raytracing engine, and Move/Drive are based on a DirectX 9 pipeline written mostly by myself.

Now, Shot's engine, though I said it is a raytracer, don't confuse it for an offline renderer. HyperShot is a realtime app. Dynamically updating with your camera moves, and continually improving the image. Let it sit for 2 seconds and you've got a brilliant image. Peruse the Gallery over at our site and you'll see the results. Go try the demo and see it in action.

But I digress, this wasn't intended to be a sales pitch. The interest for me is in the interaction between Shot and Move/Drive.

Move/Drive uses a DirectX 9 based rendering pipeline (though I'd love to be able to use 10 or 11 these days). When I started work on this engine, my goal to was to make a fast, robust, and high-quality-output engine. HyperShot did not exist at the time, and wouldn't for another year or so. So we set out to make a real time engine that focused on high quality, rather than unfallable speed or consistency. We made a lot of decisions on how to build the engine accordingly:

1. Dynamic shader construction. This lets us build shaders specifically for each exact required surface. Shader construction time is non-trivial (but caching would later make this less relevant).
2. Dynamic shader population. We use a robust and easy to adapt system for managing shading constants and parameters, but it made the typical tradeoff of generality vs. efficiency (i.e. ran like a dog). Now we have a solution that yields both, but it took 3 or 4 rewrites of the system to find something palatable.
   
3. On-demand and recursive dependent passes. Many objects in our scenes can be tagged to use a local reflection/irradiance map. This eats up lots of VRAM and rendering time, so we do this only when needed. In a game, you'd never do this, but this isn't a game, it's a visualization tool. The more interesting part here, is that we allow recursive reflections. The first frame a recursive reflection is found, an additional 2 passes will be drawn. This gives you additional 'bounces' of reflections, but it means that first frame is EXPENSIVE! At times I'd see the first frame after load take a full 3 seconds, and somewhere around 150 rendering passes, but after that, everything's there and lovely. No bad frames.
4. Cook-Torrance lighting instead of Phong. Cook-Torrance is more expensive to evaluate, but is more physically accurate. It also matched up to our rendering at the time.
5. VRAM TASTY. Oh my did we use a lot of memory.

So, we've got a 'slow' GPU renderer with good quality. Along comes HyperShot, a 'fast' CPU renderer with great quality. It's like the two were both working toward eachother from opposite ends of the spectrum. Take a game-style GPU engine, make it slow and accurate. Take a photorealistic render, take some shortcuts and make it fast and less accurate.

Isn't it, weird? If we'd succeeded we've have the same product from two completely different angles. So, we changed it up. Move/Drive focuses more on being a tool for setup, animation, and presentatio. We've focused more on speeding up the rendering and improving usability (though we retain the ability to have the high-quality GPU renders). Hypershot focuses on fast, crazy-good quality raytracing, backed by a photorealistic renderer of uncomparable quality.

Match made in heaven. The best of both worlds.

Introductions

Behold, a blog.

Herein, shall be contained the assembled rambling of one Nick Gebbie, more often known as Ventare, being composed of varying topics, including but not constrained to:

• redTOAST Studios; being myself in conjunction with some friends, developing entirely-too-complex games in our spare time. I suspect much of this blo(a)g will be spent blathering on about our TOASTprojects.
• Bunkspeed, for whom I work; I'm their Senior Graphics Programmer, so I'm basically in charge of writing the GPU rendering engine used in HyperMove and HyperDrive.
• Programming in general; graphics programming more often than not. I've had the good fortune to work with a set of excellent (and varied) programmers, so perhaps there's some wisdom to be passed forward.
• Game Design; often in the context of TOASTdevelopment, but I'd expect some WoW talk scattered about, among other games.

Let it begin.