Serialization

As I continued working on loot drops, I found it wasn’t feeling right to have items disappear after leaving a room. I don’t know for sure whether I’ll want items to persist indefinitely (in the current build, they have a ten-second lifetime before blinking out of existence), but I do feel like you should be able to walk away and return and still have a chance to collect something that dropped. In solving that problem, I’ve fallen down a rabbit hole of changes that may allow me to support complete save-and-restore-anywhere checkpointing functionality. To explain why that is, I’ll have to take a step back and talk about how things worked in Super Win.

I’ve described my entity-component system in high-level terms a few times already, but this time I’ll be going a little more in-depth with regards to the internal workings.

My level editor also serves as an entity construction tool. In Super Win, entity construction was done entirely by hand in XML markup; in Gunmetal, some aspects have been wrapped within a visual editor for ease. In both cases, however, the editor outputs definition files for each entity. These definition files consist of XML-like nested key-value pairs, written in a binary format for faster loading. As these files are generic in format, I use them for multiple purposes in my engine. When used to define an entity, each of the top-level tags define a component either in full or in part.

defviewer

At runtime, the game invokes the entity factory to produce an entity based on a provided definition file. The factory examines each tag, producing components as needed, attaching them to the output entity, and giving them a chance to use the input definition data to set up their own properties. At this point, the factory also queries the serializer to see whether it contains any additional data with which to define this entity’s components. If we’re just starting a new game and constructing an entity for the first time, it will not. We’ll come back to that in a bit.

The factory returns the fully constructed entity to the game, where it lives for a while and does whatever it’s supposed to do. (Essentially everything that can be seen or interacted with in the game is an entity, from the player character to enemies to loot drops to doors to signposts to particle effects and so on.) At some point, the entity will be destroyed. This may be because it’s something like a particle effect that has exceeded its lifetime, or because it’s an enemy who has been killed or a loot drop that has been collected, or maybe the player has walked into another room and this entity is being removed from gameplay. As part of entity destruction, each component has a chance to export data about its current state to a component delta structure. This data is passed to the serializer and stored until the next time the factory attempts to construct this entity, at which point the original definition data is supplemented with these deltas in order to restore the entity’s components to the state they were previously in.

In order to distinguish between similar entities which may share the same definition data, entities are assigned a GUID. These is done automatically when entities are placed in the level editor, and (new to Gunmetal) this can also be done at runtime for things that are dynamically spawned, such as loot drops. The serializer looks for the presence of GUID information when managing component deltas. If the entity has no GUID information, it is assumed to be a temporary entity like a particle effect that does not need to be serialized.

Not only does the serializer keep track of information about entities and components that have been created at runtime during the current session, it also handles disk access for saving and loading component deltas. This allows the previous state of entities to be restored on a future session. This was how saved games were handled in Super Win, and will likely be how they’ll be handled in Gunmetal as well.

So (minus one or two details I’m deliberately ignoring for the sake of brevity), these features as described should facilitate the complete saving and reconstructing of any game state. So why did I say it was a rabbit hole? Well…

~~~DRAMATIC PAUSE~~~

Despite the presence of this framework, only two components actually took advantage of this functionality in Super Win. One was the “delete component,” used to flag an entity as having been permanently removed from the game. This is how things like gems and powerups know to stay gone after the player collects them. The other was the “data component,” a component that allows any entity to store arbitrary key/value pairs. This was used for everything else that needed to be saved in Super Win besides existence. The player entity would store the name of the active campaign and map here, as well as the coordinates of the room where the game was last saved and the position of the checkpoint within it. NPCs would use arbitrary data to keep track of the few things they needed to know, such as quest completion status. Maybe one or two other things I’m forgetting.

And that was it. Literally nothing else about the game state was ever saved and restored, either when saving and reloading or just when walking between rooms. This is why enemies and hazards always reset to their original locations, crumbling platforms reappear immediately, and so on. In the original You Have to Win the Game, I actually found that behavior to be advantageous since several rooms were designed such that a mistake could put the game in an unwinnable state. Leaving and re-entering the room provided a convenient workaround to this problem. By the time I wrapped Super Win, it was becoming an annoyance, and as of this week, it’s become completely unacceptable for Gunmetal.

So now I have to go back through all my serializable component classes (sixty-seven at last count) and make sure they write out any relevant data that would be needed to restore them to their previous state when destroyed. For some components, this is trivial. The “spatial component” (more commonly a “transform component” but I’m bad at naming things) can serialize position, velocity, and acceleration data, and that gets me pretty far already. Where things get a little trickier is in dealing with references to other things. When the player is standing on a moving platform, a “basing component” on each is used to keep track of their relationship. These sorts of dynamics are more difficult to preserve. My current plan (which hasn’t been thoroughly tested yet) is to save out the GUID of the thing being referenced. On load, that thing may or may not yet exist, however, so a fix-up pass might be necessary for reinstating the reference once all the participants are present.

So far, this process has been going well. I have data being correctly serialized for position, velocity, and acceleration, for AI states, for animations, for current and maximum health values, and a handful of other things. That’s gotten me pretty far already. I expect that maintaining this system will be something that lasts throughout the entire development process (certainly if nothing else, it’s going to be something to take into consideration any time I add a new component class), but the initial impact has been fairly mild, as rabbit holes go.

Oh yeah. I wanted to draw a graph of how all these bits and pieces interact. It didn’t turn out very well, but uh.
entity_construction