Learning some new information about the ToD data files today.  I am hoping to absorb some of the techniques into my own game, "The Legend of Beryl Reichardt" while keeping the "unique" flare of a totally new game engine.  See, on the TI (for those of you who are unaware) Tunnels of Doom holds the dubious honor of being the only RPG put out by the company before it collapsed under it's own weight.  Subsequently, Old Dark Caves and The Legends series came out, but neither of these games were widely distributed, and remain generally obscure and relatively unknown.  I have found a love for these games over the years but no game in the entirety of my life affected me in the way ToD did so many years ago.
Anyway, back to the point... I have recently become friends with the man behind the DreamCodex remakes, Howard Kistler.  His remake of Tunnels of Doom, "Tunnels of Doom Reboot," is spectacularly constructed and has become a favorite time killer of mine.  While no game will ever replace ToD in my heart and mind, ToDR is pretty close.  He also created a piece of software that allows for original ToD games to be converted and run on the ToDR game platform!!!  This toolkit (which is free for download) holds the key for understanding the data structures of the original ToD games which in turn allows for a better understanding of how the whole shebang works.  For my purposes, this is crucial.  More on that later.
I have posted previously about the game structures and combat engines I intend to implement into "The Legend of Beryl Reichardt," and I wish to maintain that direction.  I also, however, intend to figure out the parsing methods ToD used to code entire dungeons using next to NO memory.  Remember, the TI has only 16k of console RAM.  16K!!!  And what makes this even more insane is the fact that the original ToD games were released on tape.  That means that using the tape loading method, the user did not have access to any piece of the 32k memory expansion stored in the Peripheral Expansion Box, or "PEB.". So.... What made this possible?  How did the designers implement such giant pieces of data on a cassette tape using only 16k of RAM to run the entire game?  The answer is simple.  "Data compression."  I do not mean digital compression, as nothing of the kind truly existed at the time.  The following is a concept made known to me on the Atariage thread by Codex himself, Matthew Hagerty, and Adamantyr, the developer of "Realms of Antiquity," another TI-99 RPG in the works.

Data parsing....

The premise: take a large chunk of information and break it down into uniform-length "strings" which can be deciphered by a separate program.  This program takes the strings and applies "game-values" to them.  The beat way to explain this is by an example.

For the regular monsters, they'd compress very easily with a simple ID and a few digits, plus the name string, like so:

F1312Scorpion____
F2412Skeleton____
F3212Giant_Spider

These could be compressed even further, but let's see what these do first. Take the first entry - F1312Scorpion____

F = This monster is found in the Forest World (alternatively you could number the worlds, making this a 1 instead).
1 = It is the first monster definition for this world, and since you have fewer than 10 monsters per world, you only need a single digit to distinguish them.
- Together "F1" is the unique ID of this monster, because each world has a unique letter, and within that world each monster has a unique number
3 = Hitpoints - note that the actual Hitpoints for this creature is 15. However, you've made all Hitpoints divisible by 5, so we can use that to compress them. What we store is the Hitpoints divided by 5, and when creating the creature we multiply this by 5 to get the real value. Since no regular monster has more than 45 Hitpoints (5 x 9), we can again use a single digit for this. ToD uses this technique a lot in the monster files.
1 = Attack bonus
2 = EP

The rest of the encoding is the name of the creature, with _ characters standing in for spaces so that you can see how many are required (up to the length of the longest name is what needs to be allocated).

Now, we can be even more clever in storing our monster values. None of the individual digits we've used so far comes even close to the maximum byte value of 255. So we can cut up the byte to let it represent multiple attributes. Once more, ToD leads the way in doing this, which led to some headaches in decoding it as attributes like resistance, mobility, and negotiation were all packed into one number. But to continue, we can cut a byte up into three pieces, the first of which stores the hitpoints, the second the Attack, and the third the EP.

As you know, a byte has 8 bits - 00000000. We'll use 4 to store the Hitpoints number, which has a possible range of 1 to 15 (5 to 75), more than enough given your monster list. We'll use two more to store the Attack, which has a range of 1 to 3, and the last two to store the EP, which has a range of 2 to 5. So here's how our byte gets cut up:

00 00 0000
EP AT HP

The first 4 bits are the easiest to set, because they're the low bits, so they number they represent is the same as the actual number. For the Scorpion the Hitpoints (divided by 5) are 3, so we store the number three in our bits:

00 00 0011 = 3
EP AT HP

Now the Attack. For this beastie the value is 1, so we'll store the binary representation for 1 in the two bits we set aside for Attack:

00 01 0011 = 19
EP AT HP

Notice that we can store anything from 0 (00) to 3 (11) for Attack. More than that and we'd need one or more extra bits. This is why planning in advance pays off - you need to know the maximum possible values of EVERY attribute in the game so that you can scope out their space.

Back to the number, we need to encode EP in the highest bits. Since this can range from 2 to 5, and we can store 0 to 3 in two bits, we'll store a value and then add 2 to it, so that it falls within the desired range. (0 + 2 is the lowest possible value, 3 + 2 is the highest) The Scorpion has a 2, the lowest value, so we leave it as 0 in the byte:

00 01 0011 = 19
EP AT HP

So with a single byte of the value 19, we have encoded every attribute of the Scorpion except its ID and its name. To extract these value out the parser needs to do some bit-masking and bit-shifting, but that's not too tough. 

Thanks to Codex, Adamantyr and Matthew180 from Atariage for the help on this one!