Revenant maps

[Vinylon]

Hey all, I watched the Revenant video a few days ago and I didn't have much else to do (zing), so I figured I'd take a shot at generating images of the game's maps. It's got a nice art direction, so why not? The good news: I worked out the automaps! The bad news: I couldn't figure out the full-resolution maps. I'll post what I managed to piece together in case anyone else wants to give it a shot, though.

 

So here are the actual maps; sorry that they're oddly-sized, I was lazy and used what I guess is the game's coordinate system, so I wound up just copy/pasting the map parts of some giant, mostly-black bitmaps.

 

http://imgur.com/a/s77l5

 

I generated those from the .bmp files in the Ahkuilon.rvm/Automaps folder. At first I didn't see any images at all, so I tried just grepping for '[mM]ap' in that file since it was one of the biggest. It didn't have much information, but a bunch of lines started or ended in 'PK', and some googling told me that those were the initials of one of the creators of the .zip file format. So I tried unzipping it. Believe it or not, that actually worked - this game's resource files are just renamed .zip archives! I wouldn't be surprised if that's a pretty common practice - I mean, why write your own system to compress and index tens of thousands of files?

 

So the module file contained about 3,000 64x64 pixel bitmaps with names like, '5_8_34.bmp'. It turns out that those numbers represent the map number, and an X,Y grid coordinate. So you can just loop through them and fill in a big image for each level one 64px tile at a time. This is the python script I used if anyone is interested, but it's hack-y and just does the bare minimum.

 

http://pastebin.com/V6Xvz7hq

 

I think the coordinates of maps in similar areas are relative to each other, so you could probably generate more complete maps by overlaying the right ones. You should be able to easily do that just by changing the range of 'n' and moving the 'new_im' creation and saving outside of the loop.

 

Unfortunately, the automaps are really low-res. It's better than nothing, but the actual maps are stored in .DAT files under /Map which - and this is just guesswork - seem to specify coordinates for the placement of images and game elements. The 'imagery.rvi' file is also just a zip file, and it contains all of those images, but they are all in '.i2d' format. I have no idea what that is and neither does google, so it might be a proprietary format. The headers seem to always start with the same 4 bytes, 0x43 47 53 52, but that's not a lot to go on. I tried importing them as raw RGB data, but that just looked like gibberish no matter how I massaged it. The game was 16-bit though, so...well, I don't know. Image compression is black magic to me. The 3D character models are '.i3d' files which might be associated with a toolkit called Houdini, but I couldn't make heads or tails of them.

 

Anyways, it was getting late at that point and I gave up. If anyone's seen those types of files before, I'd be interested to hear what they are. And hey, at least the automaps are something.

[Ross Scott]

Ah okay, so this is the automap data (I think I couldn't even get that working in the episode). Not as good as the full thing, but definitely cool. I actually plan on discussing the maps when I get to the followup episode.

[Vinylon]

Yeah, it's just the automaps. The map data is stored with the same sort of coordinate filenames as the automaps, so I think it would be pretty doable to generate images of them if the assets were jpegs or something. I couldn't find out anything at all about the i2d format, though. C'est la vie.

[kpcg]

Couple of useful tools for this: https://www.moddb.com/games/revenant#downloadsbrowse


the Map format reference is of particular interest: https://docs.google.com/document/d/1WAAecVsqAqZGLR4oOd3tpel7QC0YjrZOVuqwYt-sLnw/edit

Also ingame builtin editor is of interest (press enter in game, type "lookunderthehood" and press F12)

I don't have time to dive into all this now, but leaving my findings here for someone to pick up.

[kpcg]

found more lucrative things here: https://web.archive.org/web/20000817185014/http://revenantqs.telefragged.com/downloads.shtml
 

Unfortunately the i2d file format spec is unavailable here, i'll see if i can find it elsewhere. 

 

I feel like solving i2d is last piece of the puzzle here

[kpcg]

based on `map sector file format` doc, i was able to conjure up this Modules/Map/*.dat file format spec: https://pastebin.com/XeMvt9aw

sadly, i2d files are still a mystery to me. but thanks to this work: https://www.moddb.com/mods/the-forsaken/images/example-of-dat-images#imagebox image files with .dat extension (mostly in root game folder) are quite easy to extract, but they are not relevant for the task of stitching together a map (they mostly have game UI elements). a draft version of the file spec for images.dat files is here (its WIP, but with a bit of work it can be usable): https://pastebin.com/r52j4Wbt

so, at this moment 2/3 pieces of the map extraction pipeline are possible (first, extract all *.rvr files as zip files that you can find in your game dir and sub dirs):

1. read Modules/Map/*.dat files and extract TILEs from it (with coordinates, rotation, etc) - DONE (use https://pastebin.com/XeMvt9aw KaiTai scheme to generate a parser for your language of choice or just use https://ide.kaitai.io/ to check it out first)
   
    
2. Using `unique_id` of each TILE in a Modules/Map/*.dat file, get the actual tile texture path - DONE
   (open `class.def` file in game root folder and search for 0xblablabla unique_id that you got from the tile on the map in step 1)
   
    
3. Read the Imagery from the texture in path Imagery/Folder/file.i2d - NOT_DONE
   I've tried to reverse engeneer the format myself, but was not succesful in this yet. Any help of either reverse engeneering from scratch or better yet, somehow downloading this file https://web.archive.org/web/20000817185014/http://tux.telefragged.com/file.pl?filename=fileformat.zip&dir=telefragged/revenantqs/
   If you know where to find it, it would be tremendously helpful in achieving step 3 here.
   
   it's a:

   Quote

   Document detailing the structure of I3D and I2D files to allow programmers to access the files. The file is in .doc format, and expands to about 60K.

   Anyway, I started a very drafty file spec for i2d files here (but it's mostly trash at the moment): https://pastebin.com/UqCXNLry
    

[pvtjokerr]

Some good news some bad news.

I played around a bit in kaitai and got it somewhat working for i2d bitmaps, but only for really small ones. Larger ones seem to be chunked, LZ compressed and run length encoded, and I wasn't able to figure out uncompressing them.

The spec files mentioned above are gone for good I think, at least I wasn't able to find them anywhere (believe me , I  tried). Not all is lost however, as the entire (?)  Revenant source code is on github, so for someone who actually knows c++ it should be possibe to figure out the format.

Link to kaitai thingy: 

 

https://pastebin.com/7rTFwcfd

Revenant source code:

https://github.com/benjcooley/Revenant

Edited October 20 by pvtjokerr
typo (see edit history)

[kpcg]

Quote

Revenant source code:

https://github.com/benjcooley/Revenant

Wow, this is huge find! Thanks a lot @pvtjokerr! With this it shouldn't take long to crack this puzzle! I don't know how I missed it, but thanks again for pointing to it! I'll get to work in it.

[kpcg]

Quote

Larger ones seem to be chunked, LZ compressed and run length encoded, and I wasn't able to figure out uncompressing them.

okay, after spending about a week on this problem here's where we are.

I think I'm close to figuring out the compression/chunked data format of the image files.

here's the output I got for Imagery/Misc/dragonent for example:

As you can see, it's not quite correct yet, but you can still kinda see a dragon there, it's just the chunks are incorrectly aligned or appear scewed. 
Here's another example Imagery/Forest/forbirch001:

Again, you can kinda see a birch tree here. but the pixels are not aligned properly. 

What this tells me is that the decompression of each chunk works more or less as it should (I based it on ASM code in https://github.com/benjcooley/Revenant/blob/83f806bb65b762b2cc3e1ea509010f05342caa12/chunkcache.cpp#L249). The part that is missing is chunk alignment/sizing.

On another note, I'm also having some troubles with palletes for 8bit images.
For example, here's Imagery/Misc/blueflower:
 
As you can see, it's not blue, but a yellow in fact.
Here's another example Imagery/Misc/bread:

Again, not quite the color of bread.

The thing is, I'm short on time ATM, so unfortunately, I have to stop here for now. Maybe I will pick this up later when I have more free time 

In case, anyone would like to continue or learn from my efforts, I've uploaded my JS code to Github: https://github.com/WiegerWolf/revenant-map-builder and also attaching it to this post. It's a single long js file for Node.js. 

main.js.txt (mirror https://pastebin.com/UY7cbc2w)

Oh and I also found this https://github.com/IgorZyktin/Revenant which contains some useful things too in python

Edited November 9 by kpcg (see edit history)

[pvtjokerr]

That is awesome. All I got was just different images of noise when trying to decompress. :D I think i looked at thousands of them. 

The colors thing should be simple enough I think. Judging by yellow swapping to blue and vice versa, it would most likely mean that red and blue channels are swapped. Manually swapping them  results in this:




I'll try to tinker with it when I have some time. Awesome work, I'm excited.