Jump to content

ROSS'S GAME DUNGEON: TEST DRIVE 3

Recommended Posts

  Ross Scott said:
I'm thinking it might even be possible someone might be interested in porting this to another engine as a map for it, like Source or UDK or something.

Test-Life 3 Confirmed? Actually if anyone's going to port this to Source I think an L4D2 or CS:GO map could be interesting. Hell, I might even look into doing it myself it if it's not too tricky.

I have the perfect comeback. A Spaz-12.

Share this post


Link to post
  Ross Scott said:
1. The maps seem to only contain the terrain data, was it possible to extract the information for 3D objects and sprite placements or is that much more difficult?

I only managed to extract the terrain data - no props like houses, traffic signs or cows. I think tree placement is random anyway. If anyone else wants to take a stab at extracting the props, I can post some technical info about the game.

 

  Quote

2. Since I made the video, someone was kind enough to send me a copy of the Cape Cod expansion for the game. If I sent that to you, would it be difficult to map out that data as well?

I tried that already, but unfortunately the exact same script I used for extracting the standard race courses did not work out of the box. It's probably not difficult to adapt it for the expansion disk race courses, but I need to take a break from this hacking.

 

  Quote

3. The translation to blender + the lighting settings make the colors much less vibrant than the original game. Is it possible to get them closer to the original settings or is that too much of a pain? For faces that are dithered, it seems like the solution would be to just average the two colors they're using. The reason I ask is the contrast between the roads and some of the terrain is a little low. I'm thinking once this is polished a little more, it would be good to submit this to vgmaps.com either in your name, or under my account, giving you credit of course.

 

4. While I'm afraid I don't know blender, would you be interested in releasing the model files for this? I'm thinking it might even be possible someone might be interested in porting this to another engine as a map for it, like Source or UDK or something.

I re-rendered the overhead maps without the directional light source. The colors are now saturated and flat. I used averaging for extracting the dithered colors. I included the new renders as well as the old ones in the following ZIP file as lossless PNGs. Feel free to submit them to VGmaps! Also, the 3d files are included (Wavefront OBJ format), as well as the Python script I wrote for extracting the data from the game (but I doubt it will work for anyone else). I hope FileDropper doesn't suck.

http://www.filedropper.com/td3-maps

 

  Quote
All in all, thanks a bunch, it seems like I'll have to make a follow-up video to this later on.

Cool!

Share this post


Link to post

Well I was at least going to look into how difficult it would be to put it in Source, but the files aren't downloading properly. Not sure if it's an issue with filedropper or my internet.

I have the perfect comeback. A Spaz-12.

Share this post


Link to post

Got it downloaded but now when I import it into 3Ds MAX it does a bad. Specifically it looks like it got balled up and then run over by a steamroller. I was just importing the .obj files. Is there some other step I'm missing?

I have the perfect comeback. A Spaz-12.

Share this post


Link to post

Yeah, FileDropper sucks. I can't download the file any more either. Let's try sendspace:

https://www.sendspace.com/file/9ce6o5

 

As for the .OBJs. All I can recommend is to try importing the files in Blender. You can export to you favorite format from there.

 

edit: re-re-uploaded to Sendspace

edit2: sendspace sucks - OBJs available at viewtopic.php?p=227814#p227814

Edited by Guest (see edit history)

Share this post


Link to post

Ross, you were complaining about the steering being all wonky.

 

Did you not notice how fast the clock was ticking in-game? Or that every car accelerated 0-140 in like two seconds?

 

The game is running at nearly 5x speed - that's the problem. If you found a way to slow it down (where's your Turbo button?!) the steering would be fine and you wouldn't be constantly flying off the road!

Share this post


Link to post

I can't remember if he mentioned it in the video, but the speed actually doesn't help the steering. I tried slowing it down on my configuration (if you use D-Fend Reloaded, it actually makes you a keyboard shortcut) and it was still awfulllllll steering, just a bit slower.

I forget things a lot and I like chumtoads.

Share this post


Link to post
  Frostiken said:
Ross, you were complaining about the steering being all wonky.

 

Did you not notice how fast the clock was ticking in-game? Or that every car accelerated 0-140 in like two seconds?

 

The game is running at nearly 5x speed - that's the problem. If you found a way to slow it down (where's your Turbo button?!) the steering would be fine and you wouldn't be constantly flying off the road!

Everybody keeps saying this like they think they know what they're talking about. It sucks at every speed, just in different ways. I only sped the clock up to make the video more watchable. I plan on bringing this up in a follow-up video later on, (including some words from Lazy Game Reviews!).

Share this post


Link to post

Hi Ross. I'm a long time listener, first time caller. Awesome series. On the topic of hearing Freeman's voice doing game reviews: I've kinda been imagining it as a hobby that Dr. Freeman had prior to Black Mesa.

 

I actually came here explicitly looking for the map data and I was not disappointed. However, all the above links to the map files are dead. Can someone post an updated link? Along those lines what format was the data stored in? Now, I know I'm not in a position to ask for things, but I'd love to see a brief write-up of the process in extracting the map data if @ripa is up to it.

 

-SeveredBrain

Share this post


Link to post

I uploaded the ZIP archive to Sendspace again.

 

I didn't actually crack any of the file formats of the game. Instead, I executed the game in debugger-enabled version of Dosbox and dumped the memory after the game had loaded each track. The Python script included in the archive should be able to extract the track 3d data in .OBJ (+ .MTL for color) format from each memory dump. I used IDA Free 5, Dosbox debugger, and Bitmap Memory Debugger ( http://www.theycamefromhollywood.com/bmd/ ) to discover enough information about the internal data formats used by the game to be able to write the Python script.

 

Here's some copy-pasta of a PM I sent to someone else:

  Quote

Here's some instructions:

- Get a debugger-enabled version of Dosbox (e.g., http://www.vogons.org/viewtopic.php?t=7323 ).

- Set memsize=2 in the config file as well as anything else needed for TD3.

- Start the game. When you're in the menu or driving on any of the courses, hit alt-break to break into the debugger.

- Dump the 2 megabytes of emulated memory with command: memdumpbin 0:0 200000

- Now you should have a memdump.bin file in the dosbox directory. You should be able to extract the course you were driving on using memdump-to-obj.py.

- To extract all the courses, you need to repeat this for the main menu and each course.

 

Memdump-to-obj.py will work for your memory dumps if the game has loaded all the data in the same memory locations as on my computer, but I don't know if that is guaranteed or not. If it's not, the script will just print a bunch of errors and fail completely. It can be fixed by finding the correct offsets by debugger and adjusting the addresses referenced in memdump-to-obj.py.

 

Here's some plaintext information on the race courses. They consist of 16x32 grid of square tiles ("pieces" in the Python script). Each tile has an elevation (not relevant if you just want an orthogonal overhead map), can be rotated 0, 90, 180, or 270 degrees, and can be reused multiple times.

 

Ingame, the course is stored as a 16x32 entry table, where the table index gives the east-west and north-south coordinates of the tile. So the first tile, entry index 0, is one corner, and the last tile, index 16x32-1 is the opposite corner.

 

Each entry in the table consist of a z coordinate (elevation), rotation bits (0, 90, 180, 270 degrees), and some identifier or index. The identifier tells what type of tile it is.

 

There are separate data tables that contain the data for each tile. Each tile consists of several faces stored in a table. One entry in the table defines one face. Each entry refers to 3 or 4 vertices (by index) and two colors (again by indices).

 

Vertices have 16-bit x, y, and z coordinates. There's one table per dimension, and each vertex is referenced by an index into all three tables.

 

Color is stored in separate tables again using indices.

Share this post


Link to post

Well damn. I should have looked at this forum topic before having a go myself :D

 

PRESS F6 TO GET YOUR AFTERNOON BACK

 

A few notes about my version of the game

  • I found an infinite lives hex-edit guide, but it seems to be for a different version of the executable.
  • No oversteering issues until I took the clocks very high. I suspect the game was designed to run comfortably at a low framerate, something we are not used to thinking is normal.
  • My version only came with one map: Pacific Yosemite

 

 

Initial analysis

The level appeared to be segmented into square regions, ala chunks in Minecraft, and you can trace the seams across the entire map. Setting the quality to low makes this more obvious as adjacent chunks tend to stay hidden for longer.

 

It appeared like the terrain was a height-map and the roads + entities were added ontop.

 

I took the file-analysis approach rather than analysing running memory. Some rudimentary DOSBOX debugging and flower-arrangement lead me to believe the map data was stored in SCENE01.DAT

 

File analysis

Method:

1. Blank out a segment

2. Load the game

3. Observe what happens

 

This data file is amazingly resilient. I'm convinced a lot of what's in there is unnecessary :D

 

Results

Around 0x7600 in SCENE01.DAT appears to be the heightmap values for a chunk directly infront of the starting position (just before the bridge). They look like two-byte values and appear to start with the north-most row (assuming you start the game facing north) going left to right. This particular chunk is mostly the same (00 02) for all verts as it's flat.

 

I could be horribly off-track as I only dabbled into this over the last few hours. To find that someone had already completed is downright crazy -- consider yourself admired ripa!

 

Other interesting things I discovered

  • Some patches of brown land near water are marked as 'road' by the game.
  • Some interesting error messages: see attached :)
  • The road and entities are indeed sep data to the terrain. I suspect the road is rendered after the terrain to ensure the correct Z-order. Cars follow the road blindly, even through terrain.
  • Changing 0x766a (X) leads to some very fun corruption of the chunk I was playing with. It's probably a start/end marker or counter for a number of some element in the chunk. It let me 'finish' the race by driving through the chunk (beat my best time of a few seconds!).

744389992_corruptscene01_dat.png.462630aafea5ec7684c2a9c81f7f1b46.png

goback.png.feba96a967174361c4f7247e7fea2e20.png

97850353_editorlayout.thumb.png.529267dd2373072daacad09cefea23d8.png

Share this post


Link to post
  Veyrdite said:

[*] No oversteering issues until I took the clocks very high. I suspect the game was designed to run comfortably at a low framerate, something we are not used to thinking is

At lower speeds, there's clear lag between when you try to steer v. when the car actually starts moving. It's really a "pick your poison" situation between low and high framerate.

Share this post


Link to post

I just saw Ross' Test Drive 3 video and I couldn't resist the challenge of trying to extract the map data.

 

I saw this thread and I was not surprised to find out that someone already beat me to it. It looks like you're still missing a few things, though; the maps are missing some of the models and it looks like you're getting the data from memory using a hacked DOSBox rather than loading it directly from the files?

 

I started looking into it myself and I'm hoping to extract all of the data from the files including the extra models.

 

The link to that Python script appears to be dead, so I decided to try to figure it out on my own, based on ripa and Veyrdite's advice. Here's what I've got so far:

 

MSJ0bFSl.png

 

These are models loaded from SCENE01.DAT and DATAB.DAT. I'm halfway there but I don't have the data for the actual map layout yet. I was able to decode one of the models based on Veyrdite's post, then once I figured out the format I wrote a function to search the files for data that looks similar. I was able to find most of the models, but it was just a brute-force search algorithm; I don't know how the game is actually determining the file offsets when it loads the level.

 

I was able to find a lot of non-road models this way:

 

iqpKXlEm.png

 

C1RXEXYm.png

 

a01ese8m.png

 

I found most of the models, but playing through the game I can see there are a few missing ones. I couldn't find the lighthouse, some other background buildings, the other cars on the road, or the cows. I have some ideas for how I might be able to find them.

 

I did find some interesting quirks with the car models:

 

r6auHykm.png

 

TdAYBLqm.png

 

eWZZ4iDm.png

 

* There are no polygons underneath the cars, they're just thin shells of polygons going over the top.

 

* There are actually several huge gaps in between the polygons where you can see through the model, but it's so low-res that you can't see it in the game.

 

* What's that gray diamond on the hood of the Diablo? Did I do something wrong when I loaded the model?

 

1cfSu7wm.png

 

Nope! If you look at the windshield, it looks like it would be made from two triangles and a quad, but they were really stingy with the polygons, and they just modeled it with two quads that stick out past the hood. The only reason it looks correct in the game is because the z-ordering makes it draw the hood over the windshield.

 

hwOF6EZ.png

 

Most of the time, anyway:

 

Gnp2QhJ.png

 

Finally, I found a few errors in the Mythos model. Here's something that is hard to notice in the game but if you're obsessive-compulsive it might drive you crazy:

 

8QILqwcm.png

 

For one thing, one of the vertices in the very front sticks out more than it should. But also, the glowing polygon for the headlights is asymmetrical! It comes out the bottom of the right headlight and the top of the left one!

 

ConMraR.png

 

Anyway, that's what I've got so far. I'm going to try to decode the rest of the level data. If you could re-host that Python script it would make it a lot easier, otherwise I'll continue with the strategy of "blanking out parts of the file and seeing what happens."

 

If anyone here wants to see my model loading code, I can upload it somewhere once I clean it up a bit. I'm using C++ and OpenGL to render the models.

Share this post


Link to post

Awesome work Komojo. As I'm a lazy Australian, do you mind sharing the addresses of data you have found in the files?

 

You have motivated me to do something interesting, but whilst I'm getting carried away with this I should disclaim that my version of the game might be different:

-MD5sum-                          -Filename-
395bb9e79ab949c627cc8f192ee8cc36  DATAA.DAT
f10ac7b99768a8544401cf2f0026f60d  TD3.EXE
c0f1ec4e86c3b3cb8290f8506db4e2b9  SCENE01.DAT
d214c20b2fa7638cb2fac8ab129e9c2d  DATAB.DAT

 

Additionally: what 3d format are you writing into? Something nice in ASCII or are you interfacing with some easy libraries?

Share this post


Link to post

Here we go: this is SCENE01.DAT, with each byte of the file represented as a pixel. Each row is 256 bytes and the file should be about 1.5Kb. I manually drew on the index down the side so I apologise if it's off.

 

I'll do the same for the remaining files of interest. Hopefully you (and others) will find this useful for tracking down areas of data to play with.

 

Meanwhile I'll improve my tool and upload it somewhere.

SCENE01.thumb.png.235f8b6380890315e1c19436544f5436.png

Share this post


Link to post

Great job Komojo! I think you've solved the hardest part. The map layout should be a simple 32x16 table of 2-byte little-endian words. Each word encodes the elevation of a tile, rotation of the tile, and the type of the tile. The elevation should be ((word & 0x3F00) >> 8), the tile type index should be (word & 0xFF), and the rotation bits should be (word >> 14). Memdump-to-obj.py is attached. Sorry that it's a steaming pile of crap (but it worked ;-)

 

edit: These should be the offsets to the map layouts. I found them by searching the DAT files for the map layout data that I got from the memory dumps.

 

DATAB.DAT
0x216e2 main menu

SCENE01.DAT
0x1031f course 1
0x1a2ab course 2
0x1c3f8 course 3
0x21644 course 4
0x23791 course 5

SCENE02.DAT
0x1022c cape cod course 1
0x2166e cape cod course 2
0x26352 cape cod course 3
0x2849f cape cod course 4
0x2d95b cape cod course 5

 

For example, course 1 corner tile is bytes 2a 01 at SCENE01.DAT:0x1031f. Tile type (index) is 2a, elevation is 1, and rotation is 0.

memdump-to-obj.zip

Share this post


Link to post

Thanks! Both the Python script and those color charts should come in handy.

 

  Veyrdite said:

  • Changing 0x766a (X) leads to some very fun corruption of the chunk I was playing with. It's probably a start/end marker or counter for a number of some element in the chunk. It let me 'finish' the race by driving through the chunk (beat my best time of a few seconds!).

This is correct. The 'X' (0x58) is the number of vertices, and the byte just before it at 0x7669 is the number of polygons.

 

Here's an excerpt of the model rendering code. I'll post the whole project once I get it to load the full maps.

 

 

  Reveal hidden contents

 

 

I'm also not sure where it's getting the color palette from. I just figured it out through trial & error:

 

  Reveal hidden contents

 

 

Here are the offsets to the models:

SCENE01.DAT

 

  Reveal hidden contents

 

 

DATAB.DAT

 

  Reveal hidden contents

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in the community.

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×
×
  • Create New...

This website uses cookies, as do most websites since the 90s. By using this site, you consent to cookies. We have to say this or we get in trouble. Learn more.