TangledTwine

The lack of support for games without a big community around them is a big downside, yes. If you're the only one who cares about a given game, my version wouldn't really help much. The other stuff isn't as much of an issue though. Yes, games are a moving target, but if the fans care they're usually pretty good at hitting that target. The core scripting engine extender for Skyrim has to be updated every time Bethesda updates the game's exe, which for a while was VERY often, but the fans managed to keep it updated. Not saying it wouldn't be hard, it definitely would be. But it would still probably be easier than the AI approach. That said, I'd love to be wrong. Also, realized I hadn't answered the call for what MY dream software is, so here goes: an OS that "programs" itself. Monitors the web for news of security and malware issues, parses the text with a high level of understanding, and patches vulnerabilities (complete with changelog). Analyzes unfamiliar hardware and can gradually write its own drivers for said hardware, via research and testing. a-b tests new GUI ideas on the User: User is trying to find a file? Some days the dialog is a little more like this, others a little more like that, collect data on which seems more effective and enjoyed by the user, implement (major sites like google and facebook do this all the time, this would "just" be without a human in the loop). Fairly out there/sci fi at the moment, but within the horizon of possibility in a decade or so I think.

This is the video that finally pulled me into creating an account here, so well done. Love the idea, and it really gets the brain juices going. To make sure I'm understanding the idea right: Ross is proposing brute-forcing the ripping of 3d model and texture data (mostly for environments) by using an AI to analyze footage of the exploration of a game space and then attempt to rebuild it in an easily usable engine format (say unity or unreal). The advantage of this is obviously that it's universal, but it's also a hell of an ask (and obviously Ross is aware of this). However, in the spirit of going for the low-hanging fruit first, wouldn't it make more sense to just make a tool that collects, automates and standardizes ripping tools? If you have the right tools for a given engine, it's very do-able to just rip map data, models, animation, lighting etc, convert it over to a standardized format and re-assemble the pieces into a working space again (in an automated, push-button way no less), no AI magic required. Given, this would require individual attention and problem-solving on a per-engine (and maybe even per-game) basis, but fans are already doing that on a regular basis for popular games, and it would be worlds easier than trying to wrangle an AI or work with some kind of simulated photogrammetry. The website in the video showing off reconstructed Nintendo worlds shows how viable this approach could be. The real opportunity as I see it is standardizing and automating what's already out there, and what has yet to be created in terms of ripping tools. Right now if you wanted pull the worlds from say, the new Spyro remake, that's totally doable, fans have created the tools to largely re-construct and pull it all back into an Unreal project. But it's also a nightmarish hodgepodge of "use this and then this and then follow these obscure steps and then-" etc. If someone took all those tools, wove them together with some scripting and slapped a nice UI on top so it was just "Rip game map" "Play game map" or whatever, there's the dream software. This still takes real time and effort obviously, and it would have to happen in pockets: only games with a community devoted enough to build ripping tools and a plugin for this "universal" program would be accessible. But again, low hanging fruit. To steal something Ross said in the video: something is better than nothing. Ripping and reconstructing like this would be infinitely easier and more practically do-able than trying to mess with AI. Just some thoughts though. Like I said, that's why these videos are great, they get you thinking. Here's another thought: if this software was created, it could potentially ride the "just barely legal" wave in the same way emulators do. You can't share the maps, but that doesn't matter so much if anyone can create the maps themselves, at home, as long as they have both the software and the game.