MeFisto94

Some findings about big/map file packing in TDU1: Contrary to TDU2, the map files are no XMB files but instead MAP v4 (v3 is also possible), much like the Tdu1.map outlined by djey here: https://raw.githubusercontent.com/djey47/tduf/master/lib-unlimited/src/main/resources/files/structures/MAP4-map.json The additional knowledge is that one of the sizes is the compressed size and the other one the uncompressed size (sorry, can't remember the order). Also, I assumed GAP1 to be the offset into the big file (that's how it works in tdu2), but one could look at GAP1 and GAP2 and compare them against the sizes to know more. My roadblock is now when taking the first chunk out of the BIG file, it has a ".ZC" header, and I also see that in the sourcecode (`%s.zc`), it's compressing files. That's also why the compressed size is < uncompressedSize. One would have to reverse how to decompress those zc files, maybe it's a byproduct of @opentdu Additional Note: Since those files are for Console, you need to use bigEndian as opposed to littleEndian that you'd use on PC. At least for the map file

Is there finally a build of renderdoc that supports DX9? Last time I checked there wasn't and that was intentional

So you want to rewrite shaders anyway? Because obviously the best bet would've been starting with DX9 and copying the renderer verbatim to begin with. Even DX11 is already a tad more challenging in subtle ways (e.g. texture adressing changed the pivot, you do have constant groups as optimization). I know because initially I worked on a d3d9.dll to make TDU2 run using DX11. And a byproduct of that is a disassembler/decompiler for d3d9 assembly. Another interesting use would've been to patch shaders in modding, but as probably all my projects, it's unfinished One may consider in the decompilation step (that still doesn't support loops etc) to also just output glsl instead of hlsl, though differences there are probably miniscule anyway. Or was your idea to use SPIR-V with the DX Bytecode? I guess it can't right? So definetely rewriting shaders

I now also created a public channel on our TDUniverse Discord (https://discord.tduniverse.org), link to the channel https://discord.com/channels/930226011071709194/1385708087708614797 to discuss reverse engineering, as suggested.

@opentdu @fvever So it took me a while but I finally made the basic wiki at https://wiki.tduniverse.org (https://github.com/TDUniverse/EdenTduWiki and npm run build to compile it locally). I thought git is perfectly suited for such a wiki, compared to someone just hosting it, because that way the knowledge can't get lost that easily and we have better collaboration than on a github wiki alone. The file format is kind of markdown with extra features. Currently all articles are stubs, I'll see what I can contribute over the next few days. There you'd probably have more success with WebGL2/openGL ES 3.0 though probably, for old phones and because Vulkan may otherwise tempt you to write a modern rendering backend It's interesting anyway because due to both games having a "shared" xbox and pc renderer, you can see the xbox command list architecture, e.g. in tdu2 there's just a list where they append I think around 150 integers and sometimes some pointers and then number XY is means "bind index buffer", so you can probably abstract quite a bit, at least for those old APIs.

Hey @opentdu and @fvever! It seems we're 3 persons trying to achieve something similar or at least having the same requirements! That means if we're collaborating/open sourcing our findings, everyone could profit. To summarize: @fvever is working on getting TDU2 to run on PSP, right? @opentdu Your motivation is outlined in this post, + it could help building Andraste Mods, I am just a bit confused by the mix of ground truth decompilation and a vulkan rendering instead of the DX9 first. Then there's "us" (Me and the TDUniverse Team): We have a bare bones TDU1.5, which kind-of shares openTDU's goals, but not the way. The idea is to reimplement the game but without relying on decompilation and instead develop things from scratch in a modern way (and language, Rust). By that, I take the freedom to also implement things differently (maybe non p2p networking?). The name comes from the fact that it tries to bridge the gap between TDU1 and 2, too, reading both asset formats and databases but pulling the physics from TDU1 etc. Besides that we've been working in tooling since a while, we have the non-public "Modding Studio" that converts a few file formats (into gltf, e.g., for a blender pipeline) and database editing capabilities (adding cars). It doesn't support world editing, though and especially not the havok collision shapes, e.g. My suggestion would be that we at least work on a wiki documenting the file formats together, I guess we could already achieve 80% completion with our combined knowledge. TDUniverse also has "TDUFileFormats", a C# reference implementation for those files, but I guess everyone of us is working in a different language anyway. What do you think in general? WE can also provide a discord channel at TDUniverse and I guess Turboduck would do the same.

Hey 32qYVQdH! You've come to the right place. I guess Turboduck is the only remaining big TDU community and is the home of a lot top tier modders. I think I had the same problem and you indeed need the https://ntcore.com/4gb-patch/. You can read up on the backgrounds but the TLDR is that otherwise the game can only allocate 1.5 GiB of virtual address space. And PiX has an overhead, while TDU certainly had been trimmed to exactly ride the 1.5GiB border. What you can also do is inspect the capture using the 64bit PiX application and/or trying to 4GB patch the 32bit one, I think that was also a problem. FWIW TDU contains the shaders as blobs in the application, whereas TDU2 has dedicated .sob files. Furthermore static analysis will help you understand tdu1's rendering whereas tdu2 is using some xbox derived command list abstraction where it's building a linked list of "commands" that are then implemented by some backend that polls the list. Probably not the case here, but remember that asserts kind of compile to: if (IsDebuggerPresent()) { __asm int 3; // maybe there's a WinAPI function for that too } Actually, that's probably a give away for you of where anti debug resides anyway, so you can get rid of it. Cracks. Since the game is abandonware, there's probably no legit (i.e. requiring activation) version floating around anymore anyway, and depending on the crack (maybe there's multiple "vendors") they may have removed the DRM protection more or less. Could just hook it to "return false", but yeah there's a lot more Anti Debug techniques. Besides that, do a lot of static analysis and maybe try to fiddle with some less mainstream functions. Generally, DRM will only try to monitor protection worthy functions, as everything else will become too expensive.

Hello Friends! I took easter as a chance to look through all of my projects, judging their state, what needs to be done and thus writing this blog post, because I always have a lot of cool projects running, but they are barely known to most (except those that lurk on universe and turboduck discords ;)), which I really have to change because they are awesome! If you want to know more about any of these projects, ask in discord or even better: comment on this post, I'll try to answer as much as I can. So, without further ado, let's jump into the projects: Test Drive Unlimited 2: After being a complete medial disaster, we've finally managed to bring the launcher/announcement post on turboduck: In the meantime, the server has become main-feature-complete, minus the Casino. Besides that there are a lot of missing smaller things (mostly stats, custom pictures in houses, club-vs-club races (I could never test them as that would mean 8 active players), ORC (CRC works!)). If you like TDU2, also don't skip over our blog post about our dynamic matchmaking and why it's the superior solution among tdu2 servers here: The TLDR is that our approach is reactive and instead of relying on any launcher doing magic detections, we're dead easy: If there are issues with lobbying, rating goes down and someone else is chosen. Test Drive Unlimited (1): For TDU1, I also have a lot in store that you may not expect at all. Multiplayer wise, I've been working on integrating the tdu1 protocol into the existing tduniverse tdu2 code-base and to my surprise, it lines up much more than I even though. Freeride/freeroam is technically possible, however I still have a hard-to-understand issue where the peer-to-peer communication between player 1 and the host causes the host to crash. Whenever I have the time and motivation and a good idea, this may be solvable and then I can start asking for volunteers on turboduck/tduniverse to see how lobbies/p2p sync behaves with the original protocol (as opposed to the bolt-on raknet that project paradise relies on, at least partially). Why? There's a few reasons why I am reinventing the wheel here: Maintainability. As far as I know, there is someone that can restart the server, but further development is not possible. Such an addon could be an ignore mode/blacklist, that btw the original protocol even has. Stability/Improvements. It would be a stretch to believe that the original networking may be better then what pp has, because I don't know pp's internals, but I can at least get rid of the need of using a VPN in order to register an account. At least working on that is in our hands again. Completion of missing features! Once we have freeride working, we can focus on porting all other missing features over from tdu2 (e.g. races, clubs) and re-develop others (e.g. the diner, that has evolved into the CRC in tdu2 or the eBay/vehicle auction house). Modding Tools, HC Mod, Map Expansion and others that I do for TDU1 will be addressed much further down in this article. Project Riviera: Finally I can talk about riviera! Some of you may have already spotted our statusses on Discord, such as "burning tires by the sea", "eating slices of watermelon" and have wondered what that is about. It's not about adding a new island to tdu2, well, not exactly, but close. At some point in deeply modding the tdu games, I have had a lot of situations, where I've regretted "fighting the game" to achieve what I want. In fact, I've spent quite some time in producing nice tools, enabling things that weren't possible before, but the question always was: What if I spent that work in a game engine, instead, working on a blank game? This is especially relevant, when we talk about changing very fundamental parts of the game, such as adding in parts that weren't done yet (auction house, night clubs) as well as changing physics, the networking layer (no peer-to-peer or more than 8 player lobbies), the architecture (64 bit) or the rendering layer (RTX Crashes). If you factor all of those in, it really sounds stupid, trying to mod the game to that degree. And it really is, with the skillset that is available in this community, all that it takes is a vision and a team. So we've set sail into riviera: What is it? Riviera is a drive-and-vibe game as well as a love letter to the 80s and the medditarean coast. Just the dolce vita, driving your favourite car in the sun with your friends, pulling the windows down, reving the engine and powersliding down-hill next to the sea. Conceptually, we're obviously drawing inspiration from our favorite open world racers, but also trying to scale up into the megaserver principle known from some MMORPGS. That means, we want to have big lobbies (think truckersMP) and even host things like car meets, crew cruises (think: easter/christmas cruises, but also smaller inofficial cruises). Together, we'll have to find out how easily we can handle many players on a map in early access. But I will write the architecture in a way that it supports those lobbies, but at the same time we can easily have virtual smaller lobbies (e.g. just you and your crew or friends). Of course, we also want to have dealerships, a special degree of customization, a selection of daily drivers, so you can build up your collection and much more. As much as I want to tell you about all of our ideas and passion, I also have to hold back to not build promises, because the game will contain what we find out and discover together, in the end. Our perspective As you can tell from my motivation in the beginning, we're not creating riviera to satisfy some stakeholders, but instead we want to work on our vision and dream, because we're not satisfied with the games that are made nowadays. Now instead of reposting those "what's wrong with the racing game scene" videos, we try to take fate into our own hands and even if we only fulfill our niche with that, we're satisfied, as we've put our passion into the game. I say that, because we're always appreciating any help in any way, but we won't be able to finanically compensate work for the forseeable future. Instead, the drive should be to finally be able to create _the_ racing/driving game of our dreams. If you want to stay up to date (yes you want to!), discuss your ideas with us or ask us clever questions, feel free to join our discord at https://discord.tduniverse.org Riviera Tooling Tools that we develop to help us building riviera will be shared with the community, where appropriate. This is both due to our strong beliefs in open source software, as well as trying to be as open to modders as possible. What would fit better as having modders use the same tools we use to build the game and maybe even have others bringing improvements into our tooling by contributing to the repository? For now, I can't tell you too much about the tools, but I am currently fighting the mathematics of "riviera calibration", a tool that aims to creating car models from scratch (as we can't port them from existing games for legal reasons) as easy as painting by numbers! Essentially you start with a collection of reference images of your car and they compute the location of the cameras in relation to each other/the model. That way you can add them as views into blender and only need to drag vertices, so that they fit onto the image, painting by numbers. So again, as you can imagine, the math involved isn't that easy, so wish me luck Andraste Modding Framework TLDR: Andraste is the framework I wrote primarly to mod tdu games, but in general it will apply to all games, and as such, support for TDU:SC is planned as well. As the first preparation of that, I've worked on an automated build of our underlying hooking library EasyHook that is 64 bits. After that, I can work on 64bit and DX11 support for Andraste in general. What can Andraste do today? Primarly, you can use andraste to replace bnk files without touching your base game installation, so you can seamlessly add and remove mods without having to have 100 different game installs. Besides that, if you know what you are doing, you can use the very same functionalities, to extend the game with scripts written in C#. As such, I've already created plugins to unlock hardcore mode, allow for bigger bnks (BigBnk) in map sectors, dumping and changing shaders on the fly. What are future Andraste plans? A lot of my to-do list is actually Andraste related, but progress there depends a lot on adoption/use of andraste by the community. Goals that I have include: A simple GUI, comparable to Mod Organizer or other known Mod Managers Making mods available in a central repository, so you can download them from a browser, they know their dependencies and all. Conflict solving when having multiple mods (e.g. multiple car mods may have conflicting databases or conflicting bnk files). This may also be known as priority, in other mod loaders. Game specific conflict solving of the above (i.e. really _merging_ databases and common bank files, instead of choosing either or, which won't work in most cases). BNK Override mode: Have a folder with the contents of the bnk (e.g. a 2db texture) and just edit it and cause the game to reload the bnk (e.g. re-entering the dealership) and it will have the modified 2db file, essentially gettting rid of manually minibnkmanager'ing for every small change. 64bit support DX11 support, DX12 support (hooking, DearImGui) As you can see, that's quite some stuff and I'd appreciate any help and especially also just actively trying to use Andraste in your workflow! Feel free to download Andraste from below, documentation is at https://andrasteframework.github.io/ TDU Modding Studio, BNKPlorer, BigFilePacker, XMBPlorer As said, I can't go too much into detail on those tools, but you can assume that I've written a plethora of tools, all revolving around TDU and covering nearly every asset format that has ever been seen. As such, I've recently dug into "pmi" files, which, according to BEight, was a key missing part for his mapping ventures, so I'll be supplying an editor for those as well, to help him improving his great mapping even further. In theory, we could have a full coverage for almost every file format in a matter of months/years, but it's very low priority for me at the moment and it's also a mixed topic since I can't release TDU Modding Studio, as is. If you're interested in working on some of the games file formats, however, feel free to contact me, I've always wanted to write a wiki containing all file formats and the general file structure (a lot of asset files follow the same outer structure, which helped me uncover pmi quite fast). Yeah, so I guess here goes my wall of text, as mentioned multiple times already, you know where to find me, if you want to comment on or ask anything. I just wanted to put down all (or say a few of those that I can think of right now) of my projects, because I know for sure that a lot of you don't even know half of what I am doing but could be interested in a lot of those projects.

This has been solved by BEight after trying a different folder / placing Andraste next to it. I've updated the Launcher to be more verbose about such errors, it may also be that specific special characters in paths still cause issues, but so far we haven't had others with that kind of error. If someone else has that error: Check your game path. Is it very long? Does it have spaces? Other special characters? If so, try it from a very simple location like "C:\tdu" and report it back to us. Also to address the malware concerns: Andraste is a modding framework, so it invades other processes to change their behavior. This may very well trip antivirus heuristics, and since we're neither signed by a costly certificate (microsoft approved so to say), nor is the exe widespread, Anti Virus may yield a warning. If you have a technical background, feel free to inspect the DLLs (they are regular .net dlls that are not obfuscated), or even read along on https://github.com/AndrasteFramework. Note: not all parts are public on github yet.

Our Gameserver has a lot of interesting features, but so far we never shared details about them, which is what I want to change with this blog post series. In this post, we'll talk about the "automatic Lobby Karma System": In modern internet forums (such as reddit), where you can up- and downvote content and if the votes are negative enough, the content disappears. The very same is happening to the lobbies in the TDUniverse Game Server. If a player can't connect to some lobby, it is "downvoted". And if it's below a certain threshold, the lobby is closed. When player A tries to connect to player B, we can't tell if it's just player A's faulty connection or if player B is a bad host. That's why both players get negative karma, but we have big enough thresholds to account for that. Also, players with too much negative karma won't be able to vote either (i.e. increasing or decreasing other people's karma). If you want to experiment with that system, feel free to have a drive on our game server, as well as using our discord bot's debug command (!lobby). If you just want to drive, however, don't worry, the system will ensure you have a smooth ride in the background. The bot is just for curious nerds. See you on the roads, MeFisto94

View File Andraste Modding Framework Andraste Modding Framework Andraste, at it’s core, is a generic modding framework, aiming to bring a modern toolset into old (DX 9) and modern games alike. It does so by injecting a .NET FX Runtime into native applications, allowing us to code the framework in a safe language (C#), which even opens up the possibility to write code modifications in .NET languages, that interface seamlessly with the native code of the game. Note: While this is in the TDU2 category, Andraste can also be used for TDU1 and potentially a lot of other games (unless they are 64 bits or have some protective anti cheat system) How to run Andraste Andraste comes with a UI Launcher and a bundled version of the generic framework (in the most recent version). That is all you need, just start Andraste.UILauncher.exe, create a profile (you can have multiple profiles and quickly switch between them) and you are ready to go. You'll want to click "open mods folder" to download/add mods to your profile and then you can press "Start Game". If your mods don't come in an andraste ready format, check the below "Converting a Mod into the Andraste Format" and contribute the file to turboduck If you still have problems launching Andraste, see our Docs on using the Launcher and "installing" mods. Converting a Mod into the Andraste Format First, make sure that you have the permission of the original author to convert/upload the andraste version of the mod. Then, follow our Getting Started Guide What can I use it for, at the moment? If you managed to convert a mod, you can now use (and remove!) these game mods, without changing your unpacked(!) base game (the game needs to be unpacked, because the VFS feature only replaces existing files, it doesn't create packed files yet). While this feature may not be impressive yet (besides removing mods without having to have a backup of your game folder somewhere), there is much more possible with that technique, but that takes manpower and time. Internally, we're currently using the scripting functionality to find out more about the game and progress on different fronts such as multiplayer or asset manager, so definitely stay tuned for further progress, at some point, we will release the TDU2 Modding Framework, that is an additional layer on top of Andraste, providing TDU2 specific features (patching files inside BNKs, running on a packed game, hot reloading, ...) If you need further assistance, don't hesitate to read our documentation, post in the relevant forum thread, here on turboduck, or in #tdu-modding on https://discord.tduniverse.org Andraste is a TDUniverse Umbrella Project by MeFisto94 and redd Submitter MeFisto94 Submitted 06/04/2022 Category Tools / Others