Dwarf Fortress is one among those oddball passion projects that’s broken into Internet consciousness. It’s a free game where you play either an adventurer or a fortress filled with dwarves during a randomly generated phantasy world . The simulation runs deep, with new games creating multiple civilizations with histories, mythologies, and artifacts. I reached bent him to ascertain how he’s managed one , growing codebase over 15+ years, the perils of pathing, and debugging dead cats. Our conversation below has been edited for clarity.
It has become notorious, and rightly so. Individual dwarves have emotional states, favorite gems, and grudges. And it all takes place in an ASCII interface that appears imposing to newbies, but seems like the text go to bed The Matrix: craftsdwarf, river, legendary megabeast.
The entire game is product of 1 developer, Tarn Adams, aka Toady One, who has been performing on Dwarf Fortress since 2002. For the primary four years it had been a neighborhood time project, but since 2006 it’s been full time. He writes all the code himself, although his brother helps out with design and creates stories supported the sport . Up so far , he’s relied on donations to stay him going, but he’s currently performing on a version with pixel graphics and a revamped UI which will be available for purchase on Steam.
I reached bent Tarn Adams to ascertain how he’s managed one , growing codebase over 15+ years, the perils of pathing, and debugging dead cats. Our conversation below has been edited for clarity. If you would like more, we also spoke with Tarn on the podcast.
Also Read : Why HTML is called NASA hacking programming language?
Q: What programming languages and other technologies does one use? Basically, what’s your stack? Has that changed over the 15-20 years you’ve been doing this?
A: DF is a few combination of C and C++, not in some quite standard obeying way, but kind of a multitude that’s accreted over time. I’ve been using Microsoft Visual Studio since MSVC 6, though now I’m on some version of Visual Studio Community.
I use OpenGL and SDL to handle the engine matters. We went with those because it had been easier to port them to OSX and Linux, though I still wasn’t ready to do this myself in fact . I’m unsure if I’d use something like Unity or Unreal now if I had the selection since I don’t skills to use either of them. But handling your own engine is additionally a true pain, especially now that I’m doing something beyond text graphics. i exploit FMOD for sound.
All of this has been constant over the course of the project, except that SDL got introduced a couple of years in so we could do the ports. On the mechanical side of the sport , I don’t use tons of out of doors libraries, but I’ve occasional picked up some random number gen stuff—I put during a Mersenne Twister an extended while ago, and last I adopted SplitMix64, which was featured during a talk at the last Roguelike Celebration.
Q: What are the challenges in developing one project for therefore long? does one think this is often easier to try to to by yourself? that's , because you wrote every line, is it easier to take care of and change?
A: It’s easy to forget stuff! checking out ‘;’, which may be a loose method but close enough, we’re up to 711,000 lines, so it’s just impossible to stay it beat my head now. I attempt to name my variables and objects consistently and memorably, and that i leave enough comments around to remind myself of what’s happening once I reach a spot of code. Sometimes it takes several searches to seek out the precise thread I’m trying to tug on once I go and revisit some piece of the sport I haven’t touched for a decade, which happens quite bit. I’d say most changes are focused only on certain parts of the sport , so there's quite a lively molten core that I even have a way better working knowledge of. There are a couple of really crusty bits that I haven’t checked out since before the primary release in 2006.
Regarding the relative simple doing things alone , certainly on behalf of me , who has no experience performing on an outsized multi-person project, this is often the thanks to go! People obviously get good at doing it the opposite way, as an example over within the AAA games context, and clearly multiple engineers are needed over there to urge things done on time. I’d be hesitant to mention I can enter and alter stuff faster than they will , necessarily, since I haven’t worked therein context before, but it’s true that I don’t have any team-oriented or bureaucratic hurdles to leap through once I want to form an alteration. I can just go roll in the hay . But I even have to try to to it alone.
Q: What’s the most important refactor/change that you simply had to make?
A: There are some refactors that have lasted for months, redoing certain data structures then forth, though I’m unsure anything is ever a refactor strictly here since there’s always opportunities to push the mechanics forward simultaneously and it is sensible to try to to so when the code knowledge is fresh.
Adding the Z coordinate to form the sport mechanically 3D (while still being text) was another one, and really the foremost mind-numbing thing I’ve probably ever done. Just weeks and weeks and weeks of taking logic and performance calls that relied on X and Y and seeing how a Z fits in there.
Making the item system polymorphic was ultimately an error , but that was an enormous one.
Q: Why was this was a mistake?
A: once you declare a category that’s a sort of item, it locks you into that structure far more tightly than if you only have member elements. It’s nice to be ready to use virtual functions which quite thing, but the tradeoffs are just an excessive amount of . I started employing a “tool” item within the hierarchy, which began to get various functionality, and may now support anything from a stepladder to a beehive to a mortar (and pestle, separately, ha ha), and it just feels more flexible, and that i wish every crafted item within the game were thereunder umbrella.
We do tons of procedural generation, and if we wanted to, say, generate an item that acts partially like one thing and partially like another, it’s just way harder to try to to that once you are locked down during a class hierarchy. Adding things like diamond dependencies and every one that just find yourself tying you in knots when there are cleaner ways to try to to it. If different components can just be turned off and on, it’s easier, and allows you to try to to more.
I think some game developers ask this as an entity component system, though it’s my understanding that harder-core optimizer people consider that as something else where you’re actually breaking things down by individual fields. employing a single object with different allocated subobjects is nearly certainly worse for cache misses, which may be a whole other thing, but the advantages in organization, flexibility, and extensibility just can’t be ignored, and therefore the different subfields of the tool item aren’t used so often that it becomes an optimization issue.
Q: Did you run into any issues moving from 32 bit to 64 bit? That seems like one among those things that was huge at the time but has become pretty accepted.
A: Not at all! I’m struggling to consider one issue. Fortunately for us, we already had our byte sizes in check pretty much , since it comes up saving and loading the worlds; the format needed to be nailed down back once we set that up, especially because we’ve had to affect endian stuff between OSes and every one that. and that we don’t do any gnarly pointer operations or other stuff which may have gotten us in trouble. It just ended up being specialized code for 64 bit conversion thanks to our other practices, entirely accidentally . the most issue was just getting the time together to form the change, then it didn’t find yourself taking nearly as long as i assumed it might .
Q: I’ve seen other games almost like DF die on their pathfinding algorithms.What does one use and the way does one keep it efficient?
A: Yeah, the bottom algorithm is merely a part of it. We use A*, which is fast in fact , but it’s not ok by itself. We can’t cash in of a number of the innovations thereon (e.g. jump point) since our map changes such a lot . Generally, people have used approaches that add various larger structures on top of the map to chop corners, and since of the changing map, these just take too long to take care of , or are otherwise a hassle. So our approach has been to only keep track of connected components reachable by walking. These are pretty easy to update even when the map changes quickly, though it does involve some flood-filling. as an example , if water cuts the fortress in half, it must overwhelm from one side and update an entire half the fortress to a replacement index, but once that’s done, it’s good, generally. Then that permits us to chop most failed A* calls from the game—our agents just got to query component numbers, and if the component numbers are an equivalent , they know the decision will succeed.
It’s fast to take care of , but the downside is that the component indices are maintained for walking only. this suggests that flying creatures, as an example , don’t have global pathfinding intelligence that’s any different from a walker. In combat and a couple of other situations, we use short-range flood fills with their actual logic to offer them some advantages though. But it’s not ideal for them.
I’m unsure we’ll attempt other structures here to form it work any better. For our map sizes, they’ve all failed, including some outside attempts. Of course, it'd be possible with a very concerted effort, and I’ve seen other games that have managed, for instance, some rectangular overlays then forth that appear promising, but I’m unsure how volatile or large their maps were.
The most simple idea would just be something like adding a replacement index for fliers, but that’s an outsized memory and speed hit, since we’d got to maintain two indices directly , and one is bad enough. More specific overlays can track their pathing properties (and then you path through the overlays rather than the tiles), but they're hard and slow to take care of because the map changes. There are various other ideas floating around, like tracking stairs, or performing some limited path caching, and there are probably some gains to be made there. We are certainly at the sting of what we will currently support in terms of agents and map complexity, so something’ll need to give if we would like to urge more out of it.
Q: thereon note, you’re simulating tons of things all at once—how does one manage numerous numerous actors asynchronously (or do you)?
A: If we’re talking about asynchronous as in multithreading, then no, we don’t do any of that, apart from the graphical display itself. There’s tons of promise here, even with microthreading, which the community has helped me out with, but I haven’t had time to dive into. I don’t have any experience and it’s a bug-prone thing.
Q: have you ever tried other projects/technologies alongside DF?
A: Sure! The side project folder that’s migrated between computers for the last ten years approximately has about 90 projects in it. a number of them lasted for days, some for multiple years. they're mostly other games, nearly always in other genres, but there also are a couple of DF helper projects, just like the myth generator prototype. Nothing on the brink of seeing the sunshine of day, but it’s fun to fiddle .
Q: together with your ~90 side projects, have you ever explored the other programming languages? If so, any favorites?
A: Ha ha, nope! I’m more of a noodler over on the planning side, instead of with the tech. I’m sure some things would really speed up the belief of my designs though, so I should probably a minimum of learn some scripting and fiddle with threading more. People have even been kind enough to provide some libraries and things to assist out there, but it’s just difficult to dam side project outing for tech learning when my side project time is for relaxing.
Q: you've got the foremost interesting release notes. What’s your favorite bug and what caused it?
A: It’s probably boring on behalf of me to mention , but I just can’t beat the drunken cat bug. There’ve been a couple of videos made about it by now . That was the one where the cats were exposure dead everywhere the tavern floor, and it clothed they were ingesting spilled alcohol once they cleaned their paws. One number was off within the ingest-while-cleaning code, and it sent them through all the symptoms of alcohol poisoning (which we added once we dressed venomous creatures.)