It is the most demanding PC game in terms of GPU power “and for a long time it had no graphics”
It takes a lot of GPU power to run today’s most powerful AI models. Even the most extreme gaming PC build (opens in a new tab) wouldn’t be able to handle the kind of punch needed to get the best AI dungeon (opens in a new tab) operational models. For Latitude to power what is essentially this leaner, more free-to-play and creative online D&D campaign, you’d need something truly beastly.
According to Nick Walton, Latitude CEO and software engineer who helped design the initial code for the D&D-inspired story generator (opens in a new tab)“If you wanted to run the biggest AI dungeon for AI, you would need not only a powerful GPU, but a band eight or 16 massive high-powered GPUs.” Even these should be more powerful than the best graphics cards (opens in a new tab) in the consumer market.
Language models such as the AI Dungeon’s Dragon experience, based on the 178 billion parameter Jurrasic-1 Jumbo language model developed by AI21 (opens in a new tab)need enormous computing power to recognize and recall important story elements and use complex writing styles.
Nick pointed me to the DGX A100 station (opens in a new tab) as a benchmark for the type of machines Latitude uses to power the most intense games AI models (opens in a new tab). It’s a computing behemoth even with just four of Nvidia’s $10,000 A100 GPUs, the first and essentially most powerful Ampere graphics cards to surface.
“AI Dungeon, especially at launch and probably still today, is the most demanding game in terms of graphics card requirements,” Nick explains, “and for a long time it had no graphics.”
For machines like the DGX Station with, say, eight high-end A100 GPUs, you’re looking at around $150,000 in upfront tech costs, and that’s just to run the most advanced AI alone – the company has several other models of AI it needs to run concurrently for your dungeon enjoyment. This is one of the reasons the company uses AI service providers, and even those have struggled to source GPUs recently.
Obviously, running an AI business like this also comes at a huge energy cost. Nick tells me that at one point the company was spending almost as much on running the machines as it was on the payroll for the 16-person team (including supervisors).
The game’s AI image tool adds to these energy costs (opens in a new tab), which earlier this week Latitude made the decision to roll it out for free to all players. Nick was able to walk us through how it worked for us.
It says a catalog of dozens of thousands Images are pre-generated, each is captioned and then associated with stories as they are read, rather than being generated on the fly. He clarifies that generating frames during gameplay isn’t really viable for an operation like this, but that the AI models behind the process are smart enough to know he can get away with matching a image titled ‘Mountain Forest with Castle’ to a story in which a medieval kingdom is mentioned.
Nick also laments that working with AI can be very difficult. Control what the AI says to prevent it from going rogue (opens in a new tab) is difficult, because “AI can sometimes be like an unruly toddler”. You can kind of sway him in the right direction, but sometimes if you “give him instructions not to do something, you’ve now primed his brain, and he’s probably going to do that thing.”
Fortunately, the team is working on optimizations to not only make their AI more efficient, but also to keep it under control. All of this will become more important as the game unfolds. Steam (opens in a new tab) next week, July 28.
Our Fraser thinks video games need more Dungeon Masters (opens in a new tab), and Nick is inclined to agree. In fact, it was his fascination with the freedom and non-linearity of Dungeons & Dragons that led him to create AI Dungeon.
Nick really exudes a passion for AI storytelling, believing that this type of technology will change the storytelling landscape in two main ways. First, “Ai is going to enable experiences that have a freedom and dynamics that have never been possible before.” He talks about how many people have completed those same quests in World of Warcraft, for example, and how annoying it is to know other people are doing the same as you.
“I want to play an experience where I go to this small town, and I go through all these unique events that no one has gone through before, and I end up becoming the hero of the town,” Nick said. “Maybe no other player in the whole game cares about this town, but ME! This is where I had all these cool experiences that were unique and where I had the freedom to decide where goes the story.”
And it’s not just a unique experience that AI storytelling offers. “The second thing that I think is really powerful,” says Nick, “is the ability of AI to enable an entirely new, massive generation of creators…allowing each person to be the creative director of their own experience.” Nick is convinced that creators will one day be able to orchestrate their creative visions, bringing them to life through the power of AI.
Honestly, that’s a much better view of the future of AI than most people, so I appreciate it. It’s exciting to see companies like Latitude grow and evolve with something I’m so passionate about: emergent storytelling. And I’m sure the Steam launch is going to be a hit, so put AI Dungeon on your wishlist because you’re ready for a stroll.