Timothy Pruitt | Timmy Hunter-Russell · October 25, 2026 · Full Sail MSCS Capstone
The 30-Second Wall
The month started with a bug report from my own browser: pressing Start on a new campaign threw Unexpected token ‘<‘, “<!DOCTYPE “… is not valid JSON. The cause was not our code at all. Heroku cuts off any request that has not answered within 30 seconds and sends back its own HTML error page. Building a campaign is two Claude calls in a row (design the whole story arc, then write chapter one), and that regularly took about 35 seconds.
The fix was to stop making the player wait on one request. Starting a campaign now kicks off the work in the background and returns a job ID straight away, and the page checks back every two seconds until the story is ready. While it waits, players get a loading screen with a spinning d20 and rotating D&D tips, so 40 seconds of waiting feels like part of the game instead of a frozen button.
First Impressions: A Tutorial and a Survey
New players now get a guided tutorial: a welcome on the home page, a note while they build their character, and a spotlight tour of every panel in the campaign screen. It is tracked per account (0 = not done, 1 = finished, 2 = skipped, plus the step they skipped on), so we can see exactly where people bail out.
The bigger addition is the in-game survey, which is how we measure the study. At 30 minutes of play the game locks behind a required survey: it cannot be closed, the server refuses game actions until it is answered, and closing the tab just brings it back. Players can ask for “5 more minutes” twice (40 minutes at most), and every showing, extension and submission is logged. If the game ends first, for example the character dies, the survey appears right then instead, and we record which of the two triggered it.
The survey grew from 18 to 24 questions this month. The new six ask about the challenge without ever hinting that difficulty can change: whether fights felt fair, whether the challenge suited the character, and one I am especially happy with: “Over your session, how did the fights change?” (got easier / stayed the same / got harder / went up and down / not sure). Comparing that answer between groups tells us whether players notice the adjustment, without ever asking them directly.
The Mistake We Had to Own
When I sat down with our thesis and design document to plan the model, I compared what they promised with what was actually running. They did not match. Our “Group A” players had been playing with a difficulty formula I had pushed live without verifying it: every campaign started at a random difficulty between 20 and 80. Worse, difficulty only changed between chapters, and chapters almost never end inside a one-hour session, so the random number just sat there all game.
There was not much data either: 9 Group A campaigns, 8 players, and 18 finished fights in total. So we relabelled all of it pilot data, kept it out of the results, and rebuilt the study properly:
– Group A plays the static SRD game. Monsters exactly as written, nothing ever adjusts.
– Group B gets the ML model adjusting each fight.
– Both groups run at the same time, on the same interface, with the same stories, the same survey and the same tutorial. The difficulty adjustment is the only difference, so if B comes out ahead, we know what caused it.
– New players are assigned at random the first time they start a campaign, in blocks of two: you join whichever group is smaller, and a tie is a coin flip. Nobody is told which group they are in.
Every fight is now logged on its own row: the difficulty it was set at, whether it counted as a success (won without dropping to 0 HP), and for Group B, what the model predicted and which model version made the call. Every turn also logs how long the player took to decide and a play-style label (aggressive, diplomatic or exploratory) from a rule-based labeler.
What Can the Game Actually Change?
Our hypothesis says the model should hold players near an 85% fight success rate. So the first question was what the model is even allowed to change. I built a difficulty “step” for every fight, from −6 (much easier) to +3 (harder), with step 0 being the SRD as written. Each step changes enemy hit points, their to-hit bonus, their damage, and how many of them show up. The step can only move one notch per fight, because players who notice the game compensating for them report feeling cheated.
Why go all the way down to −6? Because the simulation showed that a level 1 character fighting alone at the SRD’s own numbers wins cleanly only about a third of the time. With the range stopping at −3, 85% was simply out of reach.
Training a Model Without Players
Here is the catch: 18 real fights cannot train anything. So, pairing with Claude Code again, I wrote a simulator that runs our actual combat engine offline: real character sheets, real SRD monsters, scripted player personas, no database and no API calls. It plays thousands of fights in seconds.
The first run was way off: 12% success, against roughly 50% in real play. Tracing fights blow by blow found two problems. Half the bot characters had no weapon at all (one Ranger was punching for 2 damage), and the simulated players always used their first attack, so a Wizard stabbed with a dagger instead of casting Fire Bolt. After it started drawing enemies the way real fights did and picking each character’s best attack, Warlocks went from 25% to 44% and the numbers landed much closer to real play.
The model itself is a scikit-learn random forest (tree models are the right tool for tabular data this size), trained offline on my PC. It predicts the chance that the next fight succeeds from 14 player features: level, HP, armor, best attack, recent wins, play style, decision time and how they did in earlier campaigns, plus the difficulty step. Heroku never trains anything. The trees are exported as plain JSON, and a small evaluator in Node reproduces scikit-learn’s predictions to within 0.0000000000000002.
Two results surprised me. First, the raw probabilities were too cautious: when the model said 85%, those fights were really won 92% of the time, which would have aimed Group B at the wrong place. Isotonic calibration fixed it, and now 85% means 85%. Second, my partner and I debated training mostly on static-SRD fights (80/20 and 50/50 splits) to mirror real play. I tested all three on the same 20,000 held-out fights. They ranked fights equally well, but the concentrated versions made Group B swing difficulty twice as much, right up against our limit, so we kept the even spread.
The Dry Run That Changed the Design
Before letting the model near a real player, I ran the whole study on simulated players: thousands playing as Group A and thousands as Group B. The first version of B barely beat A, and the reason was structural. Everyone opened at step 0, where a third of level 1 characters die in their very first fight, and B could only move one step per fight towards a target five steps away. Sessions ended before it got there.
So Group B’s model now also picks the opening fight’s difficulty from the character sheet and the player’s history, and then moves one step at a time. The difference was dramatic: B’s typical distance from the 85% target fell from 0.52 to 0.15, and simulated deaths dropped from 86% (Group A) to 28%.
The last tweak: “aim for 85%, but don’t force it.” The model now leaves difficulty alone while a fight is predicted within 7 points of the target, so it only moves when a player is clearly off:
// Close enough already: hold, rather than nudge towards the exact number.
const hold = options.find((o) => o.step === clampStep(step));
if (hold && Math.abs(hold.p - TARGET) <= TOLERANCE) {
return { move: hold.step - step, predicted: hold.p, options, held: true };
}
Same accuracy, and difficulty now changes direction a quarter as often (0.05 reversals per player instead of 0.20).
Teaching the Game What Level You Are
One of our pilot players had a level 20 character. Nothing stopped you from levelling up outside a campaign, so the game now caps characters at level 3, checked on the server so it cannot be bypassed by editing data in the browser. That also exposed places that ignored level entirely: the story arc always planned levels 1 to 5, and the final boss was sized for level 5. Now the arc plans level-ups from your real level to the cap, and the boss scales with you: about CR 2 with one minion at level 3, instead of a level 5 boss with up to three. I also levelled up all 26 real characters offline, so the model now learns from levels 1, 2 and 3. Its level 2 predictions went from 5 points off to within 1.
Smaller Things Players Will Notice
– DM suggestions. Under each DM message there are now two or three ideas for what to do next (one to talk, one to explore, one bold). One click sends it, and you can always type your own instead. They are deliberately balanced so they do not nudge everyone into one play style, since play style feeds the model.
– Checks always use the button. The DM once asked for a “Grapple check” in its narration. That is not an SRD skill, so the game threw the request away, no button appeared, and the player typed “Got a 15”, which the DM accepted. Now unknown actions map to the right skill (grapple becomes Athletics), a check written into the narration still gets a button, and typed rolls are ignored.
– Character wizard. Gender is now part of the character (male, female or non-binary, with pronouns passed to the DM and the matching portrait), and the wizard warns you when you pick a skill you already get from somewhere else. Along the way I found that a Human’s bonus skill was never actually granted. Fixed.
What’s Next
The study is live: new players are being assigned to A and B as they start their first campaign. From here it is recruiting play testers, watching the data come in, and getting ready to analyse it. If you have an hour, come play! You will never know which group you are in, and that is exactly the point.
Timothy Pruitt: Week by Week
9/28 – 10/4
My main focus was getting the map into play with getting it generated based on where you were at, in the story, leaflet provided Greate structure with shapes and coordination with the mapping, the below image provide, alongside with the first polished version of our 3-step survey
10/5 – 10/11
This week was a big step in our mapping, where we took our mapping to the next level, the map is created based on the story it is given, Since we were limited with our model, we have to pre generate images based on categories using Gemini and the SRD 5.1 Rule set for possible scenarios and we ran with it through our prompt so our AI could choose the appropriate image based on multiple items in the current setting. Leaflet was used to create the Map layout along side with the pre generated character image icon, of male or female based on our 18-character types, I spread the requirements of what should be implemented into our map, across the entire month this just was the start of it, each week a new feature.
Legend was added via shape based on the difficulty of each setting, grey(safe area) can be from recon to nothing really going on, Green( Easy) now we have something going on, just got to figure out what, Blue(Medium) Action be cautious and aware of what’s happening details, people, who and what it around, Red (Boss) be on your toes, battles can happen at any time, Black Final boss( these are the toughest of rooms and you can die instantly if you are not careful at least with the other rooms you have a fighting chance.
10/12 – 10/28
First set of static User testing, each person was categorized Via ID within there account, there results from the build in survey was also implemented, each persons account, provided the ID; Play time; Difficulty score; chapter they ended on survey answers; Timestamp of account creation and study group. We decided to move on forward with an A/ B testing method to prove our theory, on how ML models improved gameplay and do players accept AI models within there game. A testing was a static method while B was a dynamic method which was in the backend but not implemented in the front end quit yet.
I implemented shapes along with the categorized color of each part of the newly added map, to create a connection and a DnD feel to the storyline, UI updates were made such as adding in based character images to the main page and 3D active Dice, that still had some bugs but work, just not yet active to the Adventure Log, this week mainly became of user testing.
5)
6)
10/19 – 10/25
This week was spent on polishing the Tabletop and updating the UI, the home page was given a make over by adding in transparency DnD fade in background, floating Items and new layout of the main functional features such as the load existing campaign and have it a scrollable to view all campaigns instead of taking up extra space within the main section. The map was given a updated background Point of view for more of a DnD astatic view, provided some originality to the map, and the dice feature was updated, do to not playing the full dice roll as needed in previous week, not the dice when clicked rolls and updated the adventure log with the number it reads after it finishing rolling( the top number), when hitting check in the adventure log the dice also rolls as needed just as in another regular game, there has been some hiccup due to the physics of the dice 40 percent of the time it won’t be 100% accurate the other 50%of the time it will be accurate to the point of 1 off and the other 10% is dead on, this is due to physics of the game itself, there is a solution we are testing out if we, rework the Ai check system to read the dice after landing instead of the dice in the backend, unsure if it will work, but in theory it can.
7)
