Author: Timothy Pruitt & Timmy Hunter-Russell
Date: August 28, 2026
Project: AI-Powered Dungeon Master with Adaptive Difficulty (MSCS Capstone)
The short version
We presented our proposal to the Capstone committee this week and came out with approval to keep building. That mattered more than it might sound. If the committee had decided we didn’t have enough evidence behind the idea, we would have fallen back to our second pitch and repeated a month of research, requirements gathering, and market analysis from scratch. Instead, we spent the rest of the week writing code.
What we built this month
Choosing the project and defending it
The month opened with a decision between two pitches. We went with the AI Dungeon Master over our telemetry analytics idea, then spent a full milestone on research: competitor reviews, user stories, use cases, scope, and a hypothesis we could actually measure. That work fed directly into the requirements document and, eventually, into the proposal that the committee approved.
We also changed our stack early. We started out planning on Firebase and moved to Heroku for hosting with PostgreSQL as the database. The deciding factor was the shape of the data we need. Session telemetry, character state, and cached rules text all want relational structure and a real query language, and we did not want to fight a document store to get it.
User authentication and account creation
Timmy built account creation and authentication this month, along with the first pass of the UI around it. It’s a login screen and an account flow, which is not the glamorous part of a capstone, but it is the piece everything else depends on. Our entire research design assumes telemetry is tied to a specific player over multiple sessions. Without accounts, we’d have a pile of anonymous events and no way to say “this player, over these five encounters, kept dying to the same monster.” Auth is what turns raw events into a player history the model can learn from.
It’s built on the Express backend with the React/Vite client talking to it over the API. The backend is never exposed to the player directly, which is a rule we set in the architecture and have stuck to.
Getting the AI to read the rulebooks
This was the research-heavy piece of the month and it is going to be the hard part of the project. The DM needs to know the rules, and the rules live in large PDFs. Handing an entire rulebook to the model on every turn isn’t practical, so we spent the week working out how to parse those PDFs, break them into chunks, and store them in the database so they can be retrieved quickly and consistently.
Two requirements make this harder than a normal document search:
- Cache first, always. The AI has to check what we’ve already stored before it reaches outside for information. Otherwise responses get slow, inconsistent, and expensive, and two players asking the same rules question can get two different rulings.
- The AI has to govern, not just answer. It isn’t enough to retrieve the right paragraph about character creation. The DM has to enforce it: valid stat arrays, legal monster statistics, encounters that fit the party. That’s the step after retrieval, and it’s the one we haven’t solved yet.
IRB and consent forms
Timothy spent the week preparing our Internal Review Board documents and participant consent forms. Because we’re collecting gameplay telemetry from real people, we can’t run a single play test until that paperwork is approved. It’s easy to treat this as an administrative side quest, but it’s a hard gate on our schedule, and getting it started now rather than the week before testing is probably the smartest scheduling decision we made this month.
Retrospective
What went right
- The proposal passed. We had enough research behind the pitch to defend it, which is the direct payoff of spending an entire milestone on requirements instead of rushing into code.
- We split work cleanly. Auth and UI on one side, IRB and consent documentation on the other, with no overlap and no duplicated effort.
- We made an infrastructure decision early instead of late. Moving off Firebase in week two cost us a day. Moving off Firebase in week eight would have cost us the project.
- We have a working front door. Accounts exist, which unblocks telemetry, which unblocks the model.
What went wrong
- The PDF ingestion problem is bigger than we scoped it. We treated “store the rules in the database” as one task on the board. It’s really several: parsing, chunking, embedding, retrieval quality, and cache invalidation.
- We’re still ahead of ourselves on the governing layer. Retrieval is only half of it. Making the AI actually enforce character creation and monster stats is the harder half and we’ve mostly been describing it rather than building it.
- Play testing slipped later than we wanted. We’d hoped to be testing by now. Realistically it’s next week or the week after, and our advisor told us sessions need to run longer than 30–45 minutes, which means budgeting closer to an hour per participant. That changes recruiting and scheduling, not just the test plan.
- Month one was research-weighted. Necessary, but it means the code base has to move faster than originally planned for the remaining milestones.
How we improve
- Break the RAG work into small, testable tickets on the Trello board rather than one large card, so we can see whether retrieval is actually working before we build the governing layer on top of it.
- Define what “good enough retrieval” means with a number before we start tuning it, so we don’t polish it indefinitely.
- Schedule play test sessions as full hour blocks from the start, and line up participants now rather than when the build is ready.
- Keep the weekly advisor check-in as a hard commitment. Every course correction we’ve made this month came out of one of those meetings.
Course reflection
Time management. Front-loading research was the right call even though it felt slow, because the committee presentation depended on evidence we would not have had otherwise. The cost is that the build schedule is now tighter, and the next several milestones are coding milestones with no research cushion left in them.
Prior coursework. This project leans on almost everything we’ve taken. The machine learning and data science coursework, especially the predictive maintenance project built on drive telemetry, is the direct ancestor of what we’re doing here: collect telemetry from real usage, train a model offline, and use it to predict what happens next. The HCI coursework shapes how we’re designing the play test protocol and the survey instruments. The research methods course is why our proposal had a defensible hypothesis and IEEE-formatted references instead of a good idea and some enthusiasm.
Working with our advisor. We meet weekly with Dr. Kachirski. Three pieces of his guidance changed what we’re doing. He told us to train the model offline on a local machine and upload the trained artifact rather than training on the host, which simplified our hosting story considerably. He pushed us on what the RAG database should actually store, which is the question we spent this week answering. And this week he told us our play test sessions need to run longer than 30–45 minutes, which means an hour per session and a real recruiting plan.
Are we ready for the next phase? Yes, with one qualifier. Accounts, hosting, and database are in place, the architecture is settled, and the committee has approved the direction. The qualifier is that play testing depends on IRB approval and on the rules cache being far enough along that a session is worth running. Those two things are the whole of our next two weeks.
Next up
- Finish PDF ingestion into the rules cache and validate retrieval quality
- Start the governing layer: character creation constraints and monster statistics
- Close out IRB approval and consent forms
- Recruit participants and schedule the first hour-long play tests