Yesterday I had a playtest in Atma with a good friend Luke Mason. Luke has playtested before so he was not diving in with fresh eyes. The playtest did send him down a mostly unfamiliar route though which helped get some early insights on our recent weeks of progress.
The Playtest Goal
This playtest focused on an early tutorial system I've been working on, a first look at ship combat, and general intro pacing to the game. The tutorial system includes basic building controls, creating your first ship, gearing it up, and fighting your first pirate ships. This system had a very early playtest a few weeks ago where two of my friends went through the building section of the tutorial and gave some early feedback. That mostly developed into UX (user experience) improvements and a noticeable need for some help learning ship combat. Going into this playtest, one of my largest concerns was the new tutorial taking too long for the average player to get into their first ship combat. I have ideas on how to address it, but decided to wait until after the playtest before executing on them.
My Thoughts on Tutorials
On the topic of tutorials, I don't really like them. I prefer when games teach themselves through a player's own curiosity and exploration. Unfortunately it feels a bit naive to think Atma could get to that point in a reasonable amount of time with the minimal artistic experience that I have. So the goal for now is to introduce a traditional tutorial. I'm trying to keep it short, sweet, and to the point. As the UI and UX of the game improves I'll slowly start stripping away pieces of it until it becomes more of a generic hint system or maybe dissolves completely. Until then, the tutorial exists and needs to be playtested.
The playtest structure
I started yesterday's playtest asking Luke to record the experience. His instructions were to simply make a character on the correct realm and work his way through the first two tutorials the best he could. I informed him I would interject at one point when I needed him to make a specific decision (which was admittedly unnecessary information to give him) and that I would only jump in with hints when I felt like his experience was being tampered by a bug, significant UX issue, or he was going down a distracting path. The goal was to let him explore as much as he could on his own, but there are certain areas of the game where the existence of a distraction was a valuable enough insight and we didn't need to spend the time exploring it. For example, at one point he started fighting chickens which was noted, but land combat had minimal value to this playtest. He was also able to ask questions if he felt stuck and no longer had an idea what to explore. Again, simply knowing he was at a dead end was enough for me to move forward. Struggling for one minute compared to five minutes on how to cut a tree would not be a valuable use of time.
Other than that, the playtest was as hands off as I could make it.
Servers for the playtest were hosted in us-west-1 on Amazon EC2 instances. I used a c7g.medium (1 vCPU, 2GB ram) instance using a linux arm64 build in Test configuration. The client was also built in test configuration for windows and distributed through steam.
The Biases
To frame this playtest in a more accurate light, let's review some of the biases going into this playtest.
- Luke has played Atma before. His previous playtest was "build a house". He has experienced the building system and general controls, but has not experienced these specific tutorials or ship related content.
- Luke is invested in my project as a friend. We went to college together and kept in close touch ever since. We've always jumped on calls for big career updates or just to see how each other is doing. Luke is invested in my project and wants it to succeed. He isn't going into this project as a player trying to get their time or money's worth. He's going into it as someone who wants to really dive into a friend's creation, look through every nook and cranny, and most likely wants this playtest to be smooth.
- Luke is a VFX artist with an interest in game design. He wants to pull out and comment on the little things that matter. He's been a part of playtests before and has even conducted them himself. He explores not just the world but every visual element he can find and often sees the world through a developer's lens. Without a doubt, there is more effort put into solving a problem in this playtest than someone who just found the game on steam might be inclined to put in.
- Luke knew he was signed up for a one hour playtest. This session simply cannot test a five, 10, or 30 minute retention.
- Luke is a builder. He likes games where he can be creative and artistic. He generally avoids heavy combat games or high risk games. The harvesting and crafting tutorial we are asking him to complete are right down his alley, but the ship tutorial is fairly averse to his normal player goals.
It's important to keep these things in mind when we create inferences from the playtest. We will need to acknowledge what questions this playtest cannot answer for us and find playtest coverage in the future that can help us address those. But this leaves a lot on the table for things we can learn today.
How Did it Go?
First, I think it's valuable to share my morning-after impressions from the playtest. With a night to let the experience settle in, there are a few things that stand out to me.
Overall, the playtest went really well. There were very few stuck moments, only a couple of significant bugs, and Luke was able to destroy some pirate ships by the end of it. My favorite moment of the playtest was watching him creep up to a pirate trade route, gauge his distance carefully, and then have a tier 5 merchant ship send him into a panic. The merchant ship was completely harmless (as long as he didn't attack it), but he hadn't learned that yet. It was great to see the presence of these ships actually have impact. He was able to acknowledge which ships were strong and which were weak, who he should pick on, and showed a sense of genuine tension as he picked his first targets. There was investment in his ship.
Initial Impressions
- Tutorialization - Promising. Needs a little bit of tuning. I'm confident we can move forward with the merchant and explorer tutorials next and polish up the battleship tutorial along the way.
- Ship combat - Promising. Definitely some things learned here, but overall this was a success for the current state of the game.
- Pacing - Unclear. On one hand, it felt like it took a long time for him to get onto the high seas. On the other hand, it seemed like he enjoyed his time getting there.
- Build Quality - Fine. The server crashed during the playtest, but I already had a fix ready locally. This was more of a process issue where I didn't upload the correct build. There were quite a number of UI and UX issues that definitely need attention before the game will start feeling more professional.
Playtest Notes
Below I'm going to share my exact playtest notes from yesterday. I haven't reviewed these at all up to this point. Below the notes I'm going to pull out a few of them that I felt are worth exploring in a bit more depth. These are raw, unedited, and literally sitting in a notepad file on my desktop.
luke playtest 8-14-2026
hard to read create character text. light gray over lighter gray is bad
create character name popup takes up tons of space should be a bit smaller.
still no cursor present for hovered tiles - maybe not necessary?
panel icons are long overdue to be replaced - player wanted to check their inventory and it took some time
was about to complain that the controls disappeared when he started hovering his placements. but then he saw them in the side bar
R to rotate did not work.
Build a crafting table-> Build the crafting table
thinks construction gets faster as he levels up
equipped with a drag-drop instead of right-click equip
Build the warehouse -> Build A warehouse
Build a warehouse highlights did not show. but he found his way through it anyway
Wants to be able to hold down to move his character instead of click tiles. Or at least he expected that to work - started spam clicking to get the behavior he wanted
wasn't sure what the 8 meant (it was an experience drop)
Is a bit confused why his two wood isn't working at the warehouse. dropped it on the ground instead. The warehouse requires 2 planks
Woods in the warehouse do not have "Wood" after their name
Steel Ore -> weird.
we should center the tutorial progress drots
Battleship entry shouldn't hover at the "Use the dock to build a ship" stage. it should highlight the Build button for all 3. let the player choose
He isn't seeing that he can craft multiple resources at once with a right click
Ship type subtexts are AI written -> need to rewrite by hand
spent some time reading the ship types. acknowledged battleship is the middle of the pack but has the most canons. did notice the provisions thing - biased by previous conversation after last playtest tho
wood repair kit -> remove (a pulse every 2s) text
provisions - bad description
we need to do a pass on all items
warning texts are not red and not center justified
"all cannons draw from this shared cannon stack" - kindof unnecessary.
"They fire from any bearing around the ship" - also weird wording. not really necessary to explain that yet
dropping repair kit into ship cargo isn't moving the whole stack. right click transfer does work
tutorial title could also be centered
anvil tutorial step had no highlights. but he was still able to find it
bug happened again after docking his ship. he can only click on Z = 0 and cannot walk here on things higher than that
found the fishing spot. tried to fish. noticed the failure text saying he needed a rod. moved on with the tutorial.
Is thinking he might be able to craft a stone pickaxe as an upgrade to his wooden pickaxe. not a bad idea! maybe make it level 5?
Found the tier drop down. changing it to tier 2 didn't show stone pickaxe so he left. but it also didn't show any tools. would be good to show the tier 2 tools there.
Not clear when crafted items go into the ship cargo instead of the inventory
ship told him resources would be drawn from warehouse. but then he expected items to be drawn from warehouse for crafting as well. he figured it out. but it was a brief stumble
"craft 2 sets of cannonballs" - sets was a new word. he paid a bit of extra attention to that. didn't steer his decisions tho.
cannonballs are still really small. should make them 2x2
decided to make more cannonballs than it asked him too cause he had extra stone
"Contracts build reputation with different factions" -> "Battleship contracts build reputation with the Fighter faction"
Found the manage ships panel while looking for the contracts section. made sense to him. its pretty ugly tho
in the shop - items you can not afford should have red text on the coins. items you cannot buy at all should be grayed out (bad ship tier) with some description when you hover
Inconsistent naming between contracts - Combat as a type instead of Battleship or Fighter. lets name it for the faction - Fighter. and lets show some reputation and stuff like that.
15 ships! thats so many - first impression of the contract. and agreed. we are planning to go to 10
he's wondering if he gets the reward per ship or for doing the whole contract - interpreting it as a total value for now which is correct
not super clear when you accept the contract
Map not auto opening after reading a map. map should auto open after reading the item and the island icon should have animated
read ship movement as grid based movement but liked the feel
Immediately recognized a stronger merchant ship showing up. hovered it and saw it was tier 2 right away. also noticed some are battleships right away
is now looking for his pirate contract to figure out where to go
is trying to remember what his contract said and has no way to review it. didn't go for the tutorial - was more interested in his contract
is choosing to fight white sail ships instead of pirate ships - seems to have lost sight of the tutorial
beat the white battleship but survived with only 3 health. another battleship then came in and attacked him even tho he was out of combat - combat didn't seem to end when the first ship died?
repair at the dock is still a thing - but it was also kindof a good experience in this ship. intended to be removed but lets re-evaluate
is now realizing the white sail ships are not pirate ships
is now out exploring for pirate ships. but isn't using his map
tested collision against ships - didn't hurt each other - good
remembered the pirate island was west of his island (or just hasn't rotated camera and got lucky going left of screen) - lets follow up
remembered his map. moved over to the pirate island - is still seeing merchant ships. he found the black sailed ship and sees its a merchant. is trying to attack the level two merchant. attacked it a few times then realized it was a tier 2.
is now looking to repair. anchored. started using kits
is now using all of his repair kits at once. over-repaired. didn't get full value of his kits. definitely wasted some
am going to direct him to heal himself with a cheat since he would have gotten to full health with those repair kits that were wasted
he is now looking for a tier 1
found another pirate. is recognizing they are high level.
is running away from them. but is now thinking they might not aggro so was going to go back and check. then hit terrain and got distracted by a tier 5 ship. got scared. couldn't figure out why he couldn't move
found a tier 1 ship was a bit confused why it was called "merchant ship" but recognized the black flag.
had to remember tank controls.
fought the tier 1 merchant. is now low on cannonballs. didn't seem to immediately recognized that.
did see he should anchor to loot
tier 1 merchant hardly hit him - intended
he thought he was just invulnerable till he took a small hit.
but he did remember he put on ship armor that made him tanky. is just thinking tier 1 ships hit very low damage.
sees he is out of cannonballs so is going home
seems to remember a teleport command
found another island on the way home
thought it was his - realized it isn't - wants to mark it on his map. couldn't figure out how so he sailed on. i stopped him and asked him to find the dock - but he just docked ont othe island to explore
charting is not obvious - but he also didn't do the charting tutorial. should still call more attention to uncharted islands
is remembering he saw the zoomed out version of the island (the standin) and doesn't remember seeing valuable resources so isn't expecting to find much. partially biased from a conversation from a few weeks ago where I mentioned that was a thing. great he remembered it tho
did find chickens. asking him to act like the chickens do not exist since land combat is a distraction from this playtest and may have bugs that negatively impact the experience.
coin indicator is not updating - coins were in the ship storage. just not updating in the hud
docking on the island had the same issue trying to walk on Z != 0
realized he could have gotten some stone on the chicken island but needed his crafting bench anyway
is preparing stone to go fight the 15 ships - but i'm directing him to do a bit less because we are going to wrap up playtest soon
curios - rewrite description
he is often checking both anvil and crafting bench quite a bit to see where to make things - i'm realizing we should experiment with changing the background color or texture. some visual change. to help associate what you created with what panel it was in. and create a subconcious connection between stations and the items they make
didn't build any repair kits before leaving out the second time
wants to sell the glass bead he found
opened up the shop. couldn't figure out how to sell things
pretty confusing that he has to open the ship menu manually to sell it. but he did eventually do so.
750 provisions sounds like enough hes sailing off without selling that curio. i'm letting him since most players wouldn't have found one
hes worried if he attacks the merchant ship it might agro the tier 3 merchant next to it. he isn't sure but hes worried.
seems to be sailing around quite a bit instead of hanging out at the trade route
not getting the warning message when of provisions
saikling all the way back at low speed is a bad experience - could die or use the teleport but isn't seeing that
not getting the warning for being belo 40% health and therefore travelling slower
Q n A
-- asking him about any first impressions he has after completing things
Boat section slowed down quite a bit compared to the quick successes of the land combat
15 felt high for amount of ships he could take down in a reasonable amount of time. at least for the first tier 1 contract
wanted some more surprises during that time or more beats
did see a lot of goals being set
-- asking about what goals he has set for his player type as Luke not just what I egged on. what things he finds interesting at this point and things he might go into next
Is saying the battleship wasn't super appealing but he was interested by the merchant idea. brought up how he saw elder wood. he set that as a goal he wants to find the elder wood and saffron in the game.
wants to capitalize on his own wits and is interested in merchant
from previous playtests knows building is a big part and wants to continue doing that
is assuming other players are focused on combat so he wants to go find rare islands and stuff
did see some promises of the stuff he is interested in so that was cool. made a lot of notes in his head of things to check out later
-- what is your player fantasy for merchant in terms of how you perceive atma so far
wants to have lots of ships
some sort of established trade routes within his guild
bringing value to other players who he is working with, who are in turn focusing on things he is less interested in like fighting bosses and bringing back loot he can use to build up the base
wants to be able to find his role. do the things hes interested in and benefit the people doing things he is not interested in
-- any areas you felt lost, confused, or frustrated? anything still resonating with you when you look back?
remembers when he was initially confused by having to do things to his ship and he kept looking at ship view which was very new - a bit overwhelming - a bit confused what the tutorial wanted from him. he did open this view before the tutorial anticipated
maybe we can consider locking the menus until they get to that stage in the tutorial
definitely confused about how slow he got after provisions. would be frustrated if we didn't end thereA lot of these notes are small bug fixes or quick tweaks to wording that don't really need much explanation. Some of them might not make much sense to people less familiar with Atma. I'm going to pull out the points that read loudest to me and give some initial impressions and insights. I'm reviewing this as I'm re-reading it so these thoughts are fresh initial impressions, and my actual interpretation or plan of action might change as I let them marinate.
panel icons are long overdue to be replaced - player wanted to check their inventory and it took some time
Yep. These are pretty misleading and need a more intentional design.
equipped with a drag-drop instead of right-click equip
A sign that the player wasn't naturally exploring right-click menus. This costs time and can add friction and makes the game feel less polished.
Wants to be able to hold down to move his character instead of click tiles. Or at least he expected that to work - started spam clicking to get the behavior he wanted
I've thought about this as well and heard it from other playtests. I did try a quick implementation of it and it felt a bit odd, but it could be worth giving it another shot. Also worth acknowledging the feature might not be my preference but if players want it, it could be worth adding as an option.
wasn't sure what the 8 meant (it was an experience drop)
Experience drops like this are pretty typical in games that I play, but I have to remember that many players might not be accustomed to them. It could be worth finding a way to make this more clear but for now I'm just going to keep an eye on it.
Is a bit confused why his two wood isn't working at the warehouse. dropped it on the ground instead. The warehouse requires 2 planks
The player hasn't learned to check the chat box for failure messages yet, and the plank icon might be too small making it not read quickly as a different item than the one he's used already. For now I think the gentle nudge is to increase icon size and see how that goes.
He isn't seeing that he can craft multiple resources at once with a right click
This one stands out because right-click needs to be a clear tool for players to use. We want people to right-click everything that has additional actions to see what they are. The tutorial has a subtle mention of this but it's one of the first things the player sees, and isn't reinforced after that initial step.
Ship type subtexts are AI written -> need to rewrite by handwood repair kit -> remove (a pulse every 2s) textwe need to do a pass on all items
This one I'm a bit ashamed of. I use Claude to do a lot of my coding, which isn't my issue, but I really want everything in Atma to be hand written. It's part of my connection with the game. Unfortunately sometimes Claude puts in temporary descriptions for items and I don't notice it. I'll need to do a full review of the texts in the game and rewrite anything that I didn't write myself before the next playtest.
dropping repair kit into ship cargo isn't moving the whole stack. right click transfer does work
Inventory UX has been an issue popping up over multiple playtests. The intended design isn't unique or complicated in any way, so this is a sign that the underlying system and tests need a review. I can't keep spending time playing whack-a-mole with the inventory UI.
bug happened again after docking his ship. he can only click on Z = 0 and cannot walk here on things higher than that
Now this one is a bug that has been haunting us for a while now. The issue only happens after the player leaves their island and comes back to it. They can click to walk to any Z = 0 tile but anything greater than that seems to fail. Performing actions that requires pathing works great. So there seems to be some issue with the coordinates the client is calculating for a tile or how the server is handling terrain MoveTo queries, and the issue is somehow related to travelling between spaces.
Is thinking he might be able to craft a stone pickaxe as an upgrade to his wooden pickaxe. not a bad idea! maybe make it level 5?
A stone pickaxe makes a lot of sense. Slotting it in at level 5 could be a nice progression beat where players can chase a very early self-driven upgrade. However, the game is largely built around the tier system right now and, as often as possible, wants level requirements between tiers to stay the same. A stone pickaxe would either be tier 1.5 or take the place of tier two iron at level 10. Could it just be a stronger tier one tool? It breaks the conventions of the game, which may feel sloppier than it's worth. For now, it's tabled.
Found the tier drop down. changing it to tier 2 didn't show stone pickaxe so he left. but it also didn't show any tools. would be good to show the tier 2 tools there.
Just a nice piece of validation that a recent UX experiment is going well. Previously, recipes for all tiers were shown in the list creating quite a bit of clutter.
Not clear when crafted items go into the ship cargo instead of the inventory
More of a personal note than anything, but this caught my eye as something that will most likely confuse some players. It didn't have a negative impact on this playtest, but I'm wondering if a short animation of item icons going into their respective panel icon could help alleviate potential pains. We do have a slightly unique experience with two inventory systems existing instead of one and that's something we have to get players used to.
ship told him resources would be drawn from warehouse. but then he expected items to be drawn from warehouse for crafting as well. he figured it out. but it was a brief stumble
We have a note on the ship building panel that informs players that resources can be drawn from the warehouse. This originally served to solve a problem where ships needed more resources than you could carry at once. And I still want to preserve the ability to have a ship upgrade cost more than 10 of each resource, but in today's playtest it did cause some confusion. When crafting at the crafting bench and anvil he also assumed his resources would be drawn from the warehouse. He was able to discover the real behavior in a short amount of time but this is something we should keep an eye on. It could be worth removing that text until it is actually needed.
in the shop - items you can not afford should have red text on the coins. items you cannot buy at all should be grayed out (bad ship tier) with some description when you hover
There's no two ways about it, the shop UX is awful. The icons are all temporary and the entries don't indicate what you can and can't purchase. This needs a full pass now that the shop is involved in the core gameplay path.
15 ships! thats so many - first impression of the contract. and agreed. we are planning to go to 10
Yep. I saw this one coming. 15 is too many. I was debating changing this before the playtest and am now even more validated to do so. I think 10 might be the sweet spot, but am also considering going down to five for the first contract and moving up to 10 for higher level contracts. I want you to feel that reward of completing something early on, and I have little concerns around economic balance here since tier 1 rewards will pale in comparison to higher tier rewards.
not super clear when you accept the contract
Like the shop, this panel needs a full make-over.
Map not auto opening after reading a map. map should auto open after reading the item and the island icon should have animated
This is a significant bug. My own testing recently found that using the map item lacked significant user feedback, so I added a mechanism that automatically opens your world map and animates in the newly revealed location when you use the item. But it clearly seems bugged and none of that happened. I interjected at this point and let him know what should have happened and had him manually open the map.
read ship movement as grid based movement but liked the feel
Just an interesting note! I wasn't sure if players would pick up on the grid based movement of ships. This might be his own biased knowledge of the game kicking in, but players that know the game is grid based are going to have an easier time controlling the ships than players who think it's truly continuous movement. They'll have a little better understanding when their ship takes a bit longer to do some turns than others. And knowing he enjoyed the general feel was great.
Immediately recognized a stronger merchant ship showing up. hovered it and saw it was tier 2 right away. also noticed some are battleships right away
One of my favorite moments of the playtest for sure. I remember fondly playing Voyage Century Online and seeing the high level ships sail past. When they are hostile you can really feel a sense of tension and when they are friendly you develop a player goal. "I want to have that ship someday." The fact that he could recognize the power difference in ships quickly through the ship models and UI tier badge was a great sign. I was quite surprised that he could quickly point out other battleships too. It's a possible sign that our silhouettes are stronger than I originally thought.
is trying to remember what his contract said and has no way to review it. didn't go for the tutorial - was more interested in his contract
Definitely need a way for him to view active contracts. I'm not entirely sure how this should surface, but it's significant.
is choosing to fight white sail ships instead of pirate ships - seems to have lost sight of the tutorial
Interesting moment for sure. I was hoping the white sail ships would read as friendly. But unfortunately they were the first thing he attacked, and he got roughed up pretty bad from it since it was a battleship and not a merchant. This might deserve immediate attention because a player whose not completely sold on the game yet might see this as their breaking point if they were to have lost that battle.
is now realizing the white sail ships are not pirate ships
We're happy to see he did make the connection though eventually.
repair at the dock is still a thing - but it was also kindof a good experience in this ship. intended to be removed but lets re-evaluate
This was a feature I was testing out and actually thought I removed. But it did fix a pretty gnarly gameplay experience. I am considering leaving this in, and maybe attaching a coin cost to it. Giving the player two ways to repair (repair kits and coins) might cater to different player types more. It also reduces how much harvesting people have to do if they really just want to be fighting.
is now looking to repair. anchored. started using kitsis now using all of his repair kits at once. over-repaired. didn't get full value of his kits. definitely wasted some
Repair kits shouldn't be overlappable. And they need a bit stronger feedback of when they are working and what they are doing.
is running away from them. but is now thinking they might not aggro so was going to go back and check. then hit terrain and got distracted by a tier 5 ship. got scared. couldn't figure out why he couldn't move
Two things to unpack here. I kindof liked that he was manually testing out aggro rules. That feels like an interesting moment of player experience. And he didn't feel like being attacked was so detrimental that he should completely avoid it. Seeing the tier five ship and getting scared was also really fun to watch. The power was definitely felt. Unfortunately, the reason he couldn't figure out why he couldn't move was because his ship anchored when he hit an island. The anchoring state just isn't clear right now and needs a UX pass.
but he did remember he put on ship armor that made him tanky. is just thinking tier 1 ships hit very low damage.
Really happy to see that ship armor is doing its job and that the player was able to attribute that to his advantage in combat. This was a core goal of the ship armor item.
thought it was his - realized it isn't - wants to mark it on his map. couldn't figure out how so he sailed on. i stopped him and asked him to find the dock - but he just docked onto the island to explorecharting is not obvious - but he also didn't do the charting tutorial. should still call more attention to uncharted islands
The explorer tutorial teaches about island charting, but it was really difficult to discover at this time. The idea of charting the island was interesting to see him come up with. It felt great to have a feature implemented that he wanted but just hadn't found yet. It does need to be easier to find. I'm thinking about doing a UX pass on island stand-ins to give some better information about them and make charting clearer. I don't know exactly what that looks like yet though.
he is often checking both anvil and crafting bench quite a bit to see where to make things - i'm realizing we should experiment with changing the background color or texture. some visual change. to help associate what you created with what panel it was in. and create a subconscious connection between stations and the items they make
Throughout the playtest he was bouncing back and forth between stations to figure out where to craft things he had already seen before. I think the primary reason for this is that the different stations have identical UIs. Some visual tweaks to create a stronger connection between what recipes you are viewing and what station you are at could go a long way here. I'm thinking about recoloring the anvil panel and changing the texture possibly.
saikling all the way back at low speed is a bad experience - could die or use the teleport but isn't seeing that
Sailing*. Like I said, this is unedited. This was significant though. He was moving REALLY slow when he was both out of provisions and low on health. He did attribute this to being out of provisions but overall the player needs more feedback about what is making them slow. We also probably need a stronger solution than simply "crawl your way back to town" if they are out of provisions and repair kits. Ideally, players develop stronger planning skills over time. But we do not want this initial experience to get hit with something so disruptive. For now, I'm thinking about starting the player with more provisions and reducing provision consumption. Basically, delay this issue until later into gameplay. I will also probably weaken the movement penalty for being low health.
The Follow Up Questions
I'm paraphrasing the exact question asked
What were your first impressions after completing the playtest?
The biggest takeaway here was that he mentioned the boat section slowed down quite a bit. The initial goal of killing 15 ships sounded daunting and when he actually experienced ship combat it was reaffirmed that this was going to take a while. There was a noticeable contrast in time between interesting discoveries and gameplay beats compared to the building tutorial. On a positive note, he mentioned that the tutorial put a lot of goals into the back of his mind for him to explore later.
What are some of the goals you've created for yourself? Try to separate what this playtest specifically asked of you and what you, as Luke, want to do in this world.
Knowing Luke's player type it was unsurprising that the battleship path itself wasn't very appealing. I was happy to hear that he was excited by the idea of finding elder wood (my tier 7 wood he briefly saw in the warehouse) and saffron (a tier 8 good visible in the shop). Of course, as a builder at heart, he was excited to find himself back into building as well. He mentioned finding himself into a position where he can harvest and build for people who want to go out and fight. They take care of some roles he's not interested in and he takes care of roles they are not interested in. This is absolutely a direction I want the game to go.
Were there any areas you felt lost, confused, or frustrated?
From memory there wasn't much he could point to, which was great to hear. He did mention being overwhelmed looking at some of the ship menus and in particular opening the ship cargo view (which he had discovered before the playtest anticipated). And of course, the slow down from running out of provisions and being low health was very jarring. Understandably, he said he would have been very frustrated if he had to sail all the way back at that speed. Overall, I think this was a success. I'm considering locking some of the menus until the tutorial presents them to you to avoid some of that confusion and reduce the control space for new players.
The Analytics
A couple weeks ago I added analytics to Atma. This was my first playtest with them enabled and I figured it was a good way to both test the system as well as take a look at the playtest from a more objective point of view. There are a ton of asterisks when looking at this data. But regardless, it's fun to look at and see if it does call attention to any interesting patterns. It's also a great way for us to test how we analyze a player's session when we aren't there to watch them. So before looking at this I'd ask a few questions:
- Are there any interesting patterns showing up already?
- Are there any questions we want to ask from future playtests that analytics could help with?
- Are any of these analytics clearly misleading?
Claude was used to generate this playtest report using my analytics data. The information should be accurate, but the text should be scrutinized. Any conclusions it draws are not necessarily true, but can be worth thinking about.
The Beats
The main things that catch my eye here are the first time they hit the high seas at 47:03. That's a significant amount of time before someone gets to play with a boat, which is one of the main things the game advertises. The first pirate was then sunk at 66:19. 20 minutes between hitting the high seas and sinking a ship. Yes, the playtest itself is a factor in slowing this down. When I play a fresh character I can sink a ship in 15 minutes. But I'm the developer! I'm not learning all of this as I go. This is definitely something I want to keep an eye on and look for a way to expedite.
Server Memory
This didn't seem relevant at first but actually was a bit concerning after looking into another odd behavior. I left the server running overnight and it eventually crashed. It turns out the current ship math can cause ships to, over time, spawn more frequently than they despawn. We had a runaway memory problem where eventually the server reached over 12,000 ships and crashed. This memory growth was an early indicator of that, but stayed close enough to reasonable memory usage that I didn't see it as a problem. We'll be keeping a closer eye on this in the future.
Crafted items
It's a bit misleading to see 195 crafted cannonballs. Analytics are currently counting total crafted outputs. This means one stack of cannonballs records as 15 cannonballs crafted. The amount of "recipes" crafted is a little more useful to the way my brain thinks about this stuff. Technically, we can calculate the total cannonball count from the amount of recipes crafted, or we can calculate the recipes crafted from the total cannonball count. Both work, but I think I'll be changing this analytic to track recipes crafted instead of total output count. This will also help clarify things if we ever add multiple recipes to produce the same output item in the future.
Events Ingested
Part of running analytics during playtests is to better understand how much analytics are going to be costing us once we scale up to a higher player count. 659 events over the course of a 83 minute playtest is extremely reasonable. With today's implementation we paid somewhere around $0.00005. This price is a bit misleading because we can pay a lot less when we batch more events together in a single call. Another player going through the same experience at the same time would have negligible increase in cost because the combined events would have all fit into the same batches.
The Server Crash
Some of you might have noticed a server crash mixed in there. It turns out I actually launched the wrong server build for this playtest. Thankfully it held up for all but one issue: a crash bug that happened when you tried firing a cannonball after being at sea for a while. Thankfully I already had the fix implemented and just needed to rebuild, reupload, and relaunch the server.
Impressions after reviewing
Honestly, reviewing this was a little exhausting. Part of me wants to take a break but part of me finds this to be a valuable part of playtesting to comment on. There was a TON of information here. From it, a ton of new tasks that I have to somehow organize into actionable plans. Overall I think the playtest went well. I was happy to see a lot of previous frictions were mitigated, new features working as intended, and combat in a workable place. We discovered some new areas that need to be smoothed out and plenty of UX issues that could simplify onboarding for future players. The tutorial and combat systems themselves seem to be making good progress and landed in a spot that I think lets me move onto the next area I want to playtest.
What's next?
The next playtest is going to be a group playtest. I can't keep doing solo playtests and maintain any confidence that the team experience is holding up. The game is meant to be a social game so we need to get back into group playtesting as soon as we can. We'll continue to balance it out with some focused individual playtests but neither should dominate the other at this stage. My hope is to have the next playtest as early as this upcoming Friday but a more realistic goal would be the following week. I want to clean up the main issues from this playtest and refine the merchant and explorer tutorials so I don't need to force everyone into the combat path. The plan right now is to take Monday to tackle a lot of the quick fixes. Regardless of priority, things like small text changes, UI tweaks, highlight adjustments, and simple bug fixes deserve a quick pass. When those are done we'll move onto some of the major fixes that might require redesigns or refactors.
Improving Future Playtests
I think it's important to reflect on how we conduct playtests. One thing that stood out to me as an area I can improve is how I presented the playtest to Luke. I gave him a little more information than I needed to. While there were some known issues he was going to run into, giving him the heads up that I was going to force him into the battleship didn't provide anything useful and potentially just polluted his initial impressions going into the playtest. He was already given a goal without the playtest driving him there. In the future I'm going to spend a little more time refining what information I give ahead of time and trust that course corrections mid-playtest are a better angle than trying to prepare the player for hiccups.
Another thing I should have done was prepare my follow up questions ahead of time. I've done this in the past and it works great. After this playtest I realized I had forgotten to do just that, and it caused me to stumble a bit looking for the right things to ask. This both comes off as a little unprofessional but also meant I may have missed an opportunity to structure my questions so they provide the most value to the playtest's goals.
This post was both my way of reviewing the playtest and my way of trying to demonstrate how I work through feedback. I can't say I'll be going this in depth for every play test but hopefully this shows where we are at in the process and a bit of how my brain works as I iterate through Atma.