Developers like me are data nerds. We love crunching numbers from App Store metrics or A/B tests on a finished mobile game. But what I wish more developers realized is how powerful qualitative feedback testing can be. Listening to players in the early stages of production and continuing to solicit their opinions is key to keep them coming back and spending money.

Having worked with developers big and small, I’ve seen how effective surveys can be in driving successful mobile games. For the maximum return on investment, follow these five tips to introduce surveys and feedback testing as a critical step in your iterative process.

Include feedback testing in each stage of game design

You probably already do plenty of testing: usability tests, QA, split tests, etc. But for such a data-led industry when it comes to monetization and retention, games studios too often leave the creative side of development to chance.

Don’t miss this huge opportunity. Gather player opinions at every stage of the development process, and as you iterate, confirm your creative choices.

  • Ideation – In the concept stage, solicit feedback on your game idea, the game’s name, and storyboards. You don’t need a working build; a simple game design document is enough to ensure you’re headed in a viable direction.
  • Design – Test your target audience’s reaction to art styles, character art, and UI. Use the feedback to improve the look and feel of your game.
  • Development – Show character animations or game mechanics in action to gauge player reactions. Choose the best movements, music, voiceover, and sound effects.
  • Pre-launch – Ensure that your App Store marketing assets, including the icon, screenshots, trailer, and description, resonate with players. Test different ad creatives for Facebook and other networks if you’re planning user acquisition campaigns. Rather than testing creatives in the wild (and paying the ad spend), have confidence that your ads will communicate effectively before you start running them.

An example of feedback testing different styles of an app icon

Test with a manageable sample size

If you survey thousands of people right away, you will likely have too much data to digest in any meaningful way.

Instead, start with around 200 survey respondents. It’s a small enough pool to be manageable while still being statistically valid. Plus, as most feedback testing is priced per response, you can start with a small testing budget and expand if you think the results justify it.

Build the right test audience

All feedback is not created equal. Your test audience must accurately reflect your end-users. If a role-playing game is aimed at women, for instance, men’s feedback isn’t as valuable. Think about which demographic and behavioral traits apply to your prospective audience, such as:

Demographic traits

  • Gender
  • Age range
  • Education level
  • Marital status
  • and more…

Behavioral traits:

Maximize Your Ad Reach with ASTRAD

Harness the power of ASTRAD to optimize your campaigns and drive real results. Increase visibility and engagement with our advanced DSP tools.

Get Started Now
  • Mobile device (iOS, Android)
  • Mobile gaming habits (casual vs. hard-core gamer)
  • Favorite game genres
  • Average App Store spend
  • and more…

If you’re not entirely sure of your audience yet, start with a broad audience and see which groups are most responsive to your ideas. Hone in on specific demographics and behaviors to build a picture of your target player. For example, if you’re ideating a new puzzle game, show the designs to a broad mix of gamer types and demographics. If you find that active puzzle gamers are the most enthusiastic about the designs, then as you move on to testing game mechanics, conduct your surveys with this targeted group.

Pre-supposing your core demographic at the outset could add assumptions too early in the process. Continue to refine your game’s player persona. Once the game is launched, cross-correlate with any demographic insight you have from users.

Don’t forget about competitors

This nifty research trick can help you stay ahead of the pack: test your game assets against the competition.

Any aspect of your game, from concept through launch, can be put to the test. Collect outside feedback to understand why players prefer one game over another. If your game comes out ahead, you can proceed with confidence. If the other game beats yours, you still have time to change course.

In this example, a game developer compares its game to one of its competitors

If your game is already launched but underperforming, competitive testing can help identify areas for improvement so that you can course-correct.

Ask the right questions

Good survey answers derive from good survey questions. As you build your feedback tests, avoid these common mistakes:

  • Don’t ask leading questions that may create bias. If you ask a question like, “What do you love about this character?,” your survey will collect only positive answers. Use objective language such as “Would you want to play as this character?” to get an honest picture.
  • Don’t present assets in different formats. If you show four icon designs but they aren’t the same resolution and size, survey respondents might react to the relative size or clarity of an icon, rather than its design. Keep aspect ratios and image sizes consistent.
  • Don’t test more than one thing at a time. If you’re trying to decide on a good name for your game but you present each name in a different logo design, then survey respondents may react to the logo they like best rather than the name they like best, jumbling your results. Keep everything consistent except for the parameter you’re testing.
  • Don’t overwhelm respondents. If you submit an entire screenshot of your App Store listing, then respondents will react to different parts of it: screenshots, description, name, and potentially even fields largely out of your control, such as star rating, price, or reviews. Instead, focus their feedback on one asset or question, so that you can methodically improve each component.

In summary

Appropriately done, feedback testing is an invaluable way to perfect your game long before it gets as far as the App Store. Even a small early testing budget can generate considerable savings in resources and ad spend down the line. By confirming design choices and benchmarking your game with its intended audience, you can make data-driven decisions all along the way. And, as I said at the outset, we developers love data.