How to A/B Test App Screenshots Without Updating Your App

By Evaldo Rossi | November 4th, 2014

app screenshots

App Screenshots Are Key to App Store Optimization

Conversion Rate Optimization for the iOS App Store is a hard nut to crack. App screenshots, being the only strong visual assets at your disposal to showcase your game or app, are one of the best levers to further optimize your App Store presence. Now, and until we finally gain access to real App Store analytics (with and beyond what should already be implemented), it's hard to determine the best performing screenshot variants. Pure download numbers are affected by so many other factors that they can't be trusted to decide whether your app screenshots, icon, and app preview video are fully optimized for conversion.Even with App Store analytics, it would still be an issue— to change anything but the description, you still need to update your app, which means uploading a binary and waiting for Apple's approval. This requires you to go through the whole review process to test just one app screenshot at a time. No need to specify that this process is very slow, cumbersome and, above all, unpredictable.Thankfully, there's a much smarter way to A/B test screenshots and app preview poster frames (preview videos): running ad campaigns.Screenshots and Ad CreativesBoth app screenshots and ad creative have similar objectives as visuals: to showcase the app/game and convince the users to download it. In both situations, the attention span is very low; it will only take one or two seconds for users to make the decision to click on the ad.For this reason, the creative has to grab the users’ attention very quickly, and show only the important information likely to convert; it could for instance be the game's mechanics, the characters, or the art style.App screenshots usually have one advantage over ads: relevance. Users are pro-actively looking for apps like yours. On the downside, you have to stand out from direct competitors on the App Store’s search results page.Ad creatives appear during gameplay or loading screens, usually as an interstitial pop up. Although they don't have direct competitors in that moment, they are disrupting the user experience for that game, so they have to be really convincing in order not to get instantly closed.The fact that both visual assets share those similarities makes it very common to A/B test ad creatives as a proxy for app screenshots. The most popular performance metric is obviously the CTR, or click-through rate. For screenshots, it's basically the same: you want to know how many of the users who have seen your image were compelled to know more about your game, by clicking it to navigate further to the App Store description (or even directly downloading your app/game).

app screenshots

Testing Screenshots Using Ad CampaignsWhen designing screenshots, you will probably come up with several different compositions, and you won't be able to use them all. After launching your game, you can make all these assets compete against each other to discover the champions.You can for instance test five creatives to discover which mechanics-focused screenshot best converts users. You can make other tests for the first app screenshot, with contenders showing different characters from your game, to see which one the users like the most.You can also use this opportunity to test app preview poster frames (videos). Since these videos don’t play automatically, the frame used as a cover for your video has to be compelling just as well. It's a good idea to think about possible poster frames early in the process, when designing the video. If you don't have it yet, you can create these frames as screenshots in order to test their concepts. When you finally have created the video, you can then select the champion as a cover frame after uploading the video on iTunes Connect.ConclusionAfter running ad campaigns, it's much easier to decide which designs to go with. Conversion Rate Optimization depends a lot on these kinds of metrics, otherwise you would be pretty much guessing your way through it. You could do the research personally or ask your online community for their favorite screenshot, but the situation and context are so different that the results can never be trusted 100%.Even with access actual App Store analytics and real CTR data from inside the App Store, you can still use ad campaigns to quickly test several variants at the same time, and this without depending on Apple's app review time.Are you planning on A/B testing screenshots? Have you tried another method before? Tell us in the comments! About the Author: Evaldo Rossi is the Founder of WordData - App Store Optimization (ASO). Follow him on Google+ and check out his blog to learn more about ASO.

Evaldo Rossi

Missing person