Best Practice of App Store Optimization (ASO) – A/B Testing and A Manual


Best Practice of App Store Optimization (ASO) – A/B Testing and A Manual


Best Practice of App Store Optimization (ASO) – A/B Testing and A Manual


Best Practice of App Store Optimization (ASO) – A/B Testing and A Manual



The question of how to get users interested in an app often looms over the heads of app marketers, and it is definitely not a simple task to get around. Conducting A/B testing is one tool to provide the direct solution to the problem, giving marketers concrete data about user preferences and the market, to streamline the App Store Optimization (ASO) process.


There were not a lot of app stores that offered A/B testing functions aside from Google Play store, but starting from late January 2022, it is available on Apple App Store as well through the updated feature Product Page Optimization (PPO). In order to perform your ASO process the best way possible and avoid all the twists and turns during the experience, here are the guidelines and tips for an effective A/B Testing:


IOS App Store A/B Testing

1. Start setting up A/B test on both Google Play Store and Apple App Store

For marketers who have accessed Google Play Console before, they may be no strangers to the route that enables the configuration of A/B testing to take place. Creating an A/B test requires one to go to Google Play Console first.

Once you are in Google Play Console, select the app that you would like to run the test on. Click on the “Store presence” under “Grow” in the left menu, choose “Store listing experiments”, then “New experiment”. Underneath the option of “Default graphics”, you can select “Create” to begin setting up the attributes for your test.

During this phase of configuring the setting, you should follow the instructions that appear on screen, choosing the variants and options that match the information of your target audience to complete the settings. For each experiment, you can run 3 test variations of the store listing that exclude any functions for experimentations on text based attributes.

(Google Play – Store listing experiments)

App Store Product Page Optimization A/B test is a new function for marketers to try at on iOS, conducting it means that you have to reach the feature of PPO first in App Store Connect. Sign in App Store Connect, click on the “Product Page Optimization” right under “Features” to begin configuring the test. It is important to note that you have to be an Apple Developer Program member in order to gain access to App Store Connect on the web or its app version.

Similar to running A/B test experiments on Google Play store, please select options based on the targeting information to complete the basic settings. You can run a single test during a 90-day period, consisting of no more than 3 test variations, the test variation is called “treatment” by Apple.

(App Store Product Page Optimization, from App Store Official Website)

The details of the options within the settings will be explored in the following points.

2. Visual Assets

Graphics are more likely to catch the eyes of users, instead of long text and description. This is why visual creatives are crucial parts of metadata for marketers to optimize, an element such as the app icon is not only appearing on the product page, but also in the charts, search results and feature section, helping in forming the users’ first impression for your app.

2a. App Icon

As mentioned above, the app icon is one of the most representative items of your app at the first glimpse. To experiment with it, you can create a version of an app icon with color tone that is in contrast to your original icon, or a set of icons in multiple colors, or even reduce textual elements within the icon to enrich the selections for your test. This way, you can at least be able to tell which direction you should go into for changes, should one of the icon graphic versions not be appealing enough.

2b. Screenshots

Screenshots are your perfect opportunity to show the content of your app. But to arouse users’ interest in knowing more about the app, putting captions or wordings in the images to highlight the storyline or captivating features of your app is necessary. Varying the captions thus can be one component you test with.

Rearranging the placement or order of the screenshots is another way that is faster and less time consuming for you to immediately go for testing as there is no need for you to prepare extra materials.

(Screenshots, from App Store Official Website)


2c. Preview / Promo video

Video is a great way to show your gameplay and user experience. You can focus solely on one particular characteristic of your app for each app preview or promo video individually and then A/B test them, it will give you an idea of which feature of your app is the most attractive to the users.

(App Preview, from App Store Official Website)

2d. Feature Image

Both app stores let users test on the app icon, screenshots, preview videos. On Google Play store, however, you can test feature image as well, which is the banner image to display in your app store product page or listing. Even though adding the feature image is not compulsory, it is still highly recommended, as the first thing users see when opening your app is the feature image. You can choose either an image or a video to place with, or vice versa for the test version, and then compare their impacts on the results.

3. Text and Localizations

You can A/B test on the text based elements on the product page through localized versions on both stores. A maximum of 5 localized experiments can be run at the same time on Google Play store, you can try out multiple text descriptions and visual assets for audiences and users of certain regions or countries. PPO on Apple App Store, on the other hand, offers more or less the same function for testing in localizations for the app store product page, but with only 3 test versions available.

To see the impact of localizations, you could elect to change just one of the graphic assets or texts to test the water for a specific version of the language, to see if they perform better than the default product page. This is an effective way to tell if localizations really help with the traffic and conversion rates in some of the regions such as the United States, Singapore, Taiwan etc.

Marketers can prepare different lists of keywords to apply to each localized test version, and review the performances of these keywords. But remember not to put any wordings referencing any store performance, rankings or sales performance in the store listing on Google Play store, because Google strictly prohibits any potential activities that can manipulate the rating, review or chart rankings etc.

4. Traffic Proportion

Allocating the amount of traffic that will go to your test or experiment is an important step for your A/B test. Both stores allow you to decide the percentage of the traffic that all test variants receive, you may wish to set it above 30% – 50%, letting as many users as possible see all the versions, so that the test results can be more accurate. If you divide the percentage you allocate to the test by the number of the variants equally, you will know the actual percentage of the traffic for each test variation or treatment. For example, if you choose 45% of the traffic to go to your test, and you have three test variations, then each of them will receive 15% of the allocation.


(App Store Product Page Optimization Traffic Proportion Template, from Apple Developer)


5. Duration

All the A/B tests should be run for at least a 7-day period, no matter what assets or attributes are being tested. This will enable a comparison between the data of weekday and weekend traffic volume and impression rates, further for you to conclude on the performance of the test variations. Expect to take more time to test on a region with less users, as you need to collect sufficient data to perform an accurate analysis.

6. App Review Process

This item is noted for those who run A/B testings on the Apple app store specifically. Before any test can be successfully published, you have to submit all your “treatments” for review first except those which only have changes in screenshots and preview videos, they can be submitted separately for Apple’s approval.

7. Monitor Your Test

After your test starts running, it is important to keep track of performance metrics to evaluate the performance of all the test variations, this can help you make analysis on the preferences of the users and adjust your follow-up strategy to improve the effectiveness of optimization.

For checking on the performances of any A/B tests on Apple app store, go to App Analytics in App Store Connect, here you could view and compare the performance of all your treatments and also including your original product page, by selecting to display statistics of impressions, conversion rates, improvement rate and confidence level etc.

On Google Play store, you can monitor the numbers of installers, retained installers, first time installers etc. Basically the statistics that are available for you to monitor in Google Play Console are install-centric and user-centric. To compare the effectiveness of the test variations, check the numbers of installs divided by different audience shares.

(Monitoring Test in App Store Connect, from App Store Official Website)


Contact us with professional advice on how to improve the ASO performance.


Related articles: 
Top 7 Mistakes Made When App Rank Boosting
5 Things You Need to Know About Optimizing Your App Store Listing (ASO)
Three Key New Features You Need to Know about Apple App Store in iOS 15

About GoGoChart

GoGoChart, a tech enabled mobile marketing company headquartered in Hong Kong, being the first to be certified by Apple and the Google App partner in Hong Kong, its mission is to become the world’s leading mobile marketing technology brand. Since its establishment in 2016, the company has helped over 2,000 brands and apps in more than 60 countries optimize and promote their apps. it has also evolved into an all-rounded digital marketing service provider after winning trust from thousands of brands. Key clients of GoGoChart include Fortune 500 companies, blue chip listed companies and SMEs from the finance, consumer, ecommerce and gaming sectors.

GoGoChart was awarded the Hong Kong Technology Fast and Rising Star Program organized by Deloitte both in 2019 and 2021, the Best Use of Mobile Online Campaign (inaugural season) by Interactive Advertising Bureau Hong Kong (iAB HK) in 2021, and officially became an Affiliated Member of The Association of Accredited Advertising Agencies in 2021. The company’s regional teams are situated in the Philippines and mainland China.

Subscribe To Our Mailing List

Get updates and learn from the best

Subscribe To Our Mailing List

Get updates and learn from the best

Subscribe To Our Mailing List

Get updates and learn from the best

More To Explore

More To Explore

More To Explore

Web3 Marketing

13 Essential Terms You Must Know to Get Yourself Familiar with Web3

To adapt to a future world powered by this trending technology, you can start by getting yourself familiarized with basic concepts, and learning their terminologies and glossary. Through understanding them, you will form a better picture of the whole idea behind web3. 


The Launch of Monetized Overlay Ads in Facebook Reels Platform

Upon receiving positive feedback from the creators and users alike, Meta made announcement on February 22 – they are currently introducing the Reels platform to over 150 countries on Facebook, and testing multiple relevant updates to plan to add in the the availability of Reels Overlay Ads, brand suitability controls, full-screen and immersive ads, Stars function and expand the Reels Play bonus program.

Ready to
Level Up?

Distinguish your product or service with our wide range of award-winning solutions and technologies.

By clicking submit, we’ll use your info to reach out to you directly for your Free Consultation and to keep you updated on GoGoChart news, events, and more. You can unsubscribe from these communications at any time. For more info, please review our Privacy Policy.

Ready to
Level Up?

Distinguish your product or service with our wide range of award-winning solutions and technologies.

I declare that all the information given in this application form is true and correct