Artwork experiments
Artwork experiments
An experiment shows a share of players your variant artwork and compares its results with the live artwork. Use it to answer questions such as "does a character close-up beat a scene shot?".
Who can run one#
Games in full launch with a complete artwork set (three covers and two preview videos) and no artwork review pending.
Create an experiment#
- Open the game in the console and select Experiments, then New experiment.
- Name it (up to 30 characters) and choose what you are testing: covers, videos or both.
- Add an optional description of your hypothesis.
- Upload the variant artwork. It must follow the cover and video specs.
- Start the experiment, or save it as a draft.
Data collection begins at the next midnight UTC.
Limits#
- One experiment per game at a time.
- Two per game per calendar month.
- 14 days maximum per experiment.
Reading results#
The results table compares the live artwork with your variant and shows the difference for:
- click-through rate,
- conversion to one minute of play,
- plays per user,
- average play time,
- day-1 retention (it lags by about two days),
- average daily active users reached.
Filter by new or returning players and by desktop or mobile. A cover that wins on desktop may lose on mobile because the thumbnail is smaller.
Choosing a winner#
You pick the winner. A winning variant goes through artwork review before it replaces the live artwork.
Tips#
- Change one thing: colours, framing or title treatment, not all three.
- Let the experiment run long enough to see a stable difference; a few hundred clicks is noise.
- Keep the loser. What did not work is information too.