Measuring a metadata change honestly

The most treacherous thing about ASO is that every change can look like it worked. Rankings move every day anyway; when they move after you did something, the human mind reads it as cause and effect.

Telling them apart is not complicated but it takes discipline: measure the same words, in the same countries, the same way, before and after.

Change the title, the subtitle and the screenshots in the same version and whatever happens you will not know why. You come out of it with nothing to inform the next decision.

Because shipping a version is expensive, people want to change everything at once. It is a saving that stops learning completely.

When new metadata goes live the index does not update immediately, and rankings take a few days to settle. Movement in the first 24 hours is usually noise.

The window to look at is the week before the change against the week after. Compare a single day and even a weekday-weekend difference can invert the result.

A metadata change always works in both directions: the words you added rise, and the words you pushed out of the field fall. A report that only shows what you gained will please you every time and teach you nothing.

The right report puts two lists side by side: which keywords rose and which fell. The net gain is the difference, and sometimes it is negative. Seeing that is what tells you what to revert in the next version.

If the answer to “what did we put in the subtitle in February?” lives nowhere two months later, that period's data becomes useless too. Every metadata change should be recorded with its date, its before and its after.

rankcusp keeps this on the Outcomes screen: the field that changed, the date, and the ranks on both sides of that date. Which takes “did it work?” out of the realm of recollection.