Therefore, there may be a metric that is more sensitive (even theoretical) than the A/B test. Moreover, this metric can be simply calculated using historical data without taking into account bids. If we automate this, then it can be done in a couple of clicks.

However, since we don’t know the dependence of the number of clicks and CPC on the rate, then this metric will produce quantitative results with a rather large margin of error, but possibly quite accurate qualitative results.

In other words, this metric will answer the question “Which option is better?” Rather well, but rather poorly on the question “How much better?”.
Intermediate calculations may seem voluminous to you, but the final formula will be short.

For the sake of simplicity, let’s assume that each keyword has:
CPC and number of clicks are proportional to the bid;
conversion probability is independent of the bid.



Source link