A/B testing

C Code Feb 16, 2022 Articles

What is A/B testing?

A/B testing (also known as bucket testing or split-run testing) is a user experience research methodology. A/B tests consist of a randomized experiment with two variants, A and B. It includes application of statistical hypothesis testing or "two-sample hypothesis testing" as used in the field of statistics. A/B testing is a way to compare two versions of a single variable, typically by testing a subject's response to variant A against variant B, and determining which of the two variants is more effective.



In simpler words, If you ran into a scenario where you need to run experiments on your website or app to determine the design/interface with the best user engagement, you create variants of your site. The variants are split between the users on your site, either equally or by some formula ("60% A, 40% B"). This is what A/B testing means. After running the experiments for a specified duration, you can conclude which variant has the better user engagement through the embedded analytics i.e. (CTAs, pageviews etc).


For Rails, Sinatra or any other rack based application, split gem is a handy tool for A/B testing. 


Rails Tutorials on A/B testing:

  • https://medium.com/swlh/a-b-testing-with-ruby-on-rails-3782c6aa1066
  • https://longliveruby.com/articles/rails-ab-test

Category:  Misc