This Software Center Brown Bag seminar was hosted by theme 4, Customer Data and Ecosystem Driven Development, on August 23rd, 12:00 – 12:30.
Presenter: Yuchu Liu, industrial PhD student at Volvo Cars.
Abstract:
A/B testing is gaining attention in the automotive sector as a promising tool to measure casual effects from software changes. Different from the web-facing businesses where A/B testing is a well-established practice, the automotive domain often suffers from limited eligible users to participate in online experiments. To address this shortcoming, this talk describes how to run A/B testing with small sample sizes. In particular, the talk presents a method for designing balanced control and treatment groups so that sound conclusions can be drawn from experiments with small sample sizes. Based on case study research with companies in the automotive domain, the talk identifies the benefits and limitations of the A/B group design method.