The workshop gives you a statistical, technical, and organizational toolbox for implementing A/B testing in a medium to large organization. It is a lecture mixed with discussions, in the context of specific client cases. We run this workshop only for businesses, not individuals.

Perfect for:

Data scientist,
Data analysts,
Technical managers.

On the workshop you will

Learn from lecture supported by slides

Group discussions

During the workshop we cover:

Why do you need A/B testing?

Sequential testing
Split testing without the statistical regime
Split testing with the statistical regime

Building a hypothesis and interpreting test results

Experimentation process and hypothesis
Understanding false positives and false negatives
Calculating sample size
Calculating and interpreting the p-value

Advance hypothesis testing

Multivariate testing
Multi-metric experiment design
Segmentation
Experiment influence lifespan

Data quality

Traffic allocation
Filtering noise
Automatic error detection by sample distribution uniformity checking
Simpson’s paradox
Cross-device consistency

Frequentist statistics

Core assumptions and implications
Chi-squared
Welch’s t-test
Sample size reduction – CUPED technique

Bayesian approach to A/B testing

Cons of the frequentist approach
Interpreting Bayesian experiment results
Pros and cons of the Bayesian approach

Fully automated optimization with multi-armed bayesian bandits

Multi-armed bandits use cases
How it works mathematically

Implementing A/B testing in an organization

Building an A/B testing platform
Building a data-driven culture

Final discussion

How long does it take?

It takes 1-2 days, depending on the client's needs and experience.

Requirements

Before each workshop, the trainer meets with the group to gather requirements and understand the context.

Bartłomiej Bęczkowski

Bartek is an experienced backend/data engineer, one of the founding fathers of the Allegro experimentation and optimization platform. Currently he is leading the Allegro MLOps transformation. Along the way he has taught people A/B testing and data engineering.

Register

Choose this option if you need a workshop for your team or company. The basic price is 2500 EUR for the group for 1 day.

Reserve now

Choose this option if you need a workshop for yourself or for a small group of people. The basic price is 200 EUR per person for the whole workshop.