Instead of a Crystal Ball - A/B investigating in Practice • Sebastian Gebski • Devoxx Poland 2022

The thought of A/B investigating appears trivial: 2 versions of the same product are presented in parallel to the 2 groups of users, to find (based on output metrics - e.g. conversion rate) which 1 is more beneficial. Thanks to a series of specified experiments (we keep the better version and ditch the another one) 1 can iteratively improve the quality of the application. However, although many companies declare the willingness to effort A/B investigating in practice, very fewer do that. It turns out that the seemingly simple thought is non-trivial to implement. It requires skillful usage of feature flags, telemetry for business metrics, and of course - a appropriate setup of analytic tools.
The goal of this session is to prove that nearly everyone can easy and rapidly build an A/B investigating rig for their net application - thanks to managed, ready-to-use cloud services on AWS. We'll start with customized metrics in CloudWatch and AppConfig, only to proceed with features and experiments in Evidently. Then we'll have any fun while feeding analytical tools with A/B investigating data to gain even more interesting product-related insights.

Lecture took place on Thursday, 23rd June 2022 at 14:40 in area 2

Sebastian Gębski - chief Solutions Architect at Amazon Web Services. ex-CTO, ex-VP of Engineering. He has co-built online platforms since 2001: across all kinds of organizations - starting with startups and ending with global financial groups, on 3 continents. He was designing solutions for improvement factories up to 300 people and leading engineering organizations up to 70 people. Hyper-active blogger (No Kill Switch), co-host of CTO Morning Coffee, and a serial reader. Fan of deliberate strategy plan (e.g. Semantic Spatial Maps), System's Theory, and applicable application of DSLs.
#IT #Development #SoftwareDevelopment