Analysis of Ad AB Test Results

This project evaluates the effectiveness of a creative, interactive online advertisement for the SmartAd brand by comparing it to a standard control ad.
The goal is to determine whether the creative ad leads to higher user engagement.

Key Findings

• User Interaction: Users interacted more with interactive ads, with higher frequencies of "yes" or "no" responses.
• Statistical Significance: A significant difference in the number of "yes" responses to interactive ads was observed.
• Distribution Issues: Imbalances in user distribution by browsers, operating systems, and devices were noted. Ad impressions were uneven, though user ratios between groups were correct.

Recommendations

• Follow-Up Test: Conduct with improved technical implementation, ensuring even ad impressions and balanced data distributions.
• Extended Duration: Extend the test duration to two weeks to identify weekly and hourly variations for better ad budget optimization.

Contents

• Description
• Exploratory Data Analysis (EDA)
• Analysis of Ad Impressions and User Interactions
• Sanity Checks
• Sample Ratio Mismatch
• Distributions Balance
• Hypothesis Testing
• Conclusions and Recommendations

Tech Stack

Python Pandas Seaborn scipy.Stats