Hi I am Aditya Nambiar a third year undergraduate student pursuing Computer Science at IIT Bombay.This is final blog post on my project - Visualization for Mailing stats and A/B testing. I shall be explaining the work flow of my project in the blog post
Listing Page
The first page is the listing page where all the scheduled and drafted A/B tests are listed out. If any A/B test has been “submitted and sent” then one can view the results, if it has not yet been submitted and sent then one can edit it. Only one option edit/ result is visible at any time.
The button "New A/B Test" allows a user to start a new A/B test.
Creating a A/B test
Creating a new A/B test consists of 4 UI tabs.
First Tab
The first tab is the campaign tab. Here the user fills in the name of the A/B test so that he can identify it later to view the results or edit it. He also choses the type of testing
1)From Names (Mail A and B are sent from two different email addresses)
2)Subject Lines (Mail A and B are sent with two different subject lines)
3)Two Different Emails (Incase the user wishes to try two different emails itself he can chose this option).
On clicking Next a new A/B test is create in the civicrm_abtest table in the database if it is a new A/B test, else it simply updates the database.
Second Tab
If the user selects From Address
If the user selects Subject Lines
If the user selects Two Different Emails
In this tab the user composes the required mailings. Depending upon the option selected by the user in the first tab an appropriate view is generated. The compose screen has all the required functionality like inserting tokens , Ckeditor etc
On clicking Next 3 mails mail A, mail B, mail C are created and stored in the database. Currently the mail C is just a copy of mail C.
Third Tab
Here the users selects the recipients to whom he wishes the send the mailings . He also selects the group percentage i.e the percentage of recipients to whom he wishes to send the test mails. The winning criteria is also set in this tab . Currently I have provided 3 options –
1) Open – Number of mails opened.
2) Total number of clicks
3) Total number of clicks on a specific Url.
Also the date and time at which the mailings should start being sent either now or a later date can be set.
On clicking Next the data related the A/B test is updated in the database.
Fourth Tab
In this tab the user can see all the options he has selected at one go and change any option by clicking on the corresponding tab. He can preview the emails by clicking on the preview mails button and also send test mails
Preview the two emails
Both preview mails and send tests open in modal windows.
On clicking on submit and send the mails A and B are now scheduled. The A/B test can no longer be updated.
Clicking on scheduled and send redirects the user to the listing page.
Report A/B test
This is the tab where all results are displayed. The user can view a graph of the performance of mailing A vs mailing B where performance is measured as the winner criteria selected by the user.
There is an adjacent table which displays all the statistics.
The User if convinced can stop the test and send version A or version B to all the remaining users. Once the date till which the A/B was supposed to run is over the data will no longer be recorded and the graph and table remain the same. Clicking on either “Send Version A” or “Send Version B”
Opens a dialog where the user is asked to confirm the option and on clicking Yes the mailing C is set to scheduled and the user is redirected to the listing page.
I would like to thank my mentor Kurund Jalmi for the immense support he provided me during the 3 months. I would also like to thank Monish Deb , Siddhant Rajgopolan for helping me in the project. These 3 months were an amazing learning experience for me , I would like to thank CiviCRM for providing me with this opportunity.
