Display programmatic performance analysis

Problem Statement

The analysis was carried out for one of the most prominent US financial services industry players. The main marketing directions are massive lead generation and affiliate marketing. 

The company started investing heavily in display programmatic advertising to increase brand recognition and audience conversion intent by reinforcing upper-funnel marketing efforts. There were two display programmatic channels selected for initial exploration: Verizon Media and MediaMath. 

Our team was responsible for building performance analytics and reporting system for one of the channels - MediaMath.

 // cache.js
export function cache(func){
  let value;
  
  return () => {
    if( typeof value === "undefined" ) {
      value = func()
    }

    return value 
  }
}


// calculateHeavyStuff.js
import {cache} from './cache'

export function calculateHeavyStuff(){
  // do some heavy calculations here
}

export const cachedCalculateHeavyStuff = cache(calculateHeavyStuff)

The main challenges for the performance analysis for this programmatic test campaign were the company’s lack of a comprehensive attribution model and the fact that the company was using only a last-click attribution model for evaluating channel performance. While this approach would work well for most of the active in-house managed channels at that moment, MediaMath, being an upper funnel channel, rarely served as a click and/or conversion generation source but rather an impression and awareness-building channel. If we kept the same approach of evaluating MediaMath performance based on last-click attribution, we would end up with a channel that generated zero results since nearly 100% of the users, seeing a MediaMath ad, ended up converting through a different marketing channel (search, native, social, or affiliates). We needed to find a better way of evaluating the channel performance.

Last-click attribution may bring inaccuracies critical for performance evaluation. Consider all marketing attribution models to find the best match for your business. 

Planning

Since the company had, at the moment, neither a robust multi-touch attribution model nor a comprehensive incrementality measurement system for the upper funnel marketing efforts, our team had the following data sources and inputs for building a performance analysis model to make the most sense for the company:

• MediaMath performance data (cost, impressions, clicks at the most granular - impression-level breakdown)
• The visitor tracking ID associated with each impression or event is also shared with the internal lead data
• Internal sales/CRM and marketing data

Taking into account the availability of the data, our team confirmed that the attribution analysis approach could not be taken at that time since it would require additional data inputs from existing channels and a comprehensive ID graph for the organization’s marketing efforts. But the marketing management team had already started investing heavily into the channel and was planning for a long-time, higher-level cohort analysis to evaluate the effectiveness of the upper funnel additional efforts into the overall brand awareness and conversion rate impact. 

Our team has decided to assimilate the channel incremental expenses into the existing channel performance reporting instead of a more robust attribution of end results to specific marketing efforts.

Execution

Step 1 

We started with the identification of the final last-click channels for each converted lead that would previously have been exposed to MediaMath ads. To accomplish this step, we used log data provided by MediaMath that would indicate a specific user ID extracted from the client’s visitor tracking system for each click or conversion event on our client’s landing pages. This unique key served as a basis for merging internal/CRM lead reports with MediaMath’s log-level data and identifying the last click channel for each MediaMath conversion.

Step 2

Calculation of average weekly CPLs per each last-click channel. We took this measurement as an additional cost to be integrated into MediaMath performance reporting. 

Step 3 

Creation of final performance analysis and reporting that would go down to ad or publisher-level granularity. The study was an application of the following data:

• MediaMath spend data
• Last-click channel spend data
• Generated results (leads, contact rates, and final converted clients) on an ad or publisher-level granularity

Step 4

Having the comprehensive cost and performance data merged together, this performance report was updated weekly, allowing the MediaMath channel manager to identify lower-performing elements to maximize performance (campaigns, ad groups, ads, publishers, target audiences, creatives, themes, etc.).

Conclusion

This reporting was not an ideal solution since it did not prove incremental performance coming from upper-funnel marketing efforts. Still, it was definitely more efficient than blindly investing in a campaign and measuring incremental performance by larger-scope cohort analysis. It provided a basis for short-term optimizations of the channel towards better-performing channel management artifacts such as creatives, audiences, and publishers.

Past forward a couple of reporting cohorts, our team was also assigned to find a way to measure the efficiency of the channel in terms of actually increasing the potential lead purchase intent level.