Insights

AdOps Reporting Dashboard Case Study 

A leading video AdTech company partnered with Avenga to design and build its proprietary AdOps reporting dashboard.

  • Client NDA
  • Industry AdTech and MarTech
  • Service Media and Entertainment
  • Technologies AWS, JavaScript, Python, Apache Spark, Terraform

Introduction 

A leading video AdTech company partnered with us to build a reporting dashboard that would collect, aggregate, normalize, process, and visualize data from its ad mediator platform. 

  • 4

    Core technologies used in the tech stack (AWS, Apache Spark, Terraform, PostgreSQL)

  • 10

    Minutes maximum latency achieved between log ingestion and dashboard updates

The ad mediator platform collects data from two separate data harvesters. 

The data collected by these data harvesters would then be displayed on the reporting dashboard. 

Challenge 

  • The main challenge here was to compute and aggregate the metrics, and then display them on the reporting dashboard. 
  • Another important challenge was to create an infrastructure that is easy to implement and update. 
  • Part of the application required containerization. To solve this, we utilized Docker. 

Solution 

To handle raw header bidding logs of varying sizes and types from Amazon S3, we used Apache Spark on Amazon EMR for data processing and PostgreSQL on Amazon RDS to store the aggregated metrics. This setup ensured low-latency metric delivery to the reporting dashboard, along with smooth filtering and display functionality.
To simplify infrastructure setup and updates, we used Terraform, which made it easy to create and manage infrastructure as code.
To support the containerization needs of part of the application, we deployed a Kubernetes cluster using Amazon EKS, making the system easier to manage and scale in the future.

Process 

Defining Data Flow and Business Logic

The first step involved mapping out how data moved through the client’s ad mediator platform and identifying what needed to be tracked.

This made it possible to determine how raw header bidding logs from different sources would be turned into usable campaign metrics.

Setting Up the Data Processing Pipeline

To handle large volumes of logs stored in Amazon S3, a data pipeline was built using Apache Spark for processing and PostgreSQL for storing the results.

This setup allowed for consistent and fast metric calculation across different log formats and file sizes.

Establishing Cloud Infrastructure

Using Terraform, the entire infrastructure was configured to run on AWS, making it easy to launch, manage, and modify as needed.

The environment supported both data processing on Amazon EMR and storage on Amazon RDS to meet performance and scalability requirements.

Delivering Real-time Insights Through the Dashboard

A custom dashboard was developed to visualize the processed metrics with interactive filtering by demand partners, devices, and time.

To ensure near real-time reporting, live data from an external service was also integrated, keeping latency under the 10-minute mark.

Results 

Our client received a ready-to-use platform that meets the project’s requirements. 

Want to make sense of your AdOps data without the delays? Get a dashboard that actually keeps up.

Let`s talk

Publishers can get better insights into the performance of their inventory, view the performance of their demand partners, and filter reports by demand partners, devices, and time intervals.