I am a passionate software developer. After developing telecommunications applications for ten years, I joined Peak in 2013. At Peak, I had the opportunity of experiencing mobile game development for 5 years. Today, I am leading the Big Data and Business Intelligence teams where we develop and maintain Peak’s data infrastructure.
Big Data in the Cloud : A Glimpse of Peak’s Data Infrastructure
Improving user engagement is our number one priority while developing our products. We need to understand the interests of our users, their habits and how they interact with the product to give the users what they want and keep them engaged. It is not an easy task to accomplish, but it is definitely an essential factor in building a product that users love. We listen to our users all the time by analyzing their behaviors and generating insights to see what works and what doesn’t.
Delivering on this promise requires a two-fold approach: * Building the data pipeline and SDKs to do a lot of heavy lifting for our products when it comes to data collection and the processing of this data * Democratizing the data access within the company to enable everyone to slice & dice the data we accumulate.
As our user base grows, the volume of data increases at an ever-increasing rate. Our big data platform has evolved to keep pace. We currently collect billions of events and five terabytes of behavioral data each day. Our architecture consists of mobile SDKs, APIs, Go/Java/Python based services and a data warehouse to store and process close to a petabyte of user data. Our Business Intelligence team creates data models and uses visualization tools to make it accessible by both technical and non-technical people.
In this presentation I will give technical details about our big data pipeline. I will also talk about how our BI team transforms raw data into insightful data models.