Our client is an esteemed US-based telecom company operating in 45 states. They are one of the nation’s largest telecommunication and internet providers to residential customers and businesses. They were dealing with huge databases daily and managing them was difficult for them.
Business Challenges / Pain points
Our client was facing challenges in storing and parsing their database every day. With more than 200-300 GB of data being handled daily, this was no small task and involved a vast amount of data handling. They needed to generate and analyze accurate KPIs like Signal Strength, RSSI, RSSQ, Download KPI, Upload KPI, and the Initial connection to provide a better customer experience.
The client’s traditional data warehousing infrastructure was expensive and could not accommodate the speed or size of their telecom’s data. It was a serious challenge in terms of how to grow and scale. The need was to implement a data management and analytics platform to let the client collect the data from multiple sources and get precise insights.
To meet the client’s need, our data science team developed a big data analytics warehouse system utilizing Hadoop HDFS, Impala, and Spark. Utilizing the parallel and distributed computing capability of Spark and Hadoop, our team designed an ETL system to parse and load the data in the live warehouse. Also, we leveraged the Cassandra system for Adhoc live insights by the telecom engineers.
As the client expected that the number of users would be constantly growing, the solution was to be easily scalable to store and process an ever-increasing amount of data.
Backend: Java, Scala, Cassandra, Kafka, Spark, Hadoop, Impala
Frontend: – ReactJS, High Chart
Spring Boot, SparkSQL, Spark Streaming
Amount of Data Handled in this Project
- Around 200-300 GB of raw data from OSS is gathered & parsed. The parsed data is stored in Warehouse.
- One-month live insights reporting producing 2 TB live data in warehouse and petabytes of cold data in analytics warehouse.
Impact on Business
- Our solution reduced the overall data parsing and the report generation time to 14 hours.
- The client can now analyze the anonymized data produced by billions of networks which can provide insights into the issues being experienced by their customers and allow the operator to invest in areas that can improve the experience of customers as fast as possible.
Key Success Factors
- We are flexible to incorporate changes /functionalities demanded by the client.
- Our prior knowledge of Big Data Analytics helped in the easy adoption of a big data analytics warehouse system.