Data Science Big Data Ingestion
Customer Challenge
A utility company needed an automated method for classifying faults that were occurring regularly on their transmission grid.
Innovative Solution
Illumination Works developed automated data ingestion processes into Hadoop, resulting in a more readily consumable data schema than the software previously used by the engineers.
Benefits/Outcomes
- Drastically reduced previously manual analysis while increasing the accuracy of predictions
- Leveraged Spark to power and automate the model for transmission engineers to examine
- Enhanced with machine learning to provide insights on transmission health
- Implemented feedback loop to allow engineers to update incorrect classifications
- Improved asset renewal via enhanced fault detection and identification of at-risk locations
Toolbox
- Hadoop, Hive, Spark, PySpark, and Zeppelin
- Sensors and Spark-generated features
- Machine Learning Classifier
- Time-Series Analysis
- Discrete Fourier Transformations
