Big Data Ingestion & Cloud Architecture
Customer Challenge
A healthcare company needed to increase the speed of their big data ingestion framework and required cloud services platform migration expertise to help the business scale and grow.
Innovative Solution
Illumination Works performed architectural reviews and enhancement development for the ingestion framework and enterprise data warehouse (EDW) and provided thought leadership on the company’s migration to Amazon Web Services (AWS) cloud.
Benefits/Outcomes
- Improved Hadoop ingestion framework
- Developed incremental ingest capability
- Improved Hadoop architecture (EDW)
- Provided expertise on batch and real-time data
- Improved StreamSets architecture
- Developed ingest pipelines for streaming data
- Improved AWS cloud migration approach
Toolbox
- Big data architecture, cloud architecture, data integration
- Python, AWS, Hadoop, StreamSets, Kafka, Hive
- Big Data and Hadoop best practices
- Agile development (Scrum Master)
- Healthcare insurance data
