Large-Scale Data Integration
Customer Challenge
The USAF requires reduction in the number of systems and point-to-point interfaces across agencies, organizations, and geographic locations.
Innovative Solution
Illumination Works supported the Air Force Chief Data Office (CDO) leveraging Informatica Axon Data Governance tools hosted in the CDO data lake environment. The CDO data ingestion processes and framework were being built to accommodate an automated metadata collection process. To facilitate rapid system ingestion, our team developed templates to define common language and parameters to define any given ingestion, including transmission methods, metadata, and appropriate security controls, across all systems regardless of file size, format and transmission method.
Benefits/Outcomes
- Documented templates for persistent system interfaces
- Kylo/Nifi workflow that can be duplicated across systems
Toolbox
- Streaming data ingestion, profiling, cleansing
- Big data frameworks
- Data science exploitation (machine learning, advanced analytics, and artificial intelligence)
- Kylo, Apache Nifi, Informatica Axon