Rapid Data Ingestion to Speed Analytics
The Air Force has the need to load data of varying formats faster to enable on-demand data science, advanced analytics, and decision making.
ILW extended our proven commercial Data Framework for Air Force use by adding easily configurable sources and targets and removing Hadoop-specific dependencies. This simple framework allows analysts to create source data definition files and use the provided ingestion templates to generate fully executable scripts to ingest data.
- Usable by non-developers with minimal database knowledge
- Automated and metadata driven to provide fast and flexible capability
- Extendable and scalable to support complex data sets and data governance
- Light weight and codeless with customizable end points (source, targets, data structure types) for various end-use purposes
- Framework templates and repeatable processes
- Accelerate data ingestion tasks from months to weeks for faster analytics/decision making
- Minimize/redirect engineering support by enabling business analysts to perform ingestion tasks
- Open-source Python solution utilizing DoD compatible libraries
- Graphical user interface for use by subject matter experts to define rules