Big Data Hadoop Administration
Customer Challenge
A marketing analytics company that houses and analyzes large amounts of data for Kroger needed a big data platform to scale and deliver insights. This involved large amounts of data, hundreds of jobs running daily, and meeting security and user access requirements.
Innovative Solution
Illumination Works assisted the infrastructure team on all taks where a Hadoop Administrator was required, including install, configure, optimize, load balance, disaster recover, performance tuning, authentication, authorization, data encryption, user support, knowledge transfer, automating metadata backup, and scaling from one cluster to three.
Benefits/Outcomes
- Built big data clusters to improve speed and productivity to meet demands
- Secured and balanced clusters
- Clusters able to handle multitenancy on a large scale including shared resources between users and services
Toolbox
- 100s of terabytes―a couple of petabytes of data across all clusters
- Oracle BDA (Cloudera), Spark, Impala, Cloudera Manager, Cloudera Key Trustee, KMS, Kerberos, LDAP, Cloudera Navigator (auditing/data lineage)
- Authentication across platforms, HDFS access control lists (ACL), Sentry policies for authorization and encryption of data at rest on clusters
