Job Description

Data Engineer

Location:  Charlotte, NC or Phoenix, AZ  (Hybrid)
Duration: 12+ Months Contract

Job Summary:
We are seeking an experienced Cloudera Data Platform (CDP) Engineer with 10+ years of experience to lead data migration, optimization, and security efforts in a Big Data environment. The ideal candidate will have strong expertise in CDP, Hadoop, Spark, and scripting languages such as Shell, Python, Ansible, and Terraform.

This role requires hands-on experience with data migration from Hortonworks/MapR to CDP Cloud, securing clusters, and optimizing Big Data solutions. The ability to work cross-functionally and support team members is essential.

Key Responsibilities:

• Lead data migration from Hortonworks (on-prem) or MapR to CDP (public/private) Cloud.
• Work extensively with Cloudera Data Platform (CDP), Hadoop, Spark, and HDFS.
• Provide expertise in Kubernetes & Docker services for Big Data deployment and management.
• Write efficient scripts using Shell, Python, Ansible, and Terraform for automation.
• Implement security measures for clusters, storage, and encryption to ensure data integrity.
• Guide and mentor team members in Big Data best practices, project execution, and optimized solutions.
• Utilize DevOps tools such as Jira, Git, SVN, Liquibase, and ServiceNow to streamline workflows.
• Manage workflow orchestration with Airflow and Autosys.
• Work with Postgres, MySQL, Teradata, and Oracle databases to support data-related operations.

Required Skills and Experience:

• 10+ years of overall experience in Big Data engineering and cloud migration.
• Strong knowledge of Cloudera Data Platform (CDP), Hadoop, Spark, HDFS, and Kubernetes.
• Experience with Scala and Pyspark frameworks for data processing.
• Proficiency in scripting (Shell, Python, Ansible, Terraform) and automation tools.
• Hands-on experience with securing data clusters and implementing encryption techniques.
• Familiarity with Microsoft Stack (MS-SQL, RDBMS concepts).
• Knowledge of Agile methodologies, Scrum, Jira, and DevOps workflows.
• Experience with Airflow, Autosys, and Visio for process automation and visualization.