Job Description

Job Title: Data Engineer – Hiring FAST!
Location: Charlotte, NC
Pay Rate: W2 Rates Only – NO C2C
Setting: Hybrid and Remote Opportunities
Duration: 12+ months.

***WE DO VISA TRANSFERS***

Required Qualifications:

• 5+ years of AWS experience
• AWS services - S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight
• Experience with Kafka/Messaging preferably Confluent Kafka
• Experience with EMR databases such as Glue Catalog, Lake Formation, Redshift, DynamoDB and Aurora
• Experience with AWS data warehousing tools such as Amazon Redshift and Amazon Athena
• Proven track record in the design and implementation of data warehouse solutions using AWS
• Skilled in data modeling and executing ETL processes tailored for data warehousing
• Competence in developing and refining data pipelines within AWS
• Proficient in handling both real-time and batch data processing tasks
• Extensive understanding of database management fundamentals
• Expertise in creating alerts and automated solutions for handling production problems
• Tools and Languages Python, Spark, PySpark and Pandas
• Infrastructure as Code technology Terraform/CloudFormation
• Experience with Secrets Management Platform like Vault and AWS Secrets manager
• Experience with Event Driven Architecture
• DevOps pipeline (CI/CD); Bitbucket; Concourse
• Experience with RDBMS platforms and Strong proficiency with SQL
• Experience with Rest APIs and API gateway
• Deep knowledge of IAM roles and Policies
• Experience using AWS monitoring services like CloudWatch, CloudTrail ad CloudWatch events
• Deep understanding of networking DNS, TCP/IP and VPN
• Experience with AWS workflow orchestration tool like Airflow or Step Functions.

Desired Qualifications:

• .

Responsibilities:

• Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery
• Lead the Design, Build, Test and Deployment of components
• Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead)
• Understand requirements / use case to outline technical scope and lead delivery of technical solution
• Confirm required developers and skillsets specific to product
• Provides leadership, direction, peer review and accountability to developers on the product (key responsibility)
• Works closely with the Product Owner to align on delivery goals and timing
• Assists Product Owner with prioritizing and managing team backlog
• Collaborates with Data and Solution architects on key technical decisions
• The architecture and design to deliver the requirements and functionality
• Skilled in developing data pipelines, focusing on long-term reliability and maintaining high data quality
• Designs data warehousing solutions with the end-user in mind, ensuring ease of use without compromising on performance
• Manage and resolve issues in production data warehouse environments on AWS.