job summary: Summary:
- We are seeking a highly technical Hadoop data Engineer with hands-on experience.
- Qualified candidates must be well-versed in data warehousing and 'Big Data' distributed processing and storage technologies (spark & HDFS) as well as Data Lake design pattern. In addition, the successful candidate should have knowledge and experience with data management techniques such as Meta data management , Data Quality (DQ) management, Data Governance, Data Integration/Ingestion, Data Architecture, data profiling etc.
location: Charlotte, North Carolina
job type: Contract
salary: $55 - 65 per hour
work hours: 8am to 5pm
- This application is in the middle of a modernization, and we are looking for a well-rounded data engineer who "gets it", who thrives in an agile environment, and who doesn't recognize role boundaries, always swarming to get things done.
- Ability to develop and scripting in Python programming as well as SQL in Linux environments.
- Able to integrate data with Sqoop and ingest files with multi record types with various data formats Parquet, Avro, and Json.
- Create and maintain optimal data pipeline architecture in Cloudera CDH or similar platform with application development skills in hive, Sqoop, Pyspark
- Participate in regular status meetings to track progress, resolve issues, mitigate risks and escalate concerns in a timely manner in Scrum meetings.
- Able to understand with unit testing, release procedures, coding design and documentation protocol as well as change management procedures
- Able to using versioning tools such as GIT/Bit bucket
- Ability to setup jobs using autosys for automation
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.