Sr. Data Engineer

  • location: Houston, TX
  • type: Contract
easy apply

job description

Sr. Data Engineer

job summary:
We are currently seeking an experienced Data Engineer to join the Big Data and Advanced Analytics department. As part of the Data Engineering team, the Data Engineer will work closely with the Data Science team and Business functions solve real-world oil and gas midstream problems using machine learning, data science and artificial intelligence. This individual will provide technical and thought leadership to the team to build out a data engineering practice within the organization.

 
location: Houston, Texas
job type: Contract
work hours: 8am to 5pm
education: Bachelors
 
responsibilities:
Responsibilities:

  • Work independently on data projects for multiple business functions
  • Implement data flows connecting operational systems, BI systems, and the big data platform
  • Automate manual data flows for repeated use and scalability
  • Develop data-intensive applications with API's and streaming data pipelines
  • Prepare and transform data into a usable state for analytics
  • Document and maintain source-to-target mappings and data lineage
  • Install, configure, and administer Hadoop ecosystem tools and technologies
  • Operationalize machine learning models
  • Assists data analysts and data scientists with query optimization, performance tuning, and data processing
  • Identify opportunities for data improvements and presents recommendations to management
 
qualifications:
Qualification:

  • 5+ years of functional programming experience in Python and/or Scala
  • 5+ years of distributed computing experience with Spark and/or Map Reduce
  • 3+ years of experience developing SQL-based applications
  • 3+ years of experience developing ETL data pipelines
  • 3+ years of system administration on Linux systems
  • 2+ years of experience with multiple Big Data storage tools such as MapR-DB, HBase, MapR-FS, HDF, Parquet, Avro, and ORC
  • Experience in software development life-cycle including change management and testing
  • Experience creating data pipelines with Streamsets
 
skills:
  • Excellent communication Skills
  • Ability to listen to the business and analyze data

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

easy apply

get jobs in your inbox.

sign up
{{returnMsg}}

related jobs


    Sr. Application Developer

  • location: Houston, TX
  • job type: Permanent
  • salary: $135,000 - $140,000 per year
  • date posted: 10/8/2019

    Sr. System Engineer

  • location: Houston, TX
  • job type: Permanent
  • salary: $120,000 - $135,000 per year
  • date posted: 10/8/2019