Data Engineer

  • location: San Francisco, CA
  • type: Contract
  • salary: $130 - $140 per hour

job description

Data Engineer

job summary:
Responsibilities:

  • Owner of the core company data pipeline, responsible for scaling up data processing flow to meet the rapid data growth at Client
  • Evolve data model and data schema based on business and engineering needs
  • Implement systems tracking data quality and consistency
  • Develop tools supporting self-service data pipeline management (ETL)
  • SQL and MapReduce job tuning to improve data processing performance
Experience:

  • 5+ years of relevant professional experience
  • Experience with Hadoop (or similar) Ecosystem (MapReduce, Yarn, HDFS, Hive, Spark, Presto, Pig, HBase, Parquet)
  • Proficient in at least one of the SQL languages (MySQL, PostgreSQL, SqlServer, Oracle)
  • Good understanding of SQL Engine and able to conduct advanced performance tuning
  • skills in scripting language (Python, Ruby, Bash)
  • 1+ years of experience with workflow management tools (Airflow, Oozie, Azkaban, UC4)
  • Comfortable working directly with data analytics to bridge Client's business goals with data engineering
 
location: San Francisco, California
job type: Contract
salary: $130 - 140 per hour
work hours: 8am to 5pm
education: Bachelors
 
responsibilities:
  • Owner of the core company data pipeline, responsible for scaling up data processing flow to meet the rapid data growth at Client
  • Evolve data model and data schema based on business and engineering needs
  • Implement systems tracking data quality and consistency
  • Develop tools supporting self-service data pipeline management (ETL)
  • SQL and MapReduce job tuning to improve data processing performance
 
qualifications:
  • Experience level: Experienced
  • Minimum 5 years of experience
  • Education: Bachelors
 
skills:
  • Data Engineer

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

Qualified applicants in San Francisco with criminal histories will be considered for employment in accordance with the San Francisco Fair Chance Ordinance.

We will consider for employment all qualified Applicants, including those with criminal histories, in a manner consistent with the requirements of applicable state and local laws, including the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance.

get jobs in your inbox.

sign up
{{returnMsg}}

related jobs

    Big Data Engineer

  • location: San Francisco, CA (remote)
  • job type: Permanent
  • salary: $150,000 - $180,000 per year
  • date posted: 7/21/2021

    Data Engineer

  • location: San Francisco, CA
  • job type: Contract
  • salary: $20 - $31 per hour
  • date posted: 7/20/2021

    Sr Data Engineer

  • location: Novato, CA
  • job type: Permanent
  • salary: $125,000 - $190,000 per year
  • date posted: 7/13/2021