Sr. Data Engineer

  • location: New York, NY
  • type: Permanent
  • salary: $170,000 - $200,000 per year
easy apply

job description

Sr. Data Engineer

job summary:
Reporting to the SVP of Predictive Products, the successful candidate has an opportunity to join an innovative data science and engineering team that's operationalizing data across the organization. We see model-driven processes and workflows as a means to reduce uncertainty, reveal opportunity, and generate operational leverage that collectively create a new dimension of competitive advantage and lay the foundation for new revenue streams.

We are looking for a savvy Data Engineer to help us build a single source of truth for all data that's fully connected and version controlled. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The candidate will collaborate with our software developers, database architects, and data scientists on projects ranging from ad hoc research to deploying and monitoring production machine learning models.

The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The right candidate will be excited by the prospect of re-designing our company's data architecture to support our next generation of products and data initiatives.

Advanced working SQL knowledge and experience working with relational databases, query authoring as well as working familiarity with a broad range of databases.

  • Experience building and optimizing 'big data' data pipelines, architectures and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytical skills related to working with unstructured or "dirty" datasets.
  • Experience building processes supporting data transformation, data structures, metadata dependency, and workload management.
  • A successful history of manipulating, processing, and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
  • Strong project management and organizational skills.
  • Demonstrated ability to work closely with teammates in a highly collaborative environment and simultaneously be a self-starter with strong individual contributions.
 
location: New York, New York
job type: Permanent
salary: $170,000 - 200,000 per year
work hours: 9am to 5pm
education: Bachelors
 
responsibilities:
  • Create and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets that meet a broad range of business requirements.
  • Identify, design, and implement internal process improvements, including automating manual processes, optimizing data flows, and re-designing infrastructure for greater scalability.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS-based 'big data' technologies.
  • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
  • Work with stakeholders, including the Executive, Product, Data, and Finance teams, to understand requirements and business impact and support related data infrastructures.
 
qualifications:
Advanced working SQL knowledge and experience working with relational databases, query authoring as well as working familiarity with a broad range of databases.

  • Experience building and optimizing 'big data' data pipelines, architectures and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytical skills related to working with unstructured or "dirty" datasets.
  • Experience building processes supporting data transformation, data structures, metadata dependency, and workload management.
  • A successful history of manipulating, processing, and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
  • Strong project management and organizational skills.
  • Demonstrated ability to work closely with teammates in a highly collaborative environment and simultaneously be a self-starter with strong individual contributions.
 
skills:
  • Experience using the following software/tools:

    • Big data tools: Hadoop, Spark, Kafka, etc.
    • Relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • AWS-based cloud services: EC2, EMR, RDS, Snowflake, Redshift, etc.
    • Stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

easy apply

get jobs in your inbox.

sign up
{{returnMsg}}

related jobs

    Sr. Java Developer

  • location: Jersey City, NJ
  • job type: Contract
  • salary: $58 - $68 per hour
  • date posted: 3/19/2020


    Sr. Splunk Advisor

  • location: Florham Park, NJ
  • job type: Contract
  • salary: $55 - $57 per hour
  • date posted: 3/23/2020