We are looking for a Data Engineer to work with our data science teams to collect, store, process, and analyze huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across teams in our group.
location: Louisville, Kentucky
job type: Contract
salary: $52 - 62 per hour
work hours: 8am to 4pm
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
- Implementing ETL process and data pipelines.
- Monitoring performance and advising any necessary infrastructure changes.
- Collaborating with Data Scientists to build Machine Learning systems and run experiments. Skills and Qualifications
- strong algorithm and data structures knowledge.
- Knowledge of various databases (RDBMS, noSQL, HDFS, Cassandra, Redis).
- Experience building cloud data lakes and data warehouses is highly desirable.
- Experience with security and authentication in cloud platforms (Azure preferred).
- Bachelor's Degree or above in a technical field (Computer Science preferred).
- 5+ years of experience in data engineering.
- Experience building stream-processing systems, using solutions such as Spark-Streaming or Flink.
- Familiar with Spark ecosystem (e.g. Databricks).
- Good knowledge of Big Data querying tools.
- Understanding of Data Catalog, Data Governance, Data Lineage.
- Knowledge of various ETL techniques and data pipelines.
- Knowledge of messaging systems, such as Kafka or RabbitMQ.
- Familiarity with Machine Learning, Deep Learning and Natural Language Processing. workflows and libraries (scikit-learn, Spark MLlib, TensorFlow/Keras/PyTorch).
- Running Machine Learning tests and experiments.
- Experience level: Experienced
- Minimum 5 years of experience
- Education: Bachelors (required)
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.