senior cloud data engineer.

job summary: Own, drive, and coordinate testing on a Scrum team (Agile) or across testing phases (Waterfall) including System, Functional, Acceptance and End-to-End testing. Responsible for manual and automated testing of multiple decision support applications and systems.Responsible for designing automated tests to validate custom software products and services.Must have python experienceWorking with data processing and ETL development   location: San Francisco, California job type: Contract salary: $62 - 65 per hour work hours: 8am to 5pm education: Bachelors   responsibilities: Responsibilities: Test optimal data pipeline components primarily in Python and other languages as needed Use AWS services for testing such as including Redshift, Glue, Kinesis, Kafka (MSK) and EMR/Spark, where appropriateBuild abstractions that hide the complexity of the underlying big data processing and execution stack and that allow data consumers/partners to focus on their strengths: product, data modeling, data analysis, search, information retrieval, and machine learning.Leverage internal services and tools to help test in-house partners implement, deploy and analyze datasets with a high level of autonomy and limited friction.Support Cloud Engineers with testing efforts who deploy and manage data pipelines (monitoring, retry, failure detection) Qualifications: MUST HAVE Minimum of 5 years of overall experience in ETL/ELT testing with at least 3 years of dedicated experience testing using Python in Data Warehouse environments, preferably in AWSAdvanced knowledge of Python with a focus on data using packages like pandas, numpy, boto, s3fs, flask, fastapi, pyarrow, pyspark etc.strong SQL programming skills and experience working with relational databases, as well as working familiarity with a variety of databases (Postgres/Redshift/MySQL).Familiarity with AWS data services including Redshift, Glue, Kinesis, Kafka (MSK) and EMR/Spark, etc.Experience testing 'big data' data pipelines, architectures and data sets.strong analytic skills related to working with unstructured data. A successful history of manipulating, processing and extracting value from large, disconnected datasets.   qualifications: Experience level: ExperiencedMinimum 7 years of experienceEducation: Bachelors  skills: Automated QA TestingManual QA TestingPython (2 years of experience is required) Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status. Qualified applicants in San Francisco with criminal histories will be considered for employment in accordance with the San Francisco Fair Chance Ordinance. We will consider for employment all qualified Applicants, including those with criminal histories, in a manner consistent with the requirements of applicable state and local laws, including the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance. For certain assignments, Covid-19 vaccination and/or testing may be required by Randstad's client or applicable federal mandate, subject to approved medical or religious accommodations. Carefully review the job posting for details on vaccine/testing requirements or ask your Randstad representative for more information.

job summary: Our group enables the connection between supplier brands and retail shoppers at unprecedented scale. We are a highly motivated group of Big Data Geeks, Data Scientists and Applications Engineers, working in small agile groups to solve sophisticated and high-impact problems. We are building smart data systems that process terabytes of data per day, and generate insights using statistics, machine learning and probabilistic algorithms from online and offline user activity. We use Scala, Spark, GCP for data processing systems, and Scala framework for services.   location: San Bruno, California job type: Contract salary: $70 - 80 per hour work hours: 8am to 4pm education: No Degree Required   responsibilities: Our group enables the connection between supplier brands and retail shoppers at unprecedented scale. Senior Software Engineer - Backend/Data We are a highly motivated group of Big Data Geeks, Data Scientists and Applications Engineers, working in small agile groups to solve sophisticated and high-impact problems. We are building smart data systems that process terabytes of data per day, and generate insights using statistics, machine learning and probabilistic algorithms from online and offline user activity. We use Scala, Spark, GCP for data processing systems, and Scala framework for services.   qualifications: Experience level: ExperiencedMinimum 8 years of experienceEducation: No Degree Required  skills: HadoopJava (8 years of experience is required) Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status. Qualified applicants in San Francisco with criminal histories will be considered for employment in accordance with the San Francisco Fair Chance Ordinance. We will consider for employment all qualified Applicants, including those with criminal histories, in a manner consistent with the requirements of applicable state and local laws, including the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance. For certain assignments, Covid-19 vaccination and/or testing may be required by Randstad's client or applicable federal mandate, subject to approved medical or religious accommodations. Carefully review the job posting for details on vaccine/testing requirements or ask your Randstad representative for more information.

job summary: Responsible for building, operationalizing and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. You should be a skillful data pipeline builder and data wrangler who has experience optimizing building data systems from the ground up. You are expected to support our software developers, database architects, data analysts and data scientists on data initiatives and ensure optimal data delivery architecture is consistent throughout ongoing projects.   location: San Francisco, California job type: Contract salary: $80 - 90 per hour work hours: 8am to 5pm education: Bachelors   responsibilities: You must have excellent communication skills, have a strong sense of ownership, be self-directed and comfortable supporting the data needs of multiple cross-functional teams. This is a hands-on role and requires prior programming experience working with data on cloud. Responsibilities Data Pipeline Building, Optimization and Operationalization ? Create and maintain optimal data pipeline architecture, ? Assemble large, complex data sets that meet functional / non-functional business requirements. ? Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, ? Operationalizing, Improving the quality and reliability of data pipelines (monitoring, retry, failure detection) Data Platform and Self-serving Data Infrastructure ? Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using various data tools and cloud services. ? Design and build abstractions that hide the complexity of the underlying big data processing and execution stack and that allow data consumers/partners to focus on their strengths: product, data modeling, data analysis, search, information retrieval, and machine learning. ? Build processes and infrastructure to manage lifecycle of datasets: data structures, metadata, dependency and workload management. ? Building internal services and tools to help in-house partners implement, deploy and analyze datasets with a high level of autonomy and limited friction. Qualifications ? MS or BS in CS, Engineering, Math, Statistics, or a related field or equivalent practical experience in data engineering. ? 3-5 years of experience working in data engineering using Python on Cloud platforms like AWS. Expert in programming in Python. With experience of packages like pandas, numpy, boto, s3fs, flask, fastapi, pyarrorw, pyspark etc. ? Background in software development or engineering ? Working experience with open-source platforms for data engineering pipelines in AWS mainly with Apache Airflow. ? Working experience with Data Processing methods and techniques which includes Batch processing, online processing etc.. ? Working experience with distributed computing framework and set of libraries for real-time, large-scale data processing using PySpark. ? Working experience with Agile practices includes requirements discovery and solutions improvement through the collaborative effort of self-organizing and cross-functional teams. ? Working experience with CI/CD using AWS infrastructure and services. ? Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. ? strong sense of ownership. ? Collaborative personality with experience supporting and working with cross-functional teams in a dynamic environment.   qualifications: Experience level: ExperiencedMinimum 8 years of experienceEducation: Bachelors  skills: PythonCloudAWSairflow Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status. Qualified applicants in San Francisco with criminal histories will be considered for employment in accordance with the San Francisco Fair Chance Ordinance. We will consider for employment all qualified Applicants, including those with criminal histories, in a manner consistent with the requirements of applicable state and local laws, including the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance. For certain assignments, Covid-19 vaccination and/or testing may be required by Randstad's client or applicable federal mandate, subject to approved medical or religious accommodations. Carefully review the job posting for details on vaccine/testing requirements or ask your Randstad representative for more information.