job summary: Job Description:
Senior Data Engineer with extensive experience on the Hadoop platform\related tools, and with leading teams to deliver complex products
- Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications.
- Responsible for obtaining data from the System of Record and establishing data feed to provide analysis in an automated fashion.
- Develop Hadoop applications to analyze massive data collections
- Develop processing framework to detect conditions
- Develop techniques supporting trending and analytic decision making processes
- Apply Hadoop technologies for responsive front-end experience
- Develop within security guidelines established
- 3+ years of Hadoop experience (experience working on the Data and Analytics solutions team)
- Hive Programing
- Sqoop (utilizing various data sources)
- File Formats (e.g. ORC; Parquet; Avro; json)
- Workflow scheduling tools such as CA ESP Automation
- MapReduce and Yarn
- AWS and Azure experience
- Collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.
- Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.
- Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.
- Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools. Note :
If you have had any responsibility for or participation in judicial, administrative, regulatory or other oversight proceedings involving Client or any of its affiliates, please identify the agency or department, the dates of your employment, and the nature of your responsibilities, including a list of any proceedings involving Client or any of its affiliates. NOTE: Only answer this question if you have worked with the government or other agency involved with Client oversight.
location: Charlotte, North Carolina
job type: Contract
work hours: 9am to 5pm
- Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets
- Ability to comprehend customer requests and provide the correct solution
- Understanding of distributed computing principles
- Strong analytical mind to help take on complicated problems
- Desire to resolve issues and dive into potential issues
- Familiarity with the Agile Methodology
- Experience with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases, including SQL Server and Cassandra.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Writing Programs and solve the problems
- Write SQL queries for adhoc analysis
- Data Processing for generating the patterns
- Data Cleansing for validation and verification
- Data Acquisition to collect the data from various sources
- Define the DataFlow
- Modularize the project components
- Architect the complete project - provide solution architecture
qualifications: Additional Info:
- Workers at this level manage work in application areas of specialization, with minimal supervision.
- They are fully competent in use of concepts and procedures.
- They identify problems, develop solutions and take actions to resolve.
- Designs, writes, develops and implements innovative and advanced mobile applications.
- Maintains and enhances existing applications.
- Troubleshoots and debugs applications, including user and system interface functionality. Collaborates with cross-functional teams and clients to define, design and deliver enhancements, new applications, and technical solutions that meet the needs of the company, its clients, and the end users.
- Configures a design that satisfies business requirements and desired specifications.
- Researches, evaluates and implements new technology to optimize application efficiency and usability. Maintains records to document system and programming development and revisions.
- Designs modifications based on defined requirements and coordinates development efforts on maintained applications. Programs and directs other programmers in required programming languages, database SQL, report writing, ETL, Assists with testing and migrating code through test regions.
- Ensures adherence to appropriate design, coding and source control standards.
- Leads design and code walk-thru as a presenter or reviewer.
- Processes Change Requests and Quality Assurance (QA) documentation for project implementations and support changes.
Work experience in addition to degree:
- Bachelor degree in Computer Science or a related discipline. In lieu of a degree, 8 years of related business experience.
Job-specific knowledge, skills or abilities:
- 5-10 years as an IT Application Developer or equivalent.
- Four years coding/programming, trouble-shooting, problem-solving, logical design, SQL coding, Business Requirements analysis, vendor management, root cause analysis.
skills: Preferred Qualifications:
- Expert coding skills in 1 or more standard languages.
- Demonstrated success in complex problem solving and an understanding of quality delivery.
- Demonstrated ability to work with others having differing priorities/views/goals than your own; to meet your goals.
- Understand and lead others in use of System Development Life Cycle (SDLC).
- Effective communication skills. Experience with schema/database design, Enterprise Standards and Process Adherence.
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.