Big Data Development Senior
Our Major Financial client has an exciting opportunity for a Development, Sr (Big Data) to contribute to the development and maintenance of Big Data Applications.They are seeking a motivated individual who can solution the business need, design and develop enterprise software applications based on Big Data platforms.
location: Mc Lean, Virginia
job type: Contract
salary: $77.99 - 78.13 per hour
work hours: 8am to 6pm
Work with initiative leads, business areas, technical leads, architects, subject matter experts, developers, technical support groups and test teams to deliver value to the business.
- Deliver Hadoop components as specified in the design, functional and non-functional requirements, within agreed upon cadence based on Scaled Agile practices.
- Design, prototype, develop, test, and document product features. Make recommendations for enhancements that result in a cost-effective product delivery and/or operational efficiency.
- Develop processes to extract, transform and load unstructured data
- Cleanse, manipulate and analyze large datasets (Structured and Unstructured data - XMLs, JSONs, PDFs) using Hadoop platform.
- Develop HIVE scripts to filter/map/aggregate data. Sqoop to transfer data to and from Hadoop.
- Spark to cleanse/enable and analyze large datasets
- Manage and implement data processes (Data Quality reports)
- Develop data profiling, deduping logic, matching logic for analysis
Perform analysis of vast data stores and uncover insights.
- Maintain security and data privacy.
- Correct testing defects and support all testing, including but not limited to: Development Integration Testing, System Testing, User Acceptance Testing, End-to-End Testing, and Performance Testing.
- Identify application bottlenecks and opportunities to optimize performance.
- Perform troubleshooting of production issues.
- Provide resolution to an extensive range of complicated problems, proactively and as issues surface.
- Perform code version control activities
- Perform Exploratory Data Analysis
- Perform unit testing and document test results.
Experience with MPP databases like Vertica/Redshift
- Has experience with loading data from disparate data sets and structured and non-structed data sources
- Has experience with Pre-processing using Hive and Pig.
- Has experience with High-speed querying.
- Has experience with Managing and deploying HBase.
- Good knowledge in back-end programming, specifically java, JS, Node.js and OOAD
- Expertise in writing high-performance, reliable and maintainable code.
- Ability to write MapReduce jobs.
- Good knowledge of database structures, theories, principles, and practices.
- Hands on experience in HiveQL.
- Experience with data loading tools like Spark, Sqoop.
- Knowledge of workflow/schedulers like Oozie.
- Analytical and problem-solving skills applied to Big Data domain
- Good aptitude in multi-threading and concurrency concepts.
- Experience with Unix shell scripting and administration.
- Experience with configuration management tools & Unix shell.
- Extensive knowledge and experience with Java EE patterns.
- Extensive programming experience in Java Scripting, Python, R, etc.
- Hands-on experience with Build and Deployment tools and languages - Maven/Jenkins and Shell scripting.
- Complete code reviews, complete documentation of issues identified and action items.
- Minimum of 2 to 4 years of experience working in Agile, Lean/Kanban, or Scaled Agile organization. Demonstrates ability to use Lean/Agile delivery practices to improve teams, quality, and reliability
- Understand Business needs and processes, identify solutions, present and guide project team/sponsors in identifying the best solution in a simplified meaningful way.
- Engage with Architecture. Partner with Enterprise Architecture to define technical solutions to complex business issues that align with target state architecture and conforms with corporate best practices.
- Quick learner of new technologies and tools.
- Possess 'Can do attitude' and be able to works under minimal supervision.
Bachelor's degree in Computer Science or Engineering or equivalent work experience
skills: - Excellent communication skills that will allow the candidate to successfully document processes as well as interact with business owners, translating technical details for a non-technical audience.
- Strong problem-solving skills.
- Innovative in providing solutions, likes to take on challenges with calculated risk.
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.