Senior Software Engineer - Hadoop

Senior Software Engineer - Hadoop

What you'll do at

• Troubleshoot and triage issues related to data pipeline failures or slowness, built using Map Reduce, hive or Spark to ensure SLA adherence. These may be batch as well as streaming processes.
• Resolve issues related to several data platform components and open source database solutions like Presto/Druid or cloud native components
• Resolve issues related to commercial tools and products, which are an integral part of platform
• Be a gate keeper to ensure sanity of production systems
• Builds tools to continuously monitor and alert platform components & data pipelines • Continually improve CI/CD tools, processes and procedures
• Participate in ongoing design, implementation, and maintenance of systems and tools across our data platform
• Write and maintain infrastructure documentation
• Own production incidents/issues and provide level 2 response to infrastructure incidents and alerts

Minimum Qualifications

Bachelor s degree in Computer Science and 3 years experience in software engineering or related field OR 5 years experience in software engineering or related field, with experience in Hadoop, Driud, Presto and DevOps

Preferred Qualifications

Our Ideal Candidate
You have a deep interest and passion for production systems, one who loves to debug issues, learn new technologies and write quick utilities to manage and maintain platform. You have strong problem solving, analytical skills, decision-making and communication skills. You are self-driven and motivated with the desire to work in a fast-paced, results-driven agile environment with varied responsibilities. Also to ensure highest degree of support for this critical platform, you must be willing to be part of a team that supports 24x7 on rotation basis.
Your Qualifications
• Bachelor's/Master's Degree with 6-10 yrs. of experience in Computer Science or related streams
• 3+ years of experience of managing data platform, ETL pipelines
• Strong understanding of internals of at least 1 distributed processing framework like Map reduce, Hive, or Spark
• Strong understanding of continuous integration, deployment and operations concepts
• Experience with code repositories (Git) and continuous integration tools (Jenkins, Maven) and software provisioning and deployment automation tools (Ansible)
• Excellent knowledge of Linux system
• Experience with configuration management
• Experience with a UNIX shell scripting languages such as Bash, Ruby, Perl or Python
• System administration exposure
• Experience in building scalable/highly available distributed systems in production a plus
• Ability to work with distributed teams in a collaborative and productive manner
• A self-motivated learner and builder with strong customer focus and obsession with quality

About Walmart Labs

Imagine working in an environment where one experiment can catapult an entire industry toward a smarter future. That’s what we do at Walmart Labs. We’re a team of 5,000+ software engineers, data scientists, designers and product managers within Walmart, the world’s largest retailer, delivering innovations that improve how our customers shop and our enterprise operates.

Hello, Silicon Valley

You don’t have to choose between your career and your lifestyle. In Silicon Valley, you can have both.

Discover Silicon Valley
Silicon Valley
View of Silicon Valley from the hills after a passing storm

All the benefits you need for you and your family

  • 100% coverage for in network preventative care
  • Retirement Plan
  • Vision Plans
  • Dental Plans
  • Exclusive Discounts

Recently viewed jobs