Staff DevOps Engineer - Hadoop
- Location RESTON, VA
- Career Area Information Technology
- Job Function Information Technology
- Employment Type Full Time
- Position Type Salary
- Requisition 1298165BR
What you'll do at
The Global Data team @ Walmart Labs provides Machine Learning Platform, Customer platform, Data Foundation Infrastructure and Visualization Portal products that form part of core platforms and services that drive Walmart business. The group also develops analytical products for several verticals like supply chain, pricing, customer, HR etc.
As part of the Platform Dev-Ops Team @Walmart Labs, you’ll have the opportunity to make a difference by being a part of a critical function that would own, manage Data pipelines & various platform components that are critical to deliver data and analytical products to our consumers.
• Lead troubleshooting and triaging issues related to data pipeline failures or slowness, built using Map Reduce, hive or Spark to ensure SLA adherence. These may be batch as well as streaming processes.
• Resolve issues related to several data platform components and open source database solutions like Presto/Druid or cloud native components
• Resolve issues related to commercial tools and products, which are an integral part of platform
• Be a gate keeper to ensure sanity of production systems
• Builds tools to continuously monitor and alert platform components & data pipelines
• Continually improve CI/CD tools, processes and procedures
• Participate in ongoing design, implementation, and maintenance of systems and tools across our data platform
• Write and maintain infrastructure documentation
• Own production incidents/issues and provide level 2 response to infrastructure incidents and alerts
• Work with third-party vendors to resolve infrastructure issues
• Identify right open source tools to improve infrastructure by performing research, POC/Pilot and/or interacting with various open source forums
• Deploy and monitor products on Cloud platforms
• Mentor junior team members
• Promote and support company policies, procedures, mission, values, and standards of ethics and integrity
- Bachelor s degree in Computer Science and 4 years experience in software engineering or related field OR 6 years experience in software engineering or related field.
You have a deep interest and passion for production systems, one who loves to debug issues, learn new technologies and write quick utilities to manage and maintain platform. You should have strong desire to lead from front. You have strong problem solving, analytical skills, decision-making and communication skills. You are self-driven and motivated with the desire to work in a fast-paced, results-driven agile environment with varied responsibilities. Also to ensure highest degree of support for this critical platform, you must be willing to be part of a team that supports 24x7 on rotation basis.
• Bachelor's/Master's Degree with 10+ yrs. of experience in Computer Science or related streams
• 6+ years of experience of managing data platform, ETL pipelines
• Strong understanding of internals of at least 1 distributed processing framework like Map reduce, Hive, or Spark
• Strong understanding of MPP Databases, previous experience of administrating MPP DB will be added advantage
• Strong understanding of continuous integration, deployment and operations concepts
• Experience with code repositories (Git) and continuous integration tools (Jenkins, Maven) and software provisioning and deployment automation tools (Ansible)
• Excellent knowledge of Linux system
• Experience with configuration management
• Experience with a UNIX shell scripting languages such as Bash, Ruby, Perl or Python
• System administration exposure
• Experience in building scalable/highly available distributed systems in production a plus
• Ability to work with distributed teams in a collaborative and productive manner
• A self-motivated learner and builder with strong customer focus and obsession with quality
About Walmart Labs
Hello, D.C. Metro
National landmarks, museums, renowned restaurants—the D.C. Metro is a hub of activity and culture. It’s also a prime location for the future of tech.Discover D.C. Metro
The Lincoln Memorial
"In this temple, as in the hearts of the people for whom he saved the Union, the memory of Abraham Lincoln is enshrined forever."
The National Mall
A symbol of freedom and equality, millions visit the Mall annually to relax, enjoy its beauty, visit its museums and find voice.
Escape for a hike in the great outdoors at Great Falls, where the Potomac River spills over steep, jagged rocks through the picturesque Mather Gorge.
Air and Space
Companion to the Smithsonian National Air and Space Museum on the Mall, the Udvar-Hazy Center displays thousands of aviation and space artifacts.
Wolf Trap National Park
This 117-acre park is a majestic natural setting for performances, dining and year-round recreational opportunities like hiking and sledding.
Fishing, boating, campgrounds, mini train, carousel, volleyball courts, golf course, disk golf, snack shop, picnic areas, playgrounds and mini-golf!
Reston Town Center
Shopping, dining, ice-skating and events in a beautiful setting in the heart of Reston.
The final resting place for more than 400,000 active duty service members, veterans and their families, honoring their service to our country.
City of Alexandria
Once among the 10 busiest ports in America, the city's historic character still remains; beautiful 18th- and 19th-century buildings line the Old Town.
All the benefits you need for you and your family
- 100% coverage for in network preventative care
- Retirement Plan
- Vision Plans
- Dental Plans
- Exclusive Discounts