Principal Software Engineer - Telemetry

Principal Software Engineer - Telemetry

What you'll do at

What you'll do




As a systems telemetry principal engineer, you will be a key researcher and technical lead expert in the architecture and development of cloud native telemetry designs, managed services, and real-time telemetry software systems. You will use your depth of engineering and experience to create visionary software architectures and telemetry systems to achieve a telemetry software product portfolio. Additionally, you will design, develop and implement large-scale distributed systems that process large volumes of data focusing on scalability, latency, and fault-tolerant in every system built. You must be able to effectively communicate and build collaboration at all areas and levels of the business and engineering. An ideal candidate will be adapt at architecting large scale distributed systems and proficient in coding Java.




To achieve software solutions and designs, you will utilize multiple telemetry technologies such as: data models, metric libraries, data logging and correlation, rule based alerting engines, real-time data streaming pipelines, TSDBs, and application performance management (APM). While working in a cloud infrastructure ecosystem consisting of VMs, Kubernetes, and containers, you will create metric software designs and solutions enabling real-time monitoring and alerting of system and application metrics.




You will also utilize TSDBs and data fusion of multiple data types and heterogenous data streams coupled with Artificial intelligence (AI) and Learned Behaviors to create forward projections of system and application expected behaviors. This role will involve collaboration with enterprise architects, product managers, data scientist, engineers and business managers to bring telemetry R&D projects into production. To achieve this effect, you will use a combination of open source and COTS technologies to solve real-time telemetry problems at an enterprise wide scale. In parallel, you will lead in the design of new systems and the redesign of existing systems to meet business requirements, changing needs, and integration of state-of-the-art technology.




Work locations: successful hires can be located in Carlsbad, CA, Sunnyvale, CA, San Bruno, CA, Bentonville, AR or may potentially work remote on a part-time basis at management's discretion.




#LI-SN1

Minimum Qualifications


  • Strong architecture, programming, and engineering skills. Should have an analytical design approach and demonstrated experience in software engineering and programming
  • BS/MS in Computer Science, Engineering, or equivalent, with 12 or more years in software engineering, design and architecture
  • Full stack cloud software development experience
  • Experience in cloud technologies and cloud native designs
  • Experience with cloud infrastructures and technologies, such as OpenStack, Azure, GCP or AWS
  • Demonstrated background in developing software systems and a solid foundation in advanced SQL
  • Large scale distributed systems experience including scalability and fault tolerance
  • Deep understanding and experience with Kafka, Cassandra, and TSDBs
  • Experience with one or more TSDB (InfluxDB, Kairos, Cortex, Thanos, Prometheus)
  • Experience in extract, transform, load (ETL) processes
  • Experience with real-time telemetry pipelines and publish/subscribe models
  • Hands on experience with NoSQL (Cassandra or equivalent)
  • Experience with Storm, Spark or similar real-time streaming software
  • Experience in data warehousing, processing and data analytics
  • Experience in the design and creation of systems that orchestrate and execute complex workflows in a data driven ecosystem.
  • Experience in Unix/Linux shell scripting or similar programming/scripting knowledge
  • Experience with real-time time monitoring and alerting: metric agents, real-time dashboards, alerting rules
  • Experience with API development, integration, and utilization
  • Excellent written and verbal communication skills for diverse audiences based on engineering subject matter
  • Ability to document requirements, architectural designs, and analysis findings in both business and technical terminology
  • Software development in an Agile iterative CI/CD development environment
  • Promote and support company policies, procedures, mission, values, and standards of ethics and integrity

Preferred Qualifications


  • Fluency in Java, JavaScript, advanced shell scripting, Configuration management -Ansible, chef, puppet
  • Experience with Application Performance Monitoring (APM)
  • Experience with deployment of Kubernetes, containers, service meshes, and micro services
  • Experience of micro services architectures, Istio, and micrometer
  • Experience with data fusion and data models.
  • Experience with Go development
  • Experience in creating and maintaining managed metric services
  • UI development - JavaScript, HTML, CSS and experience with frameworks like React and AngularJS
  • Experience with observability tools and system architectures
  • Involvement and contribution with open source software communities

About Walmart Labs

Imagine working in an environment where one experiment can catapult an entire industry toward a smarter future. That’s what we do at Walmart Labs. We’re a team of 5,000+ software engineers, data scientists, designers and product managers within Walmart, the world’s largest retailer, delivering innovations that improve how our customers shop and our enterprise operates.

All the benefits you need for you and your family

  • 100% coverage for in network preventative care
  • Retirement Plan
  • Vision Plans
  • Dental Plans
  • Exclusive Discounts

Recently viewed jobs