Lead Data Engineer (SRE)

Lead Data Engineer (DevOps)

Sydney, Australia

$220,000 Including Super + Bonus

Overview

As a Lead Data Engineer reporting to our Engineering Manager, you will be responsible for building, implementing, and enhancing enterprise-scale data platforms. Working in a DevOps environment, you will have end-to-end accountability for designing, developing, deploying, and supporting your data assets. Additionally, you will create templates, implementation methods, standards, and frameworks.

You will engage with various stakeholders and become an expert in your domain while supporting the broader data engineering and Site Reliability Engineering (SRE) community. Your role will include continuously evaluating and recommending improvements to infrastructure and processes, leading incident response efforts, and participating in post-mortem analysis.

Key Requirements

• Lead significant data engineering projects in an enterprise environment.
• Mentor and guide a team on best practices for data engineering and site reliability (SRE).
• Utilize Big Data and Hadoop technologies with a focus on Spark, Hive, Presto, and big data storage formats such as Parquet, ORC, Avro.
• Manage CI/CD tools like Bamboo/Ansible.
• Operate within a cloud-based environment, preferably AWS or GCP.
• Demonstrate strong skills in Python, SQL, and shell scripting.
• Work with ETL tools such as Talend, Informatica, Datastage, and understand programming languages like Scala.
• Apply broad knowledge of technical solutions, design patterns, and coding for complex applications in a distributed computing environment.
  
  

If you feel you possess the relevant skills to the above, please respond with most updated CV!