Specialist, Data Engineering Specialist, Data Engineering …

in Sydney, New South Wales, Australia
Permanent, Full time
Be the first to apply
in Sydney, New South Wales, Australia
Permanent, Full time
Be the first to apply
Specialist, Data Engineering
Senior Data Engineer

#Closer #Braver #Faster

The Role
The Senior Data Engineer is a key member of project developer teams. The successful candidate will understand modern software development practices and work in a project team to develop data replication, streaming, transformation, CI/CD and data extraction applications.

Additional responsibilities include developing prototypes and proof of concepts for the solutions and implementing complex data delivery projects with a focus on loading, managing, analysing and maintaining large sets of data using multiple platforms.
The Senior Data Engineer role is a technical role that requires substantial expertise in a broad range of software development, data engineering and ETL/Replication skills. These professionals have knowledge of data loading, end user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solutions. They have a solid understanding of data design principles, the software development life cycle and exposure to a variety of data platforms such as DB2, Oracle, SQL Server, Postgres, Informix, AS400, Mainframe as well as AWS and/or Google Cloud environments

Key Responsibilities
  • Design and Development of scalable, secure and high performance data pipelines
  • Design and development of public, private and hybrid cloud data environments
  • Real time acquisition using CDC and Confluent KAFKA
  • Document data stores such as MongoDB
  • Relational data stores such as Postgres
  • CI/CD pattern design and implementation in Linux based environments
  • Exposure to data lake architecture principles

Skills & Experience
  • Experience with Python, Java and / or Scala
  • Experience with MPP Databases like Greenplum, MongoDB, RDS
  • Experience with Agile Development and supporting technologies
  • Exposure to relational database systems (DB2, Oracle, Postgres, SQL Server)
  • Exposure to Open Source Technologies
  • Exposure to Scheduling Tools like Active Batch, Airflow etc
  • Exposure to high performance cloud computing environments

About Us
At IAG, we believe that everyone has a unique point of view to share, shaped by their life experiences, cultures & passions. We celebrate and commit to:

#Proud to be me - we value difference, not sameness
#Together - harnessing our collective wisdom enables us to be our best for our customers & each other
#No boxes - it's not about labels, boxes or categories. It's about building a diverse and inclusive mindset into everything we do

IAG is the largest general insurance group in Australia and New Zealand. IAG owns some of the region's most trusted brands, including NRMA Insurance, CGU, SGIO, SGIC and WFI.

Let's Talk
Start your career journey with us and click 'Apply'!
In addition to a diverse and inclusive culture, some of our benefits include 13% superannuation, 50% insurance discounts, flexible work and leave options, generous parental leave and return to work program, recognition and reward program, and various corporate partner discounts.

We encourage Aboriginal and Torres Strait Islanders to apply for this position.
Please note, applications will close Sunday 17th November.