Senior Data Engineer - Contract - Auckland (Hybrid)
A leading New Zealand enterprise, focused on technology and dedicated to becoming a top provider of trusted content, is seeking an experienced Senior Data Engineer. This contract role offers the opportunity to design, develop, and optimize a modern data infrastructure, building scalable, cutting-edge solutions for a digital-first transformation. The position requires a specialist with deep expertise in Databricks, Azure, and real-time data processing.
The Role:
Reporting to the Data Engineering Lead, you will be instrumental in architecting robust, highly performant data solutions that support core business objectives, helping us build a digital-first future.
Key responsibilities include:
... - Design & Implementation: Architecting and implementing scalable data pipelines for both batch and real-time processing, leveraging Databricks and Azure Data Factory.
- Development & Optimization: Crafting and optimizing data workflows using Python, PySpark, and SQL on the Databricks platform.
- Real-Time Solutions: Developing Spark Structured Streaming solutions for real-time data processing and analytics.
- Quality & Performance: Designing data models, ensuring data quality and governance, and monitoring/tuning Spark jobs for optimal performance.
- Mentorship & Standards: Providing guidance to junior engineers, participating in code reviews, and contributing to technical standards and best practices.
Technical Foundation:
- Experience: A minimum of 5+ years in data engineering or a related field.
- Databricks Specialist: At least 1+ year of strong, hands-on Databricks experience is essential.
- Coding Expertise: Expert-level proficiency in Python and SQL.
- Spark Proficiency: Strong hands-on experience with Apache Spark and PySpark.
- Azure Data: Experience with Azure Data Factory for orchestration and familiarity with Azure cloud services (Data Lake, Synapse, Event Hubs, etc.).
Architecture & Optimization:
- Solid understanding of Databricks concepts, including CDC, Delta Lake flow jobs, and optimizing storage patterns using Azure Data Lake.
- Understanding of data modeling, data warehousing, and data lake concepts.
- Experience with version control systems (Git) and CI/CD practices.
Great to Have (Bonus Skills):
- Experience with streaming technologies (Apache Kafka, Azure Event Hubs).
- Knowledge of machine learning workflows and MLOps practices.
- Experience with On-Prem SQL servers and related technologies.
- Azure Data Engineer Associate certification or similar cloud certifications.
Contract Details
- Title: Senior Data Engineer
- Term: 6 months Contract with a chance of extension
- Location: Auckland – requiring at least 3 days work from the office.
Apply Now
If you are a talented Senior Data Engineer seeking an immediate and challenging contract, apply now or contact Amaan for more details: call on 0220607986 or Email at amaan.kazmi@randstaddigital.co.nz.
At Randstad, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.
