Manager / Assistant Manager (Data Engineering)

As Data Engineer, you will design, implement, and oversee the maintenance of data flow channels and data processing systems that support the collection, storage, batch/real-time processing, and analysis of information from structured and unstructured sources in a scalable, repeatable, and secure manner. You will work in a fast-paced and dynamic environment that requires the ability to manage multiple priorities and stakeholders simultaneously.

You will be responsible for creating end-to-end data pipelines for data ingestion into the MOH data lake, working with technologies including Databricks/IDMC, Spark, Python, and cloud data services. Key focus areas will include: • Data Pipeline Development: Design and implement data pipelines to extract, transform, and load (ETL) data from diverse sources into unified formats, supporting both historical and delta data ingestion with anonymisation capabilities to support use case needs. Identify the optimal data and information flow coordination across the organisation, ensuring scalability, accessibility, and alignment with defined business goals. • Stakeholder Collaboration: Work closely with cross-functional teams, vendors, and IT infrastructure teams to understand data structure needs and ensure seamless data flow and integration with Big Data system and Cloud Services. • Solution Design and Innovation: Provide thought determining data management techniques and solutions that will enable the achievement of defined business goals. Develop prototypes and proof of concepts for selected solutions. Explore emerging technologies, including Gen AI to minimise repetitive use and streamline pipeline development and management. • Technical Support: Assist on high-level data management or Big Data analytics solution designs or Cloud Data solutions. Research hardware and software requirements to support selected solutions. Maintain knowledge of existing and emerging data management principles and techniques. • Develop and build data ingestion pipelines to ingest data from multiple sources to support use case needs. • Production Support and Enhancement: Provide ongoing support and maintenance for data ingestion pipelines within Big Data system and Cloud Data Services environments. Execute recovery procedures for failed pipelines during business-as-usual (BAU)/BYOD and Day 2 operations. Monitor pipeline performance, troubleshoot issues, and implement corrective measures to ensure scalability, reliability, efficiency, and minimal service disruptions.

You should possess the following: Educational Qualification and Technical Requirements: • Degree/master’s in computer science, Information Technology, Computer Engineering or equivalent. • Proficient in usage of Office - Productivity Suite Software (e.g. Microsoft Excel, PowerPoint, etc). • Data engineering experience with proficient in languages such as Python, Spark and SQL. Core Competencies: • Thinking Clearly & Making Sound Judgements – Strong analytical and critical thinking abilities to support the crafting of sound policy recommendations, with the ability to grasp timelines, operating context and ground realities. • Learning & Putting Skills into Action - Possess the drive and curiosity to learn and apply new skills, knowledge and new ways of working. • Prioritising Competing Demands - Able to effectively prioritise and manage competing demands, whilst maintaining high professional standards. • Working Effectively with Stakeholders – Able to mobilise and rally partners and stakeholders towards collective objectives. • Communicating Effectively: Possess good verbal and written communication, analytical and conflict resolution skills with proven ability to translate complex, technical concepts into clear and concise communications to a variety of key stakeholders of different levels As part of the shortlisting process for this role, candidates may be required to complete a medical declaration and/or undergo further assessment.