Data Engineer
Job Description
Responsibilities
Create and maintain data architectures.
Build and maintain platforms required for storage, extraction, transformation, and loading of data from a wide variety of data sources.
Improve discoverability and distribution of analytical data to data analysts and data scientists.
Implement best practices in data security throughout the lifecycle of analytical data.
Job Requirement
Requirements
Background in Information Systems (IS), Computer Science or related field in a reputed institution.
Experience in designing and maintaining data platforms such as data lakes, data warehouses and the associated data pipelines.
Proficient in creating and maintaining complex data pipelines using tools such as Microsoft SQL Server Integration Services (SSIS).
Experience in Commercial Cloud such as Amazon web Services (AWS), Google cloud Platform (GCP) and Microsoft Azure.
Knowledge of data platforms such as Spark, Flink, Kafka, SQL and NoSQL databases, Airflow.
Knowledge of the latest developments in data engineering and best practices in data management.
Knowledge of cloud technologies in data platforms is a plus e.g., Snowflake, AWS Redshift, Google Big Query, Azure Synapse.
