Intermediate Data Engineer

July 26, 2024

Cape Town, Western Cape, South Africa, South Africa

July 26, 2024

Job description

Minimum Requirements:

Matric (Grade 12)
Masters in Software Engineering, Data Engineering, Computer Science or related field
2-3 years of relevant work experience
Strong Scala or Java background
Knowledge of AWS, GCP, Azure, or other cloud platform
Understanding of data modeling principles
Ability to work with complex data models
Experience with relational and NoSQL databases (e.g. Postgres, ElasticSearch/OpenSearch, graph databases such as Neptune or neo4j)
Experience with technologies that power analytics (Spark, Hadoop, Kafka, Docker, Kubernetes) or other distributed computing systems
Knowledge of API development and machine learning deployment

Responsibilities:

Build and scale data infrastructure that powers real-time data processing of billions of records in a streaming architecture
Build scalable data ingestion and machine learning inference pipelines
Build general-purpose APIs to deliver data science outputs to multiple business units
Scale up production systems to handle increased demand from new products, features, and users
Provide visibility into the health of our data platform (comprehensive view of data flow, resources usage, data lineage, etc) and optimize cloud costs
Automate and handle the life-cycle of the systems and platforms that process our data

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.