Senior AWS Data Engineer

ALGOTEQUE is an IT consultancy firm that helps startups, mid-sized and large corporations to create and deliver innovative technologies.

Our team has a successful track record in designing, developing, implementing, and integrating software solutions (AI, ML, BI, Web, Automation) for Telecom, Energy, Bank, Insurance, Pharma, Automotive, Industry, e-commerce. We deliver our services both in fixed-price and time-and-materials models, helping our customers achieve their business and IT strategies.

Job Description

We are looking for an experienced specialist for the position of AWS Architect, who will be responsible for designing, developing, and automating processes within the AWS environment, with a particular focus on AWS Glue pipelines utilizing data stored in S3 in ORC/Parquet/Text formats.

Responsibilities

Designing and developing processes within AWS Glue pipelines using S3.

Defining standards, guidelines, and access mechanisms.

Data extraction and aggregation using Python and pySpark.

Managing external and managed tables in S3 and Redshift.

Creating libraries for UDF functions.

Building S3 buckets and managing policies for S3 buckets.

Requirements

Profile / Requirements :

Minimum of 10 years of experience in the BI/Data field.

Minimum of 5 years of experience with AWS platform and data services.

Proficiency in S3, Redshift, Glue, and data import services (DMS, Appflow, Data Transfer/Data Sync).

Experience in creating state machines with Lambda, Glue, CloudWatch, SNS, Even Bridge.

Proficiency in scripting languages: Python, pySpark.

Familiarity with CloudWatch, SNS, and Even Bridge.

Excellent analytical and problem-solving skills.

Experience in CI/CD (preferably Gitlab), preferred AWS Code Commit, Code Build, Code Deploy services.

Familiarity with Big Data ecosystems (Lake Formation, AWS EMR).

Experience with Alembic.

Experience working in Agile, Scrum methodology.

AO2688