Senior Data Engineer

Astek Polska

We are part of the international ASTEK Group, which has been gathering experience in the global IT outsourcing and engineering services market for more than 30 years. The Astek Group is a global player in engineering and technology consulting, present on five continents.

What do we draw from the other Group entities in our daily work? First and foremost: inspiration, objectives, good practices, innovative activities and values.

In 2020, 2021, 2022 and 2023 we received the Great Place to Work certificate, and thus found ourselves among the fifteen Best Workplaces in Poland in the category: large companies

How about joining the Astek Polska Group's community of Senior Data Engineer ?

Offer details:

B2B: 800 - 1050 PLN net/MD + VAT (based on experience)

Workplace:

Remote

The day-to-day responsibilities of a Senior Data Engineer include:

• Analysis of Existing Stored Procedures: Review existing stored procedures to understand their functionality, input/output data, and dependencies.
• Identification of Conversion Candidates: Identify stored procedures suitable for migration to dbt models based on their complexity, data transformations, and frequency of use.
• SQL Conversion: Rewrite SQL code from stored procedures into dbt model definitions, taking into account differences in syntax and functionality between stored procedures and dbt.
• Model Development: Create dbt models to transform raw data from source systems into meaningful, analysis-ready datasets.
• Define and implement data transformations, including aggregations, joins, and data type conversions.
• Design dbt models to replicate the logic and functionality of stored procedures, ensuring that data transformations are accurately represented.
• Data Quality Assurance: Develop dbt tests to ensure data quality and integrity throughout the transformation process.
• Implement checks for completeness, accuracy, and consistency of data.
• Performance Optimization: Identify and address performance bottlenecks in dbt models to improve query efficiency and reduce processing time.
• Tune dbt configurations and SQL queries to optimize performance for large datasets.
• Continuous Improvement: Iterate on dbt models based on feedback and changing business requirements.

Your background? You’re ideal for this role if you have:

• 4+ years of working with programming language focused on data pipelines,eg. Python or R
• 4+ years of experience working with SQL
• 3+ years of experience in data pipelines maintenance
• 3+ years of experience with different types of storage (filesystem, relation, MPP, NoSQL) and working with various kinds of data (structured, unstructured, metrics, logs, etc.)
• 3+ years of experience in working in data architecture concepts (in any of following areas data modeling, metadata mng., workflow management, ETL/ELT, real-time streaming, data quality, distributed systems)
• 3+ years of cloud technologies with emphasis on data pipelines (Airflow, Glue, Dataflow - but also other smart solutions of handling data in the cloud - elastic, redshift, bigquery, lambda, s3, EBS etc.)
• 4+ years of experience working with SQL
• 1+ years of experience in Java and/or Scala
• Very good knowledge of relational databases (optional)
• Very good knowledge of data serialization languages such as JSON, XML, YAML
• Excellent knowledge of Git, Gitflow and DevOps tools (e.g. Docker, Bamboo, Jenkins, Terraform
• Capability to conduct performance analysis, troubleshooting and remediation (optional)
• Excellent knowledge of Unix
• Pharma data formats is a big plus (SDTM)

What benefits are waiting for you?

• trainings, certificates and upskilling if needed
• Center mentoring
• career path
• benefits package (private medical care, Multisport)
• work atmosphere, social events and team-building meetings

📩More details: pmichta@astek.pl

AO18859