TL;DR
Senior Data Engineer: Responsible for designing and building scalable data pipelines using Azure Data Factory and Azure Synapse with an accent on processing diverse drilling data formats and ensuring data quality. Focus on implementing complex logic to convert time-based drilling sensor data into depth-based datasets and optimizing data pipelines for performance.
Location: Hybrid in Bolivia
Company
Encora is a global company that offers Software and Digital Engineering solutions.
What you will do
- Design and build scalable data pipelines to ingest, process and transform high volume drilling data using Azure Data Factory and Azure Synapse.
- Implement complex logic to convert time based drilling sensor data into depth based datasets for geological and operational alignment.
- Develop ingestion frameworks to process diverse drilling data formats such as LAS, DLIS and WITSML from multiple vendors.
- Ensure data quality, availability and consistency by implementing automated validation checks and optimizing data pipelines for performance.
Requirements
- Strong experience designing and implementing data pipelines using Azure Data Factory and Azure Synapse Analytics.
- Advanced proficiency in SQL and experience working with large scale data processing.
- Strong programming skills in Python with experience using PySpark or Pandas for data processing.
- Experience working with big data technologies such as Spark for distributed data processing.
- Experience managing and organizing data in Azure Data Lake Storage Gen2.
- Understanding of ETL and ELT patterns and modern data engineering best practices.
Nice to have
- Microsoft Certified Azure Data Engineer Associate (DP 203) certification is a plus.
Culture & Benefits
- Hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.