Back to list

Azure Data Factory Developer Resume Sample

➣ Hire now

Location

Poland

Rate

$20 / per hour

Years of experience

About

I am an experienced Data Engineer with over five years of diverse experience in Information Technology. My expertise includes the development and implementation of various Big Data applications, primarily in cloud environments, and fostering a DevOps culture with CI/CD practices. I excel in both team settings and independent work, thanks to my strong communication skills and problem-solving abilities. I have successfully designed and implemented complex ETL pipelines, automated data processing, and created CI/CD pipelines in Azure DevOps, significantly improving efficiency and reliability. My career includes roles at BI4ALL, Procter & Gamble, Atos, and Nissan Sales CEE, where I developed scalable Big Data applications, optimized databases, and built data pipelines using tools like Databricks, Azure, and AWS. I hold multiple certifications, including Databricks and Microsoft credentials, and a Master’s degree in Computer Science with a specialization in Data Engineering from Warsaw University of Technology. Additionally, I am proficient in several programming languages, databases, and data visualization tools, and have a keen interest in automotive, sports, and emerging technologies like blockchain and cryptocurrencies.

Tech Stack

Cloud, AWS and Cloud, Azure, Big Data, Docker, Python, SQL

Experience

Designed and implemented complex ETL pipelines using Databricks (Python, PySpark, Spark SQL) and Azure Data Factory to efficiently ingest and transform data from multiple sources into a Data Lake and Azure SQL Database.
Developed and maintained a highly scalable Big Data application using Python, Spark SQL, PySpark, and deployed it on Azure Kubernetes Service with Spark pods implementation, handling 100 million records per processing.
Built and managed Data Pipelines and Jobs with Databricks (Delta Live Tables, PySpark, Python, SQL) and Azure Data Factory to streamline data processing.
Created CI/CD pipelines in Azure DevOps to automate development implementation with Databricks and Azure Data Factory, enabling faster and more reliable deployment of data pipelines.
Led a team of Data Engineers in refactoring the application architecture and implementing the Big Data App solution, utilizing Agile methodology.
Optimized PostgreSQL and MSSQL databases to improve query performance and efficiency in data processing tasks.
Automated data processing and improved efficiency using Databricks Jobs and Data Pipelines (DLT) with Spark, enhancing overall data processing accuracy.

Employment history

Data Engineer, BI4ALL February 2023 - Present

● Successfully designed and implemented a complex ETL pipeline usingDatabricks(Python,PySpark,SparkSQL)andAzureData Factory to efficiently ingest and transform data from multiple sources into a Data Lake and Azure SQLDatabase.
● Automated data processing using Databricks Jobs and Data Pipelines (DLT), improving overall efficiency and accuracy of the data processing with Spark.
● Created CI/CD pipelines in Azure DevOps to automate developmentimplementationwithDatabricksandAzureData Factory, enabling faster and more reliable deployment of data pipelines.

DevOps / Data Engineer / IT Manager, Procter & Gamble March 2022 - February 2023

● Developed and maintained a highly scalable Big Data application using Python, Spark SQL, PySpark and deployed it on Azure Kubernetes Service with Spark pods implementation (100M records per processing)
● Created a Docker-based app development environment (Python, Spark, YAML) and integrated it with Redis Cache for efficient app development.
● Created customlog queries usingAzureLogAnalyticsQuery Language (KQL) to search, filter, and analyze app performance.
● Ingested data with Python, PySpark, Spark SQL and optimised query, processing using Spark.
● Optimized PostgreSQL database to improve query performance.
● Built and managed Data Pipelines and Jobs with Databricks (Delta LiveTables,PySpark,Python,SQL) and Azure DataFactory to streamline data processing.
● BuiltandmaintainedCI/CDpipelinesusingAzureDevOpsand managed infrastructure environments with Terraform.
● Led a team of 2 Data Engineers in the refactoring of the application architecture and implementation of the Big Data App solution, utilizing Agile methodology.

Junior Data Scientist, Atos August 2021 - February 2022

● Designed and implemented an ETL pipeline withDatabricks (Python, Pyspark, Spark SQL) and Azure Data Factory to load data into MySQL database.
● Created reports and dashboards with Tableau and PowerBI for data-driven decision-making.
● Assisted increating MachineLearningmodelsforpredictive analytics.
● UtilizedAzureSynapseAnalyticsandAzureLogAnalyticsto troubleshoot and resolve data processing issues

Data Engineer, Nissan Sales CEE September 2019 - August 2021

● Built scripts using Python, PySpark and Spark SQL to schedule Databricks Jobs for financial calculations and task automation.
● Developed and maintained data models in MSSQL database, migrating to Azure SQL Database.
● Built data pipelines with Azure DataFactory for data processing.
● Implemented Azure Functions with Python and Spark for data extractions.
● BuiltAzureDevOpsprojectstructure andCI/CDpipelinesfor consistent deployment of code and infrastructure. Integrated environments

Education history

Warsaw University of Technology June 2023 Master of Engineering: Computer Science - Specialization in Data Engineering.
● Master's thesis: "NoSQL databases in Big Data solutions".

Warsaw University of Technology January 2022 Bachelor of Engineering: Electrical Engineering
● Engineering's thesis: "Project of software to automatically select electric cars".
● Head of Software department in Scientific Association Proton Dynamic and Aria.

Download profile as PDF ➣ Hire now

“80+ clients have experienced exceptional results from our IT recruitment, IT staffing, and software development services.”

Tom Potanski Founding Partner, DevsData LLC

Similar blind CVs/resumes

Poland

Swagger Developer Resume Sample

Senior Java Software Developer skilled in backend architecture, database management, and team mentoring.

Swagger

Angular

Bash

Docker

Git

Hibernate

Java

JavaScript

Kafka

Kubernetes

Microservices

MySQL

Oracle

PL/SQL

Python

Spring Framework

Rate $25 / per hour

Poland

CEO Resume Sample

A CEO over 15 years of leadership experience.

ERP

Asana

AWS and Cloud

CRM

Microsoft Azure

MS Office

Oracle

Salesforce Object Query Language

SAP

Slack

Rate $25 / per hour

Poland

Rest API Developer Resume Sample

Full-stack developer with experience in building REST APIs, React front-ends, and integrating cloud services.

REST APIs

API

AWS and Cloud

CSS

Express.js

Firebase

GraphQL

HTML

JavaScript

MongoDB

NestJS

PostgreSQL

React

REST

Rate $17 / per hour

Ukraine

Senior Android Developer Resume Sample

Senior Android Developer with expertise in Kotlin, app architecture, and complex backend integration.

Android

AWS and Cloud

C++

Git

Java

Jenkins

Kotlin

Mobile

Rate $15 / per hour

UK

C Level Resume Sample

Experienced CTO with 20+ years in AI, data science, and tech leadership, driving innovation, growth, and sustainability.

Management

AWS and Cloud

Azure

C#

C++

CSS

ElasticSearch

Git

GitHub

HTML

Java

JavaScript

Jenkins

MongoDB

PHP

Project Management

Project Planning

Python

SQL

Rate $60 / per hour

Poland

Chief Technology Officer Resume Sample

A results-driven technology executive with over a decade of experience.

Management

.NET

Angular

ASP.NET

C++

Docker

Lead

Project Management

Python

React

Rate $30 / per hour

Poland

Work From Home Resume Sample

Senior .NET Developer with expertise in backend architecture, business analysis, and system integration.

.NET

Angular

ASP.NET

C#

Microservices

MS SQL

TypeScript

Rate $15 / per hour

Poland

Sample COO Resume

A COO with over 20 years of experience in banking.

ERP

CRM

Google Analytics

Jira

Marketing

MS SQL

Oracle

Power BI

Project Management

ServiceNow

Workday

Rate $20 / per hour

Poland

Board Member Resume Sample

A highly experienced executive with over 20 years of leadership.

MS Office

Asana

ERP

Finance

Jira

Microsoft Excel

Oracle

Project Management

SAP

Workday

Rate $25 / per hour