Location
PolandRate
Years of experience
5+About
I am an experienced Data Engineer with over five years of diverse experience in Information Technology. My expertise includes the development and implementation of various Big Data applications, primarily in cloud environments, and fostering a DevOps culture with CI/CD practices. I excel in both team settings and independent work, thanks to my strong communication skills and problem-solving abilities. I have successfully designed and implemented complex ETL pipelines, automated data processing, and created CI/CD pipelines in Azure DevOps, significantly improving efficiency and reliability. My career includes roles at BI4ALL, Procter & Gamble, Atos, and Nissan Sales CEE, where I developed scalable Big Data applications, optimized databases, and built data pipelines using tools like Databricks, Azure, and AWS. I hold multiple certifications, including Databricks and Microsoft credentials, and a Master’s degree in Computer Science with a specialization in Data Engineering from Warsaw University of Technology. Additionally, I am proficient in several programming languages, databases, and data visualization tools, and have a keen interest in automotive, sports, and emerging technologies like blockchain and cryptocurrencies.Tech Stack
Cloud, AWS, Azure, Big Data, Docker, Python, SQLExperience
- Designed and implemented complex ETL pipelines using Databricks (Python, PySpark, Spark SQL) and Azure Data Factory to efficiently ingest and transform data from multiple sources into a Data Lake and Azure SQL Database.
- Developed and maintained a highly scalable Big Data application using Python, Spark SQL, PySpark, and deployed it on Azure Kubernetes Service with Spark pods implementation, handling 100 million records per processing.
- Built and managed Data Pipelines and Jobs with Databricks (Delta Live Tables, PySpark, Python, SQL) and Azure Data Factory to streamline data processing.
- Created CI/CD pipelines in Azure DevOps to automate development implementation with Databricks and Azure Data Factory, enabling faster and more reliable deployment of data pipelines.
- Led a team of Data Engineers in refactoring the application architecture and implementing the Big Data App solution, utilizing Agile methodology.
- Optimized PostgreSQL and MSSQL databases to improve query performance and efficiency in data processing tasks.
- Automated data processing and improved efficiency using Databricks Jobs and Data Pipelines (DLT) with Spark, enhancing overall data processing accuracy.
Employment history
● Successfully designed and implemented a complex ETL pipeline usingDatabricks(Python,PySpark,SparkSQL)andAzureData Factory to efficiently ingest and transform data from multiple sources into a Data Lake and Azure SQLDatabase.
● Automated data processing using Databricks Jobs and Data Pipelines (DLT), improving overall efficiency and accuracy of the data processing with Spark.
● Created CI/CD pipelines in Azure DevOps to automate developmentimplementationwithDatabricksandAzureData Factory, enabling faster and more reliable deployment of data pipelines.
● Developed and maintained a highly scalable Big Data application using Python, Spark SQL, PySpark and deployed it on Azure Kubernetes Service with Spark pods implementation (100M records per processing)
● Created a Docker-based app development environment (Python, Spark, YAML) and integrated it with Redis Cache for efficient app development.
● Created customlog queries usingAzureLogAnalyticsQuery Language (KQL) to search, filter, and analyze app performance.
● Ingested data with Python, PySpark, Spark SQL and optimised query, processing using Spark.
● Optimized PostgreSQL database to improve query performance.
● Built and managed Data Pipelines and Jobs with Databricks (Delta LiveTables,PySpark,Python,SQL) and Azure DataFactory to streamline data processing.
● BuiltandmaintainedCI/CDpipelinesusingAzureDevOpsand managed infrastructure environments with Terraform.
● Led a team of 2 Data Engineers in the refactoring of the application architecture and implementation of the Big Data App solution, utilizing Agile methodology.
● Designed and implemented an ETL pipeline withDatabricks (Python, Pyspark, Spark SQL) and Azure Data Factory to load data into MySQL database.
● Created reports and dashboards with Tableau and PowerBI for data-driven decision-making.
● Assisted increating MachineLearningmodelsforpredictive analytics.
● UtilizedAzureSynapseAnalyticsandAzureLogAnalyticsto troubleshoot and resolve data processing issues
● Built scripts using Python, PySpark and Spark SQL to schedule Databricks Jobs for financial calculations and task automation.
● Developed and maintained data models in MSSQL database, migrating to Azure SQL Database.
● Built data pipelines with Azure DataFactory for data processing.
● Implemented Azure Functions with Python and Spark for data extractions.
● BuiltAzureDevOpsprojectstructure andCI/CDpipelinesfor consistent deployment of code and infrastructure. Integrated environments
Education history
● Master's thesis: "NoSQL databases in Big Data solutions".
● Engineering's thesis: "Project of software to automatically select electric cars".
● Head of Software department in Scientific Association Proton Dynamic and Aria.
We’ve helped 83 clients with IT recruitment and software development.
Read about a few of them below...
37 reviews
Similar blind CVs/resumes
Poland
Swagger Developer Resume Sample
Swagger
Angular
Bash
Docker
Git
Hibernate
Java
JavaScript
Kafka
Kubernetes
Microservices
MySQL
Oracle
PL/SQL
Python
Spring Framework
Poland
CEO Resume Sample
ERP
Asana
AWS
CRM
Microsoft Azure
MS Office
Oracle
Salesforce
SAP
Slack
Poland
Chief Technology Officer Resume Sample
.NET
Angular
ASP.NET
C++
Docker
Python
React
UK
C Level Resume Sample
Java
AWS
Azure
C#
C++
CSS
Elasticsearch
Git
Github
HTML
JavaScript
Jenkins
MongoDB
PHP
Python
SQL
Spain
Snowflake Data Engineer Resume Sample
Snowflake
Azure
ETL
Informatica
Java
MS SQL
Obiee
Oracle
PL/SQL
Python
SAP
Scrum
T-SQL
Visual Basic (VB)
Poland
IT Administrator Resume Sample
DevOps
Asana
Bash
Docker
Java
Jenkins
Kubernetes
Linux
PHP
Python
SQL
Poland
Work From Home Resume Sample
.NET
Angular
ASP.NET
C#
Microservices
MS SQL
Typescript
Poland
Sample COO Resume
ERP
CRM
Google Analytics
Jira
Marketing
MS SQL
Oracle
Power BI
Project Management
ServiceNow
Workday
Poland
Board Member Resume Sample
MS Office
Asana
ERP
Jira
Microsoft Excel
Oracle
Project Management
SAP
Workday
Email copied
Or meet our local partners in other regions we serve
Consulting and tech recruitment.
Consulting and tech recruitment.
I agree to and accept that DevsData LLC will provide better user experience by collecting, analyzing and cataloging information about Internet electronic addresses that I have connected with my devices and about the type of my devices (such as the type and version of software) as well as by making automatic decisions (not involving sensitive data). The agreement applies for the legally binding period, or until either the user or DevsData LLC withdraws from the agreement. Withdrawing from the agreement will result in removing the user's data. Please see our privacy policy.
We use cookies to provide the best experience for you.
>More about cookie policy
Book a call with our team
For software development projects, minimum engagement is $15,000.
Best back-end engineers I've ever worked with...
“I interviewed about a dozen different firms. DevsData LLC is truly exceptional – their backend developers are some of the best I’ve ever worked with. I’ve worked with a lot of very well-qualified developers, locally in San Francisco, and remotely, so that is not a compliment I offer lightly. I appreciate their depth of knowledge and their ability to get things done quickly. “
Nicholas Johnson
CEO of Orange Charger LLC,
Ex-Tesla Engineer,
Mentor at YCombinator
Acknowledgments
