Location
PolandRate
Years of experience
12+About
With over a decade of IT experience, I specialize in transforming complex problems into efficient solutions, leveraging my proficiency in C#, Python, and various SQL languages to drive business potential. My role as AWS Team Lead Data Engineer at Exsell Netherlands for Majid Al Futtaim Dubai involved developing solution systems using Customer Lifetime Value (CLV) and Next Best Offer (NBO) models. I collaborated closely with Solution Architects and Data Science Teams, overseeing task creation and managing junior team members. My technical expertise includes AWS Glue PySpark jobs, AWS Step Functions, AWS Lambda, AWS Sagemaker, and infrastructure as code using Pulumi and Terraform. Previously, as a Team Lead Data Engineer at the Institute of Environmental Protection – National Research Institute, I led the PoC initiative to migrate critical data processing and analysis pipelines into AWS Cloud. I managed an ETL team, creating the National Database on products, packaging, and waste management. My background also includes roles as a Senior Full Stack C# Developer at Mera Systemy Sp. z o.o., Data Engineer at Hewlett Packard Enterprise, and Developer/Linux Administrator at the University of Warsaw. I hold a degree from the Faculty of Physics at Warsaw University of Technology and possess strong English proficiency (C2).Tech Stack
Data Engineering, Ansible, AWS and Cloud, Azure, C#, Docker, Python, SQLExperience
- Developing Solution Systems: Creating and implementing solution systems leveraging Customer Lifetime Value (CLV) and Next Best Offer (NBO) models to drive business outcomes.
- Task Management and Team Oversight: Collaborating with Solution Architects, overseeing task creation, and managing junior team members to ensure project goals are met.
- Data Ingestion and Integration: Designing and managing AWS Glue PySpark jobs, Delta Lake in AWS Glue, and Amazon S3 for comprehensive data ingestion and integration.
- Automation and Infrastructure Management: Utilizing AWS Step Functions for daily automation tasks, AWS Lambda for service calls, and infrastructure as code tools like Pulumi and Terraform.
- Cloud Migration and ETL Management: Leading PoC initiatives to migrate critical data processing and analysis pipelines into AWS Cloud and managing ETL teams for database creation and maintenance.
- Training and Mentorship: Conducting training sessions for new employees, introducing procedures and work culture, and sharing knowledge through case studies to improve professional qualifications.
- Big Data Platform Automation: Automating configuration and management of Big Data Platforms using Ansible and Docker, and tailoring platform components to meet specific client needs in cloud environments like AWS and Azure.
Employment history
• AWS Glue PySpark jobs (Creating complete data ingestion)
• Delta Lake in AWS Glue and Amazon S3
• AWS Step Functions (The primary daily automation tasks for the project involve leveraging intrinsic functions and utilizing the Context object effectively)
• AWS Lambda (Calling other aws services)
• AWS Sagemaker (training model it within the context, and incorporating recommendations)
• Pulumi & Terraform Code as Infrastructure
• Amazon RDS, Amazon S3
• AWS Identity and Access Management (IAM)
• AWS Billing and Cost Management
• Leading the PoC initiative to migrate critical data processing and analysis pipelines into AWS Cloud with AWS Glue, Lake Formation and Redshift
• Managed and empowered ETL team that created the analytical database of National Database on products, packaging and waste management (BDO database)
• Collaborated and back up other teams as Database Developer with repair errors that require most careful analysis of business logic
• Conducting training for new employees. Introducing procedures and work culture in the institution and improving professional qualifications by sharing with my knowledge and experience on the case studies (I didn’t realize before that teaching is fun and gives a ton of satisfaction)
• Lead Developer for Real-Time Data Collection Systems: Leading the development of systems that collect real-time data from 3000 ticket vending machines into a PostgreSQL database.
• Web Application Development: Creating and maintaining web applications that provide functionalities such as inspection, financial reporting, and information for field service support.
• Database Management and Optimization: Ensuring efficient data storage and retrieval by managing the PostgreSQL database and optimizing it for performance to handle real-time data inputs.
• Member of the Big Data DevOps and Administrations team
• Responsible for automating Big Data Platforms configuration and management with IaaC approach based on Ansible and Docker (Cloudera, Hortonworks)
• Lifting and shifting existing offers’ portfolio into the Cloud to match the latest clients needs (mainly AWS and Azure)
• Preparing PoC around data analysis scenarios on AWS (AWS EMR and Redshift)
• Tailoring platforms components to specific clients’ needs
• Diagnosing and solving issues on production
• Linux System Support: Providing support for scientific teams by managing and maintaining Linux systems, ensuring their smooth operation and addressing any issues that arise.
• In-Depth Learning and Application of Linux: Engaging in intensive learning and application of Linux system administration skills to support the needs of the scientific research environment.
• Troubleshooting and Maintenance: Diagnosing and resolving technical issues related to Linux systems, ensuring the reliability and performance of the computing infrastructure for research activities.
• Server and Data Storage Management: Managing and maintaining servers and data storage systems to ensure their reliability, performance, and security.
• Technical Support and Troubleshooting: Providing technical support and troubleshooting for server and storage-related issues, ensuring minimal downtime and optimal performance.
• Infrastructure Setup and Configuration: Setting up and configuring computing infrastructure, including installing and configuring hardware and software components to meet organizational needs.