Location
PolandRate
Years of experience
12+About
With over a decade of experience as a Principal Data Engineer, System Architect, and Hadoop ecosystem specialist, I have successfully led and contributed to numerous international projects across various industries, including Telco, Finance, Entertainment, Manufacturing, Pharma, and Advertisement. My expertise spans design and delivery of analytical-oriented solutions, OpenSource-based data processing, and project management methodologies such as Agile Scrum and Waterfall. Proficient in multiple programming languages and databases, I excel in architecting and implementing complex analytical platforms using cutting-edge technologies like Apache Hadoop, Apache Spark, and cloud services from Azure and AWS. In my recent projects, I've demonstrated my skills as an Architect and Data Engineer for Telcos in Germany and Poland, where I designed scalable platforms for network quality monitoring and managed data ingestion processes for multiple data sources. Additionally, I served as Lead Architect for a global truck manufacturer, designing analytical platforms for IoT sensor data analysis, with a focus on security and GDPR compliance. My extensive experience also includes roles as a Solution Architect for a global Pharma company, where I optimized historical studies data usage through a new analytical platform, and as a Team Leader and Architect for GG Network, where I spearheaded the creation of one of Poland's first big Hadoop clusters, showcasing my expertise in parallel computing and big data analytics.Tech Stack
Teradata, Ansible, Apache, AWS, Azure, MySQL, PostgreSQL, Python, R, ScalaExperience
- Design and Delivery of Analytical Solutions: Leading the design and delivery of analytical-oriented solutions, leveraging expertise in Big Data technologies and methodologies to meet business requirements across various industries.
- OpenSource-Based Data Processing: Utilizing OpenSource-based data processing tools and platforms, such as Apache Hadoop and Apache Spark, to architect and implement scalable data processing solutions.
- Project Management: Managing projects using Agile and Waterfall methodologies, ensuring timely delivery and alignment with project objectives and client expectations.
- Data-Oriented Trainings: Delivering data-oriented trainings to enhance team skills and capabilities in data processing and analytics.
- Architectural Design and Implementation: Architecting and implementing analytical platforms for clients in Telco, Manufacturing, Pharma, and other sectors, focusing on scalability, performance, and security.
- Team Supervision and Collaboration: Supervising and collaborating with teams of engineers and developers to implement data ingestion processes, data marts, and other components of analytical solutions.
- Workshop Facilitation and Roadmap Creation: Facilitating workshops with clients to document existing architecture, assess capabilities, and define recommendations and roadmaps for future analytical initiatives.
Employment history
• Responsible for creating a high-scalable environment to analyze telecommunication network data logs, ensuring the platform’s ability to monitor network quality effectively.
• Implementing technologies such as Hortonworks Data Platform (HDP), Apache Spark, Elasticsearch, and Kibana to build the analytical platform.
• Collaborating closely with stakeholders and teams to integrate various components, including Spark, Elasticsearch, and ECE (Elastic Cloud Enterprise), to ensure seamless functionality.
• Implementing and quality-assuring data ingestion and data mart creation processes for over 50 data sources as defined by the customer, ensuring accuracy and efficiency.
• Overseeing a team of engineers, providing guidance and supervision in DevOps tasks and data integration activities to ensure project success.
• Leveraging technologies such as HDP, HDF, Kylo, Nifi, Apache Spark, Semantic Layer, Kerberos, Ansible, and Jupyter to implement the required data processes and achieve project goals.
• Designing an analytical platform for analyzing data from IoT sensors installed on trucks, focusing on standard use cases like risk analysis and predictive maintenance.
• Ensuring compliance with GDPR regulations by implementing a secure access layer within the platform architecture.
• Integrating various technologies such as Hadoop, HDP, Kylo, Apache Hive, Apache Nifi, Microsoft Azure, Teradata, and IBM WMQ to develop and deploy the analytical platform.
• Analyzing the existing data system architecture across the organization, conducting workshops with key stakeholders, and preparing a target platform design and capability assessment.
• Designing a multilayered data repository to store and manage data effectively, ensuring scalability and performance.
• Delivering a detailed roadmap for the designed system, outlining steps for implementation and future enhancements.
• Designing a new analytical platform and processes to optimize the usage of historical studies data, including analysis of existing platform and implementing new tools and processes.
• Designing a new data lake to store and manage data effectively, including ELT processes for text analytics and structured data processing.
• Implementing technologies such as Cloudera, Hadoop, Elasticsearch, Kibana, Kylo, Nifi, Tikka, Maven, Apache Spark, Scala, Python, and Apache Hive to achieve project objectives.
• Conducting workshops with customers to document the existing architecture of the analytical ecosystem and create a capabilities assessment to define recommendations and roadmap.
• Creating a concept for a centralized Data Hub (Data Lake combined with Data Warehouse), outlining architecture and capabilities required to meet business needs.
• Implementing recommendations from the capabilities assessment phase, including kerberization of the HDP cluster and ELT processes running on the cluster.
• Implementing ingestion processes for the Data Lake, ensuring efficient and reliable data acquisition from various sources.
• Utilizing technologies such as CDH, Vagrant, Kylo, Apache Hive, Apache Spark, and Apache Nifi to develop and deploy the ingestion processes.
• Collaborating with stakeholders and teams to ensure alignment with project goals and requirements, and communicating progress and issues effectively.
• Designing an analytical ecosystem based on Teradata UDA solutions to combine Data Warehouse, Data Discovery Platform, and Data Lake implemented on Hadoop technology.
• Providing essential security measures to the analytical ecosystem, ensuring data privacy and compliance with regulations.
• Integrating technologies such as HDP, Teradata, Oracle, and Data Virtualization to achieve the desired analytical capabilities and functionalities.
• Conducting workshops with customers to document the existing architecture of the analytical ecosystem and create a capabilities assessment to define recommendations and roadmap.
• Planning and implementing recommendations from the capabilities assessment phase, including kerberization of the HDP cluster and ELT processes running on the cluster.
• Utilizing technologies such as HDP, Teradata, Oracle, and Data Virtualization to develop and deploy the recommended solutions.
• Working as a member of an international architecture task force to create Data Lake Design patterns using Teradata UDA stack, focusing on Storage and Data Processing layers.
• Delivering design patterns for Storage and Data Processing layers of the Data Lake, ensuring alignment with best practices and client requirements.
• Collaborating with team members and stakeholders to ensure the successful delivery and adoption of the design patterns across different projects and organizations.
• Leading the creation of one of the first big Hadoop Clusters in Poland, including designing the overall platform architecture and data flow processes.
• Implementing technologies such as CDH, Apache Hive, Apache Hadoop, Spring, Python, MapReduce, Apache Flume, MySQL, PostgreSQL, and Pentaho to develop and deploy the Hadoop Cluster.
• Designing a Lambda Architecture based analytics solution, enabling real-time data processing and analytics on the Hadoop Cluster.
Education history
We’ve helped 83 clients with IT recruitment and software development.
Read about a few of them below...