Location
AzerbaijanRate
Years of experience
12+About
I am a seasoned Data Engineer with over 12 years of comprehensive experience in managing Oracle Databases, Data Warehousing, Big Data ecosystems, and both batch and real-time streaming systems. My expertise spans cloud platforms such as Azure and AWS, as well as on-premises solutions including the Hadoop ecosystem and traditional Data Warehouses. My technical proficiency encompasses ETL processes, CI/CD pipelines, and data visualization tools like Power BI and Tableau. Throughout my career, I have developed a robust foundation in Data Warehouse methodologies, particularly Inmon, Kimball, and Data Vault 2.0. I am adept at web scraping, which complements my data engineering skills and provides valuable data insights. My hands-on experience includes a variety of data transformation tools and programming languages such as SQL, Python, and PySpark. In addition to my technical skills, I have a proven track record of successfully collaborating with IT outsourcing teams and cloud providers to build and maintain complex data infrastructures. My ability to create comprehensive internal documentation and establish benchmarks for Data Governance and Data Quality has been pivotal in ensuring the integrity and reliability of the data systems I manage.Tech Stack
Azure, Big Data, Cloud, Data Engineering, ETL, Kafka, Oracle, Python, Snowflake, Spark, SQLExperience
- Building ETL pipelines (with Python, SPARKSQL, Kafka, Azure Synapse/Databricks).
- Collaborate with company IT outsourcing to build and maintain company Insight Data Warehouse for sale.
- Over 12 years of Data Engineering with Oracle Databases, Data Warehouse, Big Data, Batch/Real time streaming system.
- Working on Cloud (Azure/AWS) and on-Premises Big Data/Hadoop Ecosystem/Data Warehouse, ETL, CI/CD and Visualization (Power BI, Tableu).
- DWH Methodologies: Inmon, Kimbal, Data Vault 2.0. Web Scraping.
- Identifying IT related risks throughout development phases. Areas include networks, operating systems, ERP, databases, security, and disaster recovery.
Employment history
-Collaborating with IT outsourcing teams to build and maintain the company’s Insight Data Warehouse for sales.
-Working closely with cloud providers to migrate existing Data Warehouses to the cloud.
-Creating internal documentation and benchmarks for Data Governance and Data Quality.
-Managing data transformation processes (ETL, ELT) using tools like Azure Data Factory, Synapse Analytics, SSIS, SSAS, SQL, Python, PySpark, MSSQL, Azure SQL, and Databricks.
-Maintaining reporting Data Warehouses and tools such as DBT, Snowflake, Azure Synapse, and Azure Fabric.
-Implementing data security measures and ensuring compliance with data protection regulations.
-Leading the design and optimization of scalable data architectures to support business intelligence needs.
-Data Transformation (ETL, ELT): Azure Data factory/Synapse Analytics, AWS GLUE,AWS Athena, Databricks, SSIS, SSAS, SSRS.
-Scripting (Python, SQL) on Spark/Pyspark, Hive frameworks for Grafana, Power BI,AWS GLUE.
-Data Storage: Data Lakes, Delta Lakes, S3, Athena, AWS Redshift, DynamoDB, RDS,AWS Lake Formation.
-Hadoop Data aggregations: Azure HDInsight, AWS EMR, Cloudera.
-Developing automated workflows to improve data processing efficiency.
-Conducting performance tuning and optimization of data pipelines.
-Oracle Big Data Appliance/Cloudera: administration, monitoring, security, and problem troubleshooting.
-Building Apache Kafka Cluster and integrating with monitoring systems.
-Building ETL pipelines (with Python, SPARKSQL, Kafka, Azure Synapse/Databricks).
-Setup and Operation support Elasticsearch 7.x, Airflow.
-Leading technical teams in the design and implementation of big data solutions.
-Ensuring the scalability and reliability of data infrastructure.
-Handled Oracle EBS Application’s code migrations using CI/CD.
-Patch/Release management through patch wizard and change management process.
-EBS R12 Applications support on 12c databases with RAC/ASM using SRVCTL.
-Implementation of external languages with EBS R12.
-Performance tuning of both Apps and DB layers.
-Handled Web logic, SOA suite, Oracle EBS R12.2 installations and configurations.
-Identify IT related risks throughout development phases. Areas include networks, operating systems, ERP, databases, security, and disaster recovery.
-Tasks audit, SQL database system Audit tasks are completed accurately and with in established timeframes.
-ERP Security: Experienced in performing ERP security reviews for system parameters and configurations, default accounts and privileged access, table/program security, batch job administration, change management.