Role name: Data Engineer - Azure -ETLLocation: Calgary - HybridDuration: Long Term contractExpereince Level: 8+yrsKey Responsibilities:Data pipeline developmento Design, build and maintain scalable ETL (Extract, Transform, Load)o Automate data ingestion from multiple sources (APIs, database, files, etc.).Data Storage & Managemento Develop and optimize data storage solutions (e.g. datalakes, warehoures)o Ensure data integrity, consistency and security across systems.o Work with Azure cloud storageData Processing & Transformationo Process and clean raw data for analysis and visualizationreporto Implement real-time and batch data processing workflows using Spark, KafkaDatabase Managemento Design and optimize relations (SQL) and NoSQL databases.o Improve database performance indexing, and query optimizationData Governance & Securityo Implement Data quality, validation and governance standards.o Ensure compliance with data privacyRequirements:Strong proficiency in Databricks, Azure Data Factory and API integration.Proficiency in programming languages such as Python, SQL and PySpark.Familiarity with data modeling, ETL processes and data governance.Design, develop and maintain robust and scalable data pipelines using Databricks, Azure Data FactoryFamiliarity with DevOps practices and tools for CI/CD pipelines.Experience with Azure cloud platformIntegrate data from various APIs and external sources to enrich and expand datasets.Project-Specific Requirements:Data Engineer will be responsible for creating a pipeline, ETL. Ingesting data from multiple sources like API, SharePoint, SAB. Communicate with stakeholders to gather and clarify requirements.Regards,Email: , Phone#: 6788247770
