
Senior Data Engineer
- Calgary, AB
- Permanent
- Full-time
- Collaborate closely with the Power Platform, Data Science, Technology Development, and various business teams to enhance and evolve MNP’s Enterprise Analytics program
- Design, develop, and maintain scalable data pipelines, leveraging Azure Databricks, Azure Data Factory (ADF), and associated Azure data platform components
- Support the development and operation of a Data Lakehouse platform, incorporating Databricks, Delta Lake, Azure Data Lake Storage, Azure SQL, and Power Platform tools
- Monitor platform performance and optimize costs to ensure efficiency and scalability
- Assist in designing and implementing robust ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes
- Support data modelling initiatives to ensure data quality and accessibility
- Implement data security best practices, including defining and managing user access controls at scale
- Contribute to automation and continuous integration/continuous deployment (CI/CD) practices using Azure DevOps
- Manage and govern data platform access through Databricks Unity Catalog
- Develop a comprehensive understanding of MNP’s services, business models, and industry context to align data solutions effectively
- Three to four years in Data Engineering, ETL development, or Business Intelligence development roles
- Hands-on experience with Azure Data Factory, Azure Databricks (Spark), Databricks Asset Bundles, Azure SQL, Azure Data Lake Storage, and Azure DevOps
- Strong expertise in working with both structured and semi-structured data formats (CSV, JSON, Parquet)
- Experience working with OAuth 2.0 APIs as data sources
- Proficiency in scripting languages, notably SQL and Python
- Familiarity with operational OLTP and analytical OLAP databases
- Understanding of Data Lakehouse architecture and Medallion architecture best practices
- Experience with Databricks; Unity Catalog, Lakebase, Apps and workflow orchestration is highly desirable
- Experience in the following would be considered an asset:
- MLOps and LLMOps within the Databricks
- Exposure to Power BI and Power BI Embedded
- Familiarity with development frameworks or languages such as .Net Core, C#.NET, JavaScript, or CSS