Data Engineer Lead (Hybrid Work Schedule)
Quorum Software View all jobs
- Calgary, AB
- Permanent
- Full-time
- Lead a team of data engineers through the full delivery lifecycle — from architecture and design through to deployment and ongoing operations
- Design and implement scalable data pipelines that ingest, transform, and serve data across Bronze, Silver, and Gold layers using Medallion Architecture
- Build and maintain the organization's Databricks environment, including Unity Catalog, Delta Lake, and Databricks Workflows
- Develop and maintain RESTful APIs that expose data assets to downstream consumers including analytics teams, applications, and external partners
- Own pipeline reliability — establishing monitoring, alerting, data quality checks, and SLA frameworks across all ingestion patterns
- Partner closely with Data Scientists to support feature engineering, model training pipelines, and the operationalization of ML models into production
- Define and enforce data engineering standards, coding conventions, and peer review practices across the team
- Collaborate with Data Architects and Analytics Engineers to evolve the centralized data model and ensure consistency across segments
- Champion data governance practices including lineage tracking, metadata management, and access control using Azure Purview and Unity Catalog
- Partner closely with our Product leaders to help guide and design needed data solutions.
- And other duties as assigned.
- 3+ years leading a data engineering team or serving as a technical lead on a major data platform initiative
- Demonstrated experience delivering an end-to-end data solution — from requirements through production — across a complex, multi-source environment
- Ability to manage competing priorities, communicate technical decisions to non-technical stakeholders, and keep a team aligned and productive
- Deep hands-on experience with Databricks or similar tools, including Delta Lake, Unity Catalog, Databricks Workflows, and cluster/job optimization
- Strong understanding of Medallion Architecture (Bronze / Silver / Gold) and how to apply it pragmatically across different data domains
- Proficiency in PySpark and Spark SQL for large-scale data transformation
- Experience building and maintaining pipelines across multiple ingestion patterns — APIs, CDC, file-based, streaming (Kafka, Event Hubs), and batch
- Familiarity with orchestration tools such as Apache Airflow, Databricks Workflows, or Azure Data Factory
- Strong understanding of pipeline reliability patterns: idempotency, incremental loading, schema evolution, and error handling
- Proven experience designing and building RESTful APIs to expose data assets, including authentication, versioning, rate limiting, and documentation
- Familiarity with FastAPI, Flask, or similar Python-based API frameworks
- Understanding of API gateway patterns within Azure (Azure API Management)
- Expert-level SQL skills including window functions, CTEs, query optimization, and data modeling
- Hands-on experience with the Azure data ecosystem: Azure Data Lake Storage Gen2, Azure Data Factory, Azure Synapse Analytics, Azure Purview, Azure Key Vault, and Azure DevOps
- Understanding of Azure networking, IAM, and security patterns as they apply to data platform architecture
- Solid grounding in data science concepts — feature engineering, model training workflows, experiment tracking, and ML model deployment
- Experience supporting or building MLflow-based experiment tracking and model registries within Databricks
- Ability to translate data science requirements into reliable, scalable data infrastructure
- Experience with Microsoft Fabric and Microsoft OneLake
- Experience in the Oil & Gas or energy industry, particularly with operational, production, or field data
- Familiarity with dbt (data build tool) for analytics engineering workflows
- Exposure to real-time / streaming data patterns using Kafka, MQTT, or Azure Event Hubs
- Experience with Terraform or Bicep for infrastructure-as-code on Azure
- Knowledge of data mesh principles and federated data ownership models
- A production-grade Medallion Architecture is live on Databricks, serving data from multiple business segments
- Pipelines are reliable, monitored, and documented — the team spends more time building than firefighting
- APIs are in use by downstream consumers internally and externally with clear ownership and versioning
- The team has a consistent engineering culture with defined standards, review processes, and a healthy backlog
- Data Scientists have a faster path from experimentation to production
- Background Check: The successful candidate will need to successfully complete the following clearances: Criminal History Check, Education Verification, Employment Verification, Driver’s License Verification and passport/ID validation.
- Visa Sponsorship: Employment eligibility to work with Quorum Software in Canada is required as the company will not pursue visa sponsorship for this position. The successful candidate will be required to ensure they maintain and renew any visas or permits that grant employment eligibility where applicable.
Quorum Software connects people and information across the energy value chain. Twenty years ago, we built the first software for gas plant accountants. Pipeline operators came next, followed by land administrators, pumpers, and planners. Since 1998, Quorum has helped thousands of energy workers with business workflows that optimize profitability and growth. Our vision for the future connects the global energy ecosystem through cloud-first software, data standards, and integration. The trusted source of decision-ready data for 1,800+ companies, Quorum Software makes the essential connections that let us work better together in the connected energy workplace. For more information, visit .Quorum Diversity Statement: At Quorum, we are committed to fostering, cultivating, and preserving a culture of belonging. We want to be the place where a diverse pool of talented people join us, stay with us and do their best work. With a diverse team of employees, we grow and learn better together. The collective sum of the individual differences, life experiences, knowledge, innovation, self-expression, and talent that our employees invest in their work represents not only part of our culture, but our reputation and our achievements. We are fully focused on equity and equality and believe deeply in diversity of race, gender, sexual orientation, age, religion, ethnicity, national origin, ability, neurodiversity and all the other characteristics that make us unique.Quorum Business Solutions and Quorum Software are Equal Opportunity Employers. All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, ancestry, veteran status, disability, genetic information, or any other basis protected by law.Those applicants requiring reasonable accommodation to the application and/or interview process should notify a member of the Human Resources DepartmentQuorum Software does not charge fees, request payments, conduct interviews via messaging apps, or request the installation of software at any stage of the recruitment process. All legitimate recruitment activities are conducted exclusively through our official careers website ( ) and email addresses ending in @quorumsoftware.com. Any communication that does not originate from these official channels should be considered unauthorized and may be reported to