Dataform Engineer/Architect
- Toronto, ON
- Permanent
- Full-time
No. of Positions: 1
Location: Toronto, RemoteDescription
We are looking for a Dataform Architect to design and optimize data pipelines on Google Cloud Platform, leveraging Dataform and BigQuery. This role involves building reusable SQLX workflows, ensuring data quality and compliance, managing performance and costs, and collaborating with stakeholders to deliver scalable data solutions while mentoring junior team members.Key Responsibilities:
- Design and develop data pipelines using Dataform and SQL on Google Cloud Platform (GCP).
- Implement reusable SQLX files for data transformation workflows and queries.
- Optimize and manage BigQuery environments for performance and cost-effectiveness.
- Collaborate with stakeholders to identify data requirements and develop data solutions.
- Maintain data integrity, security, and compliance through best practices and governance.
- Design and implement data quality checks and validation processes.
- Ensure version control and change management for data pipelines using tools such as Git.
- Develop and maintain documentation for data pipelines, including technical specifications and architecture diagrams.
- Stay up-to-date with industry trends and emerging technologies in data engineering and cloud computing.
- Mentor and provide guidance to junior team members on data pipeline development and best practices.
- 5+ years of experience in data engineering, with a focus on SQL and cloud-based data platforms.
- Strong experience with Dataform and GCP, including BigQuery and other related services.
- Experience with version control systems such as Git, GitHub, or GitLab.
- Knowledge of data security, privacy, and compliance standards.
- Excellent communication, collaboration, and problem-solving skills.
- Experience with performance optimization and cost management in BigQuery.
- Familiarity with data governance and data quality best practices.
- Ability to work effectively in a fast-paced and dynamic environment.
- Passionate about staying current with emerging technologies and trends in data engineering and cloud computing.
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field.