Senior Data Engineer(Databricks) - Toronto
Prodigy Labs
- Toronto, ON
- Permanent
- Full-time
- Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
- Optimize data pipelines for performance, scalability, and reliability
- Ensure data quality and integrity throughout the data lifecycle
- Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
- Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
- Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
- Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
- Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
- Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent work experience
- 5+ years of experience in data engineering with Databricks and Spark
- Proficient in SQL and Python, and familiar with Java or Scala
- Experience with cloud platforms, such as Azure or AWS
- Experience with big data technologies, such as Kafka, Hadoop, Hive, etc.
- Experience with data warehouse and data lake concepts and architectures
- Experience with data integration and ETL tools, such as Azure Data Factory or Talend
- Experience with data visualization and reporting tools, such as Power BI or Tableau
- Strong analytical and problem-solving skills
- Excellent communication and teamwork skills
- We are a collective group of people and collaboration is key to our process
- We don’t work for our clients, we work with them
- A Flexible Hybrid Working Environment
- Easily accessible downtown location
- Competitive compensation commensurate with experience
- Everyone brings something valuable to the table in our supportive, challenging, and collaborative, diverse work environment
- Company paid health benefits: 100% medical, dental and vision coverage
- Corporate-discounted Gym Membership through GoodLife
- Company discount program including Travel, Shopping, Attractions, Wellness, & Sporting events, just to name a few
- Access to an Employee & Family Assistance program (EAP)
- Employee Referral Program
- Employee Opportunity Program
- Professional Development Program
- Philanthropic Events
- Social Events