Staff, Machine Learning Engineer
Fullscript View all jobs
- Toronto, ON Ottawa, ON
- Permanent
- Full-time
- Lead the design, development, and deployment of production, multi-turn LLM-powered features, including summarization tools and clinician-facing conversational agents that support follow-up questions and reasoning over clinical context
- Own backend services in Python that integrate LLM agents with Fullscript's platform and support reliable production use
- Help define technical direction for prompting, grounding, safety, and orchestration strategies used across clinical AI workflows
- Establish and improve evaluation approaches for LLM outputs, including accuracy, hallucinations, edge cases, and overall feature quality
- Shape engineering patterns for model-related workflows, including testing, CI/CD, observability, and version control
- Partner with medical, product, and engineering teams to identify high-value opportunities for AI and turn them into practical, scalable product capabilities
- Work cross-functionally with engineering, analytics, and medical SMEs to refine requirements and ensure data and system design support clinical use cases
- Provide technical leadership across projects by creating clarity in ambiguous problem spaces, guiding tradeoff decisions, and raising the quality bar for the team
- Stay current with the latest LLM research and emerging AI technologies, and help assess where they can be applied effectively at Fullscript
- 6+ years of experience building and implementing machine learning applications in production, including meaningful experience with LLM-powered agents, conversational experiences, or agent-based workflows
- A track record of owning complex technical problems end to end and shaping implementation beyond your immediate code contributions
- Experience designing and deploying AI systems that answer open-ended questions, support follow-up interactions, and operate reliably in production
- Strong experience with LLM application frameworks and tooling, such as LangChain, LangGraph, or similar orchestration and RAG frameworks
- Familiarity with evaluation and monitoring frameworks for LLM outputs, conversational quality, and system reliability
- Knowledge of MCP, agent orchestration patterns, or related approaches for building multi-step AI systems
- Strong proficiency in Python and SQL
- Experience making sound technical decisions around quality, safety, maintainability, and scalability in production AI systems
- Strong communication and collaboration skills, with the ability to work effectively across technical and non-technical stakeholders
- Experience defining technical direction for AI or machine learning systems across multiple projects or teams
- Experience building clinician-facing, healthcare-adjacent, or other high-trust AI experiences
- Experience with recommendation systems, personalization, or other applied ML systems beyond LLMs
- Experience with modern retrieval, grounding, or evaluation patterns for LLM applications
- Experience working closely with domain experts to build systems in complex or highly contextual problem spaces
- SalaryFlexible PTO & competitive pay-rest fuels performance.
- RRSP match & stock options-invest in your future.
- Customizable benefits-flexible coverage, paramedical services, and an HSA.
- Fullscript discounts-save on wellness products.
- Continuous learning-training budget + company-wide initiatives.
- Wherever You Work Well-hybrid and remote flexibility.