Applied AI Engineer – GenAI Systems

• Translate business problems into a clear solution approach: user workflow, data flow, model approach, evaluation plan, and operational controls.
• Create lightweight, high-quality design artifacts (e.g., system context, runtime sequence, agent/tool map, data lineage, decision log) that make build and governance straightforward.
• Make smart design trade-offs: accuracy vs explainability, cost vs performance, speed vs robustness.

• Develop ML solutions such as forecasting, classification, NLP, optimization, anomaly detection, and scenario analysis.
• Build GenAI capabilities such as retrieval-based solutions (RAG), structured summarization, transaction understanding, variance explanations, and tool-using workflows (where applicable).
• Engineer features from structured + unstructured data and ensure solutions remain stable as data evolves.

• Define performance expectations with collaborators and implement backtesting / out-of-time testing and error analysis.
• For GenAI, design practical evaluation: scenario coverage, edge cases, human review rubrics, quality scoring, and regression testing.
• Document model limitations clearly and build guardrails for safe use.

• Collaborate with Data Engineering, ML Engineering, and Software teams to productionize: data pipelines, model packaging, CI/CD, deployment, and monitoring.
• Implement monitoring for data quality, drift, performance deterioration, and operational failures; define remediation actions when thresholds breach.
• Contribute to runbooks and support adoption and UAT with business users.

• Produce the documentation and evidence required for model risk review (assumptions, validation results, monitoring plan, UAT evidence, and approvals).
• Ensure privacy/security expectations are met through data minimization, appropriate access controls, and safe handling of sensitive information.

• Mentor junior scientists through design reviews, code reviews, and evaluation practices.
• Help standardize “how we build” (templates, checklists, examples) so delivery becomes faster and more consistent.

• 4–7 years of experience in applied data science / machine learning, with demonstrated end-to-end delivery into production (beyond notebooks), including support for UAT and post-launch iteration.
• Strong Python + SQL, with solid software engineering practices: Git-based workflows, code reviews, unit/integration testing, logging, readable code structure, and basic performance tuning.
• Hands-on experience with modern DS/ML tooling (e.g., scikit-learn, PyTorch/TensorFlow, Spark/Databricks or similar), including feature engineering and model development at scale.
• Demonstrated ability to build and communicate solution architecture. Create clear diagrams and concise specs that include data flow, runtime flow, interfaces, failure modes, and operational controls. Align collaborators on trade-offs and scope.
• Experience building and evaluating GenAI solutions, including at least one of: RAG, structured summarization/extraction, classification with LLMs, tool/function calling, or agentic workflows (multi-step orchestration with tools/data stores).
• Strong evaluation skills across ML and GenAI: backtesting/holdouts, metric selection, error analysis, and quality evaluation frameworks for GenAI (scenario coverage, edge cases, human review rubrics, regression tests).
• Understanding of production readiness: monitoring for data quality and drift, performance deterioration, cost/latency considerations for GenAI, and practical remediation approaches.
• Strong communication and collaborator management: ability to explain outputs, limitations, uncertainty, and build decisions in plain language and drive adoption in business workflows.

• Hands-on GenAI experience across multiple patterns: RAG, prompt orchestration, structured outputs, tool/function calling, agentic workflows (multi-step reasoning with tools), and practical evaluation approaches.
• Experience designing GenAI systems beyond the model: retrieval design, grounding strategies, prompt/version management, caching, and cost/latency trade-offs.
• Experience with cloud-based data/ML stacks and production patterns: model registry, CI/CD, monitoring, APIs/microservices, and automation for retraining or refresh cycles.
• Familiarity with modern GenAI engineering components such as vector databases, embedding strategies, semantic search, and orchestration frameworks (e.g., Semantic Kernel / LangChain-style frameworks).
• Experience working in Finance, Treasury, Insurance, IFRS-17, or Actuarial environments and/or within model governance practices (documentation, validation evidence, monitoring plans).
• Experience implementing GenAI guardrails: hallucination/accuracy controls, safe output formatting, data minimization, access controls, and human review workflows.
• Strong solution design influence: ability to mentor peers through design reviews, code reviews, and evaluation practices (without formal people leadership).

• We’ll empower you to learn and grow the career you want.
• We’ll recognize and support you in a flexible environment where well-being and inclusion are more than just words.
• As part of our global team, we’ll support you in shaping the future you want to see.

Manulife