Kafka Platform Engineer

Astra North Infoteck Inc.

  • Kitchener, ON
  • Permanent
  • Full-time
  • 26 days ago
Job Title: Kafka Platform EngineerPrimary Skills: AKANA API Management Platform (E2), Apache Kafka, Cloud & DevOps ToolsRole OverviewThe Kafka Platform Engineer is responsible for designing, deploying, and managing scalable, secure, and highly available Kafka-based streaming platforms across hybrid environments. This role involves platform engineering, security enforcement, automation, and enabling event-driven architectures across the organization.Key Responsibilities1. Platform Engineering & OperationsDesign, deploy, and manage Apache Kafka clusters across on-premises, cloud, and Kubernetes environments.Ensure high availability, fault tolerance, disaster recovery, and effective capacity planning.Implement and maintain Kafka ecosystem components, including Kafka Connect, Schema Registry, and ksqlDB.Automate provisioning, configuration, and lifecycle management using Terraform, Ansible, Helm, and scripting.Configure and maintain observability solutions such as Prometheus, Grafana, Splunk, ELK, and Datadog.Perform performance tuning, including partitioning, replication, retention policies, ISR management, and broker configurations.2. Security & GovernanceImplement authentication and authorization mechanisms (SASL, ACLs, RBAC).Enforce encryption standards for data in transit and at rest (TLS, AES).Collaborate with governance teams to manage schema lifecycle, compatibility policies, and data standards.3. Development & IntegrationDevelop and maintain Kafka connectors, producers, consumers, and event-streaming pipelines.Collaborate with application and platform teams to enable event-driven architecture adoption.Provide architectural guidance and best practices for scalable and resilient Kafka integrations.4. Troubleshooting & SupportDiagnose and resolve issues related to Kafka brokers, ZooKeeper/KRaft, networking, and cluster performance.Conduct root cause analysis and implement corrective/preventive actions.Participate in on-call rotations to ensure platform uptime and SLA adherence.Required Skills & TechnologiesApache Kafka & Ecosystem (Kafka Connect, Schema Registry, ksqlDB)AKANA API Management PlatformTerraform, Ansible, HelmKubernetes & Cloud Platforms (AWS/Azure/GCP)Monitoring: Prometheus, Grafana, ELK, Splunk, DatadogSecurity: SASL, TLS, ACLs, RBAC, Encryption StandardsScripting: Python, Bash (preferred)Preferred QualificationsExperience with event-driven architecture and distributed systemsStrong troubleshooting and performance tuning skillsFamiliarity with API management platforms and integration patternsExperience working in Agile/DevOps environmentsSoft SkillsStrong problem-solving and analytical thinkingEffective communication and cross-team collaborationAbility to work in high-availability, production-critical environments

Astra North Infoteck Inc.