Reference Data Engineer

Position Summary:

We are seeking an Assistant Vice President (AVP), Software Engineering to join our Operations Systems team. This role blends senior individual contribution with emerging technical leadership: you will design and deliver robust backend services and data pipelines while helping guide the technical direction of your team. The ideal candidate is a strong Python developer with hands-on experience in distributed data processing (Spark), infrastructure-as-code (Terraform), and deep data engineering fundamentals spanning data modeling, ETL, and data distribution. You will own significant workstreams end-to-end, raise the engineering bar through code reviews and design input, and mentor junior engineers as you continue to grow toward a lead or vice president level.

Key Responsibilities:

• Design, build, and own scalable backend services and data pipelines primarily in Python.
• Develop and optimize large-scale data processing workflows using Apache Spark for batch and near-real-time use cases.
• Lead data modeling and schema design decisions that support reliable analytics and downstream consumption.
• Architect and maintain robust ETL/ELT workflows with strong data quality, lineage, and observability.
• Design and implement data distribution patterns to serve data efficiently across internal systems and consumers.
• Provision and manage cloud infrastructure using Terraform and infrastructure-as-code best practices.
• Own delivery of significant workstreams end-to-end, from technical design through production and operations.
• Drive engineering quality through design reviews, code reviews, and clear technical standards.
• Mentor and support the growth of analysts and junior engineers on the team.
• Partner with product, data, and platform leaders to shape roadmap and translate requirements into scalable solutions.
• Lead root-cause analysis for complex production issues and drive continuous improvements in reliability and performance.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 4–7 years of professional software engineering experience with a strong backend focus.
• Expert-level proficiency in Python for building production services and data pipelines.
• Strong hands-on experience with Apache Spark for distributed data processing at scale.
• Proficiency with Terraform for infrastructure-as-code and cloud resource management.
• Deep data engineering fundamentals, including data modeling, ETL/ELT design, and data distribution patterns.
• Solid command of relational and non-relational data stores and SQL.
• Experience with version control (Git), CI/CD pipelines, and modern software development practices.
• Demonstrated ability to own complex workstreams and mentor other engineers.
• Strong communication skills and the ability to collaborate across technical and business teams.

Preferred Qualifications:

• Experience with cloud platforms (AWS, Azure, or GCP) and their data services.
• Familiarity with workflow orchestration tools (e.g., Airflow, Dagster).
• Experience with streaming technologies (e.g., Kafka, Spark Structured Streaming).
• Exposure to containerization and orchestration (Docker, Kubernetes).
• Prior experience in financial services or another data-intensive, regulated industry.
• Experience with Palantir Foundry or similar data integration and analytics platforms.

What We Offer:

KKR offers a collaborative, high-performance culture where engineers work on meaningful problems at scale. We provide competitive compensation, comprehensive benefits, and significant opportunities for professional growth, technical leadership, and impact.