Principal Engineer, Data Platform

About the Role: 

AlphaSense runs a large-scale, Kubernetes-native platform built on a modern microservices architecture, serving the majority of the Fortune 500. We are moving it to a cell-based architecture, where each cell is an independent deployment of the platform. 

The hardest part is data. Our platform relies on a broad ecosystem of data platforms and storage systems, including Vespa, MongoDB, ClickHouse, Redis, ScyllaDB, PostgreSQL, Temporal, and Kafka, to name a few. This role is focused on the Data Platform: the capabilities that let engineering teams build, run, and scale their storage at large scale without each team becoming an expert in every system.

You will help set the technical direction for the Data Platform and build it rather than only specify it. A central part of the work is building the capabilities for large-scale data migrations, so we can create, move, split, and rebalance data across cells and geographies as the cell-based architecture rolls out.

We prefer managed solutions over self-hosted ones, and we expect engineers to own their storage. Teams run what they build, and this role makes that model work at scale: you build it, you run it, and you scale it. Your job is to give teams the platform, standards, and tooling to do that safely, and to own the reliability and scalability of the platform itself.

Responsibilities:

• Help set the architecture and roadmap for the Data Platform and the storage ecosystem behind it
• Build the capabilities for large-scale data migrations across cells, regions, and geographies, including tooling for moving, splitting, validating, and rolling back data with minimal downtime
• Enable product teams self-service ways to provision, operate, and scale their own storage
• Drive the data-layer changes that cell-based architecture requires, including data placement, tenant routing, replication, and isolation
• Define and apply the managed-first approach: choose managed services where they fit, and build and run self-managed systems only where there is a clear reason
• Find and close scalability gaps in data stores across regions, cells and deployment models
• Build shared libraries that give product teams self-service storage capabilities 
• Closely partner with Cloud Platform, SRE, and DevEx teams
• Raise the architectural bar across the organization and mentor Staff and Senior engineers
• Apply AI tooling to data platform work, such as slow query reviews and incident diagnosis

Requirements: 

• Deep hands-on experience designing and operating data platforms and storage systems
• Production large-scale experience with storages (e.g. MongoDB, ClickHouse, Redis, PostgreSQL)
• Cloud-native data storage experience, including managed data services on AWS
• Strong background in Kubernetes and containers
• Track record of designing and running large-scale data migrations, including live migrations 
• Experience with cell-based, multi-tenant or multi-region architectures
• Experience building platforms and tooling that other engineers use, and comfort with a model where engineers own their storage and run what they build
• Demonstrated ability to land significant changes in code owned by other teams
• A record of spotting data scalability and reliability limits before they turn into incidents
• Ability and desire to design, lead, influence, and deliver on ambitious technical roadmaps

Nice to have: 

• Experience running storage systems on top of Kubernetes 
• Experience building control planes with Crossplane
• Experience with private, isolated, on-premises, or air-gapped deployments