The Role

As Staff Data Engineer, you will provide senior onshore technical leadership for the data engineering team. You will own a defined slice of our centralized Databricks data platform with full accountability for decisions and delivery, serve as a technical counterpart to the Principal Data Platform Engineer, and drive architectural judgment and independent problem-solving as platform complexity scales post-migration.

This is a hands-on data engineering role focused on building and maintaining production pipelines, exercising architectural judgment on data modeling and pipeline design, and serving as the onshore escalation point and institutional knowledge backup for platform decisions.

The Business Challenge

We operate multiple product lines (Transfer Pricing, R&D Services, RoyaltyStat, Provisioning), each with distinct databases containing enterprise financial data—journal entries, general ledgers, and financial statements. Our immediate challenge is migrating multi-terabyte datasets from legacy systems to a unified Databricks lakehouse while establishing governance patterns that enable multi-product operations at scale. As the platform matures, the data engineering team needs senior onshore technical presence to drive architecture ownership and maintain platform quality.

What You'll Build

• Production Data Pipelines: Build and maintain production data pipelines within the patterns and governance established by the Lead Data Platform Engineer, ensuring reliability and performance at multi-terabyte scale.
• Data Modeling & Architecture: Exercise architectural judgment on data modeling, pipeline design, and platform usage—translating complex business requirements into scalable data solutions across our product portfolio.
• Stakeholder Engagement: Engage proactively with product and engineering stakeholders to translate requirements into data solutions, serving as the primary onshore technical point of contact for data engineering needs.
• Platform Quality: Drive platform quality through code reviews, testing practices, and engineering standards that ensure the team delivers reliable, maintainable data infrastructure.
• Knowledge & Continuity: Serve as onshore escalation point and institutional knowledge backup for platform decisions, reducing single-point-of-failure risk and building onshore technical depth as the platform scales.

Business Problems You'll Solve

• Multi-Product Data Delivery: Implement data pipelines that serve multiple product lines (Transfer Pricing, R&D Services, RoyaltyStat, Provisioning) with distinct data requirements, ensuring each product gets the data it needs reliably and on schedule.
• Legacy Migration Execution: Lead pipeline implementation for migrating multi-terabyte datasets from legacy systems to Databricks, working within the architecture defined by the Lead Data Platform Engineer.
• Onshore Technical Leadership: Provide the senior judgment layer the current nearshore team cannot—owning problems end-to-end, making independent architectural decisions, and mentoring engineers to raise the quality bar across the team.
• Cross-Team Coordination: Bridge the gap between product teams and data infrastructure, translating business requirements into data solutions and ensuring the data platform delivers on product commitments.

Required Experience

Core Data Engineering

• SQL, Python, and PySpark—production pipeline implementation and performance optimization
• Databricks experience—Delta Lake, Workflows, and Databricks SQL; Unity Catalog familiarity preferred
• 5+ years in data engineering with demonstrated ability to own problems end-to-end without close direction
• Experience building and maintaining ETL/ELT pipelines at scale, including error handling, monitoring, and data quality validation
• Strong data modeling skills across structured and semi-structured data sources

Platform & Infrastructure

• AWS experience (S3, IAM, VPC) with ability to collaborate on infrastructure decisions
• Infrastructure-as-code experience (Terraform preferred)
• Familiarity with data governance patterns (Unity Catalog, data lineage, access controls)

Technical Leadership

• Demonstrated ability to exercise independent architectural judgment—not just ticket execution
• Experience mentoring or guiding junior and mid-level data engineers
• Strong written and verbal communication—able to document architecture decisions and engage directly with both technical and business stakeholders
• Onshore (US-based)—role requires timezone overlap, async-light communication, and direct stakeholder engagement

Preferred But Not Required

• Experience with financial data, accounting systems (NetSuite), or enterprise ERP platforms
• Background building pipelines that serve AI/ML workloads (preparing data for downstream ML consumption, RAG, and LLMs)
• Familiarity with data governance frameworks and compliance requirements for regulated industries
• Experience working alongside or transitioning from nearshore engineering teams