About the Role

NewsBreak reaches tens of millions of users every day. Every feed ranking decision, every recommendation, every A/B test, and every ML model we ship runs on the data infrastructure you'd own. We're looking for a senior engineer to build and scale the batch and streaming pipelines at the core of that platform — someone who takes end-to-end ownership, drives reliability, and can translate ambiguous business needs into production-grade data systems.

Responsibilities

• Own the streaming data backbone — design and operate high-throughput Kafka pipelines carrying user events (clicks, reads, impressions, shares) from mobile clients through to downstream consumers: the data warehouse and real-time analytics.
• Build and maintain batch pipelines at scale — author and optimize Spark jobs processing billions of rows daily: content ingestion, engagement aggregation, and user-level feature computation. Own pipeline reliability, incremental backfill strategies, and cost per TB processed.
• Drive pipeline observability — instrument data quality checks, freshness monitors, and anomaly alerts so issues are caught before they reach dashboards. Define SLOs for data freshness and completeness, and own them.
• Model structured and unstructured data — design data models that serve analytics and product use cases. Act as the data POC across teams and be the person who pushes back when a schema decision will hurt us six months later.
• Partner with data scientists and platform engineers — understand data needs across teams and drive key data infrastructure decisions. Unblock downstream teams without becoming a ticket queue.
• Improve efficiency — reduce compute costs and job latency through query optimization, smarter partitioning, better resource scheduling, and tiered storage strategies.
• Raise the platform bar — contribute to shared infrastructure: pipeline framework standards, orchestration patterns (Airflow), reusable Spark libraries, and data catalog hygiene. Mentor junior engineers on the team

Requirements

• 5+ years of data engineering experience, with at least 3 years owning production pipelines in a distributed data environment.
• Strong hands-on experience with Spark and Kafka at scale — you've debugged a production incident in both, not just run tutorials.
• Experience with big-data technologies including Hadoop, Presto/Trino, and Flink.
• Proficiency in Python and SQL; Scala a plus.
• Experience working in a cloud environment, preferably AWS (S3, EMR, Glue, Redshift).
• Track record of improving efficiency, scalability, and stability of data systems — with measurable results.
• BS or MS in Computer Science or equivalent.
• Strong communication skills; comfortable driving data infrastructure decisions across application and platform teams.

Benefits

We offer a competitive benefits package:

• Health, dental, and vision care for you and your family (100% coverage for employee)
• Top-tier 401(K) plan with company matching
• Paid time off and paid holidays
• FSA, HSA and commuter benefits programs
• Team activity budget