• Design, build, and operate resilient batch and real-time streaming data ingestion pipelines connecting to external financial market data providers via APIs and WebSockets.

• Lead the architectural migration of legacy backend infrastructure to modern cloud-native environments on AWS, establishing high standards for system reliability and scalability.

• Drive the refactoring and rewrite of the CTA positioning indicator calculation blocks from PHP to Python, ensuring strict mathematical output parity, comprehensive test coverage, and optimized latency.

• Ingest and model complex financial datasets—including futures pricing, open interest, and trading volumes—across time-series stores and search engines like QuestDB and Elasticsearch.

• Implement automated data validation, proactive alerting, and system monitoring to ensure market data feeds remain accurate, continuous, and fault-tolerant.

• Deploy, maintain, and scale containerized services across Docker and Kubernetes environments using automated CI/CD pipelines and GitOps workflows.

• Partner closely with quantitative researchers, commodity analysts, and product managers to translate complex trading models into high-performance production pipelines.

• Provide technical mentorship to peer engineers, raising team-wide engineering rigor through thoughtful code reviews, system design guidance, and architectural documentation.

What you'll need (Must-haves)

• 5+ years of professional experience building, operating, and scaling production-grade data pipelines and backend infrastructure.

• Advanced expertise in Python, with a strong background refactoring and rewriting production-critical code for performance and testability.

• Proven hands-on experience building and maintaining real-time or streaming API ingestion pipelines (REST, WebSockets) handling time-series datasets.

• Solid production experience deploying and managing containerized applications on AWS using Docker, Kubernetes, and modern CI/CD practices.

• Strong database fundamentals in SQL and experience working with distributed search or time-series databases (e.g., Elasticsearch, QuestDB, ClickHouse).

• Deep experience establishing automated data quality frameworks, observability (monitoring/alerting), and root-cause troubleshooting for production systems.

Nice-to-haves

• Domain knowledge of financial market structures, futures contracts, open interest, and commodities trading symbology.

• Experience integrating data from major financial market vendors (e.g., Bloomberg, LSEG, Enverus, Trading Technologies).

• Familiarity with legacy PHP codebase migration or reading PHP algorithms to translate into Python.

• Practical familiarity with GitOps frameworks and infrastructure-as-code tooling.