Über diese Lead Data Engineering - Product Engineering & Delivery Stelle bei Fractal
It's fun to work in a company where people truly BELIEVE in what they are doing!
We're committed to bringing passion and customer focus to the business.
Role Overview
Cogentiq I2C is Fractal’s agentic AI Product for invoice-to-cash, covering Collections, Cash Application, Deductions, Invoice Management and Credit Risk. It runs on a Next.js front end, a FastAPI service layer, the Cogentiq agentic runtime, and a data tier on Azure Databricks with PostgreSQL.
We are looking for a hands-on Data Engineering Leader with strong experience in building scalable, configurable, and cloud-agnostic data engineering solutions. The role requires a combination of deep technical expertise, product engineering mindset, and proven ownership of enterprise client deliveries.
A critical requirement is hands-on experience across at least two cloud data engineering platforms, with the ability to design portable PySpark-based pipelines and migrate workloads between platforms with minimal rework
.
Key Responsibilities
- Data Engineering & Architecture: Design and develop scalable ETL/ELT pipelines, distributed data processing frameworks, Lakehouse solutions, and Medallion architectures (Bronze/Silver/Gold).
- Cloud-Agnostic Engineering: Build reusable, metadata-driven, configuration-based data pipelines that can operate across different cloud environments and processing platforms.
- Multi-Platform Portability: Architect and migrate PySpark workloads across platforms such as Databricks, Microsoft Fabric, AWS Glue/EMR, and GCP Dataproc, managing platform-specific dependencies and optimizations.
- Product Engineering: Develop reusable frameworks, connectors, orchestration capabilities, and accelerators that support multiple enterprise clients with minimal customization.
- Enterprise Integration: Design upstream and downstream integrations with SAP ECC/S/4HANA and other enterprise systems using APIs, BAPIs, IDocs, OData, CDC, and event-driven interfaces.
- Data Governance: Implement data quality, cataloging, metadata management, lineage, security, and reconciliation capabilities.
- DevOps & DataOps: Establish CI/CD, automated testing, infrastructure automation, observability, monitoring, and production support practices.
- Engineering & Delivery Leadership: Lead engineering teams, own architecture and technical decisions, drive end-to-end client delivery, and ensure engineering quality, scalability, and production readiness.
Required Skills & Experience
- 12+ years of engineering experience, including significant hands-on Data Engineering experience and technical leadership.
- Strong hands-on expertise in Python, PySpark, Spark, SQL, distributed processing, and performance optimization.
- Hands-on experience with at least TWO data engineering platforms, such as: Azure DatabricksMicrosoft FabricAWS Glue / EMRGoogle Cloud Dataproc / Dataflow
- Demonstrated ability to migrate or adapt PySpark pipelines between platforms, understanding differences in execution environments, storage, orchestration, dependencies, and deployment.
- Experience building configurable, metadata-driven ETL/ELT frameworks, reusable components, and cloud-agnostic data pipelines.
- Strong understanding of Lakehouse architecture, Delta Lake, Medallion Architecture, data modeling, cataloging, lineage, and data quality.
- Experience with enterprise systems integration, preferably SAP (upstream ingestion and downstream write-back).
- Experience with DevOps/DataOps, CI/CD, automated testing, and production deployments.
- Proven experience in product engineering, engineering team leadership, and end-to-end enterprise client delivery.
Good-to-Have
- Experience with additional data platforms, including Snowflake, BigQuery, or Azure Synapse.
- Experience with data governance tools such as Unity Catalog, Microsoft Purview, AWS Glue Data Catalog, or Collibra.
- Experience with streaming pipelines, Kafka, CDC, and event-driven architectures.
- Exposure to ML/AI data pipelines using SageMaker, Vertex AI, or Databricks ML.
- Experience building reusable enterprise data products or accelerators for multiple clients.
Ideal Candidate
A hands-on engineering leader, not just a delivery manager or solution architect, who has built production-grade data pipelines, worked across multiple data engineering platforms, understands how to make PySpark workloads portable, and can lead both product development and complex enterprise client deliveries.
Critical Hiring Criteria: Strong PySpark engineering + hands-on experience across at least two DE platforms + configurable pipeline development + engineering delivery ownership.
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
Not the right fit? Let us know you're interested in a future opportunity by clicking Introduce Yourself in the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest!