This role is for one of Weekday’s clients

Min Experience: 12+ years
Location: Bengaluru, Karnataka, India, Bengaluru
JobType: full-time

We are looking for an experienced Senior Staff Engineer – Data Platform with 12–17 years of experience to lead the architecture, design, and evolution of our core data platform. This role will be responsible for building scalable, reliable, and high-performance data infrastructure spanning lakehouse architecture, analytical pipelines, and transactional systems. The ideal candidate will possess deep expertise in Data Engineering and Databricks, along with a strong understanding of distributed systems, cloud infrastructure, data governance, and modern data architecture. As a senior technical leader, you will drive architectural decisions, establish engineering standards, mentor technical teams, and collaborate with cross-functional stakeholders to deliver robust data solutions that support business-critical applications and advanced analytics.

Requirements

Key Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform, ensuring scalability, reliability, performance, and long-term maintainability.
• Databricks Engineering: Design, develop, and optimize Databricks-based data solutions using Apache Spark, Delta Lake, and Lakehouse architecture principles.
• Data Pipeline Development: Build and maintain robust batch and streaming data pipelines to support analytics, reporting, machine learning, and operational use cases.
• Distributed Systems: Architect high-throughput, fault-tolerant data processing systems capable of handling large volumes of structured and unstructured data.
• Transactional Systems Integration: Design efficient data integration patterns between transactional databases, analytical platforms, and downstream data consumers.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, query execution, and resource utilization to improve system efficiency and control infrastructure costs.
• Data Governance and Quality: Establish standards for data quality, lineage, security, access control, metadata management, and regulatory compliance.
• Technical Leadership: Lead complex engineering initiatives, influence architectural decisions across teams, and establish best practices for data engineering and platform development.
• Cross-Functional Collaboration: Partner with product managers, analytics teams, data scientists, and engineering leaders to translate business requirements into scalable technical solutions.
• Mentorship and Innovation: Mentor senior engineers, conduct architecture reviews, promote engineering excellence, and evaluate emerging technologies to continuously improve platform capabilities.

Must-Have Skills

• Extensive experience in Data Engineering, data platform architecture, and large-scale data processing.
• Strong hands-on expertise in Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.
• Proven experience designing and operating scalable ETL/ELT pipelines and distributed data systems.
• Strong understanding of data modeling, data warehousing, batch processing, and real-time data streaming.
• Experience with cloud-based data platforms, preferably AWS, Microsoft Azure, or Google Cloud Platform.
• Proficiency in SQL and at least one programming language, such as Python or Scala.
• Strong knowledge of data security, governance, reliability, observability, and performance optimization.
• Demonstrated ability to lead architecture initiatives and influence technical strategy across multiple engineering teams.

Experience and Qualifications

• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Significant experience building and scaling enterprise-grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission-critical data systems.
• Excellent problem-solving, communication, stakeholder management, and mentoring skills.