🚀 Senior Azure Data Engineer – Mindera

At Mindera, we build technology products that create meaningful business impact. We work with modern cloud technologies and encourage engineering ownership, collaboration, continuous learning, and self-organization.

Our Data Engineering teams build scalable, reliable, and high-performance data platforms that enable analytics, reporting, machine learning, and data-driven decision-making.

Requirements

🧠 What We’re Looking For

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows using Azure Databricks.
• Build data processing solutions using Apache Spark and PySpark for large-scale datasets.
• Develop and optimize batch and real-time/streaming data pipelines.
• Implement modern Medallion Architecture using Bronze, Silver, and Gold data layers.
• Work extensively with Delta Lake for reliable, scalable data storage and processing.
• Develop complex and optimized SQL queries for data transformation and analytics.
• Integrate data from multiple sources including databases, APIs, files, cloud storage, and streaming platforms.
• Work with Azure services such as ADLS Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure Monitor.
• Optimize Spark workloads through effective partitioning, caching, joins, cluster configuration, and file-format optimization.
• Implement data quality, validation, monitoring, logging, and observability across data pipelines.
• Apply appropriate data modelling and dimensional modelling practices for analytics workloads.
• Implement CI/CD, source control, automated testing, and deployment practices for data engineering solutions.
• Contribute to architecture and technology decisions for modern cloud data platforms.
• Troubleshoot production data pipelines and proactively improve reliability, performance, and cost efficiency.
• Collaborate with Data Scientists, Data Analysts, Software Engineers, Product Managers, and business stakeholders.
• Mentor junior and mid-level engineers and contribute to engineering best practices.
• Create technical documentation covering data pipelines, architecture, processes, and operational procedures.

Must-Have Skills

• 7+ years of professional experience in Data Engineering.
• Strong hands-on experience with Azure Databricks.
• Excellent knowledge of Apache Spark / PySpark.
• Strong programming experience in Python.
• Advanced SQL skills.
• Hands-on experience building scalable ETL/ELT data pipelines.
• Strong understanding of Delta Lake and Medallion Architecture.
• Experience with Azure Data Lake Storage Gen2 (ADLS).
• Experience with Azure Data Factory or equivalent orchestration tools.
• Strong understanding of data warehousing, data lakes, and data modelling.
• Experience with batch processing and preferably streaming data processing.
• Hands-on experience with Git and CI/CD practices.
• Experience working in Agile/Scrum or similar engineering environments.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent communication and stakeholder-management skills.

Good-to-Have Skills

• Experience with Unity Catalog and Databricks governance.
• Experience with Azure Event Hubs, Kafka, or Spark Structured Streaming.
• Experience with dbt or similar transformation frameworks.
• Experience with Terraform / Infrastructure as Code.
• Knowledge of Azure Synapse Analytics.
• Experience with data quality and observability frameworks.
• Understanding of security, access control, lineage, and governance.
• Experience with Databricks Workflows / Jobs and automated deployment.
• Exposure to machine learning or AI data pipelines.
• Experience optimizing cloud infrastructure and data-processing costs.

What We Look For

• Strong ownership and ability to work independently.
• A problem-solving mindset with attention to engineering quality.
• Ability to design solutions rather than simply implement requirements.
• Clear communication with technical and non-technical stakeholders.
• Willingness to mentor others and share knowledge.
• Curiosity and enthusiasm for learning new technologies.
• Ability to work effectively in a collaborative, self-organized engineering environment.

🌟 Nice to Have

• Experience working with feature stores and ML pipelines
• Exposure to streaming technologies (Kafka, Kinesis, etc.)
• Understanding of data governance and data quality frameworks
• Experience with LLM-based applications.

Benefits

We offer

• Flexible working hours (self-managed)
• Annual bonus, subject to company performance
• Access to Udemy online training and opportunities to learn and grow within the role

At Mindera we use technology to build products we are proud of, with people we love.

Software Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera.

We partner with our clients, to understand their products and deliver high-performance, resilient and scalable software systems that create an impact on their users and businesses across the world.

You get to work with a bunch of great people, and the whole team owns the project together.

Our culture reflects our lean and self-organisation attitude.

We encourage our colleagues to take risks, make decisions, work in a collaborative way and talk to everyone to enhance communication. We are proud of our work and we love to learn all and everything while navigating through an Agile, Lean and collaborative environment.

Our offices are located: Porto, Portugal | Aveiro, Portugal | Coimbra, Portugal | Leicester, UK | San Diego, USA | Chennai, India | Bengaluru, India