Hadoop Jobs in Pune.
All active Hadoop roles based in Pune.
Pick a job to read the details
Tap any role on the left — its description and apply link will open here.
Senior Staff Software Engineer - Data
Share this job
DeepIntent is leading the healthcare advertising industry with data-driven solutions built for the future. From day one, our mission has been to improve patient outcomes through the artful use of advertising, data science, and real-world clinical data. For more information visit, www.DeepIntent.com.
What You’ll Do:
In this role, you will take the lead in designing, building, and optimizing data pipelines, ensuring high-quality data integration and real-time analytics. As a Senior Staff Engineer, you will mentor and guide a team of data engineers, collaborate with cross-functional teams, and drive best practices for data architecture and engineering.
- Leadership & Mentorship:
- Lead and mentor a team of engineers, ensuring technical excellence and continuous improvement.
- Provide guidance on complex technical issues, and foster a culture of collaboration and innovation.
- Develop and maintain data engineering processes, practices, and standards across the team.
- Data Architecture & Engineering:
- Design and develop robust, scalable, and efficient data pipelines for ETL (Extract, Transform, Load) processes.
- Architect data solutions and ensure seamless integration of data from diverse sources, both on-premise and in the cloud.
- Optimize data storage, retrieval, and processing for performance and scalability.
- Collaboration & Stakeholder Management:
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
- Collaborate with the software engineering team to ensure smooth integration of data pipelines into applications.
- Data Quality & Governance:
- Ensure data integrity, quality, and consistency across systems.
- Implement and enforce data governance policies and best practices, ensuring compliance with data privacy regulations.
- Continuous Improvement:
- Stay updated with the latest trends in data engineering, big data technologies, and cloud platforms.
- Drive automation, testing, and optimisation within data pipelines to improve overall efficiency.
- Technology Stack:
- Lead the selection and implementation of appropriate technologies for data storage, processing, and retrieval (e.g., Hadoop, Spark, Kafka, Airflow).
- Experience working with public clouds like GCP/AWS.
- Proficient with SQL, Java, Spring boot, Python or at least one Object oriented language, Bash.
- Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc. and big data databases like BigQuery, Clickhouse, etc.
Who You Are:
- Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field (or equivalent work experience).
- 8+ years of experience in data engineering or related roles, with at least 2 years in a leadership or senior technical position.
- Extensive experience with data warehousing, ETL pipelines, data modeling, and data architecture.
- Strong experience working with large-scale data processing frameworks (e.g., Hadoop, Apache Spark, Kafka).
- Proficient in SQL, Python, Java, or other relevant programming languages.
- Hands-on experience with cloud-based platforms (AWS, GCP, Azure).
- Familiarity with data orchestration tools like Apache Airflow or similar.
- Strong knowledge of databases (SQL and NoSQL) and data storage technologies.
- Excellent problem-solving and analytical skills.
- Strong communication and interpersonal skills, with the ability to work effectively with both technical and non-technical teams.
We believe great work starts with great support. That’s why DeepIntent offers a competitive, holistic benefits package designed to empower you both professionally and personally.
Here’s what you can expect when you join our team based in the US: Competitive base salary plus performance based bonus or commission, comprehensive medical, dental, and vision insurance, 401K match program, Unlimited PTO policy and paid holidays, remote friendly culture with flexible work options, career development and advanced education support, WFH and internet stipends, plus many more perks and benefits!
Here’s what you can expect when you join our team based in India: Competitive base salary plus performance based bonus, comprehensive medical insurance, and paid holidays. Hybrid-friendly culture with flexible work options, professional development reimbursement, WiFi reimbursement and health and wellness allowance plus many more perks and benefits!
Here's what you can expect when you join our team based in Europe: Competitive base salary plus performance-based bonus, comprehensive medical insurance, and flexible PTO. Hybrid-friendly culture with flexible work options, professional development reimbursement, WiFi reimbursement, and parental leave plus many more perks and benefits!
DeepIntent is committed to bringing together individuals from different backgrounds and perspectives. We strive to create an inclusive environment where everyone can thrive, feel a sense of belonging, and do great work together. DeepIntent is an Equal Opportunity Employer, providing equal employment and advancement opportunities to all individuals. We recruit, hire and promote into all job levels the most qualified applicants without regard to race, color, creed, national origin, religion, sex (including pregnancy, childbirth and related medical conditions), parental status, age, disability, genetic information, citizenship status, veteran status, gender identity or expression, transgender status, sexual orientation, marital, family or partnership status, political affiliation or activities, military service, immigration status, or any other status protected under applicable federal, state and local laws. If you have a disability or special need that requires accommodation, please let us know in advance. DeepIntent’s commitment to providing equal employment opportunities extends to all aspects of employment, including job assignment, compensation, discipline and access to benefits and training.
CCPA Notice: If you are a California resident applying for a role at DeepIntent, we may collect personal information as part of the application process in accordance with the California Consumer Privacy Act (CCPA). To learn more about the categories of information we collect and your rights, please contact PeopleOps@deepintent.com to see our full Applicant Privacy Notice.
Ready to apply?
Apply to DeepIntent
Overview of 66degrees
66degrees is an end-to-end AI transformation partner that guides enterprises from complex business challenges to clear, quantifiable outcomes. Our company is the culmination of several successful firms, each a leader in its own right in cloud, artificial intelligence, and data. This convergence of talent and expertise is how we help businesses reach their own "inflection point," where chaotic data becomes a strategic asset, complexity becomes clarity, and AI becomes an engine for growth. Our ultimate vision is to be the catalyst for a future where every business operates as an intelligent enterprise, with autonomous systems unlocking human potential.
At 66degrees, we believe in thriving through challenges and winning together. These values not only guide us in achieving our goals as a company but also for our people. We are dedicated to creating a significant impact for our employees by fostering a culture that sparks innovation and supports professional and personal growth along the way.
Overview of Role
We are seeking a highly skilled Cloud Data Engineer to join our Managed Services team. In this role, you will be the technical backbone for our enterprise clients, responsible for designing, building, and maintaining high-performance data platforms using Snowflake and Azure Databricks. You will act as a trusted advisor, ensuring that our customers' data ecosystems are scalable, secure, and cost-optimized across multi-cloud environments.
Key Responsibilities
- Platform Engineering: Design and implement end-to-end data lakehouse architectures using Azure Databricks (Delta Lake),Azure HDInsights and Snowflake Cloud Data Warehouse.
- Pipeline Development: Build scalable, metadata-driven ETL/ELT pipelines using Azure Data Factory (ADF), PySpark, and SnowSQL, including ingestion workflows from HDInsight-managed Hadoop and Kafka clusters.
- Managed Services Support: Provide Tier-3 technical support for client environments, including performance tuning, complex bug fixes, and architectural optimizations.
- Multi-Cloud Integration: Oversee data movement and integration strategies across Azure, AWS, and GCP to ensure seamless "Data Everywhere" capabilities for clients.
- Cost & Performance Governance: Monitor and optimize Snowflake credit consumption and Databricks cluster usage to ensure clients receive maximum ROI.
- Security & Compliance: Implement enterprise-grade security including Role-Based Access Control (RBAC), Row-Level Security (RLS), and data masking (Dynamic Data Masking in Snowflake / Unity Catalog in Databricks).
Required Technical Skillset
|
Category |
Must-Have Skills |
|
Snowflake |
Snowpipe, Streams & Tasks, Dynamic Tables, Zero-Copy Cloning, and Snowpark (Python/Java), Data Sharing. |
|
Azure Databricks |
Spark SQL, PySpark, Delta Lake, Unity Catalog, and Medallion Architecture (Bronze/Silver/Gold) |
|
Orchestration |
Azure Data Factory, Airflow, or dbt (Data Build Tool). |
|
Cloud Infrastructure |
Azure Data Lake Storage (ADLS Gen2), Azure DevOps (CI/CD), and Terraform/Bicep for IaC. |
|
Languages |
Advanced SQL (Window functions, CTEs) and Python. |
Experience & Qualifications
- Experience: 5+ years in Data Engineering, with at least 3 years of hands-on experience in both Snowflake and Databricks.
- Managed Services Background: Proven experience supporting external customers with strict SLAs and complex governance requirements.
- Multi-Cloud (Plus): Direct experience deploying Snowflake or Databricks on AWS or GCP is highly desirable.
- Certifications (Preferred): * SnowPro Core or Advanced Architect
- Databricks Certified Data Engineer Professional
- Microsoft Certified: Azure Data Engineer Associate (DP-203)
66degrees is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to actual or perceived race, color, religion, sex, gender, gender identity, national origin, age, weight, height, marital status, sexual orientation, veteran status, disability status or other legally protected class.
AI Transparency & Disclosure
As an AI transformation partner, 66degrees leverages intelligent solutions to enhance our recruitment experience. We utilize AI tools—including LinkedIn Recruiter’s Hiring Assistant and interview transcription technologies—to assist with sourcing, role analysis, and capturing interview highlights.
These tools augment our process, but we "Commit to Our Craft" by ensuring all final hiring decisions are made by our human Talent Team. By applying, you acknowledge the use of these technologies to help us "Win Together" in finding the best fit for our team.
Ready to apply?
Apply to 66degrees
Job Title: Data Tester
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
MAKE AN IMPACT
Ready to apply?
Apply to CapcoAbout Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
MAKE AN IMPACT
Location: Pune
Experience Range: 5–9 years
Role Overview:
The Senior QA/Data Tester validates the functional and technical integrity of data pipelines across enterprise-scale Big Data environments. The role focuses on SQL validation, Spark testing, automation, and ensuring high-quality data delivery.
Key Skills:
• 5–9 years of QA/testing experience in Data Warehouse or Data Engineering projects
• Strong SQL expertise (Oracle/Hive) for complex data validation
• Hands-on experience in Python, Unix, and shell scripting
• Experience testing Spark/Scala applications on Hadoop or object storage
• Exposure to automation frameworks for Big Data environments
• NiFi experience (preferred)
• Strong analytical, communication, and coordination skills
• Agile delivery experience
Ready to apply?
Apply to Capco
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
MAKE AN IMPACT
Location: Pune
Experience Range: 5–9 years
Role Overview:
The Senior Data Engineer designs and develops complex data pipelines and contributes to end-to-end solution delivery across enterprise-scale Big Data environments. The role ensures data reliability, performance, and adherence to engineering best practices.
Must-Have Skills:
• 5–9 years in Data Engineering across Data Warehouse/Data Lake/Lakehouse environments
• Strong Spark development (Scala/Python/Java) on Hadoop or object storage
• Strong SQL capabilities (Oracle/Netezza/Hive or similar)
• Experience delivering large-scale data pipelines
• Agile/Scrum delivery experience
Optional Skills:
• Experience building NiFi pipelines
• Strong debugging and RCA skills
• Ability to work independently across multiple priorities
• Strong communication and collaboration abilities
Ready to apply?
Apply to Capco
Solutions Architect : Data & AI
Share this job
FEQ425R230
As a Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible data-driven outcomes through the use of our Databricks Lakehouse Platform, helping data teams complete projects and integrate our platform into their enterprise Ecosystem. You'll grow as a leader in your field, while finding solutions to our customers' biggest challenges in big data, analytics, data engineering and data science problems.
The impact you will have:
- You will be a Big Data Analytics expert on aspects of architecture and design
- Lead your clients through evaluating and adopting Databricks including hands-on Apache Spark™ programming and integration with the wider cloud ecosystem
- Support your customers by authoring reference architectures, how-tos, and demo applications
- Integrate Databricks with 3rd-party applications to support customer architectures
- Engage with the technical community by leading workshops, seminars and meet-ups
- Together with your Account Executive, you will form successful relationships with clients throughout your assigned territory to provide technical and business value
What we look for:
- Strong consulting / customer facing experience , working with external clients across a variety of industry markets
- Core strength in either data engineering or data science technologies
- 8+ years of experience demonstrating technical concepts, including demos, presenting and white-boarding
- 8+ years of experience designing architectures within a public cloud (AWS, Azure or GCP)
- 6+ years of experience with Big Data technologies, including Apache Spark™, AI, Data Science, Data Engineering, Hadoop, Cassandra, and others
- Coding experience in Python, R, Java, Apache Spark™ or Scala
About Databricks
Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook.
Benefits
At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here.
Our Commitment to Diversity and Inclusion
At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.
Compliance
If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.
Ready to apply?
Apply to Databricks