About the Team

Come help us build the world's most reliable on-demand, logistics engine for delivery! We're bringing on experienced engineers to help us further our 24x7, global infrastructure system that powers DoorDash’s three-sided marketplace of consumers, merchants, and dashers.

About the Role

The Data User Platform and Programs team works at the intersection of data infrastructure, reliability, governance, developer tooling and AI systems, supporting thousands of internal data users including engineers, analysts, data scientists and business stakeholders.

This role is part of a founding engineering team in India, offering high ownership, strong growth opportunities, and direct influence over the platform’s long-term architecture. 

You will work on systems such as:

• Platform foundations for reliability, observability, cost-efficiency and compliance
• A unified data discovery and catalog experience
• AI-powered assistants and workflows for analytics and operations

You will collaborate closely with product managers, data engineers, and analytics teams to deliver solutions that reduce friction and increase trust across the data ecosystem.

You’re excited about this opportunity because you will…

• Improve scalability, performance, and reliability of data platform services through strong engineering practices
• Build and operate data discovery and metadata platforms that help users find the right datasets, dashboards, and metrics quickly and confidently
• Develop AI-enabled services and workflows, including self-serve tooling for creating, evaluating, and improving AI assistants
• Add observability, monitoring, and operational tooling to maintain platform SLAs and reduce incident response time
• Drive cost-efficient data platform design by identifying underutilized, redundant, or idle datasets, services, and compute resources
• Embed cost awareness into the data lifecycle, including:
• Designing systems that encourage reuse over duplication
• Instrumenting usage and cost signals into platform services
• Supporting safe deprecation and cleanup of stale data assets
• Partner with cross-functional teams in a fast-moving, product-driven environment

We’re excited about you because…

• B.E./B.Tech., M.E./M.Tech, or Ph.D. in Computer Science or equivalent
• 6+ years of experience with CS fundamental concepts and experience with at least one of the programming languages of Go, Java, and Python
• Prior technical experience in Big Data infrastructure, Cost Efficiency & Governance, you've built meaningful pieces of data infrastructure. Bonus if those were open-sourced technologies like Spark, Clickhouse, Iceberg and Airflow
• Experience improving efficiency, scalability, and stability of data platforms
• Experience building or operating backend systems in production environments

Notice to Applicants for Jobs Located in NYC or Remote Jobs Associated With Office in NYC Only

We use Covey as part of our hiring and/or promotional process for jobs in NYC and certain features may qualify it as an AEDT in NYC. As part of the hiring and/or promotion process, we provide Covey with job requirements and candidate submitted applications. We began using Covey Scout for Inbound from August 21, 2023, through December 21, 2023, and resumed using Covey Scout for Inbound again on June 29, 2024.

The Covey tool has been reviewed by an independent auditor. Results of the audit may be viewed here: Covey

About the Team

Come help us build the world's most reliable on-demand, logistics engine for delivery! We're bringing on experienced engineers to help us further our 24x7, global infrastructure system that powers DoorDash’s three-sided marketplace of consumers, merchants, and dashers.

About the Role

The Data User Platform and Programs team works at the intersection of data infrastructure, reliability, governance, developer tooling and AI systems, supporting thousands of internal data users including engineers, analysts, data scientists and business stakeholders.

This role is part of a founding engineering team in India, offering high ownership, strong growth opportunities, and direct influence over the platform’s long-term architecture. 

You will work on systems such as:

• Platform foundations for reliability, observability, cost-efficiency and compliance
• A unified data discovery and catalog experience
• AI-powered assistants and workflows for analytics and operations

You will collaborate closely with product managers, data engineers, and analytics teams to deliver solutions that reduce friction and increase trust across the data ecosystem.

You’re excited about this opportunity because you will…

• Improve scalability, performance, and reliability of data platform services through strong engineering practices
• Build and operate data discovery and metadata platforms that help users find the right datasets, dashboards, and metrics quickly and confidently
• Develop AI-enabled services and workflows, including self-serve tooling for creating, evaluating, and improving AI assistants
• Add observability, monitoring, and operational tooling to maintain platform SLAs and reduce incident response time
• Drive cost-efficient data platform design by identifying underutilized, redundant, or idle datasets, services, and compute resources
• Embed cost awareness into the data lifecycle, including:
• Designing systems that encourage reuse over duplication
• Instrumenting usage and cost signals into platform services
• Supporting safe deprecation and cleanup of stale data assets
• Partner with cross-functional teams in a fast-moving, product-driven environment

We’re excited about you because…

• B.E./B.Tech., M.E./M.Tech, or Ph.D. in Computer Science or equivalent
• 3+ years of experience with CS fundamental concepts and experience with at least one of the programming languages of Go, Java, and Python
• Prior technical experience in Big Data infrastructure, Cost Efficiency & Governance, you've built meaningful pieces of data infrastructure. Bonus if those were open-sourced technologies like Spark, Clickhouse, Iceberg and Airflow
• Experience improving efficiency, scalability, and stability of data platforms
• Experience building or operating backend systems in production environments

Notice to Applicants for Jobs Located in NYC or Remote Jobs Associated With Office in NYC Only

We use Covey as part of our hiring and/or promotional process for jobs in NYC and certain features may qualify it as an AEDT in NYC. As part of the hiring and/or promotion process, we provide Covey with job requirements and candidate submitted applications. We began using Covey Scout for Inbound from August 21, 2023, through December 21, 2023, and resumed using Covey Scout for Inbound again on June 29, 2024.

The Covey tool has been reviewed by an independent auditor. Results of the audit may be viewed here: Covey

About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 7+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

Role: GCP Data Engineer 

Job Summary:

We are looking for an enthusiastic GCP Data Engineer to join our dynamic and growing project team. In this role, you will have the opportunity to develop your skills while contributing to large-scale data projects on Google Cloud Platform. You will work closely with senior engineers and data analysts to build and maintain robust data pipelines, helping to turn raw data into actionable insights. This is an excellent opportunity for a foundational career in data engineering within a supportive and collaborative environment.

Key Responsibilities:

· Develop, test, and maintain data pipelines and ETL/ELT processes using Python, SQL, and GCP services.

· Write and optimize complex SQL queries in BigQuery for data transformation, extraction, and analysis.

· Assist in the design and implementation of data models within our data warehouse.

· Collaborate with the broader project team to understand data requirements and deliver effective solutions.

· Troubleshoot and resolve issues in data pipelines and data quality to ensure accuracy and availability.

· Contribute to documentation and adhere to data engineering best practices.

· Support the team in various data-related tasks and be eager to learn new technologies and techniques.

Required Qualifications:

· Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

· 5-8 years of hands-on experience with strong programming skills in Python for data manipulation and processing.

· Proficiency in SQL for querying and data transformation.

· Hands-on experience or significant project work with Google Cloud Platform (GCP).

· Familiarity with core GCP data services, particularly BigQuery and Pub/Sub.

· Strong understanding of data warehousing concepts and ETL/ELT processes.

· A proactive and curious mindset with strong problem-solving skills.

· Excellent communication and teamwork skills, with the ability to work effectively in a large project team.

Preferred Qualifications:

· Experience with other GCP services such as Cloud Composer (Airflow) or Dataflow, Cloud Functions.

· Familiarity with version control systems, especially Git.

· Google Cloud certification (e.g., Associate Cloud Engineer) is a plus.

· A passion for data and a strong desire to grow a career in the data engineering field.

About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day :

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team :

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 5+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

What You’ll Do:

In this role, you will take the lead in designing, building, and optimizing data pipelines, ensuring high-quality data integration and real-time analytics. As a Senior Staff Engineer, you will mentor and guide a team of data engineers, collaborate with cross-functional teams, and drive best practices for data architecture and engineering.

• Leadership & Mentorship:
  • Lead and mentor a team of engineers, ensuring technical excellence and continuous improvement.
  • Provide guidance on complex technical issues, and foster a culture of collaboration and innovation.
  • Develop and maintain data engineering processes, practices, and standards across the team.
• Data Architecture & Engineering:
  • Design and develop robust, scalable, and efficient data pipelines for ETL (Extract, Transform, Load) processes.
  • Architect data solutions and ensure seamless integration of data from diverse sources, both on-premise and in the cloud.
  • Optimize data storage, retrieval, and processing for performance and scalability.
• Collaboration & Stakeholder Management:
  • Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
  • Collaborate with the software engineering team to ensure smooth integration of data pipelines into applications.
• Data Quality & Governance:
  • Ensure data integrity, quality, and consistency across systems.
  • Implement and enforce data governance policies and best practices, ensuring compliance with data privacy regulations.
• Continuous Improvement:
  • Stay updated with the latest trends in data engineering, big data technologies, and cloud platforms.
  • Drive automation, testing, and optimisation within data pipelines to improve overall efficiency.
• Technology Stack:
  • Lead the selection and implementation of appropriate technologies for data storage, processing, and retrieval (e.g., Hadoop, Spark, Kafka, Airflow).
  • Experience working with public clouds like GCP/AWS.
  • Proficient with SQL, Java, Spring boot, Python or at least one Object oriented language, Bash.
  • Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc. and big data databases like BigQuery, Clickhouse, etc.

Who You Are:

• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field (or equivalent work experience).
• 8+ years of experience in data engineering or related roles, with at least 2 years in a leadership or senior technical position.
• Extensive experience with data warehousing, ETL pipelines, data modeling, and data architecture.
• Strong experience working with large-scale data processing frameworks (e.g., Hadoop, Apache Spark, Kafka).
• Proficient in SQL, Python, Java, or other relevant programming languages.
• Hands-on experience with cloud-based platforms (AWS, GCP, Azure).
• Familiarity with data orchestration tools like Apache Airflow or similar.
• Strong knowledge of databases (SQL and NoSQL) and data storage technologies.
• Excellent problem-solving and analytical skills.
• Strong communication and interpersonal skills, with the ability to work effectively with both technical and non-technical teams.

Strategic Project Lead / Delivery Manager 

Data Annotation & AI Delivery

8+ Years of Experience

Position Overview

We are looking for a seasoned Delivery Manager (8+ years of experience) to lead large-scale data annotation and curation projects in support of cutting-edge LLM and multimodal AI research. You will oversee end-to-end delivery for projects with 100+ team members, ensure world-class data quality, and drive process and tooling innovation in collaboration with AI researchers and cross-functional teams.

This role is ideal for someone who combines deep technical expertise, proven leadership in managing large teams, and a passion for advancing AI/ML innovation. Given that our key customers operate in the data analytics, data engineering, and Business Intelligence (BI) space, the ideal candidate will bring hands-on leadership experience with platforms such as Databricks and Snowflake, enabling them to deeply understand customer domains, speak their language, and deliver annotation outcomes that are contextually relevant and technically sound.

Primary Responsibilities

1. Team Leadership & Development

• Lead, mentor, and grow large-scale teams of annotators and trainers.
• Drive performance management, career development, and targeted training.
• Foster a culture of excellence, ownership, and continuous learning.
• Design and implement motivation strategies (including gamification) to boost engagement and reward top performers.

1. Project Management & Coordination

• Define and manage project scope, objectives, timelines, and quality metrics.
• Align with cross-functional stakeholders on goals and deliverables.
• Proactively identify and resolve delivery risks, blockers, and dependencies.
• Provide clear and consistent reporting on progress, challenges, and solutions.

1. Data Quality & Integrity

• Own quality control and data integrity across the annotation lifecycle.
• Analyze datasets to identify trends, anomalies, and insights for improvement.
• Implement best practices for annotation, evaluation, and data curation.
• Share learnings and continuously improve quality processes.

1. Technical Collaboration & Innovation

• Partner with engineering and research teams to optimize internal tooling.
• Stay ahead of emerging trends in AI, ML, and multimodal processing.
• Champion innovations that reduce task completion times and improve efficiency.
• Maintain clear and scalable documentation for processes and training.

1. Customer & Domain Engagement

• Serve as a domain-knowledgeable point of contact for customers operating in data engineering, analytics, and BI environments.
• Leverage hands-on experience with platforms like Databricks and Snowflake to understand customer data workflows, terminology, and quality expectations.
• Translate customer-specific data engineering and BI contexts into clear, actionable annotation guidelines and quality benchmarks for delivery teams.
• Collaborate with customers to define annotation taxonomies and evaluation criteria that align with their data pipeline and analytics use cases.

Required Skills & Qualifications

• 10+ years of experience in engineering, program management, or delivery of large-scale projects.
• Proven ability to lead and scale large teams (100+ members) with measurable impact.
• Strong technical grounding in AI/ML, data annotation, and multimodal systems.
• Hands-on leadership experience in data analytics, data engineering, or BI domains, with working knowledge of platforms such as Databricks and/or Snowflake (architecture, pipelines, data modeling, and production workflows).
• Familiarity with the modern data stack including ETL/ELT pipelines, data warehouses, data lakes, and BI tools (Power BI, Tableau, Looker, or similar).
• Ability to translate complex data engineering and analytics concepts into annotation frameworks, quality standards, and training materials.
• Proficiency in Python (preferred) or Swift for data analysis and automation.
• Strong mathematical and analytical skills for data-driven problem solving.
• Experience in data curation, annotation pipelines, and quality control.
• Excellent communication and stakeholder management skills.
• Track record of driving high-quality outputs under tight deadlines.

Preferred Skills & Qualifications

• Advanced degree in Computer Science, Data Science, AI, or related fields.
• Experience applying gamification to workforce engagement.
• Familiarity with modern project management tools & methodologies.
• Exposure to state-of-the-art multimodal AI technologies.
• Prior experience managing annotation or data labeling projects for customers in the data engineering, analytics, or BI space.
• Familiarity with SQL at an advanced level and data orchestration tools such as Apache Airflow, dbt, or Prefect.
• Certifications in Databricks (e.g., Data Engineer Associate) or Snowflake (e.g., SnowPro Core) are a plus.

Overview of Role

We are seeking a highly skilled Cloud Data Engineer to join our Managed Services team. In this role, you will be the technical backbone for our enterprise clients, responsible for designing, building, and maintaining high-performance data platforms using Snowflake and Azure Databricks. You will act as a trusted advisor, ensuring that our customers' data ecosystems are scalable, secure, and cost-optimized across multi-cloud environments.

Key Responsibilities

• Platform Engineering: Design and implement end-to-end data lakehouse architectures using Azure Databricks (Delta Lake),Azure HDInsights and Snowflake Cloud Data Warehouse.
• Pipeline Development: Build scalable, metadata-driven ETL/ELT pipelines using Azure Data Factory (ADF), PySpark, and SnowSQL, including ingestion workflows from HDInsight-managed Hadoop and Kafka clusters.
• Managed Services Support: Provide Tier-3 technical support for client environments, including performance tuning, complex bug fixes, and architectural optimizations.
• Multi-Cloud Integration: Oversee data movement and integration strategies across Azure, AWS, and GCP to ensure seamless "Data Everywhere" capabilities for clients.
• Cost & Performance Governance: Monitor and optimize Snowflake credit consumption and Databricks cluster usage to ensure clients receive maximum ROI.
• Security & Compliance: Implement enterprise-grade security including Role-Based Access Control (RBAC), Row-Level Security (RLS), and data masking (Dynamic Data Masking in Snowflake / Unity Catalog in Databricks).

Required Technical Skillset

Experience & Qualifications

• Experience: 5+ years in Data Engineering, with at least 3 years of hands-on experience in both Snowflake and Databricks.
• Managed Services Background: Proven experience supporting external customers with strict SLAs and complex governance requirements.
• Multi-Cloud (Plus): Direct experience deploying Snowflake or Databricks on AWS or GCP is highly desirable.
• Certifications (Preferred): * SnowPro Core or Advanced Architect
• Databricks Certified Data Engineer Professional
• Microsoft Certified: Azure Data Engineer Associate (DP-203)

About HG Insights

Headquartered in Santa Barbara, California, HG Insights is the global leader in technology intelligence. We help the world’s most innovative companies accelerate their go-to-market efforts with precision through advanced data science methodologies and proprietary data assets. We offer a culture that blends innovation, collaboration, and growth, where each team member is empowered to make a measurable impact.

Role Overview

We are looking for a Staff/Senior Machine Learning Engineer to join our growing AI and Data Platform team in Pune, India. In this role, you will be responsible for designing, building, and scaling ML systems that power our core data intelligence products. You’ll work at the intersection of data engineering and machine learning, collaborating closely with data scientists, software engineers, and product teams to turn models and agents into robust, production-ready systems.

This is a high-impact role suited for someone who thrives on ownership, scalability, and deploying real-world ML and AI solutions at scale.

Key Responsibilities

ML & AI System Development

• Design and build end-to-end ML pipelines, from data ingestion and feature engineering to model training, serving, and monitoring.
• Productionize AI agents, including LLM-backed workflows, multi-step tool use, and API orchestration.
• Collaborate with data scientists to bring experimental models into stable, high-performing production systems.
  
  

MLOps & Infrastructure

• Implement and maintain MLOps workflows (CI/CD for ML), tracking experiments, managing model versions, and automating retraining.
• Set up observability for models and agents, including tracing, logging, latency/error tracking, and fallback logic.
• Ensure infrastructure supports scalability, performance, and compliance requirements.
  
  

Technical Leadership & Collaboration

• Lead technical architecture for key AI and ML systems, ensuring maintainability and extensibility.
• Partner with product and platform teams to translate business needs into reliable and performant ML solutions.
• Mentor junior engineers and help elevate technical practices across the engineering org.

Minimum Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 8+ years of industry experience, with at least 4+ years in ML engineering or MLOps.
• Strong programming skills in Python and experience with ML libraries such as PyTorch, TensorFlow, and scikit-learn.
• Experience deploying AI agents in production, including retrieval-augmented generation (RAG), multi-step reasoning, and tool-use orchestration.
• Familiarity with LangChain, LlamaIndex, Haystack, or similar frameworks for agent development.
• Hands-on with LLMs (OpenAI, Anthropic, Cohere, etc.), embedding models, and vector store integration.
• Expertise in MLOps tools like MLflow, Airflow, SageMaker, or Kubeflow.
• Experience with cloud platforms (preferably AWS), Docker/Kubernetes, and large-scale data systems (e.g., Snowflake, Databricks).
• Excellent problem-solving and system design skills, including CI/CD, testing, and infrastructure-as-code.
  
  

Preferred Qualifications

• Experience with streaming pipelines, online inference, and model monitoring at scale.
• Exposure to agent memory, feedback loops, or AI workflow orchestration in customer-facing products.
• Prior contributions to open-source ML or LLM projects are a plus.
  
  

Why Join HG Insights?

• Work on high-impact AI products used by global enterprise customers.
• Lead the ML engineering strategy and shape our AI platform.
• Competitive compensation and benefits tailored for India-based employees.
• A collaborative and innovation-focused team environment.
  
  

Ready to build the future of AI infrastructure? Apply now to join HG Insights as a Staff Machine Learning Engineer in Pune.

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

In this role, you will:

· Technical leadership for a team of engineers that focus on development, deployment and operations

· Lead and contribute to multiple pods with moderate resource requirements, risk, and/or complexity

· Interface technically with a range of stakeholders with customer and business impact

· Leading others to solve complex problems

· Experience in working within an agile, multidisciplinary DevOps team

· Migrate and re-engineer existing services from on-premises data centers to Cloud (GCP/AWS)

· ETL Engineer is responsible for performing system development work with background of various data models and data warehousing concepts.

· Write, analyse, review, and rewrite programs to departmental and Group standards

· Understanding the business requirements and provide the real-time solutions

· Following the project development tools like JIRA, Confluence and GIT

· Build and maintain operations tools for monitoring, notifications, trending, and analysis.

· Enhance programs to increase operating efficiency or adapt to new requirements

· Review code from team members Analyst/Developers as part of the quality assurance process

· Produce unit test plans with detailed expected results to fully exercise the code.

· Work closely with solution architect, business Analyst and technology lead to contribute in achieving final outcome.

· Demonstrate upward graph in adaption of various Engineering Practices.

To be successful in this role, you should meet the following requirements:

· Mandatory Skills: -

o Senior Data Engineer with Cloud (GCP) (Experience - 8 to 12 Years)

o Mandatory Google Cloud Big Query Scripting Skills or Cloud SQL hands-on experience

o Mandatory SQL PL/SQL Scripting Experience [High level expertise and hands-on work]

o Mandatory shell scripting Or Python Skills [High level expertise and hands-on work]

o Should have experience on any of the RDBMS

o Should have leadership experience.

· Good To Have: -

o GCP Data Engineer certifications is an added advantage.

o Experience/knowledge on GCP components like GCS, Big Query, Airflow, Cloud SQL and Google Cloud SDK

o Good to have knowledge - Preferable Control M or ETL Tool or Any from - UC4 Atomic, Airflow Composer

o Prefer to have Juniper Ingestion process awareness

o Experience in working within an agile, multidisciplinary ‘Dev-Ops’ team.

o Changes, Incident and Problem Management

Job Title: Control Automation Engineer (IT)

 About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the  British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

Job Location: Pune

Job Title: Control Automation Engineer (IT)

Role purpose

Build and run automation that improves the effectiveness, reliability, and auditability of IT controls. You’ll use Python/Unix scripting, CI/CD (Jenkins/Groovy), orchestration (Airflow/Composer), and ServiceNow APIs to automate control checks, evidence collection, monitoring, and remediation workflows—delivered in line with SDLC and controlled deployment (DEPL/ICE) expectations and within agreed SLA/OLA.

Key responsibilities

1) Control automation & continuous monitoring

• Engineer automated control checks across areas such as access, change, patching, backup, monitoring, vulnerability, and configuration compliance.
• Implement continuous control monitoring, automated control testing, and exception handling.
• Build automated evidence capture and retention processes to support audits and assurance activities.

2) CI/CD automation (Jenkins & Groovy)

• Develop and maintain Jenkins pipelines using Groovy for build, test, release, and operational automation.
• Embed quality gates (unit tests, linting, security checks) and ensure traceability from requirement to deployment.
• Maintain pipeline reliability, credential handling, and secure integration patterns.

3) Orchestration (Airflow / Composer)

• Design and operate workflows using Apache Airflow or Cloud Composer (DAG design, scheduling, retries, dependencies).
• Build resilient, observable orchestration with logging, alerting, and runbook-ready operational support.
• Optimise workflow performance and ensure idempotent, recoverable job execution.

4) ServiceNow automation (API-led)

• Integrate with ServiceNow via REST APIs to automate:
  
  Incident/Problem/Change workflows
  Control attestations, evidence requests, and task management
  Automated updates, approvals, and status reporting
• Troubleshoot integrations, authentication, and data mapping; ensure audit-friendly logging.

5) SDLC & controlled deployment (DEPL/ICE)

• Apply strong SDLC practices: requirements, design, build, test, release, and support.
• Ensure automation changes comply with DEPL/ICE control expectations (approvals, segregation of duties, release documentation, rollback plans, and audit trails).

6) SLA/OLA management & compliance actions

• Define, track, and report SLA/OLA for automated controls and supporting services (including over-SLA/breach management).
• Monitor control performance and compliance actions; manage exceptions, remediation plans, and closure evidence.
• Produce operational metrics and dashboards (control coverage, failure rates, MTTR, SLA compliance).

Required skills & experience

• 10+ years’ IT experience, with significant hands-on automation/scripting delivery.
• Strong Python and Unix/Linux (shell scripting, permissions, scheduling, process management).
• Proven Jenkins experience, including Groovy pipeline development and CI/CD best practices.
• Experience with orchestration tools: Apache Airflow and/or Cloud Composer (DAGs, scheduling, monitoring).
• Hands-on ServiceNow automation via APIs (REST preferred), including integration patterns and troubleshooting.
• Strong understanding of SDLC and controlled deployment/change governance (including DEPL/ICE).
• Solid knowledge of IT controls and compliance, including evidence integrity, auditability, and operational control design.
• Experience managing SLA/OLA for IT controls/services, including breach analysis and continuous improvement.

Desirable (nice to have)

• Git-based workflows, automated testing frameworks, and code quality tooling.
• Monitoring/logging platforms (e.g., Splunk, ELK, Grafana) and alerting design.
• Familiarity with CMDB concepts and data quality controls.
• Exposure to common control frameworks (e.g., ITIL control practices, ISO 27001-aligned controls).

Education / certifications (optional)

• Degree in Computer Science/Engineering (or equivalent experience).
• ITIL, ServiceNow, or security/control-related certifications.

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

In this role, you will:

Job Summary:

We are looking for an enthusiastic GCP Data Engineer to join our dynamic and growing project team. In this role, you will have the opportunity to develop your skills while contributing to large-scale data projects on Google Cloud Platform. You will work closely with senior engineers and  data analysts to build and maintain robust data pipelines, helping to turn raw data into actionable insights. This is an excellent opportunity for a foundational career in data engineering within a supportive and collaborative environment.

Key Responsibilities:

• Develop, test, and maintain data pipelines and ETL/ELT processes using Python, SQL, and GCP services.
• Write and optimize complex SQL queries in BigQuery for data transformation, extraction, and analysis.
• Assist in the design and implementation of data models within our data warehouse.
• Collaborate with the broader project team to understand data requirements and deliver effective solutions.
• Troubleshoot and resolve issues in data pipelines and data quality to ensure accuracy and availability.
• Contribute to documentation and adhere to data engineering best practices.
• Support the team in various data-related tasks and be eager to learn new technologies and techniques.

Required Qualifications:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10-12 years of experience with strong programming skills in Python for data manipulation and processing.
• Proficiency in SQL for querying and data transformation.
• Hands-on experience or significant project work with Google Cloud Platform (GCP).
• Familiarity with core GCP data services, particularly BigQuery and Pub/Sub.
• Strong understanding of data warehousing concepts and ETL/ELT processes.
• A proactive and curious mindset with strong problem-solving skills.
• Excellent communication and teamwork skills, with the ability to work effectively in a large project team.

Preferred Qualifications:

• Experience with other GCP services such as Cloud Composer (Airflow) or Dataflow, Cloud Functions.
• Familiarity with version control systems, especially Git.
• Google Cloud certification (e.g., Associate Cloud Engineer) is a plus.
• A passion for data and a strong desire to grow a career in the data engineering field.

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

In this role, you will:

· Technical leadership for a team of engineers that focus on development, deployment and operations

· Lead and contribute to multiple pods with moderate resource requirements, risk, and/or complexity

· Interface technically with a range of stakeholders with customer and business impact

· Leading others to solve complex problems

· Experience in working within an agile, multidisciplinary DevOps team

· Migrate and re-engineer existing services from on-premises data centers to Cloud (GCP/AWS)

· ETL Engineer is responsible for performing system development work with background of various data models and data warehousing concepts.

· Write, analyse, review, and rewrite programs to departmental and Group standards

· Understanding the business requirements and provide the real-time solutions

· Following the project development tools like JIRA, Confluence and GIT

· Build and maintain operations tools for monitoring, notifications, trending, and analysis.

· Enhance programs to increase operating efficiency or adapt to new requirements

· Review code from team members Analyst/Developers as part of the quality assurance process

· Produce unit test plans with detailed expected results to fully exercise the code.

· Work closely with solution architect, business Analyst and technology lead to contribute in achieving final outcome.

· Demonstrate upward graph in adaption of various Engineering Practices.

To be successful in this role, you should meet the following requirements:

· Mandatory Skills: -

o Senior Data Engineer with Cloud (GCP) (Experience - 8 to 12 Years)

o Mandatory Google Cloud Big Query Scripting Skills or Cloud SQL hands-on experience

o Mandatory SQL PL/SQL Scripting Experience [High level expertise and hands-on work]

o Mandatory shell scripting Or Python Skills [High level expertise and hands-on work]

o Should have experience on any of the RDBMS

o Should have leadership experience.

· Good To Have: -

o GCP Data Engineer certifications is an added advantage.

o Experience/knowledge on GCP components like GCS, Big Query, Airflow, Cloud SQL and Google Cloud SDK

o Good to have knowledge - Preferable Control M or ETL Tool or Any from - UC4 Atomic, Airflow Composer

o Prefer to have Juniper Ingestion process awareness

o Experience in working within an agile, multidisciplinary ‘Dev-Ops’ team.

o Changes, Incident and Problem Management

HG Insights is the pioneer of Revenue Growth Intelligence. For more than a decade, we have delivered comprehensive, AI-driven datasets on B2B buyers, technology adoption, IT spend, and buyer intent, sourced from billions of data points. Today, we are a trusted partner to Fortune 500 technology companies, hyperscalers, and innovative B2B vendors seeking precise go-to-market analytics and decision-making.

Through an evolving suite of AI agents that incorporate first-party data and buyer signals, HG Insights enables AI-powered GTM automation across sales, marketing, RevOps, and data analytics teams, modernizing GTM execution from strategy through activation.

Product Group: TrustRadius is a B2B technology decisioning platform that helps buyers make confident software purchase decisions and enables vendors to activate high‑intent, in‑market buyers through rich review content and intent data signals. As part of HG Insights, TrustRadius powers data‑driven go‑to‑market strategies for leading B2B technology brands worldwide.

Job Title: Principal Engineer

Role Overview

We are looking for a Principal Engineer – Engineering to provide technical thought leadership for our vendor‑side data platform and pipelines. This is a senior individual contributor role (IC) with Principal Engineer scope, responsible for the architecture, evolution, and reliability of our microservices and data pipelines that power vendor lead generation, intent resolution.

You will set the long‑term technical direction, mentor senior/staff engineers, and stay hands‑on in critical code paths across NestJS/Node.js, TypeScript, Python data pipelines, and event‑driven services.

Key Responsibilities

• Own the end‑to‑end architecture for data ingestion and processing pipelines, from external data sources through Airflow‑orchestrated Python workflows into storage on S3 and downstream services.
• Design and evolve microservices built with NestJS/Node.js and TypeScript, exposing APIs for lead generation, vendor management, and intent resolution at scale.
• Define and drive database architectures across PostgreSQL and MongoDB and Elastic, including complex data models for B2B lead scoring, vendor relationships, and customer analytics.
• Provide technical thought leadership to create and grow an agile, high‑performing engineering team culture around CI/CD, trunk‑based development, testing, observability, and rapid iteration.
• Lead the adoption of real‑world AI/ML in engineering workflows (code generation, code review, test generation, incident analysis) to materially improve development speed and quality.
• Partner with product and marketing to design data and service architectures that maximize content/intent data pipelines that make TrustRadius a top‑ranked destination for vendors and buyers.
• Implement and evolve secure authentication and authorization patterns using Auth0, OAuth2, and JWT for a multi‑tenant SaaS platform serving enterprise customers.
• Integrate and scale third‑party services such as LinkedIn APIs, intent enrichment, Salesforce CRM, and payment processors, with robust resilience, observability, and fault‑tolerant retry strategies.
• Architect and optimize event‑driven systems on Google Cloud Pub/Sub for real‑time data processing, enrichment, and inter‑service communication.
• Drive performance and cost optimization of APIs and data pipelines, including caching strategies, query tuning, horizontal scaling, and right‑sizing of cloud resources.
• Champion best practices for data quality, governance, and monitoring across vendor data feeds and internal platforms to ensure accurate, trusted signals for downstream analytics and GTM activation.
• Mentor Engineers, providing guidance on system design, trade‑offs, and long‑term roadmap, while influencing technical decisions.

Core Technical Skills

• Expert‑level experience designing and building large‑scale backend systems and microservices with NestJS/Node.js and TypeScript.
• Strong hands‑on experience with Python data ingestion and ETL/ELT pipelines, ideally orchestrated using Apache Airflow and cloud object storage such as S3.
• Deep understanding of relational and NoSQL database design, specifically PostgreSQL and MongoDB, including schema design, indexing, and performance tuning.
• Proven experience with secure authentication/authorization in SaaS environments using Auth0, OAuth2, and JWT.
• Strong background integrating external APIs and SaaS systems.
• Hands‑on experience with event‑driven architectures and messaging systems.
• Solid understanding of cloud infrastructure (preferably GCP), containerization, CI/CD pipelines, and production observability (logging, metrics, tracing).

Nice‑to‑Have Skills

• Experience with MCP architectures and tools development.
• Practical knowledge of machine learning techniques applied to data pipelines.
• Experience in content/intent‑data‑driven products, especially in B2B SaaS or marketplaces.
• Familiarity with data platforms in the B2B buyer intent or intent data provider space.

Business Outcomes and Impact

In this role, you will be accountable for:

• Building and sustaining an agile, high‑impact engineering culture through technical thought leadership, standards, and mentorship.
• Demonstrable improvements in engineering efficiency and quality through AI‑assisted tooling, automation, and better platform abstractions.
• A continuously evolving, scalable data and service architecture that strengthens TrustRadius’s position as the number‑one destination for SEO and GEO‑driven software buying traffic, and as a key provider of high‑quality intent data to vendors.

About Us

HG Insights is the global leader in technology intelligence, delivering actionable AI driven insights through advanced data science and scalable big data solutions. Our platform informs go-to-market decisions, and we influence how businesses spend millions of marketing and sales budgets.

What You’ll Do:

• Design, build, and optimize large-scale distributed data pipelines for processing billions of events from and to multiple sources of data.
• Architect and scale enterprise-grade big-data systems, including data lakes, ETL/ELT workflows, and syndication platforms for customer-facing products.
• Orchestrate pipelines and workflows with Airflow.
• Write optimized data into analytical databases such as ClickHouse, DuckDB, and Redshift
• Ensure data quality, consistency, and reliability across the pipeline.
• Monitor pipeline performance and troubleshoot data issues.
• Collaborate with product teams to develop features across databases, and backend services.
• Implement cutting-edge solutions for data ingestion, transformation, and analytics.
• Drive system reliability through automation, CI/CD pipelines (Docker, Kubernetes, Terraform), and infrastructure-as-code practices.

What You’ll Be Responsible For

• Development of the Data side of our Platform, ensuring scalability, performance, and cost-efficiency across distributed systems.
• Collaborating in agile workflows (daily stand-ups, sprint planning) to deliver features rapidly while maintaining system stability.
• Ensuring security and compliance across data workflows, including access controls, encryption, and governance policies.

What You’ll Need

• BS/MS/Ph.D. in Computer Science or related field, with 7+ years of experience building production-grade big data systems.
• Strong SQL skills and solid data modeling fundamentals with hands-on experience building ETL / ELT pipelines.
• Proficiency in Python for data processing and integrations.
• Familiarity with dbt, Airflow, Databricks and modern analytics engineering practices

Nice-to-Haves

• Knowledge of data governance frameworks and compliance standards (GDPR, CCPA).
• Contributions to open-source big data projects or published technical blogs/papers.
• DevOps proficiency in monitoring tools (Prometheus, Grafana) and serverless architectures.

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

Role: GCP Data Engineer 

Job Summary:

We are looking for an enthusiastic GCP Data Engineer to join our dynamic and growing project team. In this role, you will have the opportunity to develop your skills while contributing to large-scale data projects on Google Cloud Platform. You will work closely with senior engineers and data analysts to build and maintain robust data pipelines, helping to turn raw data into actionable insights. This is an excellent opportunity for a foundational career in data engineering within a supportive and collaborative environment.

Key Responsibilities:

· Develop, test, and maintain data pipelines and ETL/ELT processes using Python, SQL, and GCP services.

· Write and optimize complex SQL queries in BigQuery for data transformation, extraction, and analysis.

· Assist in the design and implementation of data models within our data warehouse.

· Collaborate with the broader project team to understand data requirements and deliver effective solutions.

· Troubleshoot and resolve issues in data pipelines and data quality to ensure accuracy and availability.

· Contribute to documentation and adhere to data engineering best practices.

· Support the team in various data-related tasks and be eager to learn new technologies and techniques.

Required Qualifications:

· Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

· 5-8 years of hands-on experience with strong programming skills in Python for data manipulation and processing.

· Proficiency in SQL for querying and data transformation.

· Hands-on experience or significant project work with Google Cloud Platform (GCP).

· Familiarity with core GCP data services, particularly BigQuery and Pub/Sub.

· Strong understanding of data warehousing concepts and ETL/ELT processes.

· A proactive and curious mindset with strong problem-solving skills.

· Excellent communication and teamwork skills, with the ability to work effectively in a large project team.

Preferred Qualifications:

· Experience with other GCP services such as Cloud Composer (Airflow) or Dataflow, Cloud Functions.

· Familiarity with version control systems, especially Git.

· Google Cloud certification (e.g., Associate Cloud Engineer) is a plus.

· A passion for data and a strong desire to grow a career in the data engineering field.