About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 7+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day :

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team :

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 5+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

About the Role

We are seeking a Senior Data QA Engineer to own and elevate data quality across our data platforms and client solutions. In this role, you will lead complex data validation efforts, shape testing strategies, and act as a trusted partner to Engineering, Product, and Operations. You’ll set standards for data quality, drive automation, and help ensure the delivery of reliable, production‑grade data solutions at scale.

Your Day to Day

• Lead data quality strategy and execution across multiple projects and datasets.
• Partner closely with Engineering, Product, Operations, and Project Management to influence data design and quality outcomes early in the lifecycle.
• Perform advanced data analysis and validation on large‑scale, client‑specific and healthcare datasets.
• Design, execute, and review complex test plans and test cases, including automated and manual testing approaches.
• Identify systemic data quality risks, drive root‑cause analysis, and partner with teams through resolution.
• Define, refine, and document QA standards, best practices, and verification protocols.
• Mentor junior QA engineers and provide guidance on testing methodologies and data validation techniques.
• Contribute to continuous improvement of QA processes, tooling, and workflows.

What You Bring to the Team

• Bachelor’s degree in Computer Science, Health Science, or a related field (or equivalent practical experience).
• 5+ years of experience in data quality, data QA, or testing roles within data‑driven environments; healthcare experience strongly preferred.
• Advanced SQL skills with the ability to validate, troubleshoot, and reason about complex datasets.
• Strong understanding of data pipelines, integrations, and large‑scale data systems.
• Experience working with healthcare data domains such as enrollment, claims, and provider data.
• Demonstrated ability to lead QA efforts, influence cross‑functional teams, and operate with a high degree of ownership.
• Excellent analytical, problem‑solving, and prioritization skills in fast‑moving environments.
• Strong communication skills, with the ability to explain data quality issues and risks to both technical and non‑technical stakeholders.
• Deep understanding of QA methodologies, testing frameworks, and best practices.

What you’ll get in return 

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements 

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

About the role 

We are seeking an accomplished Senior Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work with internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment. 

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will ensure high-quality, compliant data operations across the lifecycle. 

Your day to day 

• Architect, design, and implement high‑volume batch and real‑time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor APIs, ensuring data quality, lineage, and schema evolution.
• Design and optimize data models (star/snowflake schemas) and apply performance tuning techniques for analytical workloads on cloud data warehouses.
• Translate complex business requirements into detailed technical specifications, reusable engineering components, and implementation artifacts.
• Establish and enforce data engineering best practices, including CI/CD for data pipelines, version control, automated testing, orchestration, logging, and observability.
• Drive performance and cost optimization through profiling, cluster tuning, partitioning, indexing, caching, and compute optimization across Databricks and Snowflake.
• Ensure operational excellence and team growth by producing high‑quality documentation (runbooks, architecture diagrams), monitoring and troubleshooting production pipelines, performing root‑cause analysis, and mentoring junior engineers.

What you bring to the team 

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field, with 5+ years of hands‑on experience as a Data Engineer building and operating large‑scale, distributed data systems in modern cloud environments.
•  Proven ability to clearly communicate complex technical concepts and solutions to both technical and non‑technical stakeholders.
•  Expert‑level proficiency in Python, SQL, and PySpark, including development of distributed transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using DBT, Kafka, Delta Lake, and event‑driven or streaming architectures in cloud‑native data or platform engineering environments.
• Strong working knowledge of AWS data services (S3, SQS, Lambda, Glue, IAM or equivalents), structured and semi‑structured data formats (Parquet, ORC, JSON, Avro), schema evolution, and optimization techniques.
• Hands‑on experience with Terraform and CI/CD pipelines (e.g., GitLab), deep expertise in SQL and compute optimization (partitioning, clustering, Z‑Ordering, pruning, caching), and performance tuning on cloud data warehouses such as Snowflake (preferred), BigQuery, or Redshift

What we would like to see but not required: 

• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows. 

What you’ll get in return 

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Working Arrangements

• Standard hours: 9 hours/day, 5 working days
• Location: Onsite
• Shift: 10 AM – 7 PM local time

About the Role:

We are seeking an experienced Senior Data QA Engineer to lead the design and implementation of comprehensive data quality testing strategies for our data platforms and pipelines. In this role, you will be responsible for ensuring data integrity, accuracy, and reliability across our data ecosystem through advanced testing methodologies and automation frameworks.

Your day to day:

• Design and implement end-to-end data quality testing strategies for data pipelines, ETL/ELT processes, and data warehouses
• Develop automated data validation frameworks to verify data accuracy, completeness, and consistency
• Create and maintain data quality metrics, monitor dashboards, and alert mechanisms
• Perform complex SQL queries to validate data transformations and business logic
• Design test cases for data ingestion, processing, and consumption layers
• Collaborate with data engineers, data scientists, and analysts to understand data requirements and quality standards
• Lead root cause analysis for data quality issues and implement preventive measures
• Mentor junior QA engineers and establish data testing best practices

What We're Looking For:

• Bachelor’s or master’s degree in computer science, Business Administration, or comparable professional experience
• Minimum 5 years of data quality assurance expertise within healthcare technology companies or health insurance organizations
• Proficient in complex data transformation and manipulation techniques
• Advanced SQL expertise with proven experience in data integration and validation
• Comprehensive understanding of ETL/ELT workflows and quality assurance methodologies for data processing validation
• In-depth expertise in data pipeline architecture and implementing quality control frameworks
• Practical experience with Spark SQL for distributed data processing
• Extensive knowledge of healthcare data domains including member enrollment, medical claims processing, and prescription drug claims; familiarity with emerging data sources (wellness programs, productivity metrics, and electronic medical records) is advantageous
• Outstanding verbal and written communication abilities with demonstrated success in managing multiple priorities and delivering on tight timelines
• Exceptional collaborative and relationship-building capabilities
• Self-directed professional who excels both as an individual contributor and team player, with proven mentoring experience
• Hands-on experience with scripting languages, particularly Python and Java
• Comprehensive expertise in AWS cloud services including S3, EC2, Systems Manager, Athena, and Databricks platform
• Thorough understanding of software testing methodologies, debugging techniques, and quality assurance protocols

Nice to Have:

• Experience with Big Data, Snowflake, and ETL test automation
• Hands‑on API testing experience
• Healthcare domain knowledge, including HEDIS quality measures
• Machine Learning and Artificial Intelligence concepts with practical application experience in production environments
• Proficiency in testing automation frameworks and database technologies

What you’ll get in return 

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Working Arrangements

• Standard hours: 9 hours/day, 5 working days
• Location: Onsite
• Shift: 10 AM – 7 PM local time