About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 7+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

About the role

We are seeking an accomplished Data Engineer to join our dynamic and rapidly expanding Tech Ops division. With significant projected growth, this is an opportunity to drive meaningful technical impact. In this role, you will work directly with customers, data vendors, and internal engineering teams to design, implement, and optimize complex data integration solutions within a modern, large‑scale cloud environment.

You will leverage advanced skills in distributed computing, data architecture, and cloud-native engineering to enable scalable, resilient, and high‑performance data ingestion and transformation pipelines. As a trusted technical advisor, you will guide customers in adopting Abacus’s core data management platform and ensure high-quality, compliant data operations across the lifecycle.

Your day to day :

• Architect, design, and implement high-volume batch and real-time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks.
• Build end‑to‑end ingestion frameworks integrating with Databricks, Snowflake, AWS services (S3, SQS, Lambda), and vendor data APIs, ensuring data quality, lineage, and schema evolution.
• Develop data modeling frameworks, including star/snowflake schemas and optimization techniques for analytical workloads on cloud data warehouses.
• Lead technical solution design for health plan clients, creating highly available, fault-tolerant architectures across multi-account AWS environments.
• Translate complex business requirements into detailed technical specifications, engineering artifacts, and reusable components.
• Implement security automation, including RBAC, encryption at rest/in transit, PHI handling, tokenization, auditing, and compliance with HIPAA and SOC 2 frameworks.
• Establish and enforce data engineering best practices, such as CI/CD for data pipelines, code versioning, automated testing, orchestration, logging, and observability patterns.
• Conduct performance profiling and optimize compute costs, cluster configurations, partitions, indexing, and caching strategies across Databricks and Snowflake environments.
• Produce high-quality technical documentation including runbooks, architecture diagrams, and operational standards.
• Mentor junior engineers through technical reviews, coaching, and training sessions for both internal teams and clients.

What you bring to the team :

• Bachelor’s degree in Computer Science, Computer Engineering, or a closely related technical field.
• 5+ years of hands‑on experience as a Data Engineer working with large‑scale, distributed data processing systems in modern cloud environments.
• Working knowledge of U.S. healthcare data domains—including claims, eligibility, and provider datasets—and experience applying this knowledge to complex ingestion and transformation workflows.
• Strong ability to communicate complex technical concepts clearly across both technical and non‑technical stakeholders.
• Expert‑level proficiency in Python, SQL, and PySpark, including developing distributed data transformations and performance‑optimized queries.
• Demonstrated experience designing, building, and operating production‑grade ETL/ELT pipelines using Databricks, Airflow, or similar orchestration and workflow automation tools.
• Proven experience architecting or operating large‑scale data platforms using dbt, Kafka, Delta Lake, and event‑driven/streaming architectures, within a cloud‑native data services or platform engineering environment—requiring specialized knowledge of distributed systems, scalable data pipelines, and cloud‑scale data processing.
• Experience working with structured and semi‑structured data formats such as Parquet, ORC, JSON, and Avro, including schema evolution and optimization techniques.
• Strong working knowledge of AWS data ecosystem components—including S3, SQS, Lambda, Glue, IAM—or equivalent cloud technologies supporting high‑volume data engineering workloads.
• Proficiency with Terraform, infrastructure‑as‑code methodologies, and modern CI/CD pipelines (e.g., GitLab) supporting automated deployment and versioning of data systems.
• Deep expertise in SQL and compute optimization strategies, including Z‑Ordering, clustering, partitioning, pruning, and caching for large‑scale analytical and operational workloads.
• Hands‑on experience with major cloud data warehouse platforms such as Snowflake (preferred), BigQuery, or Redshift, including performance tuning and data modeling for analytical environments.

What we would like to see but not required:

• Experience in large-scale healthcare or payer data environments.

What you’ll get in return :

• Competitive Leave & Benefits
• Comprehensive health coverage
• Equity for every employee – share in our success
• Growth-focused environment – your development matters here

Work arrangements :

• Standard hours: 8 hours/day, 5 days/week
• Location: Pune, Hybrid (3 days a week in office)
• Shift: 12pm-9pm IST

Software Engineer III (Java + JavaScript + SQL)

TraceLink is seeking an experienced full time Software Engineer III to join our team to work on our Analytics products. If you’ve got a solid grasp of SQL, Java and Javascript and familiarity of cloud environments like AWS. If you’re highly motivated and looking for a chance to make a real contribution on a high profile project, you could be a good fit for our team.

The Software Engineer III - will be responsible for the design, development and maintenance of cutting edge cloud based Analytics applications, running in our sophisticated, Kubernetes (AWS EKS) hosted, RxJava/Vert.x cloud platform, the Digital Network Platform (OPUS).  All of this within the Life Sciences / Pharma supply chain domain.

Responsibilities:

• Proficiency in Data Structures and Algorithms.

• Develop Analytics solutions that will help clients optimize their supply chain operations and adhere to compliance regulations. 

• Work closely with Domain experts, Data scientists, Architects, Product managers and Product Engineers to design and implement Analytics products.

• As an individual contributor, you will be responsible for writing clean, high performance and scalable code across different frameworks and languages.

• Work in fast paced Agile teams and ability to quickly learn and adapt to new tools and technologies.

• Work with QA teams to align on test planning and help in test executions.

Candidates must possess the following skills and traits: 

• 4 to 6 years of experience with at least 1 year hands on experience with writing SQL queries.

• Expertise in Java & Javascript & SQL.

• Familiarity with AWS Cloud Data storage and processing services - S3, Glue, Athena, Redshift, RDS, Elastic Search, Kafka/Kinesis, SQS, SNS etc.

• Solid understanding of RESTful APIs, Microservices architecture, and containerization (Kubernetes).

• Strong understanding of event-driven architecture and asynchronous messaging

• Exposure to Javascript, GraphQL, Vert.x, Rx/Java would be preferred.

• Analytical thinking and collaborative mindset with excellent communication skills. 

• Experience working in an Agile environment with teams distributed across US and India

• Strong Computer Science fundamentals: Data Structures and Algorithms

Good to have:

• Knowledge and Experience building Agentic AI systems.

• Knowledge and Experience with Prompt/Context Engineering. 

Education:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or or equivalent experience required.

What’s in it for you?

Opportunity To Unlock Your Creativity
Think of all the times you were held back from trying new ideas because you were boxed in by bureaucratic legacy processes or old school tactics. Having a growth mindset is deeply ingrained into our company culture since day 1 so Fictiv is an environment where you have the creative liberty and support of the team to try big bold ideas to achieve our sales and customer goals.

Opportunity To Grow Your Career
At Fictiv, you'll be surrounded by supportive teammates who will push you to be your best through their curiosity and passion.

Impact In This Role

The Data Intelligence team creates reports and dashboards to provide meaningful insights from the data warehouse. This team works closely with cross-functional stakeholders and leverages multiple Business Intelligence (BI) technologies to enable data-driven decision-making across the organization.
As a Senior Data Engineer on the Data Intelligence team, you will play a key role in designing, managing, and evolving Fictiv’s reporting data warehouse and semantic layer. You will partner closely with the Data Intelligence Manager, Data Engineers, Data Analysts, and Product Managers to ensure replicated data from production systems is transformed into scalable, trusted semantic data models that power consistent, accurate self-service reporting. Acting as a bridge between business and technology, you will translate analytical requirements into well-structured data models and transformations that enable reliable dashboards and insights across multiple domains.
In addition, you will collaborate with Infrastructure, Software, and Systems Engineering teams to strengthen database design and query efficiency across the broader data ecosystem. You will provide hands-on guidance on optimal table structures, indexing and clustering approaches, and SQL query patterns that improve performance and scalability. By proactively identifying bottlenecks and recommending architectural and modeling improvements, you will help ensure that both analytical and operational workloads are efficient, maintainable, and aligned with best practices.
This is a hands-on role responsible for leading the design, transformation, implementation, and operation of data solutions across disparate source systems—driving best practices in data modeling, semantic layer design, performance optimization, documentation, and governance. You will plan, design, and operate data pipelines and models using Agile methodologies to deliver high-quality, maintainable, and secure data assets

You will report to the Data Analytics Manager.

What You’ll Be Doing

• Design efficient, scalable, and maintainable semantic data models that align with business logic and enable intuitive self-service analytics and reporting across multiple domains.
  
• Build, maintain, and optimize data transformations and pipelines that transform replicated production data into trusted, analytics-ready models.
  
• Collaborate with stakeholders across data, engineering, product, and business teams to resolve data-related issues, clarify requirements, and align on solution design.
  
• Provide technical leadership and guidance to Software Engineers and Data Engineers on query performance, optimization techniques, and efficient data access patterns.
  
• Drive ongoing improvement of database and table structures, partnering with engineers to validate and refine table design, joins, partitioning/clustering, and other performance-related considerations.
  
• Review and advise on SQL query design for both application and analytics use cases, helping teams design queries that are performant, maintainable, and cost-effective.
  
• Analyze performance metrics and execution plans to identify bottlenecks in data models, queries, and transformations, and recommend practical tuning and refactoring opportunities.
  
• Follow and contribute to SQL coding standards, data modeling best practices, change control procedures, and data warehouse governance processes.
  
• Develop and maintain data flow diagrams, data dictionaries, and schema documentation with a focus on clarity, consistency, and enterprise data management.
  
• Understand and document end-to-end data flows and dependencies from production systems through the ELT pipeline into Snowflake and downstream BI tools, supporting delivery, release, and change management.
  
• Execute and lead data design sessions and technical reviews to gather requirements, validate modeling approaches, and communicate design decisions and trade-offs to stakeholders.
  
• Research emerging technologies, patterns, and tools related to Snowflake, dbt, SQL optimization, and semantic modeling, and recommend enhancements to Fictiv’s data platform and practices.
  
• Help set broad direction and vision for the reporting data platform architecture, influencing data engineering, analytics, and application data design strategy.
  
• Partner with internal and external teams for knowledge transfer and smooth handoff in advance of new feature launches or reporting changes.
  
• Work effectively with remote and cross-functional teams across time zones, fostering strong communication and collaboration.
  
• Proven experience working with large and complex datasets and comfort writing advanced SQL queries or leveraging BI tools for analysis and transformation.
  
• Deep understanding of data and analytics architecture, including data warehousing concepts, semantic data modeling, and data integration best practices, with a strong focus on Snowflake and ELT pipelines.
  
• Strong knowledge of SQL and BI concepts, including dimensional modeling, data warehouse design, and dashboarding; ability to translate analytical needs into robust semantic models.
  
• Demonstrated ability to guide and influence database and schema optimization, including improving query performance, advising on efficient access patterns, and ensuring well-structured data that support scalable workloads.
  
• Experience mentoring and guiding Data Engineers and Software Engineers to develop robust, scalable, and maintainable data models, queries, and database structures.
  
• Ability to design and maintain data transformation and workload management processes, including job orchestration, dependency handling, scheduling, and metadata management.
  
• Excellent analytical skills and experience working with both structured and semi-structured data, from exploration through to production-ready models.
  
• Ability to evaluate new tools, database features, and methodologies relative to the existing data stack and propose detailed, pragmatic recommendations for improvement.
  
• Strong ability to perform root cause analysis on data quality, performance, and process issues to identify inefficiencies, ensure data integrity, and deliver actionable insights.
  
• Clear and effective communicator—capable of providing business context for engineers and technical context for non-engineers, especially around trade-offs in model and schema design.
  
• Confident presenting to audiences with mixed levels of technical understanding and experience, including walkthroughs of models, schemas, and query patterns.
  
• Curious, resourceful, and adaptable; comfortable navigating ambiguity, learning new technologies (e.g., new Snowflake/dbt features), and evolving processes to improve outcomes.

Desired Traits

• B.S. in Data Science, Computer Science, Information Systems, or a related field.
  
• 15+ years of experience as a Data Engineer, designing and maintaining large-scale data systems, building scalable data models, and delivering analytical solutions through modern BI and data pipeline tools.
  
• 10+ years of experience working with modern cloud and on-premise database platforms—such as Snowflake, AWS Redshift, Azure Synapse, Google BigQuery, or traditional RDBMS systems (e.g., PostgreSQL, MySQL, SQL Server, Oracle)—with a strong focus on schema design, query performance, execution plan analysis, and scalable data access patterns, rather than day-to-day operational administration.
  
• 8+ years of experience developing and maintaining data transformation workflows using dbt or a similar tool, with expertise in model design, testing, documentation, environment management, deployment automation, and enforcing version control and development best practices.
  
• Demonstrated expertise working with complex datasets and advanced SQL, including multi-level CTEs, window functions, and recursive queries, with an emphasis on performance and readability.
  
• Experience with BI platforms such as Heap, Sigma Computing, or similar technologies (e.g., Tableau, Looker, or QuickSight), and how they interact with warehouse models.
  
• Strong knowledge of BI and analytical concepts, including regressions, recursion, sliding window functions, financial metrics, marketing funnel metrics, and logistics and fulfillment analytics.
  
• Experience building and maintaining processes that support data transformation, data structures, metadata management, dependency handling, and workload optimization in a modern ELT/warehouse environment.
  
• Proven ability to lead and coordinate cross-functional initiatives, including partnering with Software Engineers to refine schemas and queries, conducting interviews, and performing analyses to develop business cases for projects and improvements.
  
• Experience delivering training sessions and technical enablement for internal and external stakeholders, especially around data models, schemas, and query best practices during system and process rollouts.
  
• Excellent communication and collaboration skills, with the ability to translate complex technical concepts (e.g., query plans, indexing/clustering trade-offs, modeling decisions) for non-technical audiences.
  
• Demonstrated leadership and mentoring abilities, fostering technical excellence and knowledge sharing across data and engineering teams, particularly in areas of semantic modeling, SQL craftsmanship, and performance optimization.

Perks and Benefits

• Medicliam
• Annual Paid holidays
• Gratuity
• Provident Fund 
  And much, much more!

Interested in learning more?  We look forward to hearing from you soon.

About Us

HG Insights is the global leader in technology intelligence, delivering actionable AI driven insights through advanced data science and scalable big data solutions. Our platform informs go-to-market decisions, and we influence how businesses spend millions of marketing and sales budgets.

What You’ll Do:

• Design, build, and optimize large-scale distributed data pipelines for processing billions of events from and to multiple sources of data.
• Architect and scale enterprise-grade big-data systems, including data lakes, ETL/ELT workflows, and syndication platforms for customer-facing products.
• Orchestrate pipelines and workflows with Airflow.
• Write optimized data into analytical databases such as ClickHouse, DuckDB, and Redshift
• Ensure data quality, consistency, and reliability across the pipeline.
• Monitor pipeline performance and troubleshoot data issues.
• Collaborate with product teams to develop features across databases, and backend services.
• Implement cutting-edge solutions for data ingestion, transformation, and analytics.
• Drive system reliability through automation, CI/CD pipelines (Docker, Kubernetes, Terraform), and infrastructure-as-code practices.

What You’ll Be Responsible For

• Development of the Data side of our Platform, ensuring scalability, performance, and cost-efficiency across distributed systems.
• Collaborating in agile workflows (daily stand-ups, sprint planning) to deliver features rapidly while maintaining system stability.
• Ensuring security and compliance across data workflows, including access controls, encryption, and governance policies.

What You’ll Need

• BS/MS/Ph.D. in Computer Science or related field, with 7+ years of experience building production-grade big data systems.
• Strong SQL skills and solid data modeling fundamentals with hands-on experience building ETL / ELT pipelines.
• Proficiency in Python for data processing and integrations.
• Familiarity with dbt, Airflow, Databricks and modern analytics engineering practices

Nice-to-Haves

• Knowledge of data governance frameworks and compliance standards (GDPR, CCPA).
• Contributions to open-source big data projects or published technical blogs/papers.
• DevOps proficiency in monitoring tools (Prometheus, Grafana) and serverless architectures.

What You Will Do and Learn

As a Data Engineer in Data Solutions, you’ll be responsible for designing, implementing, and testing proposed modern analytic solutions.  Working closely with our client partners and architects, you’ll develop relationships with key technical resources while delivering tangible business outcomes. 

Qualifications:

· 2-4 years’ experience developing industry leading business intelligence and analytic solutions

· Must have thorough knowledge of data warehouse concepts and dimensional modelling

· Must have experience in writing advanced SQL

· Must have at least 2+ years of hands-on experience on DBT (Data Build Tool). Mandatory to have most recent hands-on experience on DBT.

· Must have experience working with DBT on one or more of the modern databases like Snowflake / Amazon Redshift / BigQuery / Databricks / etc.

· Hands-on experience with Snowflake would carry higher weightage

· Snowflake SnowPro Core certification would carry higher weightage

· Experience working in AWS, Azure, GCP or similar cloud data platform would be an added advantage

· Hands-on experience on Azure would carry higher weightage

· Must have experience in setting up DBT projects

· Must have experience in understanding / creating / modifying & optimizing YML files within DBT

· Must have experience in implementing and managing data models using DBT, ensuring efficient and scalable data transformations

· Must have experience with various materialization techniques within DBT

· Must have experience in writing & executing DBT Test cases

· Must have experience in setting up DBT environments

· Must have experience in setting up DBT Jobs

· Must have experience with writing DBT Jinja and Macros

· Must have experience in creating DBT Snapshots

· Must have experience in creating & managing incremental models using DBT

· Must have experience with DBT Docs

· Should have a good understanding of DBT Seeds

· Must have experience with DBT Deployment

· Must Experience with architecting data pipelines using DBT, utilizing advanced DBT features

· Proficiency in version control systems and CI/CD

· Must have hands-on experience configuring DBT with one or more version control systems like Azure DevOps / Github / Gitlab / etc.

· Must have experience in PR approval workflow

· Participate in code reviews and best practices for SQL and DBT development

· Experience working with visualization tools such as Tableau, PowerBI, Looker and other similar analytic tools would be an added advantage

· 2+ years of Business Data Analyst experience

· 2+ years of experience writing Business requirements, Use cases and/or user stories, for data warehouse or data mart initiatives.

· Understanding and experience on ETL/ELT is an added advantage

· 2+ years of consulting experience working on project-based delivery using Software Development Life Cycle (SDLC)

· 2+ years of years of experience with relational databases (Postgres, MySQL, SQL Server, Oracle, Teradata etc.)

· 2+ years of experience creating functional test cases and supporting user acceptance testing

· 2+ years of experience in Agile/Kanban/DevOps Delivery

· Outstanding analytical, communication, and interpersonal skills

· Ability to manage projects and teams against planned work

· Responsible for managing the day-to-day client relationship on projects

· Manage the Data engineering lifecycle including research, proof of concepts, architecture, design, development, test, deployment, and maintenance

· Collaborate with team members to design and implement technology that aligns with client business objectives

· Build proof of concepts for a modern analytics stack supporting a variety of Cloud-based Business Systems for potential clients

· Team management experience and ability to manage, mentor and develop talent of assigned junior resources

· Create actionable recommendations based on identified platform, structural and/or logic problems

· Communicate and demonstrate a clear understanding of client business needs, goals, and objectives

· Collaborate with other architects on solution designs and recommendations.