Airflow Jobs in Gurugram.
All active Airflow roles based in Gurugram.
Pick a job to read the details
Tap any role on the left — its description and apply link will open here.
09.Data Engineer
Share this job
About PayPay India
Why India ?
To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated.
OUR VISION IS UNLIMITED
Job Description
PayPay's rapid growth necessitates the expansion of its product teams and underscores the critical need for a resilient Data Engineering Platform. This platform is vital to support our increasing business demands. The Data Pipeline team is tasked with creating, deploying, and managing this platform, utilizing leading technologies like Databricks, Delta Lake, Spark, PySpark, Scala, and the AWS suite.
We are actively seeking skilled Data Engineers to join our team and contribute to scaling our platform across the organization.
Main Responsibilities
- Create and manage robust data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform.
- Ensure high performance, reliability, and efficiency by optimizing large-scale data pipelines.
- Develop data processing workflows using Databricks, Delta Lake, and Spark technologies.
- Maintain and improve the Data Lakehouse, utilizing Unity Catalog for efficient data management and discovery.
- Construct automation, frameworks, and enhanced tools to streamline data engineering workflows.
- Collaborate across teams to facilitate smooth data flow and integration.
- Enforce best practices in observability, data governance, security, and regulatory compliance
Qualifications
- Minimum 7 years as a Data Engineer or similar role.
- Hands-on experience with Databricks, Delta Lake, Spark, and Scala.
- Proven ability to design, build, and operate Data Lakes or Data Warehouses.
- Proficiency with Data Orchestration tools (Airflow, Dagster, Prefect).
- Familiarity with Change Data Capture tools (Canal, Debezium, Maxwell).
- Strong command of at least one primary language (Scala, Python, etc.) and SQL.
- Experience with data catalog and metadata management (Unity Catalog, Lakeformation).
- Experience in Infrastructure as Code (IaC) using Terraform.
- Excellent problem-solving and debugging abilities for complex data challenges.
- Strong communication and collaboration skills.
- Capability to make informed decisions, learn quickly, and consider complex technical contexts.
- Leverage AI/LLM-based tools in daily workflows (e.g., code development, reviews, testing, debugging, documentation), while ensuring human oversight, judgment, and accountability drive the final product.
Remarks
＊Please note that you cannot apply for PayPay (Japan-based jobs) or other positions in parallel or in duplicate.
PayPay 5 senses
- Please refer PayPay 5 senses to learn what we value at work.
Working Conditions
Employment Status
- Full Time
Office Location
- Gurugram (Wework)
＊The development center requires you to work in the Gurugram office to establish the strong core team.
Ready to apply?
Apply to PayPay India
Lead Data Engineer
Share this job
Company Overview
Arcesium is a global financial technology firm that solves complex data-driven challenges faced by some of the world’s most sophisticated financial institutions. We constantly innovate our platform and capabilities to meet tomorrow’s challenges, anticipate the risks our clients encounter, and design advanced solutions to help our clients achieve transformational business outcomes.
Financial technology is a high-growth industry as change and innovation continue to disrupt the status-quo and prompt major transformation. Arcesium is at a particularly interesting time in our own growth as we look to leverage our successfully established market position and expand operations in pursuit of strategic new business opportunities. We value intellectual curiosity, proactive ownership, and collaboration with colleagues, and we empower you to meaningfully contribute from day one and accelerate your professional development.
Job Overview
We are looking for a bright and exceptional Engineering Lead to join our Technology team. In this role, you will work closely with Product Managers, Subject Matter Experts, Data Scientists and Software Engineers to design, build, and optimize scalable data pipelines for our Data Platform. The ideal candidate has strong hands-on experience in distributed data processing, expertise in architecting Data Pipelines and operational skills to drive efficiency and speed.
Key Responsibilities:
- Work closely with the product and other teams to understand business requirements and translate them into robust engineering solutions.
- Architect, build and maintain data solutions and pipelines to meet business requirements.
- Engage and lead a team of talented technologists and establishing engineering best practices.
- Act as a point of contact for managing and driving production defects & stability improvements to resolution
- Oversee end-to-end data workflows from ingestion, transformation, and storage to data lake in both batch and streaming environments.
- Drive improvements in performance, reliability, and scalability of data pipelines
Required Skills & Qualifications:
- A bachelor’s degree in computer science, Math, or other technical fields with 6-8 years of experience, Fintech domain is a plus.
- Demonstrated contribution to end-to-end delivery of enterprise-grade software
- Experience on Data Engineering, Data Lakes, Data Warehousing, Data Pipelines, Data Analytics tools.
- Experience using open-source Big Data technologies such as Spark, Hadoop, Elasticsearch, Kafka.
- Hands-on experience in enterprise application development
- Strong SQL skills and experience working with relational and NoSQL databases.
- Experience in cloud technologies is a plus
- Demonstrates advanced proficiency in Python for building scalable data pipelines, automating workflows
- Experience building scalable ETL/ELT data pipelines and data lakes/warehouses.
- Familiarity with workflow orchestration tools (Airflow, ADF, Glue Workflow, etc.).
- Solid understanding of CI/CD, version control, and DevOps practices.
- Experience working in agile, collaborative environments.
- Attention to detail and quality, and the ability to work well in and across teams
- Ability to advocate & influence multiple stakeholders
- Excellent problem-solving, communication, and leadership skills.
- Experience in leading a team of highly skilled engineers
Arcesium's Personal Data Privacy Notice for Candidates is linked here.
Recruiting Security
Emails from genuine Arcesium recruiters who are employees of the company will always come from the @arcesium.com domain. In some cases, you may also be contacted by independent search firms engaged to recruit on our behalf; emails from their employees should always come from their firm's applicable domain. We'll never ask for your banking information or any payment as part of the recruiting process. If something seems off or you're contacted by an unexpected third party, please reach out to us at careers@arcesium.com (US/UK) or careers-india@arcesium.com (India).
Arcesium is an equal opportunity employer.
Ready to apply?
Apply to Arcesium LLC
Inovalon was founded in 1998 on the belief that technology, and data specifically, would empower the transformation of the entire healthcare ecosystem for the better, improving both outcomes and economics. At Inovalon, we believe that when our customers are successful in their missions, healthcare improves. Therefore, we focus on empowering them with data-driven solutions. And the momentum is building.
Together, as ONE Inovalon, we are a united force delivering solutions that address healthcare’s greatest needs. Through our mission-based culture of inclusion and innovation, our organization brings value not just to our customers, but to the millions of patients and members they serve.
Job Title: Senior Full Stack Machine Learning Engineer
About Us:
Inovalon is a leading healthcare technology company dedicated to revolutionizing the healthcare industry through innovative AI and machine learning solutions. Our mission is to leverage cutting-edge technology to improve health outcomes and streamline healthcare processes. We are looking for a talented and experienced Senior Full Stack Machine Learning Engineer to join our dynamic team.
Job Description:
- As a Senior Full Stack Machine Learning Engineer, you will play a pivotal role in designing, developing, and deploying machine learning models that drive our healthcare solutions. You will work closely with data scientists, software engineers, and product managers to build scalable and robust machine learning systems. Your expertise will help us transform healthcare data into actionable insights, ultimately improving patient care and operational efficiency.
Key Responsibilities:
- Model Development: Design, implement, and optimize machine learning models for various healthcare applications, including predictive analytics, natural language processing, and generative AI.
- End-to-End Deployment: Develop and maintain the full lifecycle of machine learning solutions, from data preprocessing and model training to deployment and monitoring in production environments.
- Data Engineering: Collaborate with data engineers to build and maintain data pipelines, ensuring the availability of high-quality data for training and inference.
- Software Development: Write clean, efficient, and maintainable code for machine learning applications, ensuring seamless integration with existing systems.
- Performance Optimization: Continuously monitor and improve the performance of machine learning models and systems, addressing issues related to scalability, latency, and accuracy.
- Collaboration: Work closely with cross-functional teams to understand business requirements and translate them into technical solutions.
- Mentorship: Provide guidance and mentorship to junior engineers, fostering a culture of continuous learning and innovation.
Qualifications:
- Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field. Ph.D. is a plus.
- Experience: Minimum of 5 years of experience in machine learning, with a proven track record of deploying models in production environments.
Technical Skills:
- Proficiency in Python and relevant ML libraries (e.g., TensorFlow, PyTorch, scikit-learn).
- Proficiency in building applications leveraging generative AI technologies which includes LLM’s, prompt engineering, Vector Databases, RAG architectures and transfer learning is a plus.
- Experience with cloud platforms (e.g., AWS, GCP, Azure) and containerization technologies (e.g., Docker, Kubernetes).
- Strong knowledge of data structures, algorithms, and software engineering best practices.
- Familiarity with big data technologies (e.g., Hadoop, Spark) and data pipeline tools (e.g., Airflow, Kafka).
- Experience with frontend and backend development, including frameworks such as React, Node.js, and Django.
Domain Knowledge: Understanding of healthcare data standards (e.g., HL7, FHIR) and regulations (e.g., HIPAA) is a plus.
Soft Skills:
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities.
- Ability to work independently and as part of a team in a fast-paced environment.
What We Offer:
- Competitive salary and benefits package.
- Opportunity to work on impactful projects that improve healthcare outcomes.
- Collaborative and innovative work environment.
- Professional development and growth opportunities.
- Flexible work arrangements, including hybrid and remote options.
Join us in our mission to transform healthcare through technology! We look forward to hearing from you.
This position is not eligible for immigration sponsorship (e.g. H-1B, TN, or E-3). Applicants must be authorized to work in the United States as a condition of employment. (This is only applicable for US-based positions)
If you don’t meet every qualification listed but are excited about our mission and the work described, we encourage you to apply. Inovalon is most interested in finding the best candidate for the job, and you may be just the right person for this or other roles.
By embracing inclusion, we enhance our work environment and drive business success. Inovalon strives to provide equal opportunities to the communities where we operate and to our clients and everyone whom we serve. We endeavor to create a culture of inclusion in which our associates feel empowered to bring their full, authentic selves to work and pursue their professional goals in an equitable setting. We understand that by fostering this type of culture, and welcoming different perspectives, we generate innovation and growth.
Inovalon is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirement.
To review the legal requirements, including all labor law posters, please visit this link
To review the California Consumer Privacy Statement: Disclosures for California Residents, please visit this link
Ready to apply?
Apply to Inovalon
At Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.
What unites Anaplanners across teams and geographies is our collective commitment to our customers’ success and to our Winning Culture.
Our customers rank among the who’s who in the Fortune 50. Coca-Cola, LinkedIn, Adobe, LVMH and Bayer are just a few of the 2,400+ global companies who rely on our best-in-class platform.
Our Winning Culture is the engine that drives our teams of innovators. We champion diversity of thought and ideas, we behave like leaders regardless of title, we are committed to achieving ambitious goals, and we love celebrating our wins – big and small.
Supported by operating principles of being strategy-led, values-based and disciplined in execution, you’ll be inspired, connected, developed and rewarded here. Everything that makes you unique is welcome; join us and let’s build what’s next - together!
We are seeking an experienced Data Engineer to join our growing data team. The ideal candidate will have a strong background in building and maintaining scalable data pipelines and will be proficient in a variety of data technologies. You will be responsible for designing, implementing, and managing our data infrastructure to support our data analytics and business intelligence initiatives.
Key Responsibilities
- Data Pipeline Development: Design, build, and maintain robust and scalable ETL/ELT pipelines using tools like Informatica, Airflow, and Dataproc.
- Data Warehousing: Develop and manage our data warehouse on Google BigQuery, ensuring data is accurate, available, and easily accessible.
- Data Processing: Utilize Databricks and Python to process and transform large volumes of data efficiently.
- DevOps & Automation: Implement and manage CI/CD pipelines for data workflows using DevOps best practices and tools like Dataform and GitHub.
- Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand their data needs and provide them with high-quality data.
- Data Modeling & BI (Optional): Develop data models and support BI initiatives by creating dashboards and reports in tools like Tableau.
- EPM Integration (Optional): Integrate data with our Anaplan platform to support connected financial and operational business planning.
Required Skills and Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related technical field.
- 3+ years of proven experience in a data engineering role.
- Strong proficiency in SQL and extensive experience with Google BigQuery.
- Hands-on experience with big data technologies such as Databricks and Dataproc.
- Experience with ETL/ELT tools like Informatica.
- Proficiency in orchestrating data pipelines with Airflow.
- Strong programming skills in Python.
- Solid understanding of DevOps principles and experience with Dataform and GitHub for CI/CD and version control.
Preferred Qualifications
- Experience with BI visualization tools, particularly Tableau.
- Familiarity with EPM tools, especially Anaplan.
- Relevant certifications in Google Cloud Platform (GCP) or other data technologies.
- Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
Our Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren’t just words on paper – this is what drives our innovation, it’s how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
Anaplan does not:
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
All emails from Anaplan would come from an @anaplan.com email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to people@anaplan.com before taking any further action in relation to the correspondence.
Ready to apply?
Apply to AnaplanAt Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.
What unites Anaplanners across teams and geographies is our collective commitment to our customers’ success and to our Winning Culture.
Our customers rank among the who’s who in the Fortune 50. Coca-Cola, LinkedIn, Adobe, LVMH and Bayer are just a few of the 2,400+ global companies who rely on our best-in-class platform.
Our Winning Culture is the engine that drives our teams of innovators. We champion diversity of thought and ideas, we behave like leaders regardless of title, we are committed to achieving ambitious goals, and we love celebrating our wins – big and small.
Supported by operating principles of being strategy-led, values-based and disciplined in execution, you’ll be inspired, connected, developed and rewarded here. Everything that makes you unique is welcome; join us and let’s build what’s next - together!
We are looking for a highly skilled and motivated Data and Analytics Systems Engineer to join our growing team. The ideal candidate will have a strong background in data engineering and analytics, with hands-on experience in designing, building, and maintaining data pipelines and analytics solutions. You will be responsible for leveraging a variety of technologies, including Google Cloud Platform, Google BigQuery, Tableau, Informatica, Databricks, Python, and Anaplan, to deliver high-quality data products and insights to our business stakeholders to ensure our Data & Analytics systems and operations are effective and efficient.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in a data engineering or analytics system management role.
- Proven experience with Google Cloud Platform (GCP), including Google BigQuery, Cloud Storage, and Cloud Functions.
- Experiences with data visualization tools, preferably Tableau.
- Proficiency in ETL/ELT tools like Informatica, Airflow, Dataproc, and so on.
- Experience with big data technologies such as Databricks and Spark.
- Strong programming skills in Python and experience with data manipulation libraries (e.g., Pandas, NumPy).
- Familiarity with the Anaplan platform is a plus.
- Excellent problem-solving and communication skills.
- Ability to work independently and as part of a team.
Our Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren’t just words on paper – this is what drives our innovation, it’s how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
Anaplan does not:
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
All emails from Anaplan would come from an @anaplan.com email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to people@anaplan.com before taking any further action in relation to the correspondence.
Ready to apply?
Apply to AnaplanAt Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.
What unites Anaplanners across teams and geographies is our collective commitment to our customers’ success and to our Winning Culture.
Our customers rank among the who’s who in the Fortune 50. Coca-Cola, LinkedIn, Adobe, LVMH and Bayer are just a few of the 2,400+ global companies who rely on our best-in-class platform.
Our Winning Culture is the engine that drives our teams of innovators. We champion diversity of thought and ideas, we behave like leaders regardless of title, we are committed to achieving ambitious goals, and we love celebrating our wins – big and small.
Supported by operating principles of being strategy-led, values-based and disciplined in execution, you’ll be inspired, connected, developed and rewarded here. Everything that makes you unique is welcome; join us and let’s build what’s next - together!
We are looking for a highly skilled and motivated Data and Analytics Systems Engineer to join our growing team. The ideal candidate will have a strong background in data engineering and analytics, with hands-on experience in designing, building, and maintaining data pipelines and analytics solutions. You will be responsible for leveraging a variety of technologies, including Google Cloud Platform, Google BigQuery, Tableau, Informatica, Databricks, Python, and Anaplan, to deliver high-quality data products and insights to our business stakeholders to ensure our Data & Analytics systems and operations are effective and efficient.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in a data engineering or analytics system management role.
- Proven experience with Google Cloud Platform (GCP), including Google BigQuery, Cloud Storage, and Cloud Functions.
- Experiences with data visualization tools, preferably Tableau.
- Proficiency in ETL/ELT tools like Informatica, Airflow, Dataproc, and so on.
- Experience with big data technologies such as Databricks and Spark.
- Strong programming skills in Python and experience with data manipulation libraries (e.g., Pandas, NumPy).
- Familiarity with the Anaplan platform is a plus.
- Excellent problem-solving and communication skills.
- Ability to work independently and as part of a team.
Our Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren’t just words on paper – this is what drives our innovation, it’s how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
Anaplan does not:
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
All emails from Anaplan would come from an @anaplan.com email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to people@anaplan.com before taking any further action in relation to the correspondence.
Ready to apply?
Apply to AnaplanAt Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.
What unites Anaplanners across teams and geographies is our collective commitment to our customers’ success and to our Winning Culture.
Our customers rank among the who’s who in the Fortune 50. Coca-Cola, LinkedIn, Adobe, LVMH and Bayer are just a few of the 2,400+ global companies who rely on our best-in-class platform.
Our Winning Culture is the engine that drives our teams of innovators. We champion diversity of thought and ideas, we behave like leaders regardless of title, we are committed to achieving ambitious goals, and we love celebrating our wins – big and small.
Supported by operating principles of being strategy-led, values-based and disciplined in execution, you’ll be inspired, connected, developed and rewarded here. Everything that makes you unique is welcome; join us and let’s build what’s next - together!
We are seeking an experienced Data Engineer to join our growing data team. The ideal candidate will have a strong background in building and maintaining scalable data pipelines and will be proficient in a variety of data technologies. You will be responsible for designing, implementing, and managing our data infrastructure to support our data analytics and business intelligence initiatives.
Key Responsibilities
- Data Pipeline Development: Design, build, and maintain robust and scalable ETL/ELT pipelines using tools like Informatica, Airflow, and Dataproc.
- Data Warehousing: Develop and manage our data warehouse on Google BigQuery, ensuring data is accurate, available, and easily accessible.
- Data Processing: Utilize Databricks and Python to process and transform large volumes of data efficiently.
- DevOps & Automation: Implement and manage CI/CD pipelines for data workflows using DevOps best practices and tools like Dataform and GitHub.
- Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand their data needs and provide them with high-quality data.
- Data Modeling & BI (Optional): Develop data models and support BI initiatives by creating dashboards and reports in tools like Tableau.
- EPM Integration (Optional): Integrate data with our Anaplan platform to support connected financial and operational business planning.
Required Skills and Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related technical field.
- 3+ years of proven experience in a data engineering role.
- Strong proficiency in SQL and extensive experience with Google BigQuery.
- Hands-on experience with big data technologies such as Databricks and Dataproc.
- Experience with ETL/ELT tools like Informatica.
- Proficiency in orchestrating data pipelines with Airflow.
- Strong programming skills in Python.
- Solid understanding of DevOps principles and experience with Dataform and GitHub for CI/CD and version control.
Preferred Qualifications
- Experience with BI visualization tools, particularly Tableau.
- Familiarity with EPM tools, especially Anaplan.
- Relevant certifications in Google Cloud Platform (GCP) or other data technologies.
- Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
Our Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren’t just words on paper – this is what drives our innovation, it’s how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
Anaplan does not:
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
All emails from Anaplan would come from an @anaplan.com email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to people@anaplan.com before taking any further action in relation to the correspondence.
Ready to apply?
Apply to AnaplanAt Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.
What unites Anaplanners across teams and geographies is our collective commitment to our customers’ success and to our Winning Culture.
Our customers rank among the who’s who in the Fortune 50. Coca-Cola, LinkedIn, Adobe, LVMH and Bayer are just a few of the 2,400+ global companies who rely on our best-in-class platform.
Our Winning Culture is the engine that drives our teams of innovators. We champion diversity of thought and ideas, we behave like leaders regardless of title, we are committed to achieving ambitious goals, and we love celebrating our wins – big and small.
Supported by operating principles of being strategy-led, values-based and disciplined in execution, you’ll be inspired, connected, developed and rewarded here. Everything that makes you unique is welcome; join us and let’s build what’s next - together!
We are seeking an experienced Data Engineer to join our growing data team. The ideal candidate will have a strong background in building and maintaining scalable data pipelines and will be proficient in a variety of data technologies. You will be responsible for designing, implementing, and managing our data infrastructure to support our data analytics and business intelligence initiatives.
Key Responsibilities
- Data Pipeline Development: Design, build, and maintain robust and scalable ETL/ELT pipelines using tools like Informatica, Airflow, and Dataproc.
- Data Warehousing: Develop and manage our data warehouse on Google BigQuery, ensuring data is accurate, available, and easily accessible.
- Data Processing: Utilize Databricks and Python to process and transform large volumes of data efficiently.
- DevOps & Automation: Implement and manage CI/CD pipelines for data workflows using DevOps best practices and tools like Dataform and GitHub.
- Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand their data needs and provide them with high-quality data.
- Data Modeling & BI (Optional): Develop data models and support BI initiatives by creating dashboards and reports in tools like Tableau.
- EPM Integration (Optional): Integrate data with our Anaplan platform to support connected financial and operational business planning.
Required Skills and Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related technical field.
- 5+ years of proven experience in a data engineering role.
- Strong proficiency in SQL and extensive experience with Google BigQuery.
- Hands-on experience with big data technologies such as Databricks and Dataproc.
- Experience with ETL/ELT tools like Informatica.
- Proficiency in orchestrating data pipelines with Airflow.
- Strong programming skills in Python.
- Solid understanding of DevOps principles and experience with Dataform and GitHub for CI/CD and version control.
Preferred Qualifications
- Experience with BI visualization tools, particularly Tableau.
- Familiarity with EPM tools, especially Anaplan.
- Relevant certifications in Google Cloud Platform (GCP) or other data technologies.
- Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
Our Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren’t just words on paper – this is what drives our innovation, it’s how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
Anaplan does not:
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
All emails from Anaplan would come from an @anaplan.com email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to people@anaplan.com before taking any further action in relation to the correspondence.
Ready to apply?
Apply to Anaplandunnhumby is the global leader in Customer Data Science, partnering with the world’s most ambitious retailers and brands to put the customer at the heart of every decision. We combine deep insight, advanced technology, and close collaboration to help our clients grow, innovate, and deliver measurable value for their customers.
dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola, Nestlé, Unilever and Metro.
We’re looking for a Sr Big Data Engineer who expects more from their career. It’s chance to extend and improve dunnhumby’s Data Engineering Team. It’s an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers.
Key Responsibilities
- Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms.
- Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming to handle high-velocity data streams.
- Design /Orchestrate end-to-end automation using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies.
- Design intelligent systems that can detect anomalies, trigger alerts, and automatically reroute or restart processes to maintain data integrity and availability.
- Define and implement data governance, metadata management, and data quality standards.
- Lead architectural reviews and technical design sessions to guide solution development.
- Partner with business and IT teams to translate business needs into data architecture requirements.
- Ensure security, compliance, and regulatory requirements are addressed in all data solutions.
- Evaluate and recommend improvements to existing data architecture and processes.
Technical Expertise
- Bachelors or masters degree in computer science, Information Systems, Data Science, or related field.
- Extensive experience with high level programming languages - Python, Java or Scala
- 5+ years of experience in data architecture, data engineering, or a related field.
- Proficient in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar.
- Experience with data governance frameworks and tools (e.g., Collibra, Alation, OpenMetadata).
- Strong knowledge of cloud platforms (Azure or Google Cloud), especially with cloud-native data services.
- Experience working in Agile or DevOps environments.
- Experience with modern data stack tools (e.g., dbt, Snowflake, Databricks).
- Experience with Hive, Oozie, Airflow, HBase, MapReduce, Spark along with working knowledge of Hadoop/Spark Toolsets.
- Extensive Experience working with Git and Process Automation
- In depth understanding of relational database management systems (RDBMS) and Data Flow Development.
What you can expect from us
We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect. Plus, thoughtful perks, like flexible working hours and your birthday off.
You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.
And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One, dh Enabled and dh Thrive as the living proof. We want everyone to have the opportunity to shine and perform at your best throughout our recruitment process. Please let us know how we can make this process work best for you.
Our approach to Flexible Working
At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.
We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.
For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)
Ready to apply?
Apply to dunnhumby
dunnhumby is the global leader in Customer Data Science, partnering with the world’s most ambitious retailers and brands to put the customer at the heart of every decision. We combine deep insight, advanced technology, and close collaboration to help our clients grow, innovate, and deliver measurable value for their customers.
dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola, Nestlé, Unilever and Metro.
We’re looking for a Senior Engineer (C/C++) who expects more from their career. It’s a chance to extend and improve dunnhumby’s team. We’re a Backend Developers team working with a wide variety of tools and technologies. We handle the whole lifecycle of our teams’ solutions, from inception and development to their deployment and maintenance via Infrastructure as Code. Joining our engineering team, you’ll work with passionate people to build, extend, and maintain the platform that underpins all our client engagements.
Required Skills:
• 6-10 years of experience in software development.
• 6+ years of experience in C/C++, Python and Shell Scripting.
• Fluency in working with databases.
• Have sound understanding of Object-Oriented Programming and development principles.
• Experience working in an Agile Scrum or Kanban development environment.
• Solid understanding of high-quality coding, testing, and development practices
• HLD, LLD, Design Principles , Distributed Computing and Functional programming
Good to Have Skills:
• Experience working with version control software (preferably Git).
• Experience with CI/CD pipelines.
• Experience of working with Dockers and Containers.
• Experience of working with Kubernetes.
• Experience of working with Airflow.
• Experience of working on Cloud environments (GCP and Azure)
What you can expect from us
We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect. Plus, thoughtful perks, like flexible working hours and your birthday off.
You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.
And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One, dh Enabled and dh Thrive as the living proof. We want everyone to have the opportunity to shine and perform at your best throughout our recruitment process. Please let us know how we can make this process work best for you.
Our approach to Flexible Working
At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.
We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.
For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)
Ready to apply?
Apply to dunnhumby
Inovalon was founded in 1998 on the belief that technology, and data specifically, would empower the transformation of the entire healthcare ecosystem for the better, improving both outcomes and economics. At Inovalon, we believe that when our customers are successful in their missions, healthcare improves. Therefore, we focus on empowering them with data-driven solutions. And the momentum is building.
Together, as ONE Inovalon, we are a united force delivering solutions that address healthcare’s greatest needs. Through our mission-based culture of inclusion and innovation, our organization brings value not just to our customers, but to the millions of patients and members they serve.
Overview: The Staff Engineer L6 will lead development for various Inovalon products and platforms, innovating to overcome current technology and bring in solutions which solve resiliency, scale, security in a completely different level. This position brings subject matter expertise as a functional lead and is tasked with ensuring a high level of discipline and productivity.
Duties and Responsibilities:
- Design, build and maintain scalable data pipelines and access patterns related to Healthcare;
- Research, evaluate and utilize new technologies/tools/frameworks centered around high-volume data processing in AWS;
- Document and estimate technical scope and design as needed;
- Design and implement distributed data processing pipelines using tools and languages prevalent in the big data ecosystem;
- Involve and drive a culture of collaborative reviews of designs, code and test plans;
- Work with business stake holders and architecture engineering leads to ensure quality solutions are implemented and engineering standard methodologies adhered to;
- DevOps Focus – Utilize and advance continuous integration and deployment frameworks;
- Solve complex data issues and perform root cause analysis;
- Work across teams to resolve operational & performance issues;
- Provide strong technical leadership, raise the technical bar, maintain data and results-driven culture, nurture a team culture of high collaboration and trust;
- Communicate effectively and partner well with other disciplines of the project team to deliver high quality solutions from ideas to production code;
- Facilitate alignment and clarity across teams on goals, outcomes, and timelines;
- Influence and coach junior engineers both within and outside of your team; be a role model for an open, honest, and inclusive approach to problem solving;
- Adhere to all confidentiality, HIPAA, regulatory, and other such policies, procedures, and requirements as outlined within Employer’s Operating Policies and Procedures in all ways and at all times with respect to any aspect of the data handled or services rendered in the scope of work; and
- Maintain compliance with Inovalon’s policies, procedures and mission statement, and fulfill those responsibilities and/or duties that may be reasonably provided by Inovalon for the purpose of achieving operational and financial success.
- On Call Policy: Participate in an on-call rotation to support critical issues, including outside standard business hours (evenings, weekends, and holidays), ensuring timely response and resolution as per defined SLAs.
Job Requirements:
- Minimum of 15 years of IT experience with at least 2 years’ experience with public cloud (AWS) including serverless architecture and cloud warehousing solutions;
- Strong experience using managed services and Infrastructure as a service;
- Minimum of 6 years of strong experience with Big Data and related processing frameworks such as EMR, Spark, and Airflow;
- Strong programming experience in Python and Java/Scala preferred;
- Hands on coder who masters the concepts behind distributed databases, batch and stream processing;
- Experience in working with extremely large healthcare datasets, preferably in an architect/lead role;
- Strong familiarity with healthcare data formats including EDI and FHIR;
- Architect and code solutions for large-scale ETL pipelines with data processing frameworks and sourcing data from a diverse array of sources;
- Experience building microservices and event-based architectures;
- Experience with AWS Security Architecture including IAM, Security groups and policies;
- Hands on experience with Terraform and/or cloud formation;
- Experience building CI/CD pipelines in public cloud and creating a culture of automation;
- Experience leading and mentoring junior engineers;
- Experience working with scalable cloud-based data warehousing solutions such as Redshift, Snowflake;
- Systems engineering experience, including engineering, configuration, and management of Windows and Linux server environments;
- Prior Experience in migration of applications to AWS;
- Excellent oral and written communication skills;
- Excellent time management and problem-solving skills with the ability to adapt to changing business priorities and operational demands;
- Extensive knowledge of existing industry standards, technologies, and operational norms;
- An entrepreneurial spirit with a can-do attitude – a self-starter, project finisher and adaptive, collaborative; and
- Experienced in utilizing data and analysis to explain technical problems and providing detailed feedback and solutions.
Education:
- Bachelor's degree in computer science, Software Engineering, or Information Technology
Physical Demands and Work Environment:
- Sedentary work (i.e. sitting for long periods of time).
- Subject to inside environmental conditions.
- Travel for this position will include less than 10% locally usually for training purposes.
This position is not eligible for immigration sponsorship (e.g. H-1B, TN, or E-3). Applicants must be authorized to work in the United States as a condition of employment. (This is only applicable for US-based positions)
If you don’t meet every qualification listed but are excited about our mission and the work described, we encourage you to apply. Inovalon is most interested in finding the best candidate for the job, and you may be just the right person for this or other roles.
By embracing inclusion, we enhance our work environment and drive business success. Inovalon strives to provide equal opportunities to the communities where we operate and to our clients and everyone whom we serve. We endeavor to create a culture of inclusion in which our associates feel empowered to bring their full, authentic selves to work and pursue their professional goals in an equitable setting. We understand that by fostering this type of culture, and welcoming different perspectives, we generate innovation and growth.
Inovalon is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirement.
To review the legal requirements, including all labor law posters, please visit this link
To review the California Consumer Privacy Statement: Disclosures for California Residents, please visit this link
Ready to apply?
Apply to Inovalon
Inovalon was founded in 1998 on the belief that technology, and data specifically, would empower the transformation of the entire healthcare ecosystem for the better, improving both outcomes and economics. At Inovalon, we believe that when our customers are successful in their missions, healthcare improves. Therefore, we focus on empowering them with data-driven solutions. And the momentum is building.
Together, as ONE Inovalon, we are a united force delivering solutions that address healthcare’s greatest needs. Through our mission-based culture of inclusion and innovation, our organization brings value not just to our customers, but to the millions of patients and members they serve.
Overview: The Staff Engineer L6 will lead development for various Inovalon products and platforms, innovating to overcome current technology and bring in solutions which solve resiliency, scale, security in a completely different level. This position brings subject matter expertise as a functional lead and is tasked with ensuring a high level of discipline and productivity.
Duties and Responsibilities:
- Design, build and maintain scalable data pipelines and access patterns related to Healthcare;
- Research, evaluate and utilize new technologies/tools/frameworks centered around high-volume data processing in AWS;
- Document and estimate technical scope and design as needed;
- Design and implement distributed data processing pipelines using tools and languages prevalent in the big data ecosystem;
- Involve and drive a culture of collaborative reviews of designs, code and test plans;
- Work with business stake holders and architecture engineering leads to ensure quality solutions are implemented and engineering standard methodologies adhered to;
- DevOps Focus – Utilize and advance continuous integration and deployment frameworks;
- Solve complex data issues and perform root cause analysis;
- Work across teams to resolve operational & performance issues;
- Provide strong technical leadership, raise the technical bar, maintain data and results-driven culture, nurture a team culture of high collaboration and trust;
- Communicate effectively and partner well with other disciplines of the project team to deliver high quality solutions from ideas to production code;
- Facilitate alignment and clarity across teams on goals, outcomes, and timelines;
- Influence and coach junior engineers both within and outside of your team; be a role model for an open, honest, and inclusive approach to problem solving;
- Adhere to all confidentiality, HIPAA, regulatory, and other such policies, procedures, and requirements as outlined within Employer’s Operating Policies and Procedures in all ways and at all times with respect to any aspect of the data handled or services rendered in the scope of work; and
- Maintain compliance with Inovalon’s policies, procedures and mission statement, and fulfill those responsibilities and/or duties that may be reasonably provided by Inovalon for the purpose of achieving operational and financial success.
- On Call Policy: Participate in an on-call rotation to support critical issues, including outside standard business hours (evenings, weekends, and holidays), ensuring timely response and resolution as per defined SLAs.
Job Requirements:
- Minimum of 15 years of IT experience with at least 2 years’ experience with public cloud (AWS) including serverless architecture and cloud warehousing solutions;
- Strong experience using managed services and Infrastructure as a service;
- Minimum of 6 years of strong experience with Big Data and related processing frameworks such as EMR, Spark, and Airflow;
- Strong programming experience in Python and Java/Scala preferred;
- Hands on coder who masters the concepts behind distributed databases, batch and stream processing;
- Experience in working with extremely large healthcare datasets, preferably in an architect/lead role;
- Strong familiarity with healthcare data formats including EDI and FHIR;
- Architect and code solutions for large-scale ETL pipelines with data processing frameworks and sourcing data from a diverse array of sources;
- Experience building microservices and event-based architectures;
- Experience with AWS Security Architecture including IAM, Security groups and policies;
- Hands on experience with Terraform and/or cloud formation;
- Experience building CI/CD pipelines in public cloud and creating a culture of automation;
- Experience leading and mentoring junior engineers;
- Experience working with scalable cloud-based data warehousing solutions such as Redshift, Snowflake;
- Systems engineering experience, including engineering, configuration, and management of Windows and Linux server environments;
- Prior Experience in migration of applications to AWS;
- Excellent oral and written communication skills;
- Excellent time management and problem-solving skills with the ability to adapt to changing business priorities and operational demands;
- Extensive knowledge of existing industry standards, technologies, and operational norms;
- An entrepreneurial spirit with a can-do attitude – a self-starter, project finisher and adaptive, collaborative; and
- Experienced in utilizing data and analysis to explain technical problems and providing detailed feedback and solutions.
Education:
- Bachelor's degree in computer science, Software Engineering, or Information Technology
Physical Demands and Work Environment:
- Sedentary work (i.e. sitting for long periods of time).
- Subject to inside environmental conditions.
- Travel for this position will include less than 10% locally usually for training purposes.
This position is not eligible for immigration sponsorship (e.g. H-1B, TN, or E-3). Applicants must be authorized to work in the United States as a condition of employment. (This is only applicable for US-based positions)
If you don’t meet every qualification listed but are excited about our mission and the work described, we encourage you to apply. Inovalon is most interested in finding the best candidate for the job, and you may be just the right person for this or other roles.
By embracing inclusion, we enhance our work environment and drive business success. Inovalon strives to provide equal opportunities to the communities where we operate and to our clients and everyone whom we serve. We endeavor to create a culture of inclusion in which our associates feel empowered to bring their full, authentic selves to work and pursue their professional goals in an equitable setting. We understand that by fostering this type of culture, and welcoming different perspectives, we generate innovation and growth.
Inovalon is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirement.
To review the legal requirements, including all labor law posters, please visit this link
To review the California Consumer Privacy Statement: Disclosures for California Residents, please visit this link
Ready to apply?
Apply to Inovalon
Staff Machine Learning Engineer
Share this job
About AiDASH
AiDASH is an enterprise AI company and the leading provider of vegetation risk intelligence for electric utilities. Powered by proprietary VegetationAI™ technology, AiDASH delivers a unified remote grid inspection and monitoring platform that uses a SatelliteFirst approach to identify and address vegetation and other threats to the grid. With a prevention-first strategy to mitigate wildfire risk and minimize storm impacts, AiDASH helps more than 140 utilities reduce costs, improve reliability, and lower liability across their networks. AiDASH exists to safeguard critical utility infrastructure and secure the future of humanAIty™. Learn more at www.aidash.com.
We are a Series C growth company backed by leading investors, including Shell Ventures, National Grid Partners, G2 Venture Partners, Duke Energy, Edison International, Lightrock, Marubeni, among others. We have been recognized by Forbes two years in a row as one of “America’s Best Startup Employers.” We are also proud to be one of the few software companies in Time Magazine’s “America’s Top GreenTech Companies 2024”. Deloitte Technology Fast 500™ recently ranked us at No. 12 among San Francisco Bay Area companies, and No. 59 overall in their selection of the top 500 for 2024.
Join us in Securing Tomorrow!
The Role
We’re looking for a seasoned Staff Machine Learning Engineer to shape and scale the backbone of our production ML ecosystem. In this role, you will architect high-performing ML systems that power our geospatial intelligence platform, transforming large-scale satellite and aerial imagery into actionable insights. You’ll lead end-to-end ownership—from model deployment and MLOps to infrastructure design—while partnering closely with data science, platform engineering, and product teams to deliver reliable, scalable, and cost-efficient ML solutions. If you thrive at the intersection of deep technical expertise, system design, and cross-functional collaboration, this role is for you.
How you'll make an impact:
ML System Architecture & Production Deployment
- Design, build, and maintain end-to-end ML pipelines for batch processing of satellite and aerial imagery
- Deploy and scale ML models in production on AWS infrastructure, leveraging services like SageMaker, Bedrock,or custom-built solutions
- Implement MLflow for experiment tracking, model versioning, and model registry management
- Architect batch inference systems optimized for throughput and cost-efficiency
- Work with geospatial data formats and coordinate reference systems
- Collaborate with data scientists to transition models from research to production
- Partner with platform engineering to build scalable compute, GPU clusters, and storage layers
ML Operations & Reliability
- Implement comprehensive model monitoring systems to track performance degradation and data drift
- Design and execute canary deployments and A/B testing frameworks for safe model rollouts
- Build active learning pipelines to continuously improve model performance with minimal labeling effort
- Establish model evaluation frameworks and performance benchmarking processesCreate alerting and observability systems for production ML workloads
Technical Leadership
- Mentor ML engineers and data scientists on best practices for production ML
- Drive technical decision-making on ML infrastructure and tooling
- Collaborate with platform and data engineering teams to optimize the ML stack
- Establish engineering standards and contribute to architectural roadmaps
What we’re looking for:
- 5+ years of experience in machine learning engineering with 2+ years in a senior or lead capacity
- Proven track record deploying and maintaining ML systems in production using AWS services (SageMaker,Lambda, ECS/EKS, S3, etc.)
- Strong hands-on experience with tools like MLflow, WandB, or similar for experiment tracking and model management
- Deep expertise in image segmentation and computer vision techniques using frameworks like PyTorch or TensorFlow
- Production experience with ensemble models (xgboost, lightgbm, RF)
ML Operations Expertise
- Experience implementing model monitoring, drift detection, and alerting systems
- Hands-on experience with canary deployments, A/B testing and Shadow deployments for ML models
- Knowledge of active learning strategies and human-in-the-loop ML systems
- Strong understanding of model evaluation metrics, bias detection, and performance analysis
Technical Skills
- Expert-level Python programming with ML libraries (scikit-learn, PyTorch/TensorFlow, NumPy, pandas, etc)
- Experience with distributed batch processing frameworks (Airflow, Step Functions, Argo Workflows, Spark,Dask, Ray or similar)
- Proficiency with AWS ML ecosystem and infrastructure-as-code (Terraform, CloudFormation)
- Hands-on experience with dataset versioning tools such as DVC, LakeFS, Delta Lake, Quilt, or Pachyderm
- Strong software engineering fundamentals: unit/integration testing, CI/CD, version control, observability, designpatterns
- Experience with containerization (Docker, Kubernetes) for model deployment
- Good to have experience with ML Orchestration tools like Kubeflow, Vertex AI, etc
- Nice to have experience with GPUs: scheduling GPU jobs, optimizing GPU performance, memory profiling
We are proud to be an equal-opportunity employer. We are committed to embracing diversity and inclusion in our hiring practices, and we promote a work environment where everyone, from any race, color, religion, sex, sexual orientation, gender identity, or national origin, can do their best work.
Read our Privacy Policy here: https://www.aidash.com/policy/privacy-policy/
Ready to apply?
Apply to AiDASH, Inc.
SDE 3 - Platform
Share this job
About AiDASH
AiDASH is an enterprise AI company and the leading provider of vegetation risk intelligence for electric utilities. Powered by proprietary VegetationAI™ technology, AiDASH delivers a unified remote grid inspection and monitoring platform that uses a SatelliteFirst approach to identify and address vegetation and other threats to the grid. With a prevention-first strategy to mitigate wildfire risk and minimize storm impacts, AiDASH helps more than 140 utilities reduce costs, improve reliability, and lower liability across their networks. AiDASH exists to safeguard critical utility infrastructure and secure the future of humanAIty™. Learn more at www.aidash.com.
We are a Series C growth company backed by leading investors, including Shell Ventures, National Grid Partners, G2 Venture Partners, Duke Energy, Edison International, Lightrock, Marubeni, among others. We have been recognized by Forbes two years in a row as one of “America’s Best Startup Employers.” We are also proud to be one of the few software companies in Time Magazine’s “America’s Top GreenTech Companies 2024”. Deloitte Technology Fast 500™ recently ranked us at No. 12 among San Francisco Bay Area companies, and No. 59 overall in their selection of the top 500 for 2024.
Join us in Securing Tomorrow!
The Role
As a SDE - 3 Platform Engineer at AiDASH, you’ll design, develop, and deploy scalable distributed systems that power our AI-driven platforms. You will build robust backend services, automated data pipelines, and analytical tools to deliver actionable insights for business-critical decisions. Partnering with product, data, and design teams, you’ll own solutions end-to-end — from system design to production deployment. Your expertise in microservices, Python, SQL/NoSQL, and cloud-native technologies (AWS, Docker, Kubernetes) will help evolve our architecture as we scale. Beyond coding, you’ll drive engineering excellence through code reviews, refactoring, and performance optimization, while mentoring peers and fostering reusable, reliable system design. If you thrive in solving complex challenges, love automation, and want to shape systems that make industries climate-resilient, this role is for you.
How you'll make an impact:
-
Design, develop and deploy scalable distributed systems
-
Build scalable backend systems to support internal workflows backed by automated pipelines and ML models
-
Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics
-
Implement scalable and resilient distributed systems, ensuring fault tolerance and reliability
-
Working with stakeholders including data, design, product, and executive teams and assisting them with data-related technical issues
-
Deploy code to production and engage in code reviews
-
Refactor service to improve code quality, runtime efficiency and resource optimisation
-
Build tools that improve engineering productivity
-
Become in-house experts and provide consultation to product engineers to help them build scalable and reliable systems
-
Evolve AiDASH architecture as we continue to scale
-
Monitor, analyze, and optimize system performance and resource utilization
-
Stay updated with the latest advancements in distributed systems technologies and methodologies
What we're looking for:
Technical Expertise
-
6+ years of experience designing and building scalable, reusable platforms/components
-
Strong in system design & microservices, distributed systems, and API development
-
Solid foundation in data structures & algorithms
-
Hands-on expertise in Python (backend, data workflows), Spark/Big Data, Airflow and Snowflake (Good to have)
-
Strong with SQL/NoSQL (PostgreSQL, MySQL, MongoDB)
-
Experienced in RESTful API design, development, and maintenance
-
Proficient with cloud platforms (AWS), containers (Docker, Kubernetes)
Engineering Maturity
-
Strong debugging and optimization skills — load testing, profiling, bottleneck resolution
-
Familiar with backend security (encryption, authN/authZ, compliance)
-
Experience in CI/CD, unit testing, code coverage, and ensuring production readiness
-
Focus on scalability, reliability, and cost-efficient systems
Collaboration & Leadership
-
Works well cross-functionally to build reusable platform capabilities
-
Mentors peers, reviews with architectural thinking
-
Can break down complex product requirements into platform abstractions
Mindset & Growth
-
Thrives in fast-paced environments, balancing delivery with long-term vision
-
Curious and open to exploring new tech (e.g., AI-powered workflows
-
Bias for automation and building self-service, reusable solutions
We are proud to be an equal-opportunity employer. We are committed to embracing diversity and inclusion in our hiring practices, and we promote a work environment where everyone, from any race, color, religion, sex, sexual orientation, gender identity, or national origin, can do their best work.
Read our Privacy Policy here: https://www.aidash.com/policy/privacy-policy/
Ready to apply?
Apply to AiDASH, Inc.