Key Responsibilities: 

• Develop and optimize large-scale data pipelines and workflows using Snowflake, Google Cloud, and AWS services to support data integration and storage needs. 

• Lead development of complex ETL processes and SQL-based transformations to support data analysis, reporting, and advanced insights across the organization. 

• Implement and enhance data solutions using AWS services (e.g., S3, EC2, Lambda, Fargate, SQS), and GCP BigQuery, STS, Cloud Functions, and Vertex AI for scalable ML and data workflows. 

• Develop advanced Python scripts to automate data processing and integration, including prompt-driven interactions with generative AI models and autonomous agents. 

• Collaborate with analysts and stakeholders to support data visualization needs in ThoughtSpot and Tableau, designing robust and scalable reporting solutions. 

• Establish data quality standards and perform thorough validation and testing, including validation of AI-generated outputs. 

• Mentor and guide junior and associate data engineers, providing technical direction and promoting best practices in AI-integrated data workflows. 

• Perform code reviews, troubleshoot performance issues, and lead optimization efforts to improve data processing efficiency and scalability. 

• Stay current with emerging technologies in data engineering and AI, including generative AI principles, prompt engineering, knowledge base creation, semantic layer generation, agentic frameworks, and GCP data stack tools. 

Qualifications  

• Bachelor’s degree in Computer Science, Data Science, Information Systems, or a related field 

• 6 to 9 years of experience in data engineering, with advanced expertise in Snowflake, SQL, and ETL processes. 

• Strong proficiency in Python for complex data processing, automation, and integration tasks, including prompt engineering for generative AI workflows. 

• Hands-on experience with AWS services (e.g., S3, EC2, Lambda, Fargate); experience with GCP data tools such as BigQuery, Cloud Functions, and Vertex AI is highly desirable. 

• Experience in managing and deploying CICD pipelines across data and cloud platform services using Jenkins, Terraform, Harness and ADK 

• Proficiency with data visualization tools, particularly ThoughtSpot, for supporting scalable and impactful reporting. 

• Experience in Informatica or similar ETL tools, with the ability to lead data integration and transformation efforts. 

• Experience implementing generative AI solutions, including model orchestration, output validation, and integration into data workflows. 

• Familiarity with agentic AI frameworks and autonomous data orchestration tools. 

• Strong problem-solving skills with a focus on data quality, accuracy, and efficiency. 

• Excellent communication skills, with the ability to work collaboratively across teams and present technical concepts to non-technical stakeholders. 

• Proven experience mentoring and leading junior team members, with a commitment to fostering a collaborative team environment.