About this Principal Data Engineer role at Brillio
Brillio · Hybrid · Hyderabad, Andhra Pradesh, India
Senior Tech Specialist, Applications & Platforms
Job requirements
Experience Range: 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions Key Responsibilities:
Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirementsDevelop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMSConfigure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API GatewayMonitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open SearchAutomate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCTLead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platformsCollaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platformsEnsure data security, quality, and compliance by implementing best practices and leveraging AWS security featuresLead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organizationRequired Skills:
Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCTExpertise in designing and managing data pipelines and ETL workflows on AWSStrong experience with Spark using Scala for large-scale data processingHands-on experience with Oozie workflow scheduling and orchestrationDeep knowledge of real-time data streaming with AWS KinesisProven ability to configure and optimize Amazon Redshift for analytics workloadsExperience with DynamoDB for NoSQL database solutionsAbility to automate infrastructure and deployments using AWS CloudFormationProficient in monitoring and logging with AWS CloudWatchSkill in managing data migration and schema transformation using AWS SCT and DMSDemonstrated expertise in implementing and integrating MDM solutions using StiboPreferred Skills:
Experience integrating AWS data solutions with third-party analytics and visualization toolsProficiency in optimizing performance for large-scale distributed data systemsKnowledge of emerging AWS data services and staying current with industry best practicesExperience with Open Search for search and analytics use casesBackground in mentoring and leading engineering teams in cloud data projectsFamiliarity with advanced data governance and data quality frameworks in MDM environmentsDesired Qualifications:
Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a closely related disciplineAWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – ProfessionalCertification in Apache Spark or relevant big data technologies (preferred)
