About this Sr Data Specialist - R01564133 role at Brillio
Brillio · Hybrid · Bangalore, Karnataka, India
Lead Data Engineer
Primary Skills
Specialization
Job requirements
Job Description – AWS Senior Data Engineer
Company: Brillio Technologies
Role: Senior Data Engineer
Experience: 5–10 Years
Location: India (Hybrid/Remote)
Employment Type: Full-time
Role Overview
We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and modern data architectures, along with exposure to APIs and real-time data processing.
Key Responsibilities
- Design and build scalable data pipelines using Spark / PySpark on Databricks
- Develop and optimize ETL/ELT workflows for large-scale data processing
- Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing
- Build and maintain data lake and data warehouse solutions
- Develop and integrate APIs for data ingestion and consumption
- Implement data processing workflows using Airflow / Autosys
- Optimize data performance using partitioning, caching, and query tuning
- Handle structured and semi-structured data from multiple sources
- Ensure data quality, governance, and reliability of pipelines
- Collaborate with cross-functional teams (Analytics, Product, Engineering)
- Strong expertise in:
- Apache Spark / PySpark
- Databricks (mandatory)
- SQL (PostgreSQL or similar)
- Hands-on experience with:
- AWS EMR, S3
- Hadoop, Hive ecosystem
- Programming skills:
- Python (must-have)
- Scala (good exposure)
- Strong knowledge of:
- UNIX / Shell scripting
- ETL pipelines and data engineering fundamentals
- Experience with:
- Elasticsearch (data storage & retrieval)
- Workflow orchestration tools (Airflow, Autosys)
- Exposure to:
- API development & integration
- Version control tools (Git / SVN)
- Basic knowledge of HTML (for web/data tasks)
- Minimum 5+ years of Data Engineering experience
- Strong hands-on experience in Databricks + Big Data stack
- Proven experience working on large-scale distributed systems
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
- Work on cloud-native, large-scale data platforms
- Exposure to cutting-edge Databricks & AWS ecosystems
- Collaborative and high-performance engineering culture
Technical Skills (Must-Have)
Additional Skills
Experience Required
Education
Why Join Brillio
Ready to apply to Brillio?
Apply to Brillio
