Skillsoft are seeking an experienced Data Engineer who is excited to grow in their professional development. This is a hands-on technical role suited for someone who is comfortable writing and debugging code, eager to deepen their expertise in data engineering patterns, and looking to grow toward independent technical ownership over the next one to two years.

REPONSIBILITIES:

Data Engineering  - 60%

• Develop and maintain ETL/ELT pipelines using PySpark and Python in Databricks
• Build and extend data transformations across Bronze, Silver, and Gold layers of the medallion architecture
• Write well-structured, readable, and testable code following team standards and review feedback
• Troubleshoot and resolve data pipeline failures, with escalation to senior engineers for complex issues
• Contribute to pipeline documentation including data flow diagrams and inline code comments
• Write and optimize T-SQL queries, stored procedures, and views in SQL Server
• Support data loads and transformations that feed the SQL Server data warehouse
• Assist in identifying and resolving query performance issues with guidance from senior engineers
• Apply basic indexing knowledge and understand schema design patterns used by the team

Data Quality & Testing – 30%

• Write unit tests and data quality checks for pipelines developed or modified
• Use established data quality frameworks and tooling consistent with team standards
• Investigate and escalate data anomalies or quality issues found in pipelines

Collaboration & Communication – 10%

• Participate in code reviews — both receiving and providing constructive feedback as skills develop
• Collaborate with teammates during sprint planning, standups, and retrospectives
• Communicate blockers and progress clearly to your manager and team
• Work with data analysts and other stakeholders to understand requirements for assigned work

SKILLS & QUALIFICATIONS REQUIRED:

• 2–4 years of professional data engineering experience with a Bachelor's degree 
• Proficiency in SQL — comfortable writing multi-table joins, window functions, and subqueries
• Working knowledge of Python for data processing tasks
• Exposure to Databricks, Apache Spark, or PySpark in a professional or academic setting
• Familiarity with cloud data storage concepts; experience with Azure or ADLS a plus
• Understanding of ETL/ELT principles and data pipeline design
• Comfort using Git for version control and working within a collaborative branching workflow

PREFERRED EXPERIENCE:

• Hands-on experience with the medallion architecture (Bronze / Silver / Gold) pattern
• Experience with Delta Lake or similar table formats (Iceberg, Hudi)
• Familiarity with SQL Server, including T-SQL and stored procedure development
• Exposure to CI/CD pipelines and automated testing in a data context
• Experience with a data quality or transformation framework such as dbt or Great Expectations
• Familiarity with Azure services beyond ADLS (e.g., Azure Data Factory, Azure DevOps)