Sobre esta vaga de Senior Data Engineer na Emburse
What you will Do
Software Engineering
Architects, deploys, and maintains production-grade Python microservices, serverless components (AWS Lambda), batch workers, web services, and shared core libraries across the analytics ecosystem
Engineers RESTful interfaces and application integrations to expose dataset capabilities to platform applications, encompassing secure object storage exchange, query orchestration, automated report delivery, and embedded analytics infrastructure
Constructs backend infrastructure powering AI analytics solutions, incorporating LLM and natural language query translation engines like Snowflake Cortex, controlling dialog state, guaranteeing multi-tenant data isolation before execution, and optimizing throughput and resource utilization
Authors clean, resilient codebase backed by comprehensive test suites while directing GitHub Actions continuous integration and continuous deployment pipelines for automated compilation, verification, security inspection, and production releases
Safeguards system configuration and secrets using key-pair authentication, parameter management, and service credentials while proactively updating software dependencies based on vulnerability scanning metrics
Data Engineering
Constructs and manages ingestion and transformation workflows utilizing Airflow, AWS Glue, Spark, and Snowflake dynamic tables to feed multi-tier lakehouse environments from diverse sources
Engineers automated data cleansing and enrichment routines integrating machine learning models and record reconciliation methods, while safely executing massive historical data re-processing
Architects robust multi-tenant security frameworks in Snowflake employing row-level policies, secure functions, and isolated access controls across global cloud infrastructure regions
Implements metadata-driven automation to dynamically generate client-specific semantic layers from templates, incorporating state tracking, automated recovery, and version rollbacks
Executes data governance protocols and privacy compliance workflows, including systematic tenant data purging across all analytical storage tiers
Develops internal platform analytics modules to track, aggregate, and report infrastructure consumption and operational expenditures across cloud vendor ecosystems
Establishes proactive telemetry and alerting, drives resolution for system incidents, conducts thorough root-cause investigations, and deploys long-term technical remediation
Process
Follows SDLC processes, including adopting agile-based processes, peer code-reviews, and technical preparations required for scheduled releases and demos.
Partners with product, analytics and data science team to drive requirements that take into account all parties' needs
Establishes monitoring, responds to alerts and resolves issues within data pipelines
Uses and helps shape AI-assisted development practices and tooling for the team.
Impact
Executes intricate, multi-faceted capabilities spanning microservices, automated workflows, and analytical storage infrastructure with high autonomy while providing technical leadership across mid-scale initiatives
Assumes direct accountability for the reliability, security architecture, and long-term viability of deployed systems while systematically uncovering technical remediation opportunities
Pinpoints architectural risks across peer technical deliverables and formulates pragmatic, alternative implementation strategies
Streamlines operational processes, resolves critical system defects, performs advanced codebase refactoring, and exhibits expert troubleshooting capabilities across application code, cloud infrastructure, and analytical queries
Authors comprehensive technical documentation, operational runbooks, and detailed data dictionaries to support platform capabilities
Drives system architecture discussions and fulfills expanded peer code review obligations across the engineering organization
Spearheads onboarding initiatives and provides technical coaching and mentorship to junior staff members
What you will bring
Experience:
Demonstrated background of at least six years architecting enterprise software solutions and big data platforms in operational environments.
Advanced proficiency in Python engineering, spanning distributed microservices, internal modules, RESTful interface design, unit testing frameworks, and GitHub Actions release pipelines rather than ad-hoc scripting.
Expert SQL query optimization, data modeling techniques, and extensive background delivering cloud warehouse architectures using Snowflake.
Proven mastery deploying and orchestrating data processing flows in live production through Apache Airflow.
Hands-on expertise utilizing Amazon Web Services ecosystems, including AWS Lambda, S3 storage, Glue jobs, IAM policies, Key Management/Secrets Manager, and EventBridge integration.
Deep comprehension of multi-tenant security architecture—incorporating row-level access policies, fine-grained access governance, strict tenant isolation, and compliant handling of restricted financial assets across international cloud regions.
Track record of effective cross-functional partnership in product-driven organizations alongside engineering peers and product managers.
Preferred:
Snowflake advanced features: row access policies, secure views, dynamic tables, stored procedures, Cortex AI / Cortex Analyst, semantic views.
Experience building LLM-powered or text-to-SQL features, or other GenAI applications in production.
Spark (AWS Glue or Databricks).
Fivetran or an equivalent ELT tool; Looker or an equivalent BI platform, including embedded analytics.
Infrastructure as code and GitHub Actions.
Experience with ML-assisted data matching or entity resolution.
Financial, expense or travel industry experience.