Staff Site Reliability Engineer

As a Staff SRE at Obsidian, you will define and drive the company-wide reliability vision for a complex, multi-tenant SaaS platform serving enterprise and financial customers. You will operate as a strategic partner to DevOps and Platform Engineering leadership, shaping a unified reliability strategy that scales across the organization.

Your core mandate: ensure Obsidian detects, diagnoses, and communicates system issues before customers are impacted—consistently and predictably.

This is a hands-on technical role that involves architecting and leading the implementation of systems that handle real-world complexity, including upstream SaaS dependencies, sparse and noisy signals, and mission-critical enterprise workloads.

Key Responsibilities

• Reliability Strategy & Architecture - Define and lead long-term reliability strategy across services. Establish end-to-end system visibility frameworks and guide architecture for observability, detection, and resilience.
• Cross-Org Leadership - Partner across teams to embed reliability, standardize SLI/SLOs, and serve as a technical escalation expert.
• Detection & Observability - Build intelligent detection systems (anomaly detection, connector health models) and enable self-service observability.
• Incident Management - Define and evolve a tiered incident communication strategy, improve response practices, and lead postmortems to strengthen reliability and customer trust.
• Execution - Contribute hands-on to system design, monitoring, and debugging across distributed systems and data pipelines.

Required Qualifications

• 5+ years in SRE, Production Engineering, or related roles
• 3+ years operating at a senior or technical leadership level (Staff or equivalent scope)
• Deep expertise in:
• AWS and/or GCP
• Kubernetes and Helm
• Observability stacks (Prometheus, Grafana, or equivalent)
• CI/CD systems (GitLab CI/CD, ArgoCD, etc.)
• Proven experience designing and scaling reliability systems for multi-tenant SaaS platforms
• Strong debugging and systems thinking across distributed microservices and legacy systems
• Demonstrated ability to lead initiatives that improve incident detection, response, and system resilience
• Hands-on engineering approach with a track record of building—not just configuring—reliability systems

Preferred Qualifications

• Experience in B2B SaaS serving enterprise or financial customers
• Familiarity with third-party SaaS connector architectures and ingestion patterns
• Experience building anomaly detection or intelligent alerting systems
• Experience designing customer-facing status pages and incident communication frameworks

Why This Role

• Drive org-wide reliability strategy
• Own and build new detection & observability systems
• Tackle complex distributed systems challenges
• Safeguard critical infrastructure for financial customers

What Success Looks Like

• Issues caught and resolved before customer impact
• Reliability is measurable and continuously improving
• Teams self-serve observability with scalable tools
• Clear, proactive incident communication builds trust
• Reliability becomes a competitive advantage

About the Role

At Obsidian, we relentlessly pursue clarity in a landscape defined by complexity. We’re looking for Security Research Engineers who thrive at the intersection of security research and product engineering. You’ll dig deep into SaaS platforms, craft high-impact product features, and help shape the defensive strategies that protect our customers at scale.

This role is for someone who loves gaining a deep understanding of complex systems, who enjoys thinking like an attacker, and who wants to create category-leading security solutions for Obsidian's customers.

What You’ll Do

• Gain an in-depth understanding of major SaaS or AI platforms to identify critical security insights. 
• Explore and prototype solutions to integrate data across SaaS and AI platforms to identify toxic combinations of risks that can lead to high blast-radius compromises.
• Identify and respond to new attacker techniques against SaaS and AI platforms to build novel approaches to detect or mitigate future attacks.
• Leverage AI to turbo-charge research into SaaS and AI platform security.

What We’re Looking For

Minimum Qualifications

• 5-7 years of experience in the cybersecurity industry with either a research or data focus.
• Experience with data pipelines (dbt, dagster, clickhouse).
• Proficiency in Python/Go and SQL.
• Experience with log processing and anomaly detection.
• Fluency in using AI agents to assist in building prototype solutions.
• Ability to analyze logs and signals from IdPs, cloud, or SaaS apps.

Preferred Qualifications

• Experience with core SaaS platforms such as Google Workspace, Microsoft Office 365, Salesforce, or Workday in a security context.
• Strong understanding of identity systems (Okta, Azure AD/Entra, Google Workspace, etc.) and authentication flows.

Why Join Us

• Protect some of the world's largest organizations 
• Become an expert in SaaS security by learning from the industry’s leader.
• Have a direct impact immediately, and experience Silicon Valley pace from the UK.

About the Role - as a Senior Engineer at Obsidian, you’ll:

• Build or extend data pipelines that power Obsidian’s Agentic AI and supply chain risk products.
• Maintain, improve, and evolve existing systems to ensure performance, resilience, and scalability
• Design and implement APIs and backend services, including multithreaded applications
• Collaborate with product and engineering teams to support key product themes and ensure delivery of high-impact features
• Apply strong software engineering practices to requirements gathering, system design, and code reviews
• Contribute to a fast-moving, collaborative environment where adaptability and teamwork are essential.
• Identify and proliferate best practices in data pipeline design between both the UK and US teams.

What’s in It for You

• You will have direct impact on the core product used by enterprises worldwide
• Work alongside a talented, friendly team in a supportive and collaborative culture
• Grow your skills with opportunities to learn new technologies and engineering practices
• Be part of an innovative, fast-paced environment where your contributions are valued
• Enjoy a hybrid working, with supported remote working and great office spaces in Cheltenham and Manchester

Required Skills & Experience

• 5-7 years of experience in a software engineering role
• Proficiency in one or more modern programming languages such as Python, Go, or SQL
• Experience building backend services, APIs, and multithreaded applications
• Familiarity with containerization and orchestration technologies such as Docker and Kubernetes
• Strong knowledge of relational databases (e.g., Postgres)
• Experience collaborating in team environments and adapting to changing requirements
• Understanding of software design principles and engineering best practices
• Experience with cloud platforms (AWS, GCP), object storage (S3), or event/streaming systems (Kafka, Redis)
• Familiarity with Git for version control and deployment tooling, such as GitLab CI/CD
• Experience in influencing design patterns based on best practices.
• Experience assessing alternative data architecture designs to produce accurate and unbiased recommendations.

Desirable Experience

• Experience with system monitoring and observability tools such as Grafana, Prometheus, or similar platforms
• Understanding of quality engineering (QE) practices across development and testing lifecycles
• Exposure to large-scale distributed systems and performance optimisation
• Experience in forming strong stakeholder relationships to help promote joint working between teams.
• Experience of working alongside both UK and US development teams.

AI Skills & AI-Native Engineering Expectations: As an AI-forward engineering organization, we expect senior engineers to effectively leverage AI tools and understand foundational AI concepts to enhance development efficiency and build AI-ready systems.

AI Engineering Capabilities:

• Leverage AI tools effectively to improve development efficiency and build AI-ready systems.
• Proficient with AI-powered developer tools; able to critically evaluate and refine AI-generated outputs.
• Strong understanding of core AI/ML concepts (LLMs, embeddings, vector databases, inference, evaluation).
• Experience integrating AI/ML APIs and building AI-ready data infrastructure (e.g., for RAG).
• Ensure data quality, governance, observability, reliability, security, and performance in AI-driven systems.