What you’ll do

• Lead the design and build-out of cross-product data services for multiple product lines from one governed data plane.
• Define the “customer data plane” model: canonical customer identifiers, shared dimensions, and consistent facts used across products.
• Build and operationalize ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources.
• Own the operational playbook for data reliability: data contracts, quality checks, lineage, monitoring, and incident response.
• Implement and run access methods that make data usable: curated datasets, secure query interfaces, and product-ready data APIs where needed.
• Productize customer-facing data products (datasets, metrics, exports, and feeds) with versioning, documentation, and clear ownership.
• Design data models that fit both operational systems (RDS) and analytics stores (columnar/OLAP), including performance and cost tuning.
• Ensure data products also power ML workflows: trusted training datasets, feature-ready outputs, and consistent definitions for decision-making.
• Enable AI automation by delivering reliable, low-latency, governed data products that can be used safely in automated workflows.
• Partner closely with product, engineering, and security stakeholders to align data products to roadmap priorities and customer outcomes.
• Raise the technical bar through architecture reviews, standards, and mentoring—while staying hands-on in key systems.

Required

• Significant experience building and operating production data platforms at scale, including on-call and operational ownership.
• Strong SQL skills and strong Python skills, used to build pipelines, services, and automation.
• Hands-on experience running cloud systems on AWS and Google Cloud (IaaS level: compute, storage, networking, IAM).
• Practical experience with both operational databases (RDS-style) and analytics stores (columnar/OLAP), including performance tuning.
• Strong data modeling ability, including schema evolution, conformed dimensions, and “one source of truth” metric definitions.
• Track record of delivering data products that other teams or customers depend on, with clear contracts and reliability expectations.
• Ability to make sound engineering tradeoffs across latency, accuracy, cost, and security without creating brittle complexity.
• Experience with lakehouse patterns and open table formats (or similar), including governance and table maintenance.
• Experience with orchestration and streaming systems used in production (batch + real-time), and managing backfills safely.
• Familiarity with ML data needs (training/serving splits, feature-ready datasets, evaluation datasets) and AI-adjacent workflows.

Preferred

• Experience building self-service data platforms (catalog, discoverability, access controls) used by multiple teams.
• Experience in regulated or security-sensitive environments, including retention, auditing, and data access controls.

Work model, location & travel

• Location: Belfast, UK
• Reports to: VP of Data Engineering
• Work model: Hybrid (details TBD)
• Collaboration hours: Flexible; overlap with UK and US time zones
• Travel: Minimal

What you’ll do 

• Design and build data pipelines and datasets supporting one or more product areas within an established, governed data platform.
• Implement ingestion patterns for batch, streaming, and event data, onboarding new data sources using existing frameworks and standards.
• Build and maintain reliable data pipelines with appropriate data quality checks, basic lineage, monitoring, and alerting.
• Develop curated datasets and views that are secure, discoverable, and ready for analytics, reporting, and downstream consumption.
• Contribute to customerfacing data products such as datasets, metrics, exports, or feeds, ensuring accuracy and clear documentation.
• Implement data models for both operational databases and analytics stores, following agreed modelling conventions and best practices.
• Support ML and analytics use cases by producing trusted training datasets and featureready outputs with consistent definitions.
• Optimize pipeline performance, cost, and reliability within defined architectural patterns.
• Collaborate with product managers, engineers, analysts, and security partners to deliver data solutions aligned to roadmap needs.
• Participate in oncall rotations and operational support for data services owned by the team.
• Contribute to technical discussions, code reviews, and continuous improvement of data engineering practices. 

Required 

• Solid experience building and operating production data pipelines and datasets in a cloud environment.
• Strong SQL skills and working Python experience for building data pipelines, transformations, and automation.
• Handson experience with cloud platforms (AWS or Google Cloud), including core services such as compute, storage, IAM, and networking.
• Experience working with both operational databases (e.g., RDSstyle systems) and analytics or columnar data stores. 

The Data Science group serves under the Black Duck Data Engineering organization as a center of excellence in data analysis, statistical interpretation, machine learning engineering, attribution analysis and operational metric review.

Our core purview is the application of historical data to drive future decision making, and to develop and maintain machine learning and analytical tools to improve service delivery and advise operational processes.

We also serve as mediators of meaning across Black Duck, and as custodians of a shared data ecosystem.

This ecosystem serves to empower data consumers to explore client-operational and service-ops data in more intuitive ways, and supports their ability to share and collaborate on best practices for using this data in truthful, responsible, ethical and efficient ways.

Our Values

• Trust: Our work has no value if it is not trusted by our colleagues/customers, or if our work is not respectable. We will always rather be late than incorrect.
• Collaboration: We have a “very specific set of skills”, however we should humbly respect the subject matter experts we work with in their fields of expertise; we’re experts shuffling bits around faster/better/smarter, but they do the real work.
• Results: For our work to have value, it must speed up, augment, or replace, a current decision making process, or to demonstrate a new product/operational opportunity. While the “value” of a result may not be known for a long time, we are still outcome-driven.
• Curiosity: Exploration and Experimentation is at the heart of what we do, and we empower each other with the freedom to explore, and possibly get lost in, longer term research projects than other groups. However, even those dead ends have value when they’re written up. (It’s not science unless you write it down)
• Fun: We’re in this business because we enjoy the strange and often incongruous world of data, and what that data can tell us about ourselves. Revel in the comedy of your mistakes and discoveries, and share them with abandon.

About the Role

As a Data Engineer/Scientist, you will be a custodian of our cross-functional data regime, and a driver of innovative uses of Data across our cybersecurity platforms, ranging from predictive analytics and customer behavioural analysis, through to training customised machine learning

models on continuously evolving feedback streams from data and decisions that really matter to our thousands of customers that rely on our security assessments for safety, stability, and often, sleep.

The ideal candidate is a self-starter who is enthusiastic about exploring new technologies. They also enjoy helping their peers and their organization remain at the forefront of our industry by identifying and helping implement the latest industry best-practices that boost our organization’s productivity and team members’ experience.

The role is primarily based around our Belfast R&D Site, but UK/EMEA remote or hybrid applicants will be considered. At least Quarterly travel to the Belfast R&D Site is expected, and additional travel / conference opportunities may be available depending on your impact and collaboration.

Key Responsibilities:

• Developing and maintaining analytical data pipelines from a range of sources, internal and external
• Participate in system design discussions and contribute to architectural decisions.
• Evaluating new analytical / technological opportunities for leveraging those data for security/business impact
• Leading projects from research through to production deployment and operational handover to appropriate teams
• Partnering with R&D and Engineering teams to develop and share best practices for data tooling, from pipelines and dashboards to ML and LLM integration

Key Qualifications:

• 5+ years of experience working in Data Science, AI/Data Engineering, Data Operations, DevOps, Business Analytics, or a related field
• BSc Or MSc in Computer Science, Data Science, Artificial Intelligence, Math, Physics, Engineering or related field/degree
• Experience in a relevant analytical programming language the point where you can build / deliver a project/module from scratch that can be used by others (Python is our main daily-driver, expert-level experience in Julia or Rlang could be accepted
•  Experience in Jupyter Notebook / equivalents
• Experience in Airflow, DBT, Databricks, or equivalent stacks
• Experience in the PyData / Spark or equivalent analytical stacks
• Familiarity with Cybersecurity Governance, Application Security Testing, Quality Assurance or similar
• Experience in data modelling and working with RDBMS (PostgreSQL, Oracle or MySQL) and knowledge of NoSQL databases (e.g. MongoDB)
• Experience with Machine Learning and AI systems
• Hands-on experience with AI-assisted development tools (e.g., GitHub Copilot, Claude Code, Cursor, or similar)
• Independent project operation and cross-functional collaboration
• Strong or Developing communication skills (in-person and remote)

Nice to have:

• Familiarity with Data Mesh/Data Product concepts
• Experience in operating in Linux Command line environments
• Experience in Langchain or equivalent Agentic development stack
• Experience in training custom Machine Learning models, including familiarity with evaluation criteria and metric design
• Experience in integrating AI capabilities into software systems, including prompt engineering, API integration, and leveraging LLM-based services for automation and productivity
• Experience in Enterprise Data Visualisation such as Power BI, Tableau, Grafana, DataBricks, Snowflake etc.
• Experience deploying ML/AI models in production environments/workloads
• Experience in developing/working within large enterprise applications using microservices architecture, and container orchestration technologies, running on Kubernetes and/or cloud technologies (AWS, Azure or GCP)
• Experience in software architecture, systems design, interaction design (to the point where you can have constructive conversations with security / architecture leaders)