Octus is seeking a Principal Data Engineer to roll up their sleeves and build scalable, production-grade data pipelines and infrastructure. You'll be a hands-on technical leader — writing code daily, solving hard engineering problems, and helping elevate the team around you through doing. Spanning Snowflake, Databricks, and AWS, you'll be deeply involved in the day-to-day development of the data platform that powers Octus's products, data, and automation initiatives. The ideal candidate is an expert in Python and SQL who thrives in an execution-focused environment and has deep experience building modern data pipelines and lakehouse solutions.

Responsibilities

• Build and maintain end-to-end data pipelines — from raw ingestion through transformation and delivery — across diverse data sources (APIs, web data, internal feeds, etc.).
• Hands-on development of scalable, production-grade pipelines within Databricks, including Delta Lake table management, Workflows, and cluster optimization.
• Build and maintain data models, schemas, and transformation logic in Snowflake, optimizing for performance and reliability.
• Develop and manage Databricks environments including Unity Catalog, Delta Live Tables, and integration patterns that support both internal data consumers and external sharing use cases.
• Build and manage orchestration workflows using AWS services (MWAA/Airflow, Lambda, ECS, SQS, MSK) and Databricks-native orchestration where appropriate.
• Implement and maintain infrastructure as code (IaC) using Terraform, ensuring reproducibility and compliance with cloud standards.
• Establish and enforce best practices in data modeling, schema design, and ETL/ELT processes for high-volume structured and semi-structured data across Snowflake and Databricks.
• Ensure data quality, lineage, and observability through automated testing, monitoring, and alerting across all pipeline layers.
• Collaborate closely with technology leadership to align data platform development with business strategy and product goals.
• Stay at the forefront of industry trends in data engineering, lakehouse architecture, and cloud-native data platforms.

Requirements

• Strong foundation in software engineering principles, including SOLID design, modularity, and scalability.
• Expert proficiency in Databricks, including Delta Lake, Unity Catalog, Delta Live Tables, MLflow, and Databricks Workflows.
• Deep experience with Snowflake, including data modeling, performance optimization, and integration with upstream/downstream pipeline tooling.
• Expert proficiency in Python for data pipeline and automation development.
• Advanced SQL skills with experience optimizing complex queries and data models at scale.
• Proven experience designing and maintaining cloud-native data pipelines on AWS (e.g., MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, Redshift, etc.).
• Experience implementing and managing Terraform or similar IaC frameworks.
• Strong understanding of lakehouse architecture patterns, data ingestion, transformation, and orchestration, including familiarity with ML/AI pipeline integration patterns.
• Familiarity with CI/CD pipelines, automated testing, and modern DevOps practices.
• 8+ years of experience in data engineering or backend development, with a focus on scalable data solutions.
• Demonstrated experience leading data infrastructure projects end-to-end and mentoring senior engineers.
• Familiarity with containerization (Docker) and workflow orchestration best practices.
• Excellent communication, collaboration, and problem-solving skills.

Nice to Have

• Experience with streaming data technologies (Kafka, Kinesis, Flink).
• Exposure to ML/AI pipeline patterns (feature stores, experiment tracking, model serving) and MLOps tooling, particularly in a cross-functional team environment.
• Experience integrating data quality and observability tools.
• Experience with Databricks as a data sharing and collaboration platform (Delta Sharing, Marketplace).
• Familiarity with Claude Code or similar AI-powered developer tools for accelerating pipeline development and code workflows.

At Octus, we consider a range of factors in connection with compensation decisions, including experience, skills, location, and our business needs and limitations. As a result, compensation may vary within and across similar roles and positions. Please note that the salary range information below is a good faith estimate for this position and actual compensation for any individual may fall outside this range if warranted by the circumstances applicable to that individual. If we identify a role that would be suitable for a broader range of skills and experience such that we would consider hiring at multiple levels then the range listed below may reflect that breadth.

The salary range estimate for this position is $175,000 - $220,000

The actual compensation will be at Octus' sole discretion and will be determined by the aforementioned and other relevant factors.

Octus delivers breaking news and market-moving intelligence through cutting-edge data and technology for hedge funds, investment banks, and law firms- and we’re transforming how professionals access complex and opaque information. As part of our high-performing AI Innovation team, you’ll help design, build, and productionize modern GenAI and LLM-powered systems that support both client-facing features and internal operational efficiency. You’ll work end to end—from shaping ambiguous problems into scalable solutions to deploying reliable AI models in production - collaborating closely with product, engineering, and infrastructure teams. This is a hybrid role based in NYC (3 days in office per week). Curious about what we’re building? Check out our flagship GenAI product, CreditAI here and our AI framework here.

Responsibilities

• Apply strong problem-solving and critical thinking skills to break down complex, ambiguous requirements into clear, implementable technical components and system designs.
• Design, build, and maintain AI-powered and data-driven systems with a focus on modern language and multimodal models, including LLM-driven applications, RAG pipelines, and agentic workflows.
• Evaluate and productionize commercial and open-source LLMs, choosing appropriate models, tools, and techniques for each use case. Develop multi-step agentic workflows that incorporate tools, external data sources, memory, and control logic.
• Manage the orchestration of production LLM workflows and agentic systems, ensuring reliability and efficiency through prompt routing, state management, retries, fallbacks, and error handling. Design, test, and iteratively refine prompts and system instructions using prompt engineering and tuning techniques to improve model reliability, accuracy, and task performance.
• Maintain production-grade code and services with automated monitoring and performance tracking, using metrics and alerts to guide continuous improvements in models, prompts, and pipelines.
• Apply systems thinking to design and optimize AI and LLM systems, balancing quality, scalability, latency, cost, and operational complexity, while implementing efficiency improvements using model selection, prompt design, batching, caching, and retrieval strategies.
• Define and implement evaluation and observability frameworks for AI systems, including automated testing, task-specific benchmarks, regression testing for prompts, human-in-the-loop validation, and performance monitoring.
• Build and integrate AI models into backend systems and APIs to support both real-time and batch inference, ensuring solutions are production-ready, scalable, and efficient.
• Apply NLP and ML techniques to tasks such as information extraction, semantic search and retrieval, text classification, summarization, and reasoning over text and documents.
• Collaborate closely with engineering and infrastructure teams to deploy solutions using containerized and cloud-based environments (e.g., GitHub, Docker, AWS), applying modern deployment and infrastructure practices.
• Collaborate with product managers, business stakeholders, and domain experts to translate complex, ambiguous business problems into actionable technical solutions, and communicate progress, trade-offs, and outcomes to relevant stakeholders.
• Continuously learn and adapt to advancements in NLP and Generative AI to ensure solutions remain innovative and effective.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
• 3+ years of experience as a Data Scientist, Machine Learning Engineer, or applied AI practitioner, with a strong foundation in computer science, algorithms, and software development. 
• Advanced programming skills in Python, with experience building production-grade systems beyond research or experimentation.
• Solid understanding of machine learning and applied AI concepts, with experience taking solutions from prototype to production.
• Hands-on experience designing, building, and deploying LLM-driven or GenAI applications, including familiarity with vector databases, embeddings pipelines, or semantic search systems.
• Practical experience with cloud-based deployments and infrastructure tools (e.g., AWS, Docker, GitHub) and an understanding of modern DevOps practices, containerization, orchestration, caching strategies, and cost-aware design.
• Strong problem-solving skills and systems thinking, with the ability to balance trade-offs across model quality, scalability, inference latency, cost, and operational complexity.
• Ability to interpret and implement research ideas and algorithms, actively contributing to research and development initiatives while translating them into production solutions.
• Excellent communication and collaboration skills, with experience working closely with product managers, engineers, and domain experts to deliver actionable technical solutions.
• Passion for learning and staying current with the rapidly evolving AI/ML landscape, including emerging best practices for GenAI applications.
• Strong ownership and initiative, with the ability to independently drive projects from problem definition to delivery, while being a team player and contributing to the overall success of the data science team.

At Octus, we consider a range of factors in connection with compensation decisions, including experience, skills, location, and our business needs and limitations. As a result, compensation may vary within and across similar roles and positions. Please note that the salary range information below is a good faith estimate for this position and actual compensation for any individual may fall outside this range if warranted by the circumstances applicable to that individual. If we identify a role that would be suitable for a broader range of skills and experience such that we would consider hiring at multiple levels then the range listed below may reflect that breadth.

The salary range estimate for this position is $130,000-$145,000.

The actual compensation will be at Octus’ sole discretion and will be determined by the aforementioned and other relevant factors. This position is eligible for a performance-based annual bonus.

The Application Framework team at Octus owns the foundational platform — APIs, SDKs, and shared services — that powers application development across the organization. We're looking for a Software Engineer who enjoys solving platform problems, shipping high-quality code, and making other engineers more productive. AI is central to how we work — our team uses AI tooling throughout the entire development lifecycle, from design through code review. This isn't experimental or optional. If you're excited about building with AI as a core part of your workflow, you'll fit right in.

Build & Ship

• Contribute to projects and shared SDKs from design through production, making pragmatic tradeoffs between speed and quality
• Contribute to shared SDKs and libraries used across the organization
• Write meaningful tests — unit, integration, and contract — that give confidence to ship
• Grow toward owning larger initiatives independently over time

Collaborate & Learn

• Participate in code reviews, giving and receiving feedback that improves quality
• Collaborate with teammates through pairing, design discussions, and knowledge sharing
• Contribute to technical discussions, architecture decisions, and sprint planning

Operate & Support

• Help monitor, troubleshoot, and improve production systems in distributed environments
• Investigate and resolve technical issues systematically
• Maintain technical documentation — architecture, API specs, runbooks, and coding guidelines

Required Qualifications

• Comfortable using AI tools (e.g., copilots, code generation, AI-assisted review) as a primary part of your daily engineering workflow — not as a novelty, but as a default
• 3+ years in software engineering, with experience building backend systems, APIs, or SDKs
• Strong backend fundamentals — willing to pick up any language quickly, and go deep on C#
• Experience working with cloud-based systems (AWS preferred)
• Solid understanding of relational databases (PostgreSQL or similar)
• Familiarity with distributed systems and event-driven patterns (Kafka, RabbitMQ, or similar)

Preferred Qualifications

• Experience with multiple backend languages (C#, Python, PHP)
• Exposure to both PostgreSQL and MySQL
• Experience with caching layers (Redis or equivalent)
• Experience with CI/CD pipelines and infrastructure as code
• Interest in building internal developer tools or platforms

At Octus, we consider a range of factors in connection with compensation decisions, including experience, skills, location, and our business needs and limitations. As a result, compensation may vary within and across similar roles and positions. Please note that the salary range information below is a good faith estimate for this position and actual compensation for any individual may fall outside this range if warranted by the circumstances applicable to that individual. If we identify a role that would be suitable for a broader range of skills and experience such that we would consider hiring at multiple levels then the range listed below may reflect that breadth.

The base salary range estimate for this position is $150,000 - $175,500. This position is also eligible for an annual discretionary bonus.

The actual compensation will be at Octus’ sole discretion and will be determined by the aforementioned and other relevant factors.

As a DevOps Engineer at Octus, you will play a crucial role in the development, implementation, and maintenance of the cloud infrastructure for our products. This position involves close collaboration with Application Developers, Data Engineers, SRE, Security, and IT professionals to automate and streamline operations and processes. Your contributions will be essential in ensuring the reliability, scalability, and security of our systems and applications.

Core Responsibilities:

• Design, implement, and maintain CI/CD pipelines to automate the build, test, and deployment processes.
• Effectively prioritize workload with a constant focus on increasing developer efficiency.
• Manage and optimize cloud infrastructure to ensure scalability, performance, and cost-effectiveness. Experience with AWS is a plus.
• Work closely with SRE to develop and maintain monitoring and alerting systems to ensure the health and performance of our systems and applications.
• Work closely with Security to implement and enforce security best practices across our infrastructure and applications.
• Collaborate with software developers to optimize application performance and reliability.
• Participate in the planning and execution of disaster recovery and business continuity strategies.
• Effectively communicate with stakeholders to provide updates on project statuses, issues, and recommendations.
• Participate in on-call rotation to provide system support.

Core Requirements:

• At least 5 years of experience in a DevOps role.
• At least 2+ years of experience with Terraform.
• Expert knowledge of cloud concepts and services (EC2, ECS, S3, RDS, A/ELB, EBS, VPC, Route53 or equivalent).
• Expert level knowledge of Linux. Strong scripting skills for automation and tooling.
• Ability to troubleshoot, diagnose, resolve & document RCA of incidents.
• Good knowledge of database configuration, concepts, maintenance, and troubleshooting.
• Thorough understanding of Security principles and experience in designing, implementing, monitoring proper controls.
• Extensive experience with Infrastructure-as-code & configuration management tools (Terraform, SaltStack or equivalent).
• Experience with containerization technologies such as Docker and orchestration tools like Kubernetes, ECS including managing microservices architectures.
• Knowledge of CI/CD tools such as Jenkins, Github Actions or equivalent.
• Experience with monitoring and logging tools such as Datadog, Cloud Watch or equivalent.
• Excellent communication skill, with the ability to effectively collaborate with cross-functional teams.

At Octus, we consider a range of factors in connection with compensation decisions, including experience, skills, location, and our business needs and limitations. As a result, compensation may vary within and across similar roles and positions. Please note that the salary range information below is a good faith estimate for this position and actual compensation for any individual may fall outside this range if warranted by the circumstances applicable to that individual. If we identify a role that would be suitable for a broader range of skills and experience such that we would consider hiring at multiple levels then the range listed below may reflect that breadth.

The salary range estimate for this position is $165,000 - $190,000.

The actual compensation will be at Octus' sole discretion and will be determined by the aforementioned and other relevant factors.