Role Overview

We’re looking for a Full-Stack AI Engineer to join our team, where you’ll build the next generation of tools for developing, evaluating, and training state-of-the-art AI systems. You will own features end to end—from user-facing experiences and APIs to backend services, data models, and infrastructure. 

You’ll be at the heart of our applied AI efforts, with a particular focus on human-in-the-loop systems used to generate high-quality training data for Large Language Models (LLMs) and AI agents. This includes building a platform that enables us and our customers to create and evaluate data, as well as systems that leverage LLMs to assist with reviewing, scoring, and improving human submissions.

Your Impact

• Own End-to-End Product Features
  Design, build, and ship complete workflows spanning frontend UI, APIs, backend services, databases, and production infrastructure.
• Enable Human-in-the-Loop AI Training
  Build systems that allow humans to efficiently create, review, and curate high-quality training and evaluation data used in AI model development.
• Support RLHF and Preference Data Workflows
  Design and implement tooling that supports RLHF-style pipelines, including task generation, human review, scoring, aggregation, and dataset versioning.
• Leverage LLMs in the Review Loop
  Build systems that use LLMs to assist human reviewers—such as automated checks, critiques, ranking suggestions, or quality signals—while maintaining human oversight.
• Advance AI Evaluation
  Design and implement evaluation frameworks and interactive tools for LLMs and AI agents across multiple data modalities (text, images, audio, video).
• Create Intuitive, Reviewer-Focused Interfaces
  Build thoughtful, efficient user interfaces (e.g., in React) optimized for high-throughput human review, quality control, and operational workflows.
• Architect Scalable Data & Service Layers
  Design APIs, backend services, and data schemas that support large-scale data creation, review, and iteration with strong guarantees around correctness and traceability.
• Solve Ambiguous, Real-World Problems
  Translate loosely defined operational and research needs into practical, scalable, end-to-end systems.
• Ensure System Reliability
  Participate in on-call rotations to monitor, troubleshoot, and resolve issues across the full stack.
• Elevate the Team
  Improve engineering practices, development processes, and documentation. Share knowledge through technical writing and design discussions.

What You Bring

• Bachelor’s degree in Computer Science, Data Engineering, or a related field.
• 2+ years of experience in a software or machine learning engineering role.
• A proactive, product-focused mindset and a high degree of ownership, with a passion for building solutions that empower users.
• Experience using frontend frameworks like React/Redux and backend systems and technologies like Python, Java, GraphQL; familiarity with NodeJS and NestJS is a plus.
• Knowledge of designing and managing scalable database systems, including relational databases (e.g., PostgreSQL, MySQL), NoSQL stores (e.g., MongoDB, Cassandra), and cloud-native solutions (e.g., Google Spanner, AWS DynamoDB).
• Familiarity with cloud infrastructure like GCP (GCS, PubSub) and containerization (Kubernetes) is a plus.
• Excellent communication and collaboration skills.
• High proficiency in leveraging AI tools for daily development (e.g., Cursor, GitHub Copilot).
• Comfort and enthusiasm for working in a fast-paced, agile environment where rapid problem-solving is key.A focus on writing clean, well-tested code and delivering your work on time.

Bonus Points

• Experience building tools for AI/ML applications, particularly for data annotation, monitoring, or agent evaluation.
• Familiarity with data infrastructure components such as data pipelines, streaming systems, and storage architectures (e.g., Cloud Buckets, Key-Value Stores).
• Previous experience with search engines (e.g., ElasticSearch).
• Experience in optimizing databases for performance (e.g., schema design, indexing, query tuning) and integrating them with broader data workflows.

Engineering at Labelbox

At Labelbox Engineering, we're building a comprehensive platform that powers the future of AI development. Our team combines deep technical expertise with a passion for innovation, working at the intersection of AI infrastructure, data systems, and user experience. We believe in pushing technical boundaries while maintaining high standards of code quality and system reliability. Our engineering culture emphasizes autonomous decision-making, rapid iteration, and collaborative problem-solving. We've cultivated an environment where engineers can take ownership of significant challenges, experiment with cutting-edge technologies, and see their solutions directly impact how leading AI labs and enterprises build the next generation of AI systems.

Our Technology Stack

Our engineering team works with a modern tech stack designed for scalability, performance, and developer efficiency:

• Frontend: React.js with Redux, TypeScript
• Backend: Node.js, TypeScript, Python, some Java & Kotlin
• APIs: GraphQL
• Cloud & Infrastructure: Google Cloud Platform (GCP), Kubernetes
• Databases: MySQL, Spanner, PostgreSQL
• Queueing / Streaming: Kafka, PubSub

Role Overview

As an Applied Research Engineer at Labelbox, you will be at the forefront of developing cutting-edge systems and methods to create, analyze, and leverage high-quality human-in-the-loop data for frontier model developers. Your role will involve designing and implementing advanced systems that align human feedback into AI training processes, such as Reinforcement Learning from Human Feedback (RLHF), Direct Preference Optimization (DPO), etc. You will also work on innovative techniques to measure and improve human data quality, and develop AI-assisted tools to enhance the data labeling process. Your expertise in machine learning, frontier model training, and advanced human data alignment techniques will be crucial in pushing the boundaries of AI capabilities and delivering state-of-the-art solutions to meet the evolving needs of our customers.

Your Impact

• Advance the field of AI alignment by developing cutting-edge methods, such as RLHF and novel approaches, that ensure AI systems reflect human preferences more accurately.
• Improve the quality of human-in-the-loop data by designing and deploying rigorous measurement and enhancement systems, leading to more reliable AI training.
• Increase efficiency and effectiveness in AI-assisted data labeling by creating tools that leverage active learning and adaptive sampling, reducing manual effort while improving accuracy.
• Shape the next generation of AI models by investigating how different types of human feedback (e.g., demonstrations, preferences, critiques) impact model performance and alignment.
• Optimize human feedback collection by developing novel algorithms that enhance how AI learns from human input, improving model adaptability and responsiveness.
• Bridge research and real-world application by integrating breakthroughs into Labelbox’s product suite, making human-AI alignment techniques scalable and impactful for users.
• Drive industry innovation by engaging with customers and the broader AI community to understand evolving data needs and share best practices for training frontier models.
• Contribute to the AI research ecosystem by publishing in top-tier journals, presenting at leading conferences, and influencing the future of human-centric AI.
• Stay ahead of AI advancements by continuously exploring new frontiers in human-AI collaboration, human data quality, and AI alignment, keeping Labelbox at the cutting edge.
• Establish Labelbox as a thought leader in AI by creating technical documentation, blog posts, and educational content that shape the industry's approach to human-centric AI development.

What You Bring

• A strong foundation in AI and machine learning, backed by a Ph.D. or Master’s degree in Computer Science, Machine Learning, AI, or a related field.
• Proven experience (3+ years) in solving complex ML challenges and delivering impactful solutions that improve real-world AI applications.
• Expertise in designing and implementing data quality measurement and refinement systems that directly enhance model performance and reliability.
• A deep understanding of frontier AI models—such as large language models and multimodal models—and the human data strategies needed to optimize them.
• Proficiency in Python and experience with deep learning frameworks like PyTorch, JAX, or TensorFlow to prototype and develop cutting-edge solutions.
• A track record of publishing in top-tier AI/ML conferences (e.g., NeurIPS, ICML, ICLR, ACL, EMNLP, NAACL) and contributing to the broader research community.
• The ability to bridge research and application by interpreting new findings and rapidly translating them into functional prototypes.
• Strong analytical and problem-solving skills that enable you to tackle ambiguous AI challenges with structured, data-driven approaches.
• Exceptional communication and collaboration skills, allowing you to work effectively across multidisciplinary teams and with external stakeholders.

Labelbox Applied Research

At Labelbox Applied Research, we're committed to pushing the boundaries of AI and data-centric machine learning, with a particular focus on advanced human-AI interaction techniques. We believe that high-quality human data and sophisticated human feedback integration methods are key to unlocking the next generation of AI capabilities. Our research team works at the intersection of machine learning, human-computer interaction, and AI ethics to develop innovative solutions that can be practically applied in real-world scenarios.

We foster an environment of intellectual curiosity, collaboration, and innovation. We encourage our researchers to explore new ideas, engage in open discussions, and contribute to the wider AI community through publications and conference presentations. Our goal is to be at the forefront of human-centric AI development, setting new standards for how AI systems learn from and interact with humans.

Role Overview

As an Applied Research Engineer at Labelbox, you’ll sit at the junction of advanced AI research and real product impact, with a focus on the data that makes modern agents work—browser interactions, SWE/code traces, GUI sessions, and multi-turn workflows. You’ll drive the data landscape required to advance capable, adaptable agents and help shape Labelbox’s strategy for collecting, synthesizing, and evaluating it. You will possess expertise in LLM agents and planning/execution loops, plus creativity in tackling problems across data design, interaction, and measurement. You’ll publish meaningful results, collaborate with customer researchers in frontier AI labs, and turn prototypes into reliable, scalable features.

Your Impact

• Create frameworks and tools to construct, train, benchmark and evaluate autonomous agent capabilities.
• Design agent-focused data programs using supervised fine-tuning (SFT) and reinforcement learning (RL) methodologies.
• Develop data pipelines from diverse sources like code repositories, web browsers, and computer systems.
• Implement and adapt popular open-source agent libraries and benchmarks with proprietary datasets and models.
• Engage with research teams in frontier AI labs and the wider AI community to understand evolving agent data needs for frontier models and share best practices.
• Collaborate closely with frontier AI lab customers to understand requirements and guide model development.
• Publish research findings in academic journals, conferences, and blog posts.

What You Bring

• Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or related field.
• At least 3 years of experience addressing sophisticated ML problems with successful delivery to customers.
• Experience building and training autonomous agents—tool use, structured outputs, multi-step planning—across browsers/GUI, codebases, and databases using SFT and RL.
• Constructed and evaluated agentic benchmarks (e.g. SWE-bench, WebArena, τ-bench, OSWorld) and reliability/efficiency suites (e.g. WABER). 
• Adept at interpreting research literature and quickly turning new ideas into prototypes.
• Deep understanding of frontier models (autoregressive, diffusion), post-training (SFT, RLVR, RLAIF, RLHF, et al.), and their human data requirements.
• Proficient in Python, data science libraries and deep learning frameworks (e.g., PyTorch, JAX, TensorFlow).
• Strong analytical and problem-solving abilities in ambiguous situations.
• Excellent communication skills.
• Track record of publications in top-tier AI/ML venues (e.g., ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, etc.).

Labelbox Applied Research

At Labelbox Applied Research, we're committed to pushing the boundaries of AI and data-centric machine learning, with a particular focus on advanced human-AI interaction techniques. We believe that high-quality human data and sophisticated human feedback integration methods are key to unlocking the next generation of AI capabilities. Our research team works at the intersection of machine learning, human-computer interaction, and AI ethics to develop innovative solutions that can be practically applied in real-world scenarios.

We foster an environment of intellectual curiosity, collaboration, and innovation. We encourage our researchers to explore new ideas, engage in open discussions, and contribute to the wider AI community through publications and conference presentations. Our goal is to be at the forefront of human-centric AI development, setting new standards for how AI systems learn from and interact with humans.

Role Overview

As an Applied Research intern at Labelbox, you will design, build, and productionize evaluation and post‑training systems for frontier LLMs and multimodal models. You’ll own continuous, high-quality evals and benchmarks (reasoning, code, agent/tool‑use, long‑context, vision‑language, et al.), create and curate post‑training datasets (human + synthetic), and prototype RLHF/RLAIF/RLVR/RM/DPO‑style training loops to measure and improve real‑world task and agent performance.

Your Impact

• Build and own evaluation and benchmark suites for reasoning, code, agents, long‑context, and V/LLMs.
• Create post‑training datasets at scale: design preference/critique pipelines (human + synthetic), and target hard failures surfaced by evals.
• Experiment and prototype RLHF/RLAIF/RLVR/RM/DPO‑style training loops to improve real-world task and agent performance.
• Land research in product: ship improvements into Labelbox workflows, services, and customer‑facing evaluation/quality features; quantify impact with customer and internal metrics.
• Engage with customer research teams: run pilots, co‑design benchmarks, and share practical findings through internal research reports, blog posts, talks, and published papers.

What You Bring

• A strong foundation in AI and machine learning, backed by a Ph.D. or Master’s degree in Computer Science, Machine Learning, AI, or a related field (in progress degrees are acceptable for intern positions).
• A deep understanding of frontier autoregressive and diffusion multimodal models, along with the human and synthetic data strategies needed to optimize them.
• Passion and experience for LLM evaluation and benchmarking.
• Expertise in training data quality construction, measurement and refinement.
• The ability to bridge research and application by interpreting new findings and translating them into functional prototypes.
• A track record of publishing in top-tier AI/ML conferences (e.g., NeurIPS, ICML, ICLR, ACL, EMNLP, NAACL) and contributing to the broader research community.
• Proficiency in Python and experience with deep learning frameworks like PyTorch, JAX, or TensorFlow.
• Exceptional communication and collaboration skills.

Applied Research at Labelbox

At Labelbox Applied Research, we're committed to pushing the boundaries of AI and data-centric machine learning, with a particular focus on advancing human-AI interaction techniques. We believe that high-quality human data and sophisticated human feedback integration methods are key to unlocking the next generation of AI capabilities. Our research team works at the intersection of machine learning, human-computer interaction, and AI ethics to develop innovative solutions that can be practically applied in real-world scenarios.