Jobs Companies TwelveLabs Computational Linguist, AI Evaluation

Über diese Computational Linguist, AI Evaluation Stelle bei TwelveLabs

TwelveLabs · Hybrid · San Francisco

Who We Are:

Video is 90% of the world's data. Most of it is invisible to machines.
TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound, and motion — and power production-scale AI workloads across media, entertainment, sports, security, and government.

We have raised more than $210 million from NEA, Radical Ventures, Amazon, NVIDIA, Snowflake, Databricks, Index Ventures, NAVER Ventures, Korea Investment Partners, Quadrille Capital, Red Bull Ventures, and AI pioneers including Fei-Fei Li, Silvio Savarese, and Alexandr Wang.

We are a global company, headquartered in San Francisco with offices in Seoul, New York, and London, and employees around the world. We believe the differences in our cultural, educational, and life experiences make our products stronger. Building technology that understands the world in all its complexity requires people who see it from every angle. We are looking for individuals who are driven by hard problems and want their work to matter. Come build it with us!

About the Role:

You will be a vital member of our ML Data Operations Team – which leads the full spectrum of video-language data collection, labeling operations, and model quality measurement. This role comes with high ownership and includes responsibilities such as defining dataset and evaluation requirements in consultation with our research and product teams; designing and building data pipelines; and coordinating with our vendor partners that execute at scale. You will also be responsible for automating as much of the repetitive partnership and annotation-quality-evaluation work as possible. A desire to work cross functionally and to build relationships is critical for success in this position.

Position is hybrid in San Francisco - onsite Tuesdays & Thursdays.

In this role, you will:

  • Evaluation Strategy & Design: Define and build evaluation protocols for video-language model quality, working with Research and Product teams to translate ambiguous quality questions into measurable, repeatable benchmarks or evaluation flows.

  • Data-to-Insight Pipelines: Turn raw customer and usage data into structured signal, building pipelines and analyses that surface where our models are underperforming and working with XFN partners to translate these insights into concrete improvement plans.

  • Labeling Operations: Design and execute video-language data collection and labeling projects, automating repetitive processes so the team can focus on higher-leverage work.

  • Vendor & Partner Collaboration: Coordinate with vendor and outsourcing partners executing at scale, keeping quality high through clear guidelines and feedback loops.

  • Cross-Functional Partnership: Work closely with Engineering and Research teams to align on top-priority data and evaluation needs, communicating findings through dashboards and reports that drive decisions.

You may be a good fit if you have:

  • Direct experience building or running model evaluation pipelines (benchmark design, human eval frameworks, automated scoring systems), and translating ambiguous quality questions into measurable criteria.

  • 5+ years of experience working in an AI focused data operations organization.

  • A proven track record designing and executing large scale data projects, including gathering, labeling, and post-processing data.

  • The ability to analyze messy and complex data, identify overarching patterns, and distill your findings into crisp annotation guidelines or other accessible documentation.

  • Proficiency with Python, agentic coding, or other popular industry tools for automation.

  • Excellent communication and project management skills, and the ability to support several projects simultaneously.

  • A foundational understanding of and interest in LLMs/VLMs and multimodal AI.

  • Conviction that data is the key ingredient for the performance of AI models.

Preferred Qualifications:

  • Experience in data collection and labeling for multimodal language models.

  • Experience working with research scientists and engineers.

  • Experience planning and operating new data tools and labeling systems.

  • Expertise or interest in video-centric domains, such as sports, advertising, and content creation.

Tech Stack:

  • Technical: Python, FFMPEG, AWS (S3, EC2)

  • Project Management: Linear, Notion, Google Suite, Encord

Even if there are a few checkboxes that aren’t ticked through your prior experience, we still encourage you to apply! If you are a 0-1 achiever, a ferocious learner, and a kind and fun team player who motivates others, you will find a home at TwelveLabs.

We are a global company that values the uniqueness of each person’s journey. It is the differences in our cultural, educational, and life experiences that allow us to constantly challenge the status quo. We are looking for individuals who are motivated by our mission and eager to make an impact as we push the bounds of technology to transform the world. Join us as we revolutionize video understanding and multimodal AI.

Benefits and Perks:

🤝 An open and inclusive culture and work environment

🚀 Work closely with a collaborative, mission-driven team on cutting-edge AI technology

🏥 Full health, dental, and vision benefits

🌴 Extremely flexible PTO and parental leave policy. Office closed the week of Christmas and New Years

💪 Monthly wellness stipend

📚 Annual Learning & Development stipend to invest in your growth

💼 Global offices in San Francisco and Seoul, and co-working office memberships for remote team members

🚆 Transportation stipend (SF onsite/hybrid only)

🍲 Daily lunch & dinner provided (SF onsite/hybrid only)

Bereit, sich bei TwelveLabs zu bewerben?
Bei TwelveLabs bewerben

Ähnliche Jobs

Registrieren für Vorschläge, die auf die von Ihnen geöffneten Jobs und gespeicherten Suchen zugeschnitten sind.

Mehr Jobs bei TwelveLabs

Alle Jobs bei TwelveLabs ansehen →

Jetzt bewerben
🤖

Moment — langsam

JobsRadar wurde für echte Menschen gebaut, die eine schwere Zeit bei der Jobsuche haben — nicht für automatisierte Anfragen. Sie klicken viel zu schnell und sind jetzt vorübergehend blockiert.

Kommen Sie später wieder. Wenn Sie wirklich auf Jobsuche sind, stehen wir hinter Ihnen — verhalten Sie sich einfach wie ein Mensch.

Catch your next role the second it’s posted.

Create a free account and we’ll watch the boards for you — the instant a job matches your search, it lands in your inbox or Telegram. No digging, no refreshing.

Create free account

Free forever · takes 30 seconds · already have one?

Verschaffe dir einen Vorsprung bei der Jobsuche.

Tritt unserem Telegram-Kanal bei für das, was dir hilft, die Stelle zu bekommen — Gehaltsbenchmarks, den wöchentlichen Marktpuls und neue Feature-Drops. Kein Spam, nur Signal.

Dem Kanal beitreten — kostenlos