PyTorch Jobs in Lima.
All active PyTorch roles based in Lima.
Pick a job to read the details
Tap any role on the left — its description and apply link will open here.
Lead Machine Learning Engineer
Share this job
Lead Machine Learning Engineers at Thoughtworks use modern architectures to develop end-to-end scalable machine learning systems and applications. They use their specialized depth and breadth of knowledge to impact the achievement of client, project or service objectives and advocate for ways of working to promote and deliver excellence. They operate within the framework of functional policies, navigate through intricate challenges and apply their proficiency to contribute to the success of high-stakes projects. Their leadership extends beyond technical prowess, encompassing strategic thinking and effective collaboration to drive innovation and deliver solutions that meet and exceed organizational goals.
As a lead machine learning engineer on projects, you will be leading the design of technical solutions or perhaps overseeing a program inception to build a new system and/or application. Alongside hands-on coding, as a key influencer, you will shape the trajectory of machine learning engineering initiatives, playing a pivotal role in advancing the field and ensuring impactful outcomes for the broader objectives of the company.
Job responsibilities
- You will embrace a strategic mindset, contributing to the direction of machine learning (ML) initiatives and aligning technical solutions with broader organizational goals
- You will play a pivotal role in program inception, shaping the development of new systems and applications from idea to reality, overseeing technical feasibility and resource allocation
- You will leverage your deep understanding of modern architectures to lead the development of scalable and maintainable ML systems, ensuring optimal performance and efficiency
- You will translate client needs into technically feasible and impactful ML applications, driving solution design and deployment within complex, high-stakes projects
- You will own the development and maintenance of ML applications, including ML pipelines, model training and deployment, and monitoring and evaluation
- As a key influencer, you will champion Responsible AI and effective ways of working within the team, advocating for a culture of excellence and continuous improvement
- You will navigate intricate technical challenges with proficiency, employing your specialized knowledge to troubleshoot issues and guide the team towards successful resolutions
- You will stay at the forefront of the evolving field of machine learning, actively seeking out and implementing new technologies and advancements to ensure Thoughtworks remains a leader in innovation
- You will foster a collaborative environment, effectively leading your team through hands-on coding alongside mentorship and guidance, empowering individual growth and knowledge sharing
- You will measure and analyze the impact of ML initiatives, iteratively refining approaches and ensuring solutions deliver tangible value to clients and the organization
Job qualifications
Technical Skills
- You have experience in developing a technical vision and strategy, keeping it relevant and aligned to the business needs
- You can design and execute cross-functional requirements based on business priorities
- You have experience in writing clean, maintainable and testable code, demonstrating attention to refactoring and readability of the code using Python or Shell
- You have experience with distributed systems and scalable architectures to handle large-scale ML applications
- You have experience with building, deploying and maintaining ML systems using relevant ML techniques and platforms, i.e.: Scikit-learn, Tensorflow, MLFlow, Kubeflow, Pytorch
- You have experience with building, deploying and maintaining ML systems and experience with application of MLOps principles and CI/CD to ML
- You have experience in machine learning engineering and data science, are familiar with key ML concepts, algorithms and frameworks, and understand ML model lifecycles
- You have experience with designing and operating the infrastructure required to run different types of ML training and serving workloads, i.e.: on-premise vs. cloud infrastructure, infrastructure as code, monitoring, etc.
- You have hands-on experience with on-premise and cloud services for building and deploying ML pipelines, i.e.: Azure, AWS, GCP or Databricks and associated ML managed services
Professional Skills
- You understand the importance of stakeholder management and can easily liaise between clients and other key stakeholders throughout projects, ensuring buy-in and gaining trust along the way
- You are resilient in ambiguous situations and can adapt your role to approach challenges from multiple perspectives
- You don’t shy away from risks or conflicts, instead you take them on and skillfully manage them
- You are eager to coach, mentor and motivate others and you aspire to influence teammates to take positive action and accountability for their work
- You enjoy influencing others and always advocate for technical excellence while being open to change when needed
- You are a proven leader with a track record of encouraging teammates in their professional development and relationships
- Cultivating strong partnerships comes naturally to you; You understand the importance of relationship building and how it can bring new opportunities to our business
Other things to know
Learning & Development
There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you. But we also balance autonomy with the strength of our cultivation culture. This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow. We see value in helping each other be our best and that extends to empowering our employees in their career journeys.
Travel
While we’ve traditionally been a traveling consultancy, we have adopted a hybrid working model with the majority of work being completed remotely from either home or local Thoughtworks offices. However, business travel to client locations should be expected when required by Thoughtworks or our clients’ needs.
Company Policies
We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
About Thoughtworks
Thoughtworks is a global technology consultancy that integrates strategy, design and engineering to drive digital innovation. For 30+ years, our clients have trusted our autonomous teams to build solutions that look past the obvious. Here, computer science grads come together with seasoned technologists, self-taught developers, midlife career changers and more to learn from and challenge each other. Career journeys flourish with the strength of our cultivation culture, which has won numerous awards around the world.
Join Thoughtworks and thrive. Together, our extra curiosity, innovation, passion and dedication overcomes ordinary.
#LI-Remote
Ready to apply?
Apply to Referrals OnlySenior Machine Learning Engineer
Share this job
Senior Machine Learning Engineers at Thoughtworks build, maintain and test the architecture and infrastructure for managing machine learning applications. They are involved in supporting and contributing to the design of the end-to-end applications and products. They are responsible for building core capabilities including technical and functional machine learning systems and applications, being the anchor for functional streams of work and are accountable for timely delivery.
As a senior machine learning engineer, you will work on the latest tools, frameworks and offerings while also being involved in enabling credible and collaborative problem solving to execute on a strategy.
Job responsibilities
- You will contribute to design and drive the development of robust scalable architectures and infrastructure for deploying and managing machine learning (ML) applications, ensuring high availability, performance and security.
- You will collaborate with data scientists and engineers to translate business needs into effective and efficient ML systems and applications.
- You will own the development and maintenance of core functionalities within ML applications, including ML pipelines, model training and deployment, and monitoring and evaluation.
- You will drive the functional stream of work by providing technical expertise, handling team discussions and ensuring timely delivery of assigned tasks.
- You will stay ahead of the curve by actively exploring and implementing the latest tools, frameworks and offerings in the ML landscape.
- You will facilitate collaborative problem solving within the team by actively listening, communicating effectively and mentoring other engineers.
- You will contribute to the development and execution of the team's overall ML strategy, aligning technical capabilities with business objectives.
- You will proactively identify and address challenges related to ML systems and applications, proposing solutions and implementing improvements.
Job qualifications
Technical Skills
- An advanced English level is required.
- You must have a strong experience with LLM and AI.
- You have experience in writing clean, maintainable and testable code, demonstrating attention to refactoring and readability of the code.
- You are proficient in scripting languages such as Python or Shell for automation and task streamlining.
- You have knowledge of distributed systems and scalable architectures to handle large-scale ML applications.
- You have experience with building, deploying, and maintaining ML systems using relevant ML techniques and platforms, i.e.: Scikit-learn, Tensorflow, MLFlow, Kubeflow, Pytorch.
- You have experience with building, deploying and maintaining ML systems and experience with application of MLOps principles and CI/CD to ML.
- You have experience in machine learning engineering and data science, are familiar with key ML concepts, algorithms and frameworks, and understand ML model lifecycles.
- You have experience with designing and operating the infrastructure required to run different types of ML training and serving workloads, i.e.: on-premise vs. cloud infrastructure, infrastructure as code, monitoring, etc.
- You have hands-on experience with on-premise and cloud services for building and deploying ML pipelines, i.e.: Azure, AWS, GCP or Databricks and associated ML managed services.
Professional Skills
- You understand the importance of stakeholder management and can easily liaise between clients and other key stakeholders throughout projects, ensuring buy-in and gaining trust along the way.
- You are resilient in ambiguous situations and can adapt your role to approach challenges from multiple perspectives.
- You don’t shy away from risks or conflicts, instead you take them on and skillfully manage them.
- You are eager to coach, mentor and motivate others and you aspire to influence teammates to take positive action and accountability for their work.
- You enjoy influencing others and always advocate for technical excellence while being open to change when needed.
Other things to know
Learning & Development
There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you. But we also balance autonomy with the strength of our cultivation culture. This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow. We see value in helping each other be our best and that extends to empowering our employees in their career journeys.
About Thoughtworks
Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech. As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work. For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator. Bring your brilliant expertise and commitment for continuous learning to Thoughtworks. Together, let’s be extraordinary.
#LI-Remote
Ready to apply?
Apply to Referrals Only
Senior ML Researcher (Spectrum)
Share this job
Software engineers and AI agents alike suffer from the same problem: finding that one person or place that will answer their tough, specific question. Many solutions promise to solve this with similarity search in vector databases. Unfortunately, finding the answer is often a puzzle with pieces to be collected across a myriad of contradictory sources and cannot be solved without surgical search and careful reasoning.
Spectrum collects data from an organization's code, docs, and issues, and organizes knowledge in a unified ontology that AI agents can efficiently search through and reason over. We aim to revolutionize the semantic layer space for software-building organizations and move beyond specs that fall out of sync with code, introducing a living spec – one that's extracted from the whole system and used to keep it aligned. Spectrum is meant to be the single source of truth for all product and architectural knowledge.
A resident of JetBrains' startup incubator, Spectrum enjoys startup speed and autonomy, and is backed by 25 years of developer tooling expertise. We are looking for a Senior ML Researcher to develop the core methods that make Spectrum possible – novel approaches to temporal ontology extraction, contradiction detection, and semantic alignment across heterogeneous software artifacts. You will help define and execute the research agenda, while also collaborating with JetBrains Research and external academic advisors.
Your responsibilities will include:
- Developing methods for LLM-based ontology construction from code, documentation, and issue trackers, including contradiction detection and reasoning over the resulting knowledge graph.
- Creating datasets, metrics, and benchmarks that drive measurable improvements across the system.
- Prototyping and validating research ideas, then collaborating with ML and software engineers to bring them to production.
- Collaborating with JetBrains Research and external academic advisors to shape our research agenda.
- Publishing research findings at top venues and representing Spectrum in the research community.
We expect you to have:
- A PhD (or equivalent research experience) in NLP, knowledge graphs, ontology learning, information extraction, or a closely related field.
- A strong publication record in at least one of the following areas: knowledge graph construction, ontology learning, information extraction, or NLP.
- Experience applying LLMs to structured knowledge extraction tasks.
- Experience designing and running rigorous experiments, including hypothesis formulation, ablation studies, and statistical evaluation.
- Strong Python and PyTorch skills, with the ability to implement research ideas from scratch.
- Excellent communication skills, with the ability to explain complex technical concepts to diverse audiences.
- Proficiency in English, both written and verbal.
Our ideal candidate would also have:
- Experience with ontology engineering, semantic web technologies (OWL, RDF, SPARQL), or ontology alignment.
- A background in code analysis, developer tools, or software engineering research.
- Experience with knowledge graph embedding methods or graph neural networks.
- Experience in early-stage startups – you enjoy the zero-to-one phase.
- Actively contributed to relevant open-source projects.
What we offer:
- A competitive salary and JetBrains benefits.
- A generous runway and corporate resources with startup autonomy.
- Full support for publishing ML research and attending industry conferences.
- Collaboration with JetBrains Research and external academic advisors in the knowledge graph and ontology learning community.
- A motivated team that moves fast, with short feedback loops, rapid prototyping, and lots of ownership.
- A rare applied research role where your methods ship in a real product used by software teams.
Why join JetBrains?
- Strong base salary. We offer competitive pay that reflects your skills and experience.
- Flexible work location. Enjoy the freedom to work from home or from the office.
- Remote work. Spend up to 30 days per year working remotely from abroad.
- Extra time off. More days to relax, recharge, and do the things you love.
- Medical insurance allowance. Enjoy peace of mind for you and your family
- Learning and development opportunities. Access to conferences, courses, and language classes.
- Language classes. Pick up the local language or sharpen your English skills.
- Fuel your day. Enjoy a hot meal or receive a lunch allowance on workdays.
- Mental health support. To help you feel your best, we provide easy access to professional mental health services.
- Sports benefit. Enjoy an on-site gym or sports club stipend.
- Internal events. Join company-wide celebrations and team gatherings.
*Some benefits may vary depending on location.
#LI-DNI
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Ready to apply?
Apply to JetBrains
Research Engineer (LLM Training and Performance)
Share this job
At JetBrains, code is our passion. Ever since we started back in 2000, we have been striving to make the strongest, most effective developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create.
We’re looking for a Research Engineer who will own the training stack and model architecture for our Mellum LLM family. Your job is easier said than done: make training faster, cheaper, and more stable at a large scale. You’ll profile, design, and implement changes to the training pipeline – from architecture to custom GPU kernels, as needed.
As part of our team, you will:
- Be responsible for improving end-to-end performance for multi-node LLM pre-training and post-training pipelines.
- Profile hotspots (Nsight Systems/Compute, NVTX) and fix them using compute/comm overlap, kernel fusion, scheduling, etc.
- Design and evaluate architecture choices (depth/width, attention variants including GQA/MQA/MLA/Flash-style, RoPE scaling/NTK, and MoE routing and load-balancing).
- Implement custom ops (Triton and/or CUDA C++), integrate via PyTorch extensions, and upstream when possible.
- Push memory/perf levers: FSDP/ZeRO, activation checkpointing, FP8/TE, tensor/pipeline/sequence/expert parallelism, NCCL tuning.
- Harden large runs by building elastic and fault-tolerant training setups, ensuring robust checkpointing, strengthening reproducibility, and improving resilience to preemption.
- Keep the data path fast using streaming and sharded data loaders and tokenizer pipelines, as well as improve overall throughput and cache efficiency.
- Define the right metrics, build dashboards, and deliver steady improvements.
- Run both pre-training and post-training (including SFT, RLHF, and GRPO-style methods) efficiently across sizable clusters.
We’ll be happy to bring you on board if you have:
- Strong PyTorch and PyTorch Distributed experience, having run multi-node jobs with tens to hundreds of GPUs.
- Hands-on experience with Megatron-LM/Megatron-Core/NeMo, DeepSpeed, or serious FSDP/ZeRO expertise.
- Real profiling expertise (Nsight Systems/Compute, nvprof) and experience with NVTX-instrumented workflows.
- GPU programming skills with Triton and/or CUDA, and the ability to write, test, and debug kernels.
- A solid understanding of NCCL collectives, as well as topology and fabric effects (IB/RoCE), and how they show up in traces.
Our ideal candidate would have experience with:
- FlashAttention-2 and 3, CUTLASS and CuTe, TransformerEngine and FP8, Inductor, AOTAutograd, and torch.compile.
- MoE at scale (expert parallel, router losses, capacity management) and long-context tricks (ALiBi/YaRN/NTK scaling).
- Kubernetes or SLURM at scale, placement and affinity tuning, as well as AWS, GCP, and Azure GPU fleets.
- Web-scale data plumbing (streaming datasets, Parquet and TFRecord, tokenizer perf), eval harnesses, and benchmarking.
- Safety and post-training methods, such as DPO, ORPO, GRPO, and reward models.
- Inference ecosystems such as vLLM and paged KV.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Ready to apply?
Apply to JetBrains
Staff Research Engineer (LLM Pre-Training)
Share this job
At JetBrains, code is our passion. Ever since we started back in 2000, we have been striving to make the world’s most robust and effective developer tools. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create.
We are working on an ambitious new platform that provides AI capabilities to all JetBrains products. Our platform is based on models developed in-house for writing and coding assistance, as well as integration with our strategic partners.
We are looking for a Research Engineer who can contribute to training foundation models for coding tasks. You’ll be working on developing Large Language Models from scratch and deploying them into production environments where they will be accessible by end users across the globe.
We value engineers who:
- Can plan projects and make decisions independently, consulting with others if needed.
- Identify customer needs and prioritize their tasks accordingly.
- Start with the simplest solutions and gradually add complexity as needed.
- Take sole responsibility for an entire subsystem.
- Have a passion for learning and a desire to stay up to date with the latest developments in the LLM field.
In this role, you will:
- Work with stakeholders to convert business requirements into technical specifications.
- Train LLMs from scratch on a large GPU cluster.
- Collect and process pre-training and fine-tuning datasets.
- Support and improve existing subsystems.
We’ll be happy to have you on our team if you have:
- Experience in design, deployment, and support of production ML systems.
- A strong theoretical background in NLP and transformer-based approaches.
- Proficiency with modern deep learning frameworks such as PyTorch and common libraries for NLP.
- Experience in distributed training of multi-billion parameter models.
- Attention to detail in everything you do and great communication skills.
We’d be especially thrilled if you have experience with:
- LLM inference frameworks such as vLLM, DeepSpeed, TensorRT.
- LLM alignment techniques such as RLHF/RLAIF.
- MLOps tools and practices, including CI/CD for ML.
- K8s and Kubeflow.
- Scientific publications in the NLP field.
How we develop JetBrains AI:
- A cluster of hundreds of NVIDIA GPUs as training infrastructure.
- Git for source control management.
- Python, PyTorch, and HuggingFace as an ML stack.
- Kubeflow and Weights & Biases for experiment tracking.
- TeamCity as a CI Automation system.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Ready to apply?
Apply to JetBrains
Research Engineer (Agentic Models)
Share this job
At JetBrains, code is our passion. Ever since we started, back in 2000, we’ve been striving to make the strongest, most effective developer tools on earth. Today, AI-powered assistance and agents are becoming a core part of how developers work in our IDEs.
We’re building multi-step coding agents that can understand large codebases, plan changes, call tools, and iterate with the user. As a Research Engineer in the Agentic Models team, you’ll be responsible for the models, training loops, and evaluation pipelines that power these agents.
You’ll work at the intersection of SFT and RL-style post-training, and product-driven evaluation, using our distributed GPU and MapReduce clusters to ship models into JetBrains products.
As part of our team, you will:
- Design, implement, and maintain SFT and RL post-training pipelines for multi-step coding agents.
- Train and adapt LLMs for agent workflows, including planning, tool use, and multi-step interactions inside JetBrains IDEs.
- Build and develop evaluation and simulation environments where coding agents can act, be measured, and compared on realistic developer tasks.
- Design evaluation frameworks and metrics for agent behavior, analyze traces and logs, and close the loop from evaluation back into training, data, and reward design.
- Analyze training and evaluation results to propose and implement improvements to model architectures, training recipes, and datasets.
- Work with large-scale infrastructure, including distributed training on GPU clusters and large MapReduce-style data processing for pre-training and fine-tuning datasets.
- Collaborate closely with research, product, and infrastructure teams to turn high-level product visions into concrete models, experiments, and shipped features.
We’ll be happy to bring you on board if you have:
- Extensive hands-on experience training LLMs (pre-training, fine-tuning, or post-training) in a research or production setting.
- Deep expertise in modern deep learning frameworks such as PyTorch, and specialized LLM training stacks (e.g. Megatron, NeMo, verl, or similar).
- Strong theoretical and practical understanding of LLM fundamentals: architectures, tokenization, data pipelines, batching, mixed precision, distributed training, and debugging unstable runs.
- The ability to own projects end to end, starting from a high-level problem or product pain point and overseeing it through the design, experimentation, implementation, and iteration phases.
- A product-aware mindset – you care about how developers actually use agents and can translate product needs and failure modes into modeling and evaluation work.
- At least 3 years of Python experience writing clean, maintainable code in modern ML codebases.
Our ideal candidate would have experience with:
- ML orchestrators and workflow tools such as Kubeflow, Dagster, Airflow, ZenML, and/or job schedulers like Kubernetes or SLURM.
- Large-scale data and training pipelines, e.g. MapReduce-style clusters, multi-node GPU training, or workloads on the order of 1M+ CPU/GPU hours.
- Designing and maintaining evaluation pipelines for LLMs or agents, including metrics, dashboards, experiment tracking, and automated regression checks.
- AI agent development, such as tool-using agents, planners, or multi-step coding workflows, and familiarity with agentic frameworks or patterns.
- Experiment tracking and observability using tools like Weights & Biases, MLflow, Langfuse, or similar.
- Inference optimization and serving optimized models in production.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Ready to apply?
Apply to JetBrains
Research Engineer (Agentic Behavior – Kotlin AI Value Stream)
Share this job
At JetBrains, code is our passion. Ever since we started, back in 2000, we've been striving to make the strongest, most effective developer tools on earth. Today, AI-powered coding agents are becoming a core part of how developers write Kotlin – and we want to make sure they write it well.
The Kotlin AI Value Stream team is responsible for how AI agents understand, generate, and improve Kotlin code across all platforms: Android, Kotlin Multiplatform, server-side, web, desktop, and others. We build the evaluation infrastructure, error analysis tools, and post-training pipelines that measure and improve agent behavior on real Kotlin developer tasks.
As a Research Engineer on this team, you'll own the end-to-end loop: Analyze how agents fail on Kotlin → build evals that capture those failures → research and implement methods to fix them → measure the improvement. Your work will directly shape how millions of developers experience Kotlin through AI coding agents.
As part of our team, you will:
Build tools for agentic error analysis
- Design and implement tooling to systematically capture, classify, and analyse errors that AI coding agents make when generating Kotlin code.
- Build observability pipelines over agentic traces – mining patterns from agent sessions in JetBrains IDEs, Junie, Claude Code, Cursor, and other coding agents.
Build evaluation pipelines
- Design, implement, and maintain evaluation pipelines that measure Kotlin code generation quality across dimensions, including correctness, idiomaticity, build success, framework usage, and test coverage.
- Build simulation environments where coding agents can be measured on realistic Kotlin developer tasks – from greenfield KMP projects and Gradle dependency management to migrating Spring applications from Java to Kotlin.
- Own evaluation infrastructure: metrics, experiment tracking, automated regression checks, and reproducible benchmarking.
Research methods for improving agent and model behavior on Kotlin
- Experiment with post-training techniques (SFT, DPO, GRPO) to improve how models handle Kotlin-specific patterns, idioms, and frameworks.
- Investigate context engineering approaches: CLAUDE.md/AGENTS.md files, compiler-as-verifier feedback loops, Kotlin LSP integration, and MCP-based tooling.
- Run experiments to measure impact: A/B comparisons, benchmark suites, and before/after analyses on real codebases.
- Collaborate with model providers (Anthropic, OpenAI, and Google) to translate Kotlin-specific findings into model improvements.
Build public Kotlin benchmarks
- Design and build open-source benchmarks that measure AI coding agent performance on Kotlin tasks and eventually become the standard reference for the ecosystem.
- Create task datasets covering the breadth of Kotlin usage: the server side (Spring, Ktor), multiplatform projects (KMP), build systems (Gradle), Android, library development, and others.
- Include both mined real-world tasks and carefully designed synthetic tasks that test specific Kotlin capabilities.
- Maintain and evolve benchmarks as models improve, ensuring they remain challenging, relevant, and contamination-resistant.
We'll be happy to have you on board if you have:
- Hands-on experience building evaluation or analysis pipelines for LLMs or AI coding agents in a research or production setting.
- Strong Python engineering skills (at least three years), with the ability to write clean, maintainable code in data-heavy and ML-adjacent codebases.
- Experience with data analysis at scale: querying large datasets (SQL/Athena), building data pipelines, and performing statistical analysis of experimental results.
- The ability to own projects end to end – from identifying a problem in agent traces to designing an eval, running experiments, and shipping a fix.
- A product-aware mindset: You care about how agents are actually used by developers and can translate real failure modes into evaluation and training work.
- Familiarity with Kotlin or a strong willingness to develop deep Kotlin expertise (you'll be living in Kotlin codebases daily).
Our ideal candidate would also have experience with:
- Post-training LLMs: SFT, RLHF, DPO, GRPO – either hands-on training or designing the data and reward pipelines that feed into training.
- Modern deep learning frameworks (PyTorch) and LLM training stacks (TRL, verl, Megatron, or similar).
- AI agent development: tool-using agents, multi-step coding workflows, agentic frameworks.
- Evaluation frameworks and tools: Inspect AI, Promptfoo, LM-evaluation-harness, or custom eval pipelines.
- Experiment tracking and observability: Weights & Biases, MLflow, Langfuse, or similar.
- The Kotlin ecosystem: Android, Gradle, KMP, Spring, Ktor – with an understanding of the developer workflows that agents need to support.
- Contributing to or maintaining open-source projects, especially benchmarks or evaluation tools.
Don't check every box? That's okay – if you're excited about this work and bring strong fundamentals, we'd love to hear from you. We're happy to talk and provide the training you need to grow into the role.
Why join JetBrains?
- Strong base salary. We offer competitive pay that reflects your skills and experience.
- Flexible work location. Enjoy the freedom to work from home or from the office.
- Remote work. Spend up to 30 days per year working remotely from abroad.
- Extra time off. More days to relax, recharge, and do the things you love.
- Medical insurance allowance. Enjoy peace of mind for you and your family
- Learning and development opportunities. Access to conferences, courses, and language classes.
- Relocation support. We help make your move as smooth and stress-free as possible.
- Language classes. Pick up the local language or sharpen your English skills.
- Fuel your day. Enjoy a hot meal or receive a lunch allowance on workdays.
- Mental health support. To help you feel your best, we provide easy access to professional mental health services.
- Sports benefit. Enjoy an on-site gym or sports club stipend.
- Internal events. Join company-wide celebrations and team gatherings.
*Some benefits may vary depending on location.
#LI-DNI
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Ready to apply?
Apply to JetBrains