This role is for one of our clients

We are seeking experienced Engineering Experts to design, develop, and validate complex engineering simulation problems for training and evaluating advanced AI models and coding agents. This role combines deep engineering expertise, scientific computing, simulation-based design, and AI evaluation to create challenging tasks that test an AI system's ability to interpret technical requirements, solve engineering problems, use simulation tools, and produce physically valid solutions.

Experts will work across Electrical, Mechanical, and Aerospace Engineering, developing realistic, multi-constraint problems with measurable objectives and objectively verifiable outcomes. The role also involves analyzing AI execution logs, identifying reasoning and tool-use failures, and refining evaluation benchmarks to improve model performance.

Job Details

• Role: Engineering Expert
• Experience: 10+ years of hands-on engineering design experience
• Education: Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering, or a closely related discipline
• Locations: Bangladesh, India, Indonesia, Egypt, Ghana, Nigeria, Turkey, Brazil, Colombia, and approved US states
• Employment Type: Short-term contractor engagement
• Contract Duration: Up to 24 weeks
• Start Date: Immediate
• Compensation: USD 500 per task, based on an estimated 30–40 hours of work per task
• Availability: Approximately 40 hours per week, with at least 4 hours of overlap with US Pacific Time (PST)
• Work Arrangement: Remote, with access to a personal computer and reliable high-speed internet

Requirements

Key Responsibilities

1. Engineering Problem Design

• Develop original, self-contained engineering design problems with realistic technical requirements, competing constraints, and clearly defined optimization objectives.
• Create challenging tasks that require AI models to interpret specifications, evaluate design trade-offs, configure simulation environments, and iteratively improve engineering solutions.
• Establish validated reference solutions and objective automated grading mechanisms to assess correctness, performance, and compliance with engineering requirements.

2. Simulation and Computational Engineering

• Build, configure, execute, and validate simulation-based problem environments using relevant open-source engineering tools.
• Develop Python scripts, custom test benches, and supporting utilities to automate simulations and verify expected results.
• Ensure that simulation outputs satisfy physical plausibility, dimensional consistency, boundary conditions, convergence requirements, and other domain-specific constraints.
• Troubleshoot simulation failures, numerical instability, dependency issues, and computational errors.

3. AI Evaluation and Failure Analysis

• Evaluate the performance of large language models (LLMs) and coding agents across repeated engineering problem-solving trials.
• Analyze execution trajectories, tool interactions, simulator feedback, and intermediate outputs to identify systematic reasoning failures.
• Investigate issues such as incorrect interpretation of engineering specifications, premature convergence, invalid geometries, inconsistent units, and physically impossible solutions.
• Apply evaluation concepts such as pass@k, failure-mode classification, and analysis of nondeterministic model behavior.

4. Benchmark Development and Refinement

• Calibrate task difficulty using empirical model performance and evaluation results.
• Refine engineering problems to improve their ability to distinguish between successful and unsuccessful AI solutions.
• Ensure that tasks remain unambiguous, technically rigorous, reproducible, and objectively gradable.
• Collaborate with AI researchers, technical leads, and subject-matter experts to integrate validated benchmarks into AI evaluation workflows.

Required Qualifications and Skills

• Master's degree or PhD in Electrical Engineering, Mechanical Engineering, Aerospace Engineering, or a closely related technical discipline.
• At least 10 years of practical engineering design experience.
• Strong Python programming and scripting skills.
• Hands-on proficiency with at least one relevant open-source engineering simulation or design tool, such as:
• Electrical simulation: ngspice, PySpice
• Mechanical and computational engineering: OpenFOAM, FEniCSx, CalculiX
• Control systems: python-control, OpenModelica
• CAD and design automation: CadQuery, build123d
• Thermodynamics and meshing: Cantera, Gmsh
• Experience developing, executing, debugging, and validating simulation-based engineering workflows.
• Familiarity with modern LLMs, AI coding agents, and AI evaluation methodologies.
• Ability to analyze model execution logs and identify underlying reasoning, planning, and tool-use failures.
• Strong understanding of physical constraints, units, boundary conditions, numerical convergence, and engineering validation.
• Excellent analytical, problem-solving, and technical documentation skills.
• Ability to work independently and deliver technically accurate results within defined timelines.

Preferred Qualifications

• Experience in AI model evaluation, benchmark development, automated grading, or coding-agent assessment.
• Familiarity with repeated-trial evaluation, pass@k metrics, failure taxonomies, and nondeterministic model behavior.
• Experience building automated testing frameworks and reproducible computational environments.
• Knowledge of optimization methods, numerical analysis, simulation automation, and engineering design validation.
• Experience collaborating with multidisciplinary engineering and AI research teams.

Engineering Domains

Candidates should have strong expertise in one or more of the following areas:

• Electrical Engineering
• Mechanical Engineering
• Aerospace Engineering

Work Commitment and Contract Terms

• Expected commitment of approximately 40 hours per week.
• At least 4 hours of daily overlap with US Pacific Time.
• Weekend on-call availability is required; part-time engagement may be acceptable where agreed.
• The engagement is contractor-based and may last up to 24 weeks.
• Candidates must have a personal desktop or laptop and a stable, high-speed internet connection.
• Compensation is USD 500 per task, with an estimated 30–40 hours of effort per task. Actual task allocation and payment terms should be confirmed before engagement.