Senior Machine Learning Engineer - Model Evaluations, Public Sector

The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, and multimodal pipelines—into mission-critical government environments. We build evaluation frameworks that ensure these models operate reliably, safely, and effectively under real-world constraints. As an ML Engineer, you will design, implement, and scale automated evaluation pipelines that help customers trust and operationalize advanced AI systems across defense, intelligence, and federal missions.

You will:

• Develop and maintain automated evaluation pipelines for ML models across functional, performance, robustness, and safety metrics, including LLM-judge–based evaluations.
• Design test datasets and benchmarks to measure generalization, bias, explainability, and failure modes.
• Build evaluation frameworks for LLM agents, including infrastructure for scenario-based and environment-based testing.
• Conduct comparative analyses of model architectures, training procedures, and evaluation outcomes.
• Implement tools for continuous monitoring, regression testing, and quality assurance for ML systems.
• Design and execute stress tests and red-teaming workflows to uncover vulnerabilities and edge cases.
• Collaborate with operations teams and subject matter experts to produce high-quality evaluation datasets.
• Comfortable with light travel (approximately 10%) for customer interaction and team needs.

This role will require an active security clearance or the ability to obtain a security clearance.

Ideally you’d have:

• Experience in computer vision, deep learning, reinforcement learning, or NLP in production settings.
• Strong programming skills in Python; experience with TensorFlow or PyTorch.
• Background in algorithms, data structures, and object-oriented programming.
• Experience with LLM pipelines, simulation environments, or automated evaluation systems.
• Ability to convert research insights into measurable evaluation criteria.

Nice to haves:

• Graduate degree in CS, ML, or AI.
• Cloud experience (AWS, GCP) and model deployment experience.
• Experience with LLM evaluation, CV robustness, or RL validation.
• Knowledge of interpretability, adversarial robustness, or AI safety frameworks.
• Familiarity with ML evaluation frameworks and agentic model design.
• Experience in regulated, classified, or mission-critical ML domains.

Position Summary

We are seeking a skilled Machine Learning Engineer with approximately three years of hands-on experience designing, deploying, and maintaining production-grade machine learning systems. In this role, you will collaborate closely with data scientists, software engineers, and product teams to translate research models into reliable, scalable, and high-impact applications. You will be deeply involved in the end-to-end ML lifecycle—from data ingestion and feature engineering to deployment, monitoring, and continuous improvement—playing a critical part in shaping our machine learning platform and capabilities.

Primary Responsibilities

• Develop, deploy, and optimize machine learning models for real-world business use cases and client-facing applications.
• Partner with data scientists to operationalize predictive models and ensure scalable, maintainable, and performant production deployments.
• Design and implement data pipelines and workflows that support training, inference, and model lifecycle management.
• Work with large, complex datasets to ensure data quality, reproducibility, and reliable version control across ML workflows.
• Implement model monitoring, logging, and alerting strategies to track performance, detect drift, and support retraining cycles.
• Leverage cloud platforms (AWS, Azure, GCP) to build scalable ML solutions using managed services and infrastructure-as-code practices.
• Write clean, modular, and well-documented code aligned with MLOps and software engineering best practices.
• Stay current on emerging ML tooling, frameworks, and industry best practices to continuously enhance our platform and capabilities.

Qualifications

• Master’s degree in Computer Science, Data Science, Engineering, or a related technical field.
• 6+ years of experience in machine learning engineering, applied ML, or related software engineering roles.
• Strong proficiency in Python and experience with modern ML frameworks such as TensorFlow, PyTorch, or scikit-learn.
• Experience with distributed data processing and compute frameworks (e.g., Pandas, Spark, Dask).
• Hands-on experience with containerization and orchestration technologies such as Docker and Kubernetes.
• Familiarity with CI/CD pipelines, testing automation, and version control using Git.
• Experience working with cloud-based ML platforms or services (e.g., SageMaker, Vertex AI, Databricks, or Snowflake ML) is preferred.
• Strong understanding of model evaluation, feature engineering, and performance optimization in production contexts.
• Excellent analytical, communication, and collaboration skills, with the ability to work effectively in cross-functional teams.

This is an exempt position. The annualized base pay range for this role is expected to be between $140,000-$180,000.  Actual base pay could vary based on factors including but not limited to experience, subject matter expertise, geographic location where work will be performed, and the applicant's skill set.  The base pay is just one component of the total compensation package for employees.  Other rewards may include an annual cash bonus and a comprehensive benefits package, including but not limited to medical, dental, vision, life and 401(k). Please note that the job title is subject to change based on the selected candidate’s experience and education.

Data Scientist - MID

Location: Springfield, VA 

TS/SCI REQUIRED

Position Responsibilities:  

• Support NSG strategies through the creation of automated collection models, dynamic analytic models, workflow automations, and any other automation processes and products as assigned.  
• Refine, enhance and improve the operational performance of any automated solution through the evaluation of performance data, regular customer interaction, and a standardized maintenance cycle.  
• Apply data science and visual programming tradecraft to support and streamline analysis tasks as identified by stakeholders and the Government.  
• Enhance technical solutions to problems related to IC intelligence integration, automated collections, tipping and cueing, information sharing, and visualization.  
• Conduct extensive collections and analytic modeling, data processing,  

Required Skills. Proficiency in common geospatial software applications and tools, such as visual programming (JEMA, FADE/MIST, ECO/ETAS), Python, SQL, Git, GIMS, AWS Sagemaker, AWS Cloud, ESRI ArcGIS, statistics (descriptive, Bayesian), Markov-Chain modelling, TensorFlow, Linear Algebra, R, SAS, NLP.