About this Senior Engineer, Compute role at Thinkahead
The High-Performance Computing Compute Engineer is primarily responsible for the overall health and maintenance of the infrastructure that powers high-performance computing, artificial intelligence, machine learning, and large-scale data workloads in our managed services customer environments. Our HPC Compute Engineers are a valued member of the Managed Services Infrastructure Practice responsible for Tier 3 incident management, service request management and change management infrastructure support for all Managed Services customers.
The successful candidate will own critical components of the compute environment—from Linux hosts, GPUs, HPC clusters, and physical hardware through schedulers, containers, monitoring, automation, and version maintenance. The engineer will work closely with our internal infrastructure teams, vendors, and customers to deliver reliable, secure, and efficient AI compute services.
Key Responsibilities
-
Provide enterprise-level operational support to Managed Services customers for incident, problem, and change management activities
-
Manage and maintain large-scale HPC and GPU compute clusters for AI training, inference, simulation, and data-intensive workloads
-
Deploy and administer Linux-based compute environments, including operating systems, drivers, firmware, CUDA or equivalent accelerator software, libraries, and runtime dependencies
-
Operate workload scheduling and orchestration platforms such as Slurm, Kubernetes, Ray, or comparable technologies
-
Benchmark and tune end-to-end workload performance
-
Troubleshoot complex failures from bare metal and firmware through the operating system, drivers, middleware, distributed frameworks, and user applications in step with other teams
-
Build automation for provisioning, configuration management, patching, validation, monitoring, and lifecycle management using Python, Bash, Ansible, Terraform, or similar tools
-
Develop operational dashboards, alerts, health checks, and capacity-planning processes that improve availability and resource utilization
-
Establish repeatable processes for incident response, root-cause analysis, change management, and operational readiness
-
Create technical documentation, runbooks, architecture diagrams, and user guidance for HPC and AI platforms
-
Participate in on-call rotation
-
Completion of training and certification as assigned to further skills and knowledge
Required Qualifications
-
Bachelor’s degree in computer science, computer engineering, electrical engineering, or a related technical field; equivalent practical experience may be considered
-
Four or more years of experience deploying, operating, or engineering HPC, GPU, distributed systems, or cloud infrastructure
-
Strong Linux or UNIX systems administration experience in production environments
-
Hands-on experience with GPU computing, accelerator infrastructure, or large-scale AI workloads
-
Experience with at least one workload manager or orchestrator, such as Slurm, Kubernetes, Run:ai, or PBS
-
Strong scripting and automation skills in Python, Bash, or a comparable language
-
Working knowledge of networking fundamentals, including TCP/IP, DNS, DHCP, routing, firewalls, Ethernet, and high-speed interconnects
-
Experience diagnosing performance, reliability, and capacity issues across multiple infrastructure layers in a physical datacenter environment
-
Ability to document systems clearly and communicate effectively with both technical and nontechnical stakeholders
-
Demonstrated ability to work independently, prioritize operational risk, and collaborate in a fast-moving environment
Preferred Qualifications
-
Experience with NVIDIA GPU platforms, CUDA, NCCL, DGX, HGX, or comparable accelerator technologies
-
Experience with InfiniBand, RoCE, RDMA, congestion control, or lossless network fabrics
-
Experience with Slurm configuration, fair-share scheduling, QoS, accounting, job arrays, and high-throughput workload management
-
Experience with Kubernetes, GPU operators, container runtimes, Helm, or cloud-native infrastructure
-
Experience with parallel and distributed storage platforms such as Lustre, GPFS, BeeGFS, Ceph, ZFS, or XFS
-
Familiarity with PyTorch, TensorFlow, MPI, OpenMP, UCX, NCCL, Ray, or distributed model-training frameworks
-
Experience with observability platforms, telemetry pipelines, and performance-profiling tools
-
Experience with configuration management, infrastructure as code, CI/CD, GitOps, or immutable infrastructure practices
-
Experience supporting LLM training, fine-tuning, retrieval-augmented generation, model serving, or inference platforms
