Gruve
Pick a job to read the details
Tap any role on the left — its description and apply link will open here.
Director of Product Management, AI Inferencing
Share this job
About Gruve
Gruve is an innovative software services startup dedicated to transforming enterprises to AI powerhouses. We specialize in cybersecurity, customer experience, cloud infrastructure, and advanced technologies such as Large Language Models (LLMs). Our mission is to assist our customers in their business strategies utilizing their data to make more intelligent decisions. As a well-funded early-stage startup, Gruve offers a dynamic environment with strong customer and partner networks.
About the Role
As the Director of Product Management, AI Inferencing you will play a pivotal role in defining and delivering next-generation full-stack (hardware + software) datacenter solutions that combine innovative GPU platforms, high-speed networking, scalable storage, and advanced cooling technologies. You will help shape the future of AI infrastructure—supporting AI inference pipeline—by aligning customer needs with Gruve’s engineering innovation and operational capabilities.
This role requires deep knowledge of the AI cloud software ecosystem—both public and on-premises—as well as the hardware that powers it. Success demands a blend of product leadership, market research, project coordination, and executive-level communication. You will lead through influence to develop and drive to market tightly integrated solutions built on. This is an opportunity to join an early-stage startup and play a key role in shaping and defining its product strategy.
Key Responsibilities
- Product Strategy for Modern Data Centers work with internal stakeholders and external partners to shape the product vision and contribute to its roadmap for cutting-edge datacenter solutions that integrate GPU-accelerated computing, high-speed networking and cooling building blocks. Stay ahead of industry trends in AI hardware infrastructure, AI software trends, and the competitive offerings landscape.
- Monitoring, Analytics & Optimization Work with internal groups across the full stack to ensure it produces monitoring and analytic data specific to AI operations and inference, supporting future product, development, and operational decisions.
- AI Inference and Data Workflow Enablement Work closely with AI/ML customers to identify the latest trends in infrastructure requirements for AI inference, and data processing workloads.
Basic Qualifications
- Bachelor’s degree in Computer Science or Computer/Electrical Engineering
- 7+ years of experience in product management and solutions architecture
- Strong background in datacenter infrastructure (servers, networking, storage), AI platforms, or cloud-native environments
- Demonstrated experience driving complex projects in cross-functional, matrixed organizations
- Excellent presentation skills, with proficiency in PowerPoint and storytelling for executive and technical audiences
- Ability to perform market research, competitive analysis, and customer engagement activities
- High-level understanding of product lifecycle—from concept to launch
Preferred Qualifications
- Experience with Supermicro/Dell/Lenovo hardware, AMD-based/NVIDIA-based/RTX-based AI infrastructure, Kubernetes, or cloud orchestration
- Familiarity with Agile methodologies and product management tools (e.g., Jira, Clickup, Microsoft Project)
- Prior experience in enterprise IT, datacenter design, or cloud infrastructure product development, network management
- MBA or advanced degree in technical or business field.
Salary Range
$220,000 - $275,000 USD + Benefits + Equity
Why Gruve
At Gruve, we foster a culture of innovation, collaboration, and continuous learning. We are committed to building a diverse and inclusive workplace where everyone can thrive and contribute their best work. If you’re passionate about technology and eager to make an impact, we’d love to hear from you.
Gruve is an equal opportunity employer. We welcome applicants from all backgrounds and thank all who apply; however, only those selected for an interview will be contacted.
Ready to apply?
Apply to Gruve
About Gruve
Gruve is an innovative software services startup dedicated to transforming enterprises to AI powerhouses. We specialize in cybersecurity, customer experience, cloud infrastructure, and advanced technologies such as Large Language Models (LLMs). Our mission is to assist our customers in their business strategies utilizing their data to make more intelligent decisions. As a well-funded early-stage startup, Gruve offers a dynamic environment with strong customer and partner networks.
Position Summary:
We are looking for a highly experienced Network Architect & Operations Lead to drive the design, operation, and optimization of large-scale, AI inferencing data center environments. The ideal candidate will have a strong background in architecting and running complex distributed infrastructures supporting GPU-as-a-Service (GPUaaS) workloads, high-throughput AI inferencing traffic, GPU cluster networking, and globally distributed network topologies.
This role is central to ensuring the uptime, reliability, and scalability of our inferencing data centers. This role requires a senior professional with CCIE-level expertise (or equivalent capability) who can combine architecture, operations, team leadership, and automation skills to ensure network reliability, scalability, and operational excellence across a 24/7 GPU inferencing environment. The ideal candidate will lead and mentor a tiered team of L1, L2, and L3 network engineers, owning end-to-end network operations and SLA commitments for our AI inferencing data centers. Experience in high-scale environments such as Lambda Labs, Nvidia, Equinix, AWS, Azure, Google Cloud, or similar large AI infrastructure or hyperscale organizations is highly preferred.
Key Responsibilities:
- Operate large-scale AI inferencing data center network environments across multiple sites.
- Lead and manage a tiered NOC team (L1, L2, L3), driving operational discipline, escalation processes, and continuous uptime of inferencing infrastructure.
- Own uptime and SLA commitments for GPU inferencing data centres, ensuring high availability and rapid incident response.
- Design and maintain scalable network topologies supporting high-throughput GPU cluster traffic, east-west data flows, API traffic, VPN connectivity, and GPUaaS tenant workloads.
- Lead technical design decisions for leaf-spine data center architectures using Arista switching platforms.
- Configure, manage, and troubleshoot advanced routing protocols with a strong focus on BGP, EVPN, VXLAN, and large-scale traffic engineering.
- Manage and optimize Cisco firewall platforms (Firepower/FTD) to ensure secure and efficient traffic flows across tenant and infrastructure networks.
- Support GPU cluster networking including high-bandwidth, low-latency east-west traffic between GPU nodes, with familiarity in RDMA over Converged Ethernet (RoCE) or similar low-latency fabrics.
- Drive operational excellence through structured NOC methodologies, runbook standardization, incident management, and continuous optimization.
- Improve network efficiency through automation, tooling, and operational workflows.
- Perform deep troubleshooting and debugging of complex network and performance issues across Arista, Cisco, and Dell infrastructure.
- Lead network capacity planning, upgrades, and infrastructure scaling initiatives in line with GPU compute growth.
- Collaborate with cross-functional engineering teams (compute, storage, platform, security) to support business growth and ensure high availability.
- Document architecture standards, operational procedures, runbooks, and best practices.
Basic Qualifications:
- 10+ years of hands-on experience in network architecture or network operations within large-scale enterprise, cloud, or AI/HPC data center environments.
- Strong expertise in advanced routing & switching technologies, including BGP, OSPF, EVPN, and VXLAN.
- Deep operational understanding of multi-site, high-scale data center network infrastructure.
- Hands-on experience with Arista EOS and Arista switching platforms in a data center environment.
- Proven experience managing complex network topologies and distributed environments.
- Experience leading and managing NOC teams (L1/L2/L3), including escalation frameworks, shift management, and SLA ownership.
- Familiarity with high-performance compute (HPC) or GPU cluster networking and associated traffic patterns.
- Strong troubleshooting, debugging, and analytical skills across multi-vendor environments.
- Practical experience with network automation and operational optimization.
Preferred Qualifications:
- CCIE certification (or equivalent real-world expertise).
- Arista ACE (Arista Certified Engineer) certification or equivalent hands-on Arista expertise.
- Experience with GPU-as-a-Service (GPUaaS), AI/ML inferencing platforms, or hyperscale compute environments.
- Hands-on experience with Cisco Firepower / FTD firewall platforms and enterprise security frameworks.
- Experience with Dell PowerEdge or similar GPU server environments and their network integration.
- Familiarity with RDMA over Converged Ethernet (RoCE), InfiniBand, or other low-latency GPU interconnect fabrics.
- Experience with Arista and Cisco routing and switching platforms.Leaf-spine data center fabric design for AI/ML compute clusters.
- Experience with BGP EVPN and VXLAN overlay architectures. Network automation frameworks and scripting (Python, Ansible, Terraform)
- Experience with Tenant network isolation, VLANs, and multi-tenancy in GPUaaS environments.
- Additional certifications such as CCNP, Arista ACE-L, or equivalent advanced networking certifications.
Why Gruve
At Gruve, we foster a culture of innovation, collaboration, and continuous learning. We are committed to building a diverse and inclusive workplace where everyone can thrive and contribute their best work. If you’re passionate about technology and eager to make an impact, we’d love to hear from you.
Gruve is an equal opportunity employer. We welcome applicants from all backgrounds and thank all who apply; however, only those selected for an interview will be contacted.
Ready to apply?
Apply to Gruve
Senior Engineering Leader - AI Infrastructure and Inferencing
Share this job
About Gruve
Gruve is an innovative software services startup dedicated to transforming enterprises to AI powerhouses. We specialize in cybersecurity, customer experience, cloud infrastructure, and advanced technologies such as Large Language Models (LLMs). Our mission is to assist our customers in their business strategies utilizing their data to make more intelligent decisions. As a well-funded early-stage startup, Gruve offers a dynamic environment with strong customer and partner networks.
About the Role
We're seeking an exceptional Senior Engineering Leader to build and lead a high-performing engineering team focused on design and development of a distributed multi-tenant AI inference SaaS platform. Platform development responsibilities include but are not limited to software design, development and testing for multiple domains such as inference engines (AI/ML, program and compiler analysis), core platform services, and observability. This role sits at the intersection of systems engineering, AI/ML operations, and product development, requiring both deep technical expertise and proven leadership capabilities.
As a leader at Gruve, you'll drive the technical vision and execution of critical infrastructure that enables our AI capabilities at scale. You'll work closely with cross-functional partners including research scientists, product managers, and other engineering leaders to deliver robust, performant systems that power our AI products.
This position is based in the United States and reports to the SVP of Inferencing and Infrastructure Management
Key Responsibilities
- Team Leadership & Development: Build, mentor, and scale a world-class engineering team of 10-15+ engineers. Foster a culture of technical
- Excellence, collaboration, and continuous learning. Conduct performance reviews, career development planning, and succession planning.
- Technical Strategy & Architecture: Define and execute the technical roadmap for AI inference infrastructure, AI toolchains, and AI software development. Make critical architectural decisions that balance performance, scalability, maintainability, and cost.
- Compiler Design & Optimization: Lead the development of AI inference systems and optimizations for AI workloads, including graph optimization, kernel fusion, and hardware-specific code generation to maximize inference performance.
- AI Model Development & Deployment: Oversee the end-to-end lifecycle of AI models from development through production deployment, including model fine-tuning, quantization, distillation, and serving infrastructure.
- Inference API & Platform Development: Drive the design and implementation of scalable, low-latency inference APIs and platforms that serve models reliably at production scale with strict SLA requirements.
- Spec-Driven Development: Champion rigorous engineering practices including comprehensive technical specifications, design reviews, and documentation to ensure alignment and quality across complex projects.
- Cross-Functional Collaboration: Partner effectively with research, product, and business stakeholders to translate requirements into technical solutions and communicate progress, trade-offs, and risks clearly.
- Delivery & Execution: Own quarterly planning, roadmap prioritization, and on-time delivery of major initiatives. Establish metrics and KPIs to measure team performance and system health.
Basic Qualifications
- 10-15+ years of software engineering experience with at least 5+ years in engineering leadership roles managing teams of 5+ engineers
- Proven track record of building and scaling high-performing engineering teams in high-growth technology companies
- Deep expertise in systems programming languages (C++, Go, Rust, or similar) and architecture design
- Strong background in AI model design, optimization, or adjacent systems-level programming (LLVM, MLIR, XLA, or similar frameworks)
- Hands-on experience with AI/ML model development, training, and inference systems
- Experience with model fine-tuning techniques and deployment optimization (quantization, pruning, etc.)
- Demonstrated ability to design and build production-grade APIs and distributed systems
- Strong understanding of spec-driven development processes and engineering best practices · Excellent communication skills with ability to influence across all levels of the organization
- Demonstrated ability to work effectively with teams across multiple time zones
- Bachelor's degree in Computer Science, Engineering, or related technical field (or equivalent practical experience)
Preferred Qualifications
- Master's or PhD in Computer Science, Machine Learning, or related field
- Experience at leading AI/ML companies or research labs (OpenAI, Google DeepMind, Meta AI, Anthropic, etc.)
- Direct experience with modern ML frameworks (PyTorch, JAX, TensorFlow) and their compilation stacks
- Background in GPU programming (CUDA, Triton) and hardware acceleration for ML workloads
- Experience with transformer architectures and large language model (LLM) inference optimization
- Track record of shipping production ML systems serving millions of requests per day
- Contributions to open-source compiler or ML infrastructure projects
- Experience with cloud infrastructure (AWS, GCP, Azure) and containerization/orchestration (Kubernetes, Docker)
- Previous experience scaling teams from <10 to 20+ engineers
Salary Range
$240,000 - $300,000 USD + Benefits
This position is with Gruve and is an on-site role based at the Gruve headquarters in Redwood City, California.
Why Gruve
At Gruve, we foster a culture of innovation, collaboration, and continuous learning. We are committed to building a diverse and inclusive workplace where everyone can thrive and contribute their best work. If you’re passionate about technology and eager to make an impact, we’d love to hear from you.
Gruve is an equal opportunity employer. We welcome applicants from all backgrounds and thank all who apply; however, only those selected for an interview will be contacted.
Ready to apply?
Apply to Gruve