Jobs › Companies › Portpro › Director of Software Engineering (Node.js & Web Scraping Expert)

Sobre este puesto de Director of Software Engineering (Node.js & Web Scraping Expert) en Portpro

Portpro · Remoto · Los Angeles, California
We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.

Key Responsibilities- Software Engineering & Web Scraping Leadership:

  • Architect, develop, and maintain scalable and distributed web scraping systems using Node.js.
  • Design and implement data extraction pipelines to process large volumes of structured and unstructured data.
  • Develop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation.
  • Optimize scraping processes for performance, reliability, and efficiency while managing proxy services(residential, datacenter, rotating).Oversee data storage and processing strategies, ensuring high availability and consistency.
  • Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications.
  • Implement best practices for microservices, API integrations, and real-time data streaming.
  • Key Responsibilities- Scalability, Security & DevOps:

  • Lead the transition to cloud-native, containerized, and serverless architectures for web scraping.
  • Ensure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.).Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping.
  • Manage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks.
  • Work closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability.
  • Key Repsonsibilities- Engineering Team Management & Strategy:

  • Lead, mentor, and grow a high-performance engineering team.
  • Define and execute the technology roadmap, aligning with business objectives.
  • Foster a culture of continuous learning, collaboration, and innovation.
  • Implement agile development methodologies (Scrum, Kanban) to optimize project execution.
  • Ensure code quality, security, and best practices across all engineering efforts.
  • Qualifications & Experience- Technical Expertise:

  • 10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction.
  • Strong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation.
  • Extensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques.
  • Deep knowledge of proxy management (residential, datacenter, rotating, and VPNs).Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.).
  • Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.).Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure).
  • Qualifications & Experience- Leadership & Soft Skills:

  • Proven track record of scaling engineering teams and leading complex projects.
  • Strong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecks.
  • Excellent communication and stakeholder management skills.
  • Passion for mentorship, team development, and continuous learning.
  • Preferred Qualifications:

  • Experience with machine learning for data extraction and NLP.
  • Knowledge of browser fingerprinting and bot detection mechanisms.
  • Familiarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.).
  • Prior leadership experience in data-driven businesses or web scraping startups.
  • ¿Listo para postularte en Portpro?
    Postúlate en Portpro

    Empleos similares

    Portpro
    Engineering Manager
    Portpro
    ⚡ Postúlate pronto New York City, New York · restringido por ubicación
    ● Nuevo 👁 Visto ✓ Postulado hace 3 meses
    Portpro
    Staff Engineer
    Portpro
    ⚡ Postúlate pronto New York City, New York · restringido por ubicación
    ● Nuevo 👁 Visto ✓ Postulado hace 3 meses
    Portpro
    Data Scientist
    Portpro
    ⚡ Postúlate pronto New York City, New York · restringido por ubicación
    ● Nuevo 👁 Visto ✓ Postulado hace 3 meses
    Portpro
    Director of DevOps
    Portpro
    ⚡ Postúlate pronto Los Angeles, California · restringido por ubicación
    ● Nuevo 👁 Visto ✓ Postulado hace 3 meses
    ALO
    Asset Protection Specialist - The Grove
    ALO
    ⚡ Postúlate pronto Los Angeles, California, Unite... Presencial $58,240–$62,400
    ● Nuevo 👁 Visto ✓ Postulado hace 3h
    The Nature Conservancy
    Associate Director, Principal Gift Development
    The Nature Conservancy
    ⚡ Postúlate pronto Los Angeles, California Presencial
    ● Nuevo 👁 Visto ✓ Postulado hace 4h
    Oscar Health
    Product Designer
    Oscar Health
    ⚡ Postúlate pronto Los Angeles, California, Unite... Híbrido $123,372–$161,926
    ● Nuevo 👁 Visto ✓ Postulado hace 5h
    Celsius
    Field Marketing Manager - Celsius
    Celsius
    ⚡ Postúlate pronto Los Angeles, California, Unite... · restringido por ubicación $60,000–$70,000
    ● Nuevo 👁 Visto ✓ Postulado hace 7h
    Fullmind
    Virtual SLP (Speech Language Pathologist) California K-12 Schools
    Fullmind
    ⚡ Postúlate pronto Los Angeles, California, Unite... · restringido por ubicación
    ● Nuevo 👁 Visto ✓ Postulado hace 7h

    Regístrate para recibir sugerencias adaptadas a los empleos que abres y las búsquedas que guardas.

    Más empleos en Portpro

    Ver todos los empleos en Portpro →

    Postúlate ahora
    🤖

    Un momento — para

    JobsRadar se creó para personas reales que están pasando un mal momento en su búsqueda de empleo — no para solicitudes automatizadas. Estás haciendo clic demasiado rápido y ahora estás bloqueado temporalmente.

    Vuelve más tarde. Si de verdad estás buscando empleo, cuentas con nosotros — solo compórtate como una persona.

    Catch your next role the second it’s posted.

    Create a free account and we’ll watch the boards for you — the instant a job matches your search, it lands in your inbox or Telegram. No digging, no refreshing.

    Create free account

    Free forever · takes 30 seconds · already have one?

    Toma ventaja en tu búsqueda de empleo.

    Únete a nuestro canal de Telegram para lo que te ayuda a conseguir el puesto — referencias salariales, el pulso semanal del mercado y avisos de nuevas funciones. Sin spam, solo señal.

    Únete al canal — es gratis