Skip to content

perf(onboard): reduce repeated DNS and TLS overhead in provider validation probes #3771

Description

@wscurran

Problem

#2001 identifies provider validation as a source of onboard latency and fragility. Validation can run multiple sequential probes, and each probe may spawn a fresh curl process with its own DNS lookup, TCP connection, TLS handshake, request setup, and teardown.

On slower or virtualized networks, especially WSL2, corporate VPN, proxy, or remote-development environments, repeated connection setup can make validation slower and more fragile than the provider itself requires.

This issue focuses on making validation probes cheaper and less repetitive. It is separate from adaptive timeout calibration, which decides how long NemoClaw should wait.

Scope

Reduce repeated DNS, TCP, TLS, and process setup overhead in provider validation probes.

Candidate areas:

  • Inventory provider-validation probe sequences that repeat DNS/TCP/TLS setup.
  • Add DNS pre-resolution where safe.
  • Evaluate passing resolved addresses to curl with --resolve for known host/IP pairs.
  • Evaluate curl --next or equivalent batching for related probes.
  • Consider native Node HTTP keepalive if curl-based reuse becomes too awkward.
  • Preserve current provider-validation semantics and fallback behavior.
  • Add tests for optimized and fallback paths.

Expected Behavior

Provider validation should avoid repeating expensive connection setup work when multiple probes target the same provider endpoint.

Optimizations should be conservative: if DNS pre-resolution, address pinning, batching, or connection reuse is not safe for a provider path, NemoClaw should fall back to the existing behavior.

Related Work

This issue is related to #3770 but distinct: #3770 decides the timeout budget, while this issue reduces the amount of repeated work each validation attempt performs.

Acceptance Criteria

  • Provider-validation probe sequences that repeat DNS/TCP/TLS setup are identified.
  • Where safe, validation avoids unnecessary repeated DNS resolution or connection setup.
  • Existing provider fallback behavior is preserved.
  • Slow-network and failure-path behavior remains debuggable.
  • Tests cover successful validation, fallback probe path, DNS/pre-resolution failure, and providers that cannot safely use the optimized path.
  • perf: investigate and reduce networking latency during onboard and validation #2001 is updated with implementation notes or trace evidence.

Non-goals

  • Changing validation timeout policy.
  • Replacing readiness polling loops.
  • Adding profiling trace output.
  • Parallelizing onboard orchestration.
  • Changing provider support policy.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area: cliCommand line interface, flags, terminal UX, or outputarea: inferenceInference routing, serving, model selection, or outputsarea: performanceLatency, throughput, resource use, benchmarks, or scaling

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions