IPIP-0476: Delegated Routing DHT Closest Peers API

Related Issues
ipfs/specs/pull/476
ipfs/specs/pull/497
History
Commit History
Feedback
GitHub ipfs/specs (inspect source, open issue)

1. Summary

Add a new HTTP endpoint to the Delegated Routing API that allows clients to find the closest peers to a given peer ID without being full DHT clients.

2. Motivation

Browser nodes and other resource-constrained clients need to perform peer discovery operations without the overhead of being full DHT clients. The primary use case is for browser nodes performing random walks to find peers that they can make circuit relay reservations on.

Currently, to find peers close to a particular key in the DHT keyspace, a node must:

  1. Be a full DHT client with all the associated overhead
  2. Maintain connections to many peers
  3. Handle the complexity of the DHT protocol

This is particularly problematic for:

3. Detailed design

This IPIP introduces a new "DHT Routing API" section to the Delegated Routing V1 HTTP API specification with the following endpoint:

3.1 GET /routing/v1/dht/closest/peers/{key}

3.1.1 Path Parameters

  • key is a CID or Peer ID to find the closest peers to.
    • CID SHOULD be a CIDv1 in any encoding.
    • Peer ID can be represented as a Multihash in Base58btc, or a CIDv1 with libp2p-key (0x72) codec in Base36 or Base32.
    • Arbitrary multihash lookups can be performed by wrapping the multihash in a CIDv1 with raw (0x55) codec.

3.1.2 Response

The response follows the same format as the existing peer routing endpoints, returning a JSON object with a Peers array containing peer records conforming to the Peer Schema.

3.2 Specification Changes

The following changes are made to src/routing/http-routing-v1.md:

  1. Add a new "## DHT Routing API" section after the "Peer Routing API" section
  2. Document the /routing/v1/dht/closest/peers/{key} endpoint with its parameters and response format
  3. Update the document date to reflect the modification

4. Design rationale

The design follows several key principles:

4.1 Simple MVP Approach

The endpoint provides the minimum viable functionality needed for the primary use cases. It can be expanded later if more complex routing scenarios emerge.

4.2 Future-Proofed Path Structure

The path /routing/v1/dht/closest/peers/{key} is intentionally structured to allow future expansion:

4.3 Routing-Agnostic Implementation

While the endpoint is in the DHT namespace, implementations have flexibility in how they determine "closest" peers. This allows for optimization based on the specific routing system being used.

4.4 Consistency with Existing API

The endpoint follows the established patterns of the Delegated Routing API:

4.5 User benefit

This enhancement provides significant benefits to end users:

  1. Browser Compatibility: Browser nodes can discover circuit relay servers without implementing the full DHT protocol
  2. Reduced Resource Usage: Light clients save bandwidth and processing power by delegating peer discovery
  3. Faster Peer Discovery: Delegated routing servers can provide cached results more quickly than performing DHT walks
  4. Simplified Implementation: Application developers can implement peer discovery with simple HTTP requests instead of complex DHT logic

4.6 Compatibility

This change is fully backward compatible:

4.7 Security

The new endpoint introduces no additional security considerations beyond those already present in the Delegated Routing API:

4.8 Alternatives

Several alternatives were considered:

  1. Full DHT Client Implementation: Rejected due to excessive resource requirements for browser and mobile environments

  2. Custom libp2p Protocol: Would require all nodes to implement a new protocol, creating adoption barriers

  3. Extension of Existing Peer Routing: The /routing/v1/peers/ endpoint serves a different purpose (finding specific peers rather than closest peers)

  4. Amino-Specific Endpoint: Initially considered /routing/v1/amino/ namespace but rejected in favor of the more generic /dht/ approach

  5. Query Parameters for Filtering and Limiting: During design discussions, count and closer-than query parameters were considered:

    • count: limit the number of results returned
    • closer-than: filter peers to only return those closer than a reference peer

    These were omitted from the initial implementation (boxo v0.35.2, someguy v0.11.0) for simplicity. The endpoint defaults to returning up to the DHT bucket size (20 for Amino DHT). Future backwards-compatible updates MAY add these parameters if use cases emerge that require more fine-grained control.

4.9 Peer Ordering

Implementations SHOULD return peers sorted by closeness to the key. For Kademlia-based DHT implementations (such as Amino DHT), this means sorting by XOR distance with the closest peers first.

5. Test fixtures

This IPIP does not deal with content-addressed data, so specific test CIDs are not applicable. However, implementations should test:

  1. Valid CID and Peer ID inputs return appropriate closest peers
  2. Results are limited to the DHT bucket size
  3. Peers are sorted by closeness (XOR distance for Kademlia DHTs)
  4. Invalid keys return appropriate error responses
  5. Both streaming (NDJSON) and non-streaming (JSON) response formats work correctly

A. References

[rfc2119]
Key words for use in RFCs to Indicate Requirement Levels. S. Bradner. IETF. March 1997. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc2119

B. Acknowledgments

We gratefully acknowledge the following individuals for their valuable contributions, ranging from minor suggestions to major insights, which have shaped and improved this specification.

Editors
Alex Potsides (Shipyard) GitHub
Marcin Rataj (Shipyard) GitHub