The first version was a minimal layer to do just the DHT/NAT and tried to be very generic and extensible, deferring identity and transport to the app. This second version simplifies the base protocol and incorporates the identity and transport elements into the core.
Every packet must begin with two bytes that are a short unsigned integer representing the length of bytes that follow it. That length of bytes are UTF8 encoded JSON and can be parsed by any JSON parser. Any remaining bytes on the packet are considered raw binary and referenced as the ‘body’ during content transport.
The top level keys of the JSON are for switch use only, the presence of different keys trigger different responses:
The value is any string, and is the thread id as defined by the sending switch. When received and any response is generated at all, the same tid value must be returned so the sending switch can match up the response to it’s request. The value format can be anything and is the domain of the sending switch only.
This is to initiate a new session with any switch, and can be any arbitrary string value, but should be sufficiently unique and random per use so that it can’t be guessed by anyone. Along with start, the to and from keys should also be set to the IPP of the sending and receiving switches, so that the public IPP can be identified and verified.
Upon receiving a start to create a session one must be generated to return and create the shared sid value, which is calculated as the SHA1 hash of both starts combined in either order. To validate, either side should combined the start values sent and received and compare to the sid value.
Once a sid is validate, it should be used for all future packets in that session.
These are only allowed along with a sid and are the mechanism for reliable packet delivery. The seq must always be increased incrementally with every packet sent. A buffer of sent packets must be kept keyed by the seq value until the receiving switch has responded confirming them in a range and not in the miss.
The range is an array of two integers, the first seq received in the session, and the last one received.
The miss is an array of integers and must be sent along with any range if in the process of receiving packets there were any missing sequences, containing in any order the missing sequence values within the associated range. Upon receipt those packets should be resent verbatim.
By default a session should be invalidated if a sequence has been missed three or more times, or there’s more than 100 missed packets by default (senders cannot send more than that without a confirming range). When there’s consistently missing packets, senders should not send very many packets beyond the confirmed range. (TBD)
When any switch is told about another, there may be a NAT between them, so they must both be told to send a packet to each other to make sure the path between them is open, this is called “hole punching.” The pop value is used to do this, containing the IPP of the switch being contacted for the first time.
The local switch is A, the existing one in the DHT is B, and this happens when B tells A about C, a new switch that A may be interested in. When A goes to contact C it should send the packet directly, as well as send a packet containing a pop:”IPP of C” to B. Upon receiving a pop with an IPP value that is not itself, it sends a new packet with a pop and the same value to the given IPP, as well as setting the source value to that of A. When C receives a pop value of itself, it sends a packet with popped:true to the value of source. If the first packet A receives from C is popped:true, it should resend the original packet since it was likely dropped due to the NAT, but the path is open now.
Only valid with a sid, the recv value is a hash to match against any incoming send value. All received values are associated with the active sid, so that once the session is over, they are all invalid as well. In order to validate a recv was accepted, the matching send and a tracking tid can be sent too, as any receiving switch should apply the recv before processing the rest of the packet, which would then match and be relayed back.
To stop receiving any hash on a session, just send stop with the hash.
On any packet a bind can be sent, this is an object containing keys/values of a base URI and crypto signatures to validate it. The recipient should cache the validated mapping between the contained base uri and the IPP it came from.
A bind enables the recipient to verify the sender’s id and trust the IPP of that sending id. It can be sent proactively with any send, and can also be sent in response to any request to verify the responder.
The contents of the bind are TBD.
Any HTTP request can be performed within a session using the req and res objects. A tid is required and must be unique to each request.
The req is an object that looks like:
"req": {
"uri":"http://telehash.org/"
"m":"GET",
"h":{ "User-Agent": "testing" }
}
It can also be a POST or PUT, and any content body on the request should be included as binary appended to the packet, and spread across subsequent packets containing only just the same tid value if it’s larger than a single one. When all the data has been sent, a done of true must be set on the last packet (or first one if it was all self contained).
Any response works identically, a res object like:
"res": {
"s":200,
"h":{ "Content-Type": "foo", "Content-Length":100 }
}
The binary body is appended to the packet, and when broken across multiple they must all contain just the same tid and the body is appended in order based on the seq until the last one containing the done of true.
The URI in the request doesn’t need to be just HTTP either, this pattern can be used for streaming or other content transport patterns, and application specific schemes.
To reach any other switch you have to use the DHT to find it or the switches close to the same hash you’re both using. You start by contacting any switch closest to the target hash and a send with the hash value (and any bind and tid and body as needed). If that switch knows any closer, it will respond with a see array of other switches closer than it. Take the top three closest and repeat the process until nobody responds or you get the tid returned with the reciprocating bind of the target you were trying to reach, connected to a new session with the sending IPP.
When a recv is matched with an incoming send, a new packet is sent in the session the recv came from copying in the send, tid, bind, and any body data, along with a source of the switch it came from.
This is the same as telehash v1 and needs to be re-described here yet…