String representation of addresses and routes

[hairyhum]

Route string format:

For user APIs we want to be able to serialize routes into strings or have some sort of string representation of routes.

Routes consist of series of addresses, so we need a string representation of address and some separators between them.

Rust library already using # to separate address type and address value. Address value currently can have any symbols other than # and format depends on the transport.

This means that # - type and value separator for address cannot be used in the address value

Also the separator between addresses in the route cannot be used in the address value.

In order to enforce that we have a few options:

1. Enforce limited characters to be used by transport to encode string address and make sure they don't contain the separators. For example use ASCII characters and URLEncoding.

This means that every transport now needs to implement a to_string and from_string API for its addresses and string addresses have to follow strict rules

2. Allow addresses to have string representation and do additional encoding for separators when creating routes. In this case addresses still would have to provide a to_string functionality

This would cause an inconsistency in the format and might cause situations when you cannot just take a string address from a string route because it was encoded, which is not too convenient.

3. Have binary representation for transport addresses and transform them to string only for routes. This might be less convenient when working with addresses separately.

This would mean string addresses will be bigger and less readable as they will likely be something like Base64

If we're fine with forcing all transports to provide "address_to_string" returning something like "NUMBER#URL_ASCII" (option 1), then we can use something like ; for a separator:

0#foo;1#1.node.ockam.network;0#echoer

Also we can have every address to be started with the separator, so just concatenating addresses would create a route string. For example if separator is $:

$0#foo$1#1.node.ockam.network$0#echoer

While addresses are $0#foo, $1#1.node.ockam.network

Otherwise binary address encoded to string makes more sense (option 3), but it would make addresses in the route not readable.

[thomcc]

Sorry in advance if I'm misunderstanding here.

Have binary representation for transport addresses and transform them to string only for routes.

This seems like the best option IMO — just use this serialization for display to the user and/or user input in some APIs. Then, you can perform whatever escaping you need when displaying without it causing any additional cost for transport.

Everywhere else any byte should be allowed in the addresses contents — That way, you only need a single to_string/from_string, and there's no extra validation needed to decode an address thats already in binary format — win/win. (or am I missing something?)

Actually using the string repr for anything is going to be wasteful — for example, Rust already generates addresses by generating 16 random bytes, and then hex-encoding them (to get ASCII). The ASCII is which is what gets sent over the wire, which is 2x bigger than it needs to be because of the hex-encoding.

It's goofy/pointless, but needed because there's no single decision for how this stuff should be represented as a string (which I don't have strong opinions on — various ways of escaping the problematic bytes would work i guess, as you suggested)

Anyway, If we care enough about keeping these small to want address type to stay as a single byte (which IIRC @mrinalwadhwa said is important on bluetooth), it genuinely doesn't make sense to use the string representation of these for anything beyond display and accepting input from a user (esp because the string repr of a u8 will be up to 3 bytes — "123#asdfasd").

[hairyhum]

If we have a route level (and not transport level) encoding of addresses, we might lose infomation in addresses like 1#my.node.com or 1#127.0.0.1 because route level encoder might not know that address is supposed to have a meaningful string format. The routers to_string function would have no idea that 4 bytes in address is an ipv4 and could be formatted as x.x.x.x unless transport provides this information.

That's the main downside this of generic binary-to-string approach.

On the other hand, a transport may not be available for some nodes in the system, so those nodes would not be able to format the route if address format is transport-specific.

I think it's a good point that we're currently using some sort of a mix of binary and string formats and it's confusing, and I agree that we could use binary format more instead of the string format for inter-machine communication.

For machines there is not much benefit of strings over binaries, the string representation is for humans.

[spacekookie]

I definitely agree that addresses should be binary encoded. Currently they kinda are (well it's utf-8 string encoding but we only see an address as a byte array).

But most importantly: addresses can contain the # character! The Rust parsing is currently not clever enough to accept you creating an address from the string 1#foo#bar but this is a totally valid address to create and use via the Address::new(1, "foo#bar") API.

That is something we should fix, but it's a problem that purely exists in the conversion layer between strings and the underlying address.

[hairyhum]

Maybe we should actually use the prefix separator then, so the format becomes something like $<type>#<value> matching ^\$[0-9]+\#.*$ ?

This way the value can be anything except $ and we can just concat addresses to make a route

[mrinalwadhwa]

If we can push UTF8 String <-> Route encoding/decoding to the edges of the system - to CLIs and UIs

Then I think we should explore the possibility of addresses being displayed as URLs - something like tcp://ip:port, udp://ip:port, ws://, ockam-worker:// etc.

And routes can be some concatenations of URIs with a separator.

[hairyhum]

This looks nice. Although if we require urlencoded values, then : in ip:port is going to be %3A or we'll have to go with inconsistent urlencoding.

[thomcc]

I suspect we should use a URN rather than a URI, as to not imply availability of some resource. This would also avoid technically needing to go through through IANA for some of these.

takes off former browser developer hat