7a: Replication: Sharing Data With Other Computers

Now to the fun part: let's share this hypercore of chat messages with another machine (or peer). Hypercore supports this, and calls the process of two machines exchanging the messages of a feed replication.

Hypercore and the whole Dat ecosystem uses an asymmetric key encryption scheme to sign and encrypt all the data. This guarantees data integrity when we share data with another peer.

So, let's dive deeper. There are 3 keys to keep in mind when using hypercore:

Public Key πŸ”‘

This key is globally unique.

It is used to identify our feed publicly, and also allows us to verify that data has not been tampered with when replicating with peers we do not nessesarily trust.

We can also encrypt a message using someone else's public key so that nobody else but them can decrypt it.

Secret Key πŸ”

In hypercore, only the feed’s owner (whoever has the secret key) can write to the log. This key is kept only on the person’s local machine and it is never shared with anyone.

The secret key is used to decrypt messages encrypted with our public key, as well as for signing messages we've written to prove that we are the original author.

Discovery Key 🌍

We recognize our feed by its unique public key and allow only those peers who know this key to be able to exchange information.

We might not want other parties to know our public key, since it uniquely identifies us. So we make use of another key, derived via a one-way function from our public key, that we can freely share. This is called the discovery key. It is a hash of the public key. This key is used to discover peers without leaking the public key.

Hypercore and the Public Key

If someone gives us their public key, we can read their feed.

To achieve this, hypercore accepts a second parameter in its constructor:

hypercore(<storage>, <public key>, <options>)

Each machine compares the parts of the hypercore they have locally to the full set of parts that are known to exist for that hypercore. They then proceed to ask for the parts they are missing from their peers. They also give back to the network by sharing those parts that they have locally but which other nodes are missing and have asked for. This is how all the peers in a P2P network can eventually converge on the same set of data, even if some of the peers are not always online.

Exercise

Let's look at these keys in your feed!

Try extending single-chat.js to print them out:

feed.ready(function () {
  console.log('public key:', feed.key.toString('hex'))
  console.log('discovery key:', feed.discoveryKey.toString('hex'))
  console.log('secret key:', feed.secretKey.toString('hex'))
})

The keys are also stored in the same folder as the data but are stored in binary. You can view them with a tool like xxd.

xxd single-chat-feed/key
00000000: 23af 00b1 46c9 56a8 dc57 8bd1 8ffa b722  #...F.V..W....."
00000010: 34eb 550b be55 6933 6d9d 5183 c156 935b  4.U..Ui3m.Q..V.[
xxd single-chat-feed/secret_key
00000000: a330 e73a 2ca9 8b4b 5083 ebf0 75a4 159e  .0.:,..KP...u...
00000010: 2ab5 d2f2 2c55 4b47 d79c 8d48 81e2 8485  *...,UKG...H....
00000020: 23af 00b1 46c9 56a8 dc57 8bd1 8ffa b722  #...F.V..W....."
00000030: 34eb 550b be55 6933 6d9d 5183 c156 935b  4.U..Ui3m.Q..V.[

Once you solve this exercise continue to exercise 7b