The Network

A lot has happened in the last month or so. For various reasons I was able to spend a bit more time on Cinode and make some good progress with this project. In this and some following posts I will outline what the current status of Cinode is, what it is currently capable of, how you can play with it yourself.

Will try to keep this as generic as possible, no deep math this time, I promise 😉.

Major milestones

There are two major milestones that I managed to reach.

First one is the initial implementation of dynamic links. Those are, in my humble opinion, the most powerful concept in Cinode allowing distributed propagation of dynamically changing data. All of that without the need of global ledger that almost all blockchains require (and let’s be clear about it - Cinode is NOT a blockchain).

In addition to dynamic links, components of cinode now start working together and propagate the data between themselves. Currently it is just a simple implementation where one can add sibling or secondary nodes to query for data that could not be found locally. This itself though already allows creating whole Cinode networks.

See it in action

This blog (when viewed through https://blog.cinodenet.org/) is actually served through Cinode network utilizing dynamic links. Let me picture this:

And here’s some more info about all the components.

Blog deployment pipeline

The deployment of new content in this blog happens in few steps.

The first one is to get the source of this blog, compile it with hugo and generate static set of compiled files.

Once done, those files are then compiled with an encrypting tool. It takes the directory structure and creates corresponding dataset. This dataset itself contains only encrypted blobs and can not be used alone to access any plaintext data. For that reason, an additional entrypoint information is generated which allows decrypting whole directory structure.

The compilation process also requires additional information, called WriterInfo - it contains necessary data that allows creating the root dynamic link (contains things such as the private key). This WriterInfo is only used during the compilation process and must not be revealed elsewhere.

The final step in the pipeline is the release of upgraded dataset. In case of this blog instance it is about creating a docker image with static dataset and deploying it in some tiny kubernetes cluster.

Static datastore

Once compiled and encrypted, the dataset must become available to its consumer (so the backend for the blog page). This is done using a simple container with nginx that simply exposes the directory with encrypted files. The way the dataset is laid out in the folder makes it compatible with the Cinode datastore API thus can be accessed directly through https without any proxy code. It could actually be called a cinode network component. It can not update its dataset since the dataset is static but can be read from by other nodes.

Worth noting is that this component does not have access to any information that would allow decrypting the data in its blobs. It is thus the part of the public layer of cinode.

Blog backend

The blog backend component is another piece of the network. It does not come with any dataset at all, the only information it contains is the entrypoint dataset and the address of the other node that contains the data. To avoid querying the static datastore for data all over again, it also contains its own in-memory datastore that acts here as a cache.

For the backend component it really does not matter where the data comes from. It is currently served from the static datastore. This datastore could actually be split into multiple independent components (e.g. by storing only blobs with given name prefix in each), it could be a mix of static, dynamic nodes. In the future I can forsee some more advanced setups such as DHT based on Kademlia.

It is here - in the memory if the cinode_web_proxy process where all the decryption takes place. This component is partially part of the public network layer (it has its own datastore that holds encrypted blogs) and the private one since it can decrypt the data. This is enabled by the entrypoint information which, next to other necessary things, contains the encryption key for the root of the directory.

The entrypoint points out to a dynamic link, the root one for the blog content. It does not change (at least it should not change frequently) which means the data behind this entrypoint can change but the entrypoint will stay the same. This is what currently happens - while new blog content is being published, only the static datastore needs to be updated. The backend will periodically update its in-memory blob cache catching up with changes made to the root link.

The update of the root link can only be performed with the knowledge of WriterInfo that is used during the deployment pipeline. Since the blog backend does not know that information, it can not generate new valid link for the same entrypoint. This puts that backend into a private Cinode layer but not the publisher one.

Frontend LB

If you visit https://blog.cinodenet.org page (or the accompanying https://blog-datastore.cinodenet.org) it goes through Cloudflare. This proxy is just a safety fuse. Cinode is a very immature project, it has many places that require optimization and may contain some hidden bugs. Cloudflare will help dealing with spontaneous traffic spikes and will act as a sanitization layer for the incoming traffic.

What’s next

Cinode made a big progress in recent weeks and already has a first working (and a little bit decentralized) implementation. What I’ve put into this post today only scratches the surface and I already have a lot of ideas what else to write about.

Can not guarantee frequent updates here since I won’t have as much time to spend on Cinode in the following weeks as in the last month, but let’s stay a bit optimistic here. Making small concrete steps is the way to go I believe.