March 2021. The days are longer, Covid shows the first signs of receding and Spring is coming. For the Ancients, this was the beginning of the new year, a time of renewal and hopes. So too it is for Coder’s Errand.
This month marks the third anniversary of this blog, and with it new changes are coming. It is not so much about the style of the blog. That was the big opus of last year and I’m happy with it.
Nor with the general direction. Despite my extra-curricular interests in Physics, Music and a few other things, I am still focused on my traditional topics of Blockchain, Computer Science and Cryptography. What has changed, though, is my professional focus, and that, I’m sure, will have an impact.
Over the last year, there were a couple of transformations. For one, I am now a people’s manager. My time is divided between mentoring and leading others, and the hard tech development that I did before.
Secondly, my focus at work is now on developing a whole new Blockchain, reducing my focus on the Ethereum platform. Therefore, I expect more posts dedicated to the intricacies of Aventus, and the architecture and design of blockchain features. It will be an interesting ride.
The Aventus Network (AvN)
Another change that happened at the beginning of the year was my employer. I moved from Artos, together with the whole team, to Aventus Network Services, to develop, deliver and grow the Aventus Network.
The first steps were taken last month, with the release of the mainnet and the onboarding of the first transactions. It is amazing to see the fruits of a year-long work, and of many discussions in esoteric topics.
Frequently, when discussing a new feature, we delved deep into some aspects that we thought were edge cases, and would hardly come up. Still, we cared for them, and took our time to protect against those cases, just to be safe.
How surprising, chastening and, in a sense, exhilarating to see those precise “edge cases” emerge in the first month of production! And how glad we were we had the foresight to think of them, and the humility to acknowledge that they still deserved our care and respect. The result is that our chain is running robustly and happily.
I expect that many of these issues will at some point make an appearance in these pages. Stay tuned, there are many lessons here for those developing their own chains or interested in architecture and design.
So, what is Substrate after all?
The Aventus Network, affectionately called the AvN, is built on Substrate, so we didn’t have to implement everything from scratch. But what is Substrate?
For those up to date with the blockchain world, the short answer is that it is the technology behind Polkadot. But this is hardly satisfying for more curious readers, so let me explain a little more.
Substrate is a framework for developing Blockchains, and is developed by Parity based on research by the Web3 Foundation.
As is the aim of every good framework, Substrate intends to offer you something (in this case, a blockchain) running out of the box, that you merely have to customize in order to suit your own objectives and business model.
Notably, Substrate offers these things out of the box:
- logic for implementing a BFT consensus protocol between network participants
- a voting mechanism to support the consensus protocol
- a networking layer, handling all the communication between network nodes, including discovery, gossipping protocol and replication
- a runtime environment, that is executed inside every chain full-node, and executes the business logic. This is compiled into Web Assembly to facilitate runtime upgrades without stopping the chain and avoid hard-forks.
- the ability to quickly start a new blockchain and integrate it into the Polkadot ecosystem
I intend to talk about some of these aspects in future posts, and so I won’t delve into them here. But all of this is remarkable. It’s basically abstracting the essentials and basic architecture of a blockchain in pretty much the same spirit that the internet has been abstracted in the last 2 or 3 decades.
If blockchain technology ends up developing in the same way most technology has, unifying and consolidating, then some project like Substrate will be at the centre of it.
Polkadot
It is noteworthy that chains built with Substrate are compatible with each other in some way. Polkadot, itself, is the project that spearheaded the development of Substrate. If I recall correctly, it emerged as a tool during Polkadot’s development, to save developers from having to solve the same problems over and over again.
But Polkadot has a different aim, and is independent of Substrate. It is positioned as a competitor to Ethereum, in that it can provide an execution environment for any blockchain logic. It can host and execute smart contracts and even emulate the Ethereum’s EVM. But its most notable feature, in my opinion, is that it acts like a federation of other chains. Instead of, like Ethereum, trying to solve everyone’s problems by having a Turing-complete execution environment, it orchestrates dozens of other chains, each dedicated to a single use case.
This is not so easy to explain, so let me try. I believe it is more efficient to explore a certain niche with logic focused on exactly that one use case. Ethereum derives many of its problems with scalability from having to maintain the Turing-complete EVM and prevent exploitation of its resources by the use of gas. Focused chains can be much lighter.
But every newcomer faces an uphill battle to attract enough users to ensure the security of the chain. This is the genius of Polkadot. A niche-chain can become a para-chain of Polkadot. This is the relay-chain at the centre of the ecosystem, offering these chains the ability to communicate but, most importantly, the critical mass they need. This is done by allowing Polkadot validators to check and validate the blocks of each Parachain as if they existed in the relay-chain itself. In essence, a single set of validators is pooled to ensure the security of all the para-chains.
My Experience Developing the AvN on Substrate
Substrate is great for what it offers the community. You can take it, customize it and quickly launch a blockchain that can stand on its own or connect to Polkadot.
With the AvN, though, we chose a different path. We offer a fast chain that can scale to hundreds of transactions per second, at a very low cost, but still derive its security from the enormous amount of Ethereum users. In a sense, we upgrade a subset of Ethereum, by building the AvN as a Layer 2.
This blog, however, is not the place to advertise and promote the use of the AvN. It started as a personal endeavour and for the most part it will remain so. But I will use the AvN as a source of inspiration for posts. I will touch on concepts of general use for other developers, both on Substrate and other chains.
Indeed, I have been using carelessly in this post many terms that need refinement or are not general knowledge. Concepts like “Layer 2”, “BFT Consensus”, etc. all are worthy of their own posts or even series. They will come, if time and energy allow.
For now, I will close just with some thoughts on what it has been like to work with Substrate.
First of all, let me extend my thanks to all the colleagues who have accompanied and supported me in this journey, and shared the despair and joys that regularly come with complex projects like this.
There will be many hurdles for those wanting to take the path we did. Substrate is written in Rust, which in itself is a growing and fascinating language. I have no complaints with Rust. It has some quirks that will take time to learn (like references, borrowing and lifetimes), but I find it elegant and efficient.
Substrate is not a language. It is built in Rust, but comes with its own macros, definitions, modules and their own public functions. This makes it complex, big and unforgiving. The documentation is still not at the level I would like, and Substrate can feel bloated for many projects.
In trying to do everything, you may find, as we did, that some things are just too much. And disentangling your project from all the unneeded code will be hard work. Mastering Substrate is a long and demanding challenge.
Substrate does not solve everything either. During our progress, we found that we are probably the only Substrate project out there seeking to not only read but also write to Ethereum, and as a consequence we had to research and solve many problems outside of Substrate’s scope. They were fascinating but also provided many rewards. In time, I hope, they will be exciting matter for this blog.
I hope I have whetted your eagerness for what is to come. Stay tuned, if you are really interested in blockchain design. Travel safe.