The Internet of Things is expected to consist of 30 billion connected devices by 2020. How would a distributed ledger ever be able to handle such a large scale of transactions, while running on low-end devices? In this blog, I will continue my explanation on how IOTA is planning to improve the maximum throughput of the system to scale towards mass adoption.
Vision of IOTA
The IOTA platform is specifically designed for the Internet of Things (IoT). Any device connected to the internet is an IoT device. In recent years, many household devices have become IoT devices to create home automation. But IoT goes far beyond that. The Internet of Things enables the development of smart cities, automated manufacturing, and mass data collection. But, in order to do so, IoT devices need a platform that facilitates the (micro)payments of services and data exchange between machines, or so-called machine-to-machine transactions. Therefore, IOTA is a distributed ledger without transaction fees, great scalability, and secure data transfers. In my previous IOTA Deepdive blog, I already explained some of the technical aspects of IOTA that eliminates the need for transaction costs and removes the inherited scaling issues from Blockchain. But in order to become the platform for all IoT devices, IOTA needs to scale beyond the current state.
Besides the transaction throughput, one of the more growing scaling issues is the size of the blockchain. At the time of writing, running a Bitcoin node will cost you 172 GB of disk space and is growing at a rate of about 1 GB per week. Meanwhile, Ethereum is at 75 GB, growing at a speed of about 1.7 GB a week. Both Blockchains are working on their own scaling solutions like the Bitcoin Lightning network, to move as many transactions of the ledger as possible, and Ethereum Sharding, which splits the entire ledger over multiple nodes. For IOTA, this size of the ledger is (going to) grow even faster. The current size of the complete ledger is around 100 GB, but it can grow at an unlimited speed since it doesn’t have a set amount of transactions per second (e.g. 1 GB growth at the day of writing).
To counter this problem, IOTA is currently using a feature called snapshotting. When the ledger grows too big, the IOTA Foundation proposes a snapshot of the ledger. This means that all the transactions are added up, and a list of addresses with the number of IOTA’s owned is generated. All the nodes verify the list in order to check if the correct addresses have the correct number of IOTAs. Only when a consensus is reached across the network, the Tangle will be reduced to only a list of the addresses and all the transactions will be removed. It is also a common misconception that transaction that transfer value survive a snapshot since all transactions are removed regardless of the values sent. This is a very important feature for IOTA to consider when you design an IOTA application. However, historical transactions are still stored in what is called a permanent node that can be used to check transactions from before a snapshot. This is probably going to be a paid service in the future. It is currently not possible yet to run a permanent node, but this is currently being developed.
However, IOTA is made to work on very small IoT devices. When the Tangle starts to handle more and more transactions a manual snapshotting feature would not be ideal. Automated Snapshotting has been on the roadmap of the IOTA Foundation for quite a while and will allow nodes to choose how much of the ledger they want to store, automatically removing old transactions and keeping track of the current addresses with the owned IOTA’s. While the IOTA Foundation has yet to release the Automated Snapshotting feature, SemkoDev (a third party developer) has recently shown a working IOTA implementation on the Raspberry Pi. The Raspberry Pi runs a full IOTA node using an automated snapshotting algorithm to keep the ledger at a fixed size (e.g. 10 GB) while using only 300 MB of memory. These optimization steps make IOTA more and more compatible for the Internet of Things.
One of the more controversial design decisions of IOTA has been the fact that IOTA transactions are sent in Trinary. Trinary is a way to encode data using 3 symbols. This is different from Binary, which uses the two symbols (0 and 1) to represent the data, and the decimal system (0-9). Every computer is currently made to understand the Binary system, however, research shows that Trinary encoding is about 6% more efficient in storing the data. The speed increase that we have experience in the last few decades in computer power seems to have hit its limit. This limit is caused by the fact that the computer parts are already at an atomic level and can’t be shrunken down further. For this we can conclude that space is very important for computer speed, so why not switch from Binary to the more compact Trinary? For one, It would be a very difficult paradigm shift since Binary and Trinary computers would have inefficient communication. The Internet of Things, however, is relatively young and could still handle such a paradigm shift, which might cascade to more electronic devices in the long run.
IOTA is built with Trinary to grasp every bit of optimization that they can find to allow IOTA to run on low-end devices, but little is known about the exact ways that Trinary would improve upon Binary. It has been a long-standing rumor that the IOTA Foundation is working together with JINN labs to create a Trinary processor that would be able to solve IOTA’s PoW in the fastest and most energy-efficient way. The progress is still under NDA, but the project is all but confirmed. The IOTA Foundation has already mentioned a reduction of 34% in the wiring inside Trinary processors, which brings energy-efficiency benefits. What is not yet considered is how more compact storage of values in Trinary could directly increase the speed of computers.
In the next section, I am briefly going to explain a personal theory about how Trinary might improve computational speeds. The goal of this blog is to explain how the IOTA Foundation is making sure that IOTA is compatible with the future, which includes Trinary, but you can skip the next section as it goes a bit deeper. IOTA will bring you as a developer, enthusiast or business, in contact with Trinary and it is good to realize how this initial hurdle might benefit you in the future.
Trinary Storage Optimization
In Binary, a number is represented with 0’s and 1’s. We currently use 64 units (bits) to encode every number with a really high number as a maximum. Representing an even higher maximum number in Trinary would take only 42 units (trits). This would save us 34% storage space, assuming that a bit and trit would take on the same physical space inside the computer. One of the slowest processes for the processor in a computer is data retrieval from memory or disk. In order to reduce the latency from reading data, computers have different levels of storage that differ in size and distance to the processor. A shorter distance reduces the latency. Any frequently read data will be stored in the register (which can only hold 16 values of 64 bits). Less important data on the Level 1 cache (L1), which is a bit further and therefore slower, but can hold more data (32 KB). Loading data from the L1 cache takes 4 processor cycles, which it would have to wait. Level 2 cache is again further, slow and bigger with 256 KB and 12 processor cycles. The point of all this is that you want to store as much data possible at close distance to the processor. With Trinary, we could make an upgrade in cache storage (Table 1). This will reduce the distance and latency of memory lookups, allowing the processor to spend more time on calculations and generally speed up.
|Location||# 64-Bit values||# 42-Trit values|
Table 1: Comparing Binary and Trinary storage of the cache in processors. Assuming that Bit and Trit storage would have the same physical size.
With snapshotting and Trinary the IOTA Foundation will improve the scaling in regard to storage and PoW computation time/efficiency, however, if IOTA wants to be the platform for all IoT devices, it needs to scale beyond that. Economic Clustering (EC) was proposed in a recent blog post from Come-from-Beyond. He suggests that IOTA can reach true infinite scaling, by only processing transactions from users/machines that are physically near the node in so-called clusters. The idea is that an American node doesn’t really care that someone paid for parking in the Netherlands. If we can create consensus in a more local manner, we can keep the transaction traffic local. The more activity there is in the IOTA network, the more local the clusters might become.
Let’s say we have a cluster per country (which is a very arbitrary, but easy to follow example). My IOTA’s currently exist and are verified in the Dutch cluster. Clusters also process neighboring cluster transactions, so for the Dutch cluster, it also processes the Belgium, English and Norse transactions. If I now get in my car and drive to Germany, my funds would need to be transferred to that cluster through a transaction. Similarly, if I wanted to send IOTA’s from the Netherlands to Switzerland, I would first hop the money through Germany (Which neighbors both the Netherlands and Switzerland). This means that you can still pay someone on the other side of the world through a chain of hops. With Economic Clustering, you can split and merge clusters depending on the throughput of the system. This allows real infinite TPS throughput, while also keeping the storage space and broadband required to a minimum. The idea still has some kinks that need to be worked out before it can be made, but the system is probably not needed until widescale adoption starts happening.
Overall, we see how the IOTA Foundation has created a product that already scales really well. The difficult goal of being able to handle worldwide IoT scale applications sounds impossible, but the IOTA Foundation has a solid plan. With the upcoming updates of automated snapshotting, trinary hardware and economic clustering, the biggest obstructions to handle the Internet of Things are fixed. In short, IOTA is a great platform for any (IoT) application that wants to be built with the future in mind.
If you want to learn more about developing with Blockchain or IOTA:
- Check out part 1 of the IOTA Deepdive series: IOTA Deepdive: Consensus on the Tangle.
- Contact me for consultancy or development for your projects via the information below.