How Tagion works: step-by-step
Written by Decard - the initial stewardship company of the Tagion platform.
All technical terms will here be written in italics. Some of them are explained at the beginning, some first appear "in inverted commas" in the text and are explained there.
If not mentioned specifically, links in the text go to the technical documentation pages of the Tagion project. There, full technical details and codes are provided but - in contrast to this introductory text - a high level of technical understanding is assumed.
There is also a complementary Glossary for terms which are here not touched upon.
Up first: a bit of terminology
User: anybody making use of Tagion software, without being a node operator
Node: instance of main Tagion software (for Tagion Mainnet and Subsystems). Several nodes can run on one computer, but in a "distributed" system they will be on different computers connected via the internet
Subsystem: a bespoke network of nodes for a particular use-case, independent of the Tagion Mainnet in terms of rules and settings. It hosts its own database and sends regular hashed/encrypted status updates to the Mainnet (for validation and notary functions, further explained below). Because of this, Subsystems are here called "federated" with the Tagion Mainnet.
Database: structured system to store, manage and retrieve data. Akin rather to a filing cabinet with labeled drawers - not the records in those drawers.
"DART" is the name of Tagion's custom-designed database system (further explained below) maintained in/by the Tagion node software
App: user-interface to make changes to the database, it sends info about intended changes to a node
Transaction: unit of database-changes (add, edit, delete), including but not limited to currency related transactions
... that much said, we now have enough words to begin ...
Starting from a general use-case
A user (see above) wants to enter some new piece(s) of information to the system. Let's assume that this information is related to a particular use-case for which a federate Subsystem (see above) has been set up.
The user submits the information via an app (see above), e.g. on their mobile phone. This information can be new (it will be added to the database - see above), or it can be a change to (or deletion of) existing data. Along with the information itself, the user submits their electronic signature which will be linked to the data as a way to manage access rights and authentication.
The app they use for this is built upon - and sometimes still called - a "wallet application", a term inherited from the Tagion Mainnet with its principle use-cases centered around financial transactions (see above. "JustPay" is such a wallet application - more information about it can be found on the Decard website). The application packages the information to be entered into the database and sends it to a node (see above, or here for all details) of the Subsystem network.
Next to the basic data-input and additional required information (timestamp, user identity, references to other data, required permissions) there can also be "payload"-information which might contain conditional instructions that are far more complex than simply "add this information to the database". The whole information package is here called a "contract" - a term inherited from "blockchain" and "DLT" concepts, where the compound term "smart contracts" is used to imply "self-execution" when the payload contains (complex) conditional instructions.
To enable swift transmission of contracts to nodes under most bandwidth conditions, they are structured/written according to our custom "HiRPC" communication format, which in turn is based on our general data format called "HiBON". We felt compelled to design those new formats to achieve the highest level of security and flexibility for control of data at the record level. At the same time, these formats allow for the utmost efficiency in terms of storage and computing requirements for the network.
Distribution: from one node to many
The receiving node checks the information sent in the contract against the information already existing in its locally stored database (e.g. if user identities and permissions are valid and other data is referenced correctly). Results are preliminarily stored in the memory of the node's "virtual machine" - the computation and execution software interpreting the operations specified in a contract. The Tagion Virtual Machine (TVM) is built on the popular open WebAssembly standard.
However, for the moment none of the results of executing a contract is committed to the Tagion database. Because in a distributed system without central control, all nodes need to make sure they maintain their individual copies of the database in accurate synchronisation with the whole network and don't make changes without coordination. Thus, the Tagion nodes will first notify other nodes in the network about the receipt of any new contract and their contents.
This continuous communication between nodes does not only concern contracts they received from users (what is here called "gossip"), but also the contracts submitted to other nodes which they consequently heard about. This propagation of information between nodes is here called "gossip about gossip". And in this way, all nodes will eventually know of all recent submissions to the whole network - at least up to a certain point in the (not-so-distant) past.
For everything before that point, a network-wide agreement about everything to be changed in the database will have been reached, thanks to the gossiping between nodes. In distributed systems, such a process of reaching agreement is called "consensus mechanism", and the time-span from one moment of consensus to the next is called an "epoch".
In Tagion, only for the contracts submitted within a concluded epoch will nodes commit the new information to their database. Later submissions, which at this moment have not been shared widely across the whole network, will be dealt with in the next epoch. This way the information in the network is synchronised up to the recent moment of consensus, and the resulting shared, ornised version of all individual database(s) is called their new "state" (as in "state of play" or "the current situation").
Consensus: how nodes know what other know
How exactly consensus on the state of the network is achieved is different from one distributed network to the next. So called "blockchains" offered the first popular solution for consensus without a central authority. Later, when the practical limitations of blockchains became obvious and ever new solutions were sought, the collective term "DLTs" was introduced - which stands for "distributed ledger technologies" . The "ledger" part of that term was again inherited from the early use-cases of the technologies in currency transactions and accounting. Tagion, which is not a blockchain but a network of distributed databases, has utility far beyond ledger based accounting. But to understand the concept and implementation of its consensus mechanism it remains useful to see it in context of this heritage.
Common to all consensus mechanisms is that nodes constantly communicate with each other about how their local versions of their shared database have evolved or should evolve. One example of that is the gossip about gossip described above. The scenario in which some nodes fail to do so (be it deliberate/maliciously, or because of lost connectivity), is described as the so-called "Byzantine Generals Problem". Consequently, DLTs that solve that problem and can deal with a certain number of divergent nodes are called "byzantine fault tolerant".
For Decard's platform two innovative "protocols" (meaning sets of rules about the communication between different software entities) enable a fast, secure and modular consensus. It is highly byzantine fault tolerant, as it still functions with up to 1/3 of all nodes being out of sync or not playing by the rules. Additionally, it is also most efficient in terms of required storage capacity, internet bandwidth and energy consumption.
The first of those two protocols, developed and patented by Decard, is called the "Wavefront" protocol and it manages the details of the gossip-about-gossip communication. It swiftly conveys how many new contracts each of the two corresponding nodes have received (directly or indirectly) since the last epoch, and then enables them to efficiently ronise their local knowledge.
The second protocol then establishes consensus not by comparing the nodes' stored databases themselves (or, in the case of many blockchains, by looking for the largest number of blocks in a chain) but by tracking the level of shared knowledge in across the network, as communicated in the wavefront protocol. This consensus protocol used by Tagion was originally developed by Leemon Baird (co-founder of the "Hedera" project), under the now well established term "Hashgraph consensus algorithm".
Together, the two protocols establish, swiftly and securely, which changes are to be introduced to the database with the conclusion of the next epoch. Because the hashgraph provides a network-wide communication history from which each node can reconstruct by and for itself, and with mathematical certainty, what other nodes in the network know. And once information about new contracts/transactions has spread to enough nodes in the network (namely to 2/3+1 of all nodes) - and consequently consensus about those contracts is algorithmically established, all nodes will commit the same information/changes to their respective copies of the database.
DART: smarter than blockchains
Everything up to that recent epoch is thus fixed in the distributed database and no competing versions can occur (or what is called "forks" in blockchain systems). This certainty about the state of the network is what is called "finality" in DLTs. But while traditionally blockchains rely on "probabilistic finality" (because there is always a slight chance that a fork can gain consensus), Tagion's consensus database features an absolute certainty called "deterministic finality". For this, a unique and final step is introduced, made possible by Tagion's patented database architecture:
Once the changes agreed on in the hashgraph consensus mechanism have been submitted to the database, each node calculates a signet of the complete new version of their database and submits it to the whole network for comparison. These signets are mathematical fingerprints (called "hashes") of the entire database. For Decard's database called "DART" (which stands for "Distributed Archive of Random Transactions"), this fingerprint is called the "bullseye" (a play on the metaphorical DART-board used to depict our database structure, see last link).
Thanks to the way data is stored and indexed in the DART, the calculation and sharing of a bullseye requires only minimal computing resources. This way, the network can quickly identify nodes that have fallen out of sync (because their bullseye would be different). And because the bullseyes of each epch are stored and linked in what is here called an "epoch chain", finality of the DART database is absolute and the history of past communications/gossip, also containing the dealt with contracts, does not have to be retained.
In many blockchain systems, where both the consensus mechanism and data-storage happen "on the blocks", finality entails that information (included in earlier blocks) can never be altered or deleted. This is different with Tagion because the hashgraph consensus relates to the submitted/prospective changes to the database without determining what those changes can or cannot do/be. This makes Tagion a modular system where a) the storage of data in a database and b) the consensus about its evolving state are two separate functions. In this way, data in the DART can always be changed or deleted (given the correct permissions, of course), and finality refers to reliability of each new state of the database without limiting data management and control.
Another effect of the way Tagion achieves consensus is that the exact order of changes to the database is determined by the time at which most nodes have received knowledge about contracts affecting those change. We call this "fair ordering", as it cannot be manipulated by individual node operators or distorted by the amount of fees offered by users for the priority-execution of their transactions (as in many popular blockchain systems).
Tagion: best of two worlds
Another unique design feature related to the DART data architecture is that all entries and edits have an "owner", typically the user as specified in the contract which created/modified the data entry. Access rights to each data point can thus be managed like the file permissions in a conventional computer system (e.g. read, write, own). In this regard, rules concerning privacy, anonymity and security can be determined individually for each (type of) entry, or for each Subsystem database, depending on its particular use-case.
Only for contracts/transactions of "TGN", the native utility token on the Tagion Mainnet, the rules are fixed: the amounts of the transactions are revealed, along with the identifiers of the payer and payee. But their actual identities are protected because, different from most blockchains, only the resulting balances of a transaction/contract are stored in the new state of the database, not the contracts with the actors' information themselves.
In a Subsystem, the actors can be made visible (only to the participants of that Subsystem or more widely), while the content kept in the Subsystem's own database can be protected as appropriate for the use-case or industry. The level of data-protection available can be configured to be compliant with any privacy and data-protection regulation or even up to government agency levels of classification - including but not limited to access-control, authentication and sharing services, and multi-signature requirements.
Each Subsystem thus benefits from having its own distributed database, shared and synchronised across its own network of nodes. The set of participants of that "sub-network" can be freely defined (e.g in a "permissioned" network) while also freely determining its own rules and governance as needed. The only mandatory information shared with the Tagion Mainnet is the bullseye-signet calculated for each Subsystem's epoch. As these Subsystem bullseyes get recorded on the Tagion Mainnet they provide a continuous quasi-public audit trail of the Subsystem's accurate operation. This is why the Mainnet is said to fulfill the function of a "public notary" to its federated Subsystems.
But public here goes beyond any nation state's public institutions and power. The governance rules and protocols being developed for the Tagion Mainnet make it "decentralised", meaning no entity owns or controls the network and anybody can participate. Tagion achieves decentralisation by its "proof of community" requirements that determine who can become a node operator on the Mainnet, and by the randomised rotation of validator nodes that participated in the hashgraph consensus of a given epoch. Consequently, the trust and reliability provided by the Tagion platform to multi-stakeholder environments extends beyond the classical boundaries between public and private, or nation states and industries.
Such a move away from central authority to an infrastructure that is run and governed by its users and stakeholders had been one of the impulses for the development of blockchain technologies. However, the particular way in which DLTs manage data and achieve consensus has been riddled with inefficiencies from the beginning. No DLT solution has achieved scalability without compromising on security or decentralisation.
This is why Tagion is such a novelty in combining the best features of central databases and blockchain/DLT ambitions: Built from the ground up with security and efficiency as its foundational design principles, it achieves the speed, reliability and energy efficiencies of traditional server based databases, including a full suite of data control and privacy options- while staying true to the promises of DLTs and providing the cooperative data infrastructure needed for multi-stakeholder industries and their individual use-cases.
If you are now inspired to deploy Tagion for your business, project or community but do not have the expertise to set-up a node and Subsystem for yourself, contact Decard: the stewardship entity to drive the development and adoption of Tagion.