Have you ever been in a situation where you are starving, open the freezer and pull out a box of pizza rolls just to find that your roommate (I am looking at you Dave) put it back EMPTY. How did it make you feel? Frustrated, nay, hangry. Suddenly feeling an irresistible urge to jump out of the nearest window?
Now imagine a system, by which information about the number of pizza rolls is automatically updated every 5 seconds. So that you and everyone else in the household always knows the status of the pizza rolls? Once the pizza rolls start dipping under dangerously low levels, the system automatically re-balances and re-distributes the pizza rolls. And even replenishes them ending your lifelong battle with unreliable roommates …? Not only that, but this system would also communicate about and balance out other aspects of your life. Wouldn’t that be amazing? (And perhaps slightly creepy…)
This is essentially what TAP does in the Timpi Search Engine. Not only does it communicate usage, latency, and distance to all nodes within the Timpi Search Engine. It updates, in real time, the connections between the nodes so that the workload, latency and distance is constantly balanced.
When I embarked on writing this article, I asked Joerg to explain TAP to me as if I was a child. Joerg, who is the CEO and CTO of Timpi has spent a whooping 1.5hrs explaining TAP to me. I must applaud his patience. He answered all my questions calmly (while I stared at him with a mostly blank expression). After blood, sweat and metaphorical tears, Joerg managed to make me understand how TAP works. Now buckle up as I relay the information.
Let’s start at the beginning.
There is a big problem facing our decentralized search engine. Namely, the question of how we make sure that each member of the network knows about each other. And how these different parts can respond autonomously to the information they receive to balance out the network?
In other words, if one of our GeoCore nodes is overloaded, how can we ensure that other nodes step in to take off the load? Without which our search engine would experience regular system failures. Doing this all manually would be impossible — especially as our network grows.
For this reason, Joerg built Timpi’s own proprietary communication protocol called TAP — Timpi Autonomous Protocol (“No biggy”, he says.). By the way, he built this using the Gossip Protocol Principles as its basis. As the name suggests this is a group of principles that work together to disseminate information — like gossip. Having said this, the Gossip Protocol Principles is not a framework to build onto. Rather a group of principles that can be used to build a system from the ground up.
TAP is the communication and system-balancing protocol that Joerg built (have we mentioned from scratch?). It ensures that GeoCores, Collectors and Guardians are running efficiently, smoothly and in balance.
Hold up. GeoCores, Collectors, Guardians? What are those?
Essentially, these are the different nodes that make-up our decentralized search engine. To put simply, Collectors crawl websites (index); Guardians store the data; and the GeoCores oversee these processes. They also ensure that Collector and Guardian nodes are safe from possible attacks by separating them from the search users.
Now back to TAP.
Let us show how TAP works with a real-life example:
Let’s say that there are 5 GeoCores. They are working as a group of neighbors in the same region with a total of 100 Collectors and 100 Guardians. Let’s assume for the sake of simplicity that all GeoCores have the same capacity and specs as each other. All Collectors have the same capacity and specs as each other. And all Guardians have the same capacity and specs as each other. Now, let’s say that this is the arrangement of the Nodes:
GeoCore 1 — 5 Collectors, 5 Guardians
GeoCore 2 — 2 Collectors, 2 Guardians
GeoCore 3 — 20 Collectors, 20 Guardians
GeoCore 4 — 50 Collectors, 50 Guardians
GeoCore 5 — 23 Collectors, 23 Guardians
You can see above that there is a workload imbalance in which GeoCore 4,5 and 3 work with much more Collectors and Guardians than GeoCore 1 and 2.
Enter TAP.
Within 5 seconds, TAP goes through the following processes:
· TAP sends information about all 5 GeoCores to each other
· TAP recognizes the discrepancy & works to workload, location, and latency balance
· TAP requests Collectors and Guardians from overloaded GeoCores to move to GeoCores that are less loaded (& closest in location). In this case GeoCore 4 and 5 would send Collectors and Guardians to GeoCore 1 and 2
· TAP sends information about incoming Collectors and Guardians to GeoCore 1 and 2
And Voila. Within 5 seconds you now have a balanced system that looks like this:
GeoCore 1–20 Collectors, 20 Guardians
GeoCore 2–20 Collectors, 20 Guardians
GeoCore 3–20 Collectors, 20 Guardians
GeoCore 4–20 Collectors, 20 Guardians
GeoCore 5–20 Collectors, 20 Guardians
Remember, this process repeats every 5 seconds, ensuring that the system is kept up to date in real time.
In Joerg’s own words “I think it’s pretty cool” (he is obviously being too modest!).
In addition to the above, what happens if a GeoCore goes down? Once again, the GeoCores balance out and make sure that they have equal amounts of workload.
Further, each Collector and Guardian have back-up GeoCores that they can connect to.
Another cool feature of TAP is the ability of Special Collectors to be temporarily transformed into GeoCores should the system need more GeoCores.
What if, for whatever reason, ALL GeoCores in a region go offline? Timpi will always be running 3 Master GeoCores. This will ensure that even if all GeoCores in a region go down there will be backups for Collectors and Guardians to connect to.
In addition to this, if all GeoCores are busy and overloaded, the GeoCores also have the ability via TAP to tell the Collectors to slow down.
I know, multiple minds blown at this point.
Joerg tells me these are only the base functionalities and when you investigate more into the details, there is much much more to TAP! For the sake of keeping it to one article we won’t go into the exact details of the other functionalities. OK, one more, just because I can’t help myself…
Another functionality of TAP is a consensus process to make sure every Collector redeems the right amount of $TIMPI tokens. AKA, people can’t go in and fraudulently claim more $TIMPI than they are entitled to. For a Collector $TIMPI claim to be approved, it has to be checked by the GeoCore it is working with AND three other GeoCores that are selected at random. This means that it is very hard (one could say almost impossible), for hackers to game the system.
So, to summarize.
What are the benefits of TAP
TAP stops our system from being overloaded. It load balances autonomously. Fixes itself, detects and reacts to bottlenecks.
Removes the need for centralized management. Grows autonomously and can deal with unexpected loads. It sets up Timpi to work autonomously, without the need for Joerg to manage things in the background. It automates its own growth and takes care of itself.
Further, since we are decentralized and because of the dynamic nature of our system due to TAP we can’t be DDOSed easily. (Distributed Denial of Services — when a group of malicious computers try to bring the webserver down.) This makes us very secure. There is no central part of the network that can be hit. Every region can work on its own (a region can be a city, country, town, etc.) and can balance and manage itself. We are excited to share this new technology with you as we move towards releasing our closed beta search engine in Q1 of next year.
