Collector UI is the first piece of technology that Timpi makes open source

If you’ve been following us for a while, or have read our WhitePaper you will know that the Collector, Guardian and GeoCore nodes work together to create Timpi’s search engine. Last week we published an article explaining how these different parts are managed by TAP (Timpi Autonomous Protocol).

This week, we are excited to announce that we are making part of Timpi’s technology open source!

GeoCore, Collector and Guardian node contributors in Timpi’s search engine

Let’s back track for a moment and return to Collectors, Guardians and GeoCores.

As the world’s first decentralized search engine, it is paramount that the various parts of our search engine are completely decentralized. It is due to this that we want our community to run all three types of nodes.

So far, in the past months, we have released the Collector and Guardian nodes for our community to beta test in preparation for our private beta launch in Q1 of next year.

About the Collector

Until recently the Collector was one application; a combination of our proprietary technology and the front-end UI (the dashboard if you will). After some testing, we decided to divide it into two parts: our proprietary technology as one part and the front-end UI as the other.

Timpi’s Collector UI becomes open source.

Why did we decide to divide the Collector into 2 parts?

We decided to separate the Collector into two parts for simple reasons. Firstly, because of the system usage and performance. Secondly Since we have separated the Collector into two parts, we have seen considerable improvements in all cases.

Collector nodes before vs. now

Both our proprietary technology as well as the Collector UI use memory. This is an opportunity to make our Collector more RAM efficient. By separating the Collector into two parts, we can now switch off the UI part of the Collector. This frees up memory and also improving stability. There is no reason for the UI to always run while the Collector is running. With this the UI can be stopped. This means that our community is now able to run more workers on their Collector nodes.

An application or service?

Previously we the Collector was run by an application. Now, it is only the Collector UI that is run by an application (Timpi Manager). The Collector proprietary technology is now run as a service. This means that you can run a virtual machine and the Collector will run automatically. In other words, you no longer have to start and stop the Collector — it can run at all times in the background. As for the UI, you can now switch it off completely without it affecting the Collector workers. After all, the UI is only used to check statistics and to adjust the configurations of the Collector.

Moving to RUST: A change in programming language

With the move to separate the Collector into two parts we also decided to build the Collector UI on a different programming language. The UI, as part of the Collector application, was previously built with C-Sharp, but now the UI will move to RUST.

The reason being, that C-Sharp has over the top functionality that we don’t need for the Collector UI. It is the wrong tool for the job. It made more sense to use RUST to make the Collector UI slimmer and more efficient.

Why make the Collector UI open source?

If you are in our Telegram or Discord group, you will know our superstar community manager SpellignError. It was his brilliant suggestion to make the UI part of our Collector open source. After some consideration we decided to green light the idea.

The reason we will make the Collector UI open source is to give our node operators an opportunity to extend UI functionality. There are many ways that the functionality can be extended. A few examples are (by no means exhaustive):

  • Customization of the look
  • Open source home automation (e.g.: Home Assistant)
  • Extension of functionality so that other platforms can run it (e.g.: raspberry pi or other low power systems)
  • Creation of remote controls

Who can extend the functionality of the Collector UI?

Anyone with the technical know-how can extend the Collector UI’s functionality. To do so you will need to understand RUST and have general programming and html knowledge.

We encourage our community take a dive into extending the functionalities. We welcome forks and would love to see those made available via GitHub. We are a community-oriented organization, and we’d love to see our members make use of other UI versions developed by the community.

When will the Collector UI be available on GitHub?

According to our estimations the GitHub Repository for the beta version of the new UI will be available in 4 weeks. Keep your eyes peeled for when that drops.


Timpi Autonomous Protocol: A revolutionary technology

Have you ever been in a situation where you are starving, open the freezer and pull out a box of pizza rolls just to find that your roommate (I am looking at you Dave) put it back EMPTY. How did it make you feel? Frustrated, nay, hangry. Suddenly feeling an irresistible urge to jump out of the nearest window?

Now imagine a system, by which information about the number of pizza rolls is automatically updated every 5 seconds. So that you and everyone else in the household always knows the status of the pizza rolls? Once the pizza rolls start dipping under dangerously low levels, the system automatically re-balances and re-distributes the pizza rolls. And even replenishes them ending your lifelong battle with unreliable roommates …? Not only that, but this system would also communicate about and balance out other aspects of your life. Wouldn’t that be amazing? (And perhaps slightly creepy…)

This is essentially what TAP does in the Timpi Search Engine. Not only does it communicate usage, latency, and distance to all nodes within the Timpi Search Engine. It updates, in real time, the connections between the nodes so that the workload, latency and distance is constantly balanced.

When I embarked on writing this article, I asked Joerg to explain TAP to me as if I was a child. Joerg, who is the CEO and CTO of Timpi has spent a whooping 1.5hrs explaining TAP to me. I must applaud his patience. He answered all my questions calmly (while I stared at him with a mostly blank expression). After blood, sweat and metaphorical tears, Joerg managed to make me understand how TAP works. Now buckle up as I relay the information.

Let’s start at the beginning.

There is a big problem facing our decentralized search engine. Namely, the question of how we make sure that each member of the network knows about each other. And how these different parts can respond autonomously to the information they receive to balance out the network?

In other words, if one of our GeoCore nodes is overloaded, how can we ensure that other nodes step in to take off the load? Without which our search engine would experience regular system failures. Doing this all manually would be impossible — especially as our network grows.

For this reason, Joerg built Timpi’s own proprietary communication protocol called TAP — Timpi Autonomous Protocol (“No biggy”, he says.). By the way, he built this using the Gossip Protocol Principles as its basis. As the name suggests this is a group of principles that work together to disseminate information — like gossip. Having said this, the Gossip Protocol Principles is not a framework to build onto. Rather a group of principles that can be used to build a system from the ground up.

TAP is the communication and system-balancing protocol that Joerg built (have we mentioned from scratch?). It ensures that GeoCores, Collectors and Guardians are running efficiently, smoothly and in balance.

Hold up. GeoCores, Collectors, Guardians? What are those?

Collector, Guardian and GeoCore nodes run the Timpi search engine.

Essentially, these are the different nodes that make-up our decentralized search engine. To put simply, Collectors crawl websites (index); Guardians store the data; and the GeoCores oversee these processes. They also ensure that Collector and Guardian nodes are safe from possible attacks by separating them from the search users.

Now back to TAP.

Let us show how TAP works with a real-life example:

Let’s say that there are 5 GeoCores. They are working as a group of neighbors in the same region with a total of 100 Collectors and 100 Guardians. Let’s assume for the sake of simplicity that all GeoCores have the same capacity and specs as each other. All Collectors have the same capacity and specs as each other. And all Guardians have the same capacity and specs as each other. Now, let’s say that this is the arrangement of the Nodes:

GeoCore 1 — 5 Collectors, 5 Guardians

GeoCore 2 — 2 Collectors, 2 Guardians

GeoCore 3 — 20 Collectors, 20 Guardians

GeoCore 4 — 50 Collectors, 50 Guardians

GeoCore 5 — 23 Collectors, 23 Guardians

You can see above that there is a workload imbalance in which GeoCore 4,5 and 3 work with much more Collectors and Guardians than GeoCore 1 and 2.

Enter TAP.

Within 5 seconds, TAP goes through the following processes:

· TAP sends information about all 5 GeoCores to each other

· TAP recognizes the discrepancy & works to workload, location, and latency balance

· TAP requests Collectors and Guardians from overloaded GeoCores to move to GeoCores that are less loaded (& closest in location). In this case GeoCore 4 and 5 would send Collectors and Guardians to GeoCore 1 and 2

· TAP sends information about incoming Collectors and Guardians to GeoCore 1 and 2

And Voila. Within 5 seconds you now have a balanced system that looks like this:

GeoCore 1–20 Collectors, 20 Guardians

GeoCore 2–20 Collectors, 20 Guardians

GeoCore 3–20 Collectors, 20 Guardians

GeoCore 4–20 Collectors, 20 Guardians

GeoCore 5–20 Collectors, 20 Guardians

Remember, this process repeats every 5 seconds, ensuring that the system is kept up to date in real time.

In Joerg’s own words “I think it’s pretty cool” (he is obviously being too modest!).

In addition to the above, what happens if a GeoCore goes down? Once again, the GeoCores balance out and make sure that they have equal amounts of workload.

Further, each Collector and Guardian have back-up GeoCores that they can connect to.

Another cool feature of TAP is the ability of Special Collectors to be temporarily transformed into GeoCores should the system need more GeoCores.

What if, for whatever reason, ALL GeoCores in a region go offline? Timpi will always be running 3 Master GeoCores. This will ensure that even if all GeoCores in a region go down there will be backups for Collectors and Guardians to connect to.

In addition to this, if all GeoCores are busy and overloaded, the GeoCores also have the ability via TAP to tell the Collectors to slow down.

I know, multiple minds blown at this point.

Joerg tells me these are only the base functionalities and when you investigate more into the details, there is much much more to TAP! For the sake of keeping it to one article we won’t go into the exact details of the other functionalities. OK, one more, just because I can’t help myself…

Another functionality of TAP is a consensus process to make sure every Collector redeems the right amount of $TIMPI tokens. AKA, people can’t go in and fraudulently claim more $TIMPI than they are entitled to. For a Collector $TIMPI claim to be approved, it has to be checked by the GeoCore it is working with AND three other GeoCores that are selected at random. This means that it is very hard (one could say almost impossible), for hackers to game the system.

So, to summarize.

What are the benefits of TAP

TAP stops our system from being overloaded. It load balances autonomously. Fixes itself, detects and reacts to bottlenecks.

Removes the need for centralized management. Grows autonomously and can deal with unexpected loads. It sets up Timpi to work autonomously, without the need for Joerg to manage things in the background. It automates its own growth and takes care of itself.

Further, since we are decentralized and because of the dynamic nature of our system due to TAP we can’t be DDOSed easily. (Distributed Denial of Services — when a group of malicious computers try to bring the webserver down.) This makes us very secure. There is no central part of the network that can be hit. Every region can work on its own (a region can be a city, country, town, etc.) and can balance and manage itself. We are excited to share this new technology with you as we move towards releasing our closed beta search engine in Q1 of next year.