The DSN Traveller travels the Matrix network under the name of
and writes a travel report of what it saw. 📝
In case you have encountered Travis Ralston's Matrix Traveler Bot, yes, they're quite similar.
The bot makes a measurement of “how big” matrix.org and other matrix servers are, and how centralized or disributed the matrix network currently is. To do that, it needs to be part of many rooms, as room membership can't be queried without being in the room.
So first, the
DSN Traveller (bot) will try to join all public rooms it can find.
This phase will probably finish around the end of May 2018.
The bot will then query which servers are in the rooms, and how many users are there from which servers. While that is still in progress, the bot will stay in the rooms until I'm sure everything has gone right. This is done to avoid needing to rejoin all rooms in case something went wrong, cluttering the room histories and stressing the federation .
Because it anonymizes the collected data, the bot can't do partial scanning of the network by joining and leaving batches of rooms. The resulting data could not be merged together, and therefore the bot needs to be in all rooms at once.
After data gathering is finished, the bot will leave all rooms again. If all goes well, this should happen by no later than the middle of June 2018.
Why does the bot stay around?
For not having to to rejoin on every adjustment I do on the data collection part of the bot. After the bot development is finished, it could join all rooms, then collect data, then leave all rooms without idling in between.
How to make the bot join a room
If you want the bot to visit you, you can invite the bot into your room. As the DSN Traveller does not travel continuously, though, it will not join the room immediately, but on its next trip. 💼
How to make the bot leave a room
If you don't want to have the Bot in your room, please pardon the inconvenience, and just throw it out! 😓
You can make it leave immediately by:
- Kicking it (someone could still invite it back) 📤
- Banning it (if you'd like it to stay gone forever) ❌
As I don't require exact data at all, and an approximation to the Matrix network is sufficient for what I want to do, this won't affect my research significantly.
Why? Who are you?
I want to find out whether the Matrix network scales, i.e. whether everything still works when the network is ten times bigger, or ten times more decentralized. I'm Florian Jacob, an informatics student at the Karlsruhe Institute for Technology, and this research question is the core of my Master's thesis at the Decentralized Systems and Network Services Research Group. 📑
How does this work?
The DSN Traveller tries to get a rough overview of how the Matrix network is structured today. It records how many rooms it finds, how many users and servers take part in those rooms, and how they relate to each other, meaning how many users a server has and of how many rooms it is part of.
All room, server and user IDs get pseudonymized immediately after receiving them using a randomly-chosen hash function on each trip. After the Traveller has finished a trip, and before storing it on disk, the resulting room-server-user network is anonymized by hashing each ID together with an individual random value (Salt) which is immediately thrown away. 🧂
This room-server-user network observed by the DSN Traveller will be fed into my simulation of the Matrix network as an anchor point to see what happens when I change the network's size and decentrality, to answer the main question of my thesis.
Any questions? 🤔 Feel free to ask me (
in #dsn-traveller:dsn-traveller.dsn.scc.kit.edu! 💡
Note: In case you have problems joining that room, I also created the alias
as a workaround until the cause is found. Please try that and report any problems. If that does not work
either, you can also find me in #Help:matrix.org as
@florianjacob:matrix.org so we can debug this.
The source code is available under AGPLv3+.
The bot was written in Rust using ruma-client from the Ruma project.
As of 2018-08-31, my thesis is handed in! I analyzed the Matrix Event Graph as a replicated data structure, examined the structure of the current Matrix network with the DSN Traveller crawler bot, and simulated the network with a simplified behavioural model.
In summary, I could show that Matrix has few large but many small servers. Large servers reduce the overall network load, but a significant fraction of the load is concentrated in them. Introducing more small servers would further increase the load concentration. The Matrix event graph as a Conflict-Free Replicated Data Type showed to be well-suited for reliable messaging and history synchronization, and is applicable beyond Matrix.
I'm now working on a scientific paper on the results, which will boil down the more than 80 pages of the thesis to something much more digestible. 🔬 I won't be able to keep this website here running forever, but you'll hear through #TWIM:matrix.org when and where to find the publication and the results.
- Travis Ralston's Matrix Traveler Bot was used for getting the initial room list for the DSN Traveller.
synapse-workaroundbranches on his ruma forks used for fest were a great help to get ruma-client and synapse to talk to each other.
- The logo is the U+1f4bc 💼 “briefcase” emoji from Noto Color Emoji.
- This site is using the Solarized color scheme by Ethan Schoonover.