What is VerliLinker

VerliLinker is a Lua script written for Verlihub, that allows different hubs to be linked into one consistent network. The work is done using LuaSockets library in non-blocking mode. That means this script will NOT block Verlihub while executing network operations, like listen(), accept(), connect(), etc.

Consider the following image:


Each hub will display all the users on the network in its nicklist and users that are logged on the network via another hub will be shown with a prefix of that hub, before their nick (ie. [A]nick).

The idea

The idea behind this is very simple. Each hub can connect to any number of other hubs, via hublink connections, so it is even possible to join 2 existing networks, linked with VerliLinker, into a bigger network. The only thing that can cause a trouble is a loop in a network (a loop is any closed circuit between two hubs in the network). VerliLinker has got an anti-loop protection, so all admins will be aware of the loop, but you should try to always avoid it by planning your network more carefully.

The implementation

Each hub is listening on specific ports in order to establish links with another hubs. Each listening port is used for 1 remote hub only, for security reasons. For each new hub you want to link with your hub, you need to open a new listening port, with !links command. So, if you want to link your hubs with 3 more hubs, you need to create 3 links, using 3 different listen_ports, with command !links add.

listen_port is used as a security measure. Each link has a different listen_port assigned. Why? Whenever a remote computer tries to connect to that listen_port, hub only needs to check if the remote ip address is the one that is allowed to connect to that port. dns lookup for the remote "host" field of that link, should return an IP address, which can be compared to the connecting ip address. That way the remote hub doesn't need to have a static ip address. This way we are certain that there will be no brute force attacks in order to get unauthorized access into the network.

To connect to another hub, in order to establish a hublink, you can use !links connect. If both you and another hub's admin try to connect to each other's hubs (to establish that hublink), one of those connections will be dropped and another will succeed (no point of having 2 tcp connections for 1 thing).

You can type !links, after you load the lua script, for more info on the syntax.

Authentication

After a successful tcp connection between 2 hubs, a simple authentication protocol is used, for example:

For the sake of simplicity, the hub that accepted the tcp connection (a server-side hub) will be named "Server" and another hub, that initiated tcp connection, will be named "Client", S and C respectively.

So, the Server will first send the $AuthRequest command, followed by a random string parameter (parameter cannot contain spaces). Client replies to this request with $AuthReply followed by an md5 hash. Hash is created using random string from $AuthRequest command and the password, associated with this hub link (password can be set when adding new link with !links add):

This is used to avoid sending plain passwords over insecure internet.

If authentication fails, Server closes connection with simple error message, otherwise, $AuthOK is sent as a reply and now hubs continue to negotiate some options that will be used in the linking process.

Negotiating hublink options

In order to avoid misconfigurations in the network, before any successful link establishment, there are some things that need to be checked for validity first. One of those things is a hub prefix on the network. Obviously, 2 hubs cannot have the same prefix, so, each of those need to check if their prefixes are unique on the network.

You can set a prefix for your hub by editing the script file "links.lua", changing the value of "links_my_prefix" constant. There are some limitations on types of characters used in a prefix. Generally, avoid using: $ (dollar), <space>, <tab>, | (vertical line), & (ampersand). Also, keep in mind that visitors of your hub could be from around the world, so, it would be the best to use only 7-bit printable ASCII characters.

Here is a simple example of the options negotiation:

This shows that Client hub is responsible for all the users prefixed with prefixes: [Alfa], [Gama], [G] and [T] and Server is responsible for prefix "[Beta]". If any of the hubs detects that any of those prefixes of remote hub is already in use, by some other hub on the network, it will close the connection with a simple error msg, otherwise, a $LoginOk is sent and a negotiation process is complete. Hubs are now successfully connected into a network.

Note that Client hub wants to register several prefixes to the Server, that means the Client has already some hubs connected in his network and is now trying to connect to a Server hub to join that hub in the existing network.

Notice: So far $MyPrefixes is the only thing that is being negotiated during this process. In future versions, more things can be checked for validity, so, it's best to rely on $LoginOk message before trying to send anything through that link.

It's alive

If properly authenticated and negotiated, hubs can now broadcast to their established hublink connections whatever they want, main chat messages, pm, search, myinfo and other messages. Most of things that flow over hublinks are regular NMDC messages, but some control messages can be spoted too.

For a start, hubs send their nick list to each other. Also, hubs will inform all the other linked hubs of new hubs/users in the network. If a hub link is broken, hubs don't need to send mass $Quit to each others, instead they only send $PrefixDel message, for each broken hub, to save bandwidth. All the other hubs, that receive $PrefixDel, will remove nicks prefixed with that prefix.

Here is a brief list of some messages that can be seen over a hublink connection:



Basically, most of NMDC messages are broadcasted through the hub links unmodified. Main chat messages are broadcasted in a form suitable for faster checks of nicks/prefixes. There are also some new messages introduced, like "$Kicked", which is sent in a response when an operator kicks a user, so that all other hubs are informed that a particular user has been kicked and why (cross-hub kicks/bans are possible). For the complete list of hub-to-hub messages, using VerliLinker protocol (VLP), take a look here.

Usage

The script introduces a new command: !links
Usage:

Examples:


Flags

Since version 1.4, flags are implemented, to allow hub admins to configure what gets broadcasted from/to their hubs. The usage is fairly simple:

The list of flags follows (lower case flags are used to allow/forbid messages FROM our hub to be broadcasted to other hubs, and uppercase flags are used to allow/forbid messages from other hubs to get INTO our hub):

For example:

this command will set following options for link named "alpha":
- don't allow MainChat messages from my hub to be broadcasted to other hubs (missing lowercase 'm')
- allow MainChat messages from other hubs INTO my hub (uppercase 'M')
- allow Search messages from/to my hub (both 's' and 'S')
- allow ConnectToMe messages from/to my hub (both 'c' and 'C')
- don't broadcast MyINFO messages from/to my hub (both 'i' and 'I' missing)
- allow my OPs to kick remote users (lowercase 'k')
- don't allow remote OPs to kick my local users (missing uppercase 'K')
- don't show my/remote OPs in remote/my hub (both 'o' and 'O' missing)

Example

Let's assume we have 5 hubs: hub1, hub2, hub3, hub4 and hub5. We would like to connect them like on the following diagram:


Now, let's assume that addresses of those hubs are:
-for hub1: dchub://hub1.domain1.org:111
-for hub2: dchub://hub2.domain2.org:222
-for hub3: dchub://hub3.domain3.org:333
-for hub4: dchub://hub4.domain4.org:444
-for hub5: dchub://hub5.domain5.org:555

Also, let's assume that passwords, used for hublinks, are:
- for link hub1-hub2: "pass12"
- for link hub2-hub3: "pass23"
- for link hub3-hub4: "pass34"
- for link hub3-hub5: "pass35"

Further more, let's assume that prefixes for each hub are:
-for hub1: [Alpha]
-for hub2: [Beta]
-for hub3: [Gamma]
-for hub4: [Delta]
-for hub5: [Epsilon]

And finally, let's assume that each hub has listen ports for each of their's links:
- hub1's local listen port for link hub1-hub2: "201"
- hub2's local listen port for link hub1-hub2: "301"

- hub2's local listen port for link hub2-hub3: "401"
- hub3's local listen port for link hub2-hub3: "501"

- hub3's local listen port for link hub3-hub4: "601"
- hub4's local listen port for link hub3-hub4: "701"

- hub3's local listen port for link hub3-hub5: "801"
- hub5's local listen port for link hub3-hub5: "901"

Now, let's see how the configuration of each hub looks like.

In hub1, an admin would have to type:

Note that if admin doesn't provide a listen_port parameter, hub will not listen for incoming connections, for that link. This is useful if you want to have some kind of "master" hub, that is the only one that can accept connections from other hubs.

In hub2, an admin would have to type:


In hub3, an admin would have to type:


In hub4, an admin would have to type:


In hub5, an admin would have to type:


One more thing. Each admin will need to set the prefix of his hub on the network. They will each open their own script file "links.lua" and edit it, to change the value of "links_my_prefix" constant to what they want to be their prefix ([Alpha], [Beta], [Gamma], [Delta], [Epsilon]).

That's it. After successful configuration, they can connect their links to build a network.

Admin in hub1:

Admin2 doesn't need to do: !links connect hub1, because the admin from hub1 already established that link.

Admin in hub3:


After successful connections, the network is up and running

Final word

I hope you like this project that makes it able to create better p2p networks, which can act as a single body, like IRC networks. You can send your bug reports, suggestions, etc. to help improve the VerliLinker.

NOTE:
The oldest version of Verlihub which is tested with this script is Verlihub v0.9.8d-RC2 and VH plugin for Lua v1.7rc1

Finally, this version works 100%

That means, all users, who have a valid hub prefix, can search across hubs in the network. Those, who choose not to put the prefix in their nick, can also use the hub, without network support, so they can only search/download in the current hub. For those who choose to put the prefix of their hub (to which they connect) in their nick, they can enjoy the full power of the dc hubs network

Steps to make things work:


Install Lua

Go to: http://www.lua.org/download.html and download "lua-5.1.4.tar.gz"
Read: INSTALL file

most likely you'll end up typing this:
# make (or make linux)
# sudo make install
# ldconfig

test (if successfully installed):
# lua -v
Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio


Install LuaSocket

Go to: http://luaforge.net/projects/luasocket/ and download "luasocket-2.0.2"
Read: Install instructions

most likely you'll end up typing this:
# make
# sudo make install
# ldconfig

test (if successfully installed):
# cd test
# lua testsrvr.lua
server: waiting for client connection...
(CTRL+C)

also, type this:
sudo ln -s /usr/local/lib/lua/5.1/socket/core.so /usr/local/lib/libluasocket.so
sudo ln -s /usr/local/lib/lua/5.1/mime/core.so /usr/local/lib/libluamime.so

to make sure everything is back compatible

Install Verlihub

If you are installing older versions of Verlihub, you might end up using a very new gcc compiler which will complaint about some things with older versions of Verlihub, and if so, take a look here: viewtopic.php?f=3&t=4272&p=8475

Go to: http://www.verlihub-project.org/doku.php?id=download or http://downloads.sourceforge.net/project/verlihub/ and download the version of Verlihub you want.
Read: Install instruction

most likely you'll end up typing this:
# ./configure
and if successful, you'll see this:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Configure has completed successfully. Now you must run make.
If make fails with errors, please consult the forums and see
if anyone else has had the same issue. The forums are located
here http://forums.verlihub-project.org
Extensive documentation is also available at the project website
which can be found here http://www.verlihub-project.org
Please feel free to join and register at the Verlihub suport
hub located here dchub://hub.verlihub-project.org:7777
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


then type:
# make
# sudo make install
# ldconfig

now, reading the INSTALL file, we can see (at the end):
* run the mysql server
* run vh_install and let yourself guide
* run vh_regnick -f -c 10 -n "yournick"
* run vh_runhub 1234 # where 1234 is the listening port of your choice
* connect to the hub with "yournick" use command +passwd yourpassword and reconnect
* use hub command != listen_port 1234
* now on you can use vh_runhub without an argument..


Install Lua plugin

# ./configure --with-luasocket
# make
# sudo make install
# sudo ln -s /usr/local/lib/liblua_pi.so /etc/verlihub/plugins/liblua_pi.so
# ldconfig

and then in the hub:
!plugin /etc/verlihub/plugins/liblua_pi.so

Load the script

- download this package (3 files: links.lua, links_nmdc.lua.lib and utils.lua.lib) and put it into the Verlihub's script folder
- edit links.lua file and configure your hub prefix (the prefix of your hub in the future network)
- load links.lua with !luaload
- type !links and follow the help in that command and from the example here in the previous post to correctly execute the "!links add" command
- after adding desired links, type: "!links connect <linkname>" for each link in the links list, to connect to those hubs (all hubs need to have this script loaded in order to connect to each others)

Now, try search/download/pm/mainchat.. And if you find any bugs, please report them in the Bugs section.

Have fun with networking

Version 1.4 released
Added FLAGS to fine-tune each hublink connection.

Added instructions how to install/load everything, and also made the script compatible with Verlihub v0.9.8d-RC2 and VH plugin for Lua v1.7rc1

i have misunderstand one thing how registered nicks are managed?
for example we connect two hubs and one nick is registered on both, how this will be processed?

That's the point of using prefixes in nicks. Each hub has its own prefix on that network and all nicks in a hub, that are prefixed with prefix of that hub, can use network. Nicks, that don't have a valid prefix, can't use network functionality, but they are still able to use that local hub as they used it before.

Very unconvenient, don't you think so?
Especially when two big hubs want to connect but they have huge regs base or when you want to split one big hub into several small ones. We should think out another way of users aggregation without manual nick changing.

be my guest
any suggestion is very welcome

the most stupid and easiest thing is to force linking hub adding prefixes to users nicks but i think this will consume a lot of extra cpu resources.
anyway users should be somehow marked as linked from the outside

solution that you propose has already been considered (in earlier versions of VerliLinker) and the main disadvantage of that proposal is that $ConnectToMe messages will not work, because DC clients impose a security protection that prevents CTM messages to be correctly handled.. more info here: http://forums.apexdc.net/topic/3423-hel ... d-problem/

the problem is that users in their own hub "see" their nick as usual (not prefixed), but users on another hub, see their nicks prefixed with prefix of that remote hub, so when p2p connection is established, nicks don't match, and that's the basic flaw in that proposal :/

Ok, i see.
Then maybe we can think of some hash for each user of linked hub for his unique identification like TTH file or something... i am not sure yet how can it be done but will think futher.
anyway linked (not host) hub should take all operations and handling of hash creation for users and so on.

p.s.
CTM will work properly if both hubs will have tables of comparison for nicks and will change CTM command according to it when users from linked hub needs to establish connection

any nick mangling done by hubs will simply not work because clients will establish a p2p connection and they are gonna check each other's nick, so that's where the problem arises..

consider the case where nick "TheNick" from hub A wants to connect to nick "TheNick" from hub B.. after they establish a p2p connection, they are going to send their nick to the other side, and they will both assume the remote nick is invalid (because its the same as their own)..

Version 1.6 released.
Added flags 'o' and 'O' to control the appearance of operators in the OpList.
For more info, see here: viewtopic.php?f=103&t=4795

Version 1.8 released.
Corrected all those bugs, related to Verlihub v0.9.8d-RC2, so it now runs pretty stable.

Version 1.10 released.
Corrected a bug, related to "loop message" bogus error