Author Topic: Experimental UDP code is working!  (Read 19825 times)

0 Members and 1 Guest are viewing this topic.

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Experimental UDP code is working!
« on: August 10, 2008, 07:03:36 PM »
The new experimental UDP code is working and tested on Linux. For windows, let me know how it goes.

EDIT: Sept 01, 2008 - Updated this post for the new reliable UDP code for V0.19.27.

Why UDP? It may solve some problems for people who have different network setups. Like getting around some NAT routing situations, ISP blocking, Wifi Mesh networks or LAN sharing.

Thanks to the team at the University of Illinois, we can now have reliable connections via UDP. They created some C++ code called "UDT".

If you don't want to wait to see if it gets included in the next release you can compile it yourself, I posted a patch file at
http://sourceforge.net/tracker/index.php?func=detail&aid=2087537&group_id=96735&atid=615726
(see bottom of that page for file)

To patch V0.19.27, make a new directory called "test1", download OFFSystem-0.19.27-src.tar.gz and the patch file into it, like this...

cd test1
tar -xzvf OFFSystem-0.19.27-src.tar.gz
patch -p1 < version_diffs.patch

Then download the two other files "udt.cxx" and "udt.h" that are on these patch pages.
http://sourceforge.net/tracker/index.php?func=detail&aid=2087541&group_id=96735&atid=615726
http://sourceforge.net/tracker/index.php?func=detail&aid=2087546&group_id=96735&atid=615726

Download them to off_network/src/ then do the normal compile...

cd off_network/build_gtk
make
(then do whatever you normally do with a new compiled version)

The OFF client will operate normally via TCP unless you change one configuration option.

When the program is not running, in the "config/offsystem.conf" file, near the end, edit (or make) the use_udp option to enable UDP.

 use_udp: 1

You should also rename the "config/known_nodes.txt" file, it contains the IPs of other nodes on the normal OFF network. Renaming it lets you put it back when you want to go back to normal.

You want to start with no "config/known_nodes.txt" or a empty file.

Start the OFF client from a terminal and you will see a lot of debug messages about UDP, showing socket open and a few other messages. UDP is working if you see those messages.

Go to the "Known Nodes" screen and right click then select "Add Node". Type in the other node's IP (and port if different) and a connection should start in about 5 seconds. Both nodes need to be in UDP mode of course.

This code will handle multiple node connections, so have a party and test like mad!

If the other node has no blocks in it's cache then you will see messages and network activity as it pushes randomly selected blocks to the other node.

Have fun and please post on how your test went.


Some things that aren't working yet in the new test code:

No combo TCP and UDP, just one or the other.

Loopback/web server doesn't work since it would be a UDP socket.

WARNING, privacy risk if you are only running two nodes. Don't post URLs that you don't want others to know you have since it's easy to know who has what URL.

Technical notes:

The largest packet I've seen sent is 1040 Bytes so no worries about exceeding the normal MTU so far. However, the IP layer is suppose to break up any packets and handle it if necessary and on Wifi there's more code I'm sure.

TCP and UDP are both a little slow with no throttling set on a local LAN. I'm not sure if it's something I am doing wrong but the CPU usage is low so it isn't processing much. I will be looking into some of the places in the code there could be delays so don't get too worried yet.

Wild ideas:

It may be possible to get around some NAT routing problems by initiating a UDP port scan to some destination from a fixed local port. Wait for one response packet. Keep the UDP hole open by sending UDP packets at a low rate to the remote port that responded. You may now have a open UDP port to the world.

Other clients can do the same (possibly port scan you until they find your open OFF UDP port) and then the connect may go through.

You may be able to forge the sending IP so the other end thinks it's coming from the local or other network. That may take two "connections" to work and both ends would probably have to know what is going on.

You could have "fake" packets that look like they are from other nodes or IP addresses but actually do nothing. With encryption on this would be hard to sort out for anyone "listening in".

It may be possible to add FEC to each packet so that single bit errors are corrected on the fly.

If you have problems with the patch, I posted this file just in case
http://sourceforge.net/tracker/index.php?func=detail&aid=2087553&group_id=96735&atid=615726

« Last Edit: September 01, 2008, 09:20:31 PM by scripter »

Offline da1l6

  • Newbie
  • *
  • Posts: 5
  • Karma: +0/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #1 on: August 13, 2008, 09:44:05 PM »
Hello

Wow, thats a pretty impressive work you brought up there.

I havn't tried it yet, but i am going to test it as soon as possible. I will also try to find out what is wrong with the RSA key exchange, if you havn't already.

With some work this patch could be integrated in the OFF client.
Most important is the possibility to use UDP and TCP connections at the same time to avoid splitting the network.

If you would like to discuss future development in real time, you should come to the project IRC channel #thebighack on irc.p2pchat.net
Note that the main developer (Spectral_Morning) is currently on vacation, he will be back within a week. But he still comes online in the channel about once per day, so you can reach him. He is also highly interrested in your work.

da1l6

Offline OFF-meister

  • Regular
  • **
  • Posts: 98
  • Karma: +3/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #2 on: August 14, 2008, 09:21:20 PM »

Hi scripter,

thanks a lot for this work, our main dev is on vacation right now, but apparently your
code has been integrated into the main OFF tree, and some limited tests have been done.

Most likely the first use we will put this to is UDP hole-punching for firewall-to-firewall transfers,
which we currently cannot do.

 Thanks again, and stop by the chatroom anytime. :)

   OFF-meister

P.S. this message was not brought to you by your usual OFF-meister!

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #3 on: August 15, 2008, 02:44:07 AM »
I don't want to slow things down, and no promises here, but I found some "reliable" UDP code called "UDT" and integrated it into the OFF base code, it's compiling nicely including one socket call so far.

All I have to do is properly tie it into the places I already tied in the other UDP code in and it should work nicely. As long as I can keep the OFF code from messing with low level stuff by using "sockfd" (which I think is covered by the original UDP mods I made).

UDT handles things by emulating TCP almost exactly on the code side, and can do all the stuff we want from one open port.

UDT is currently maintained, was developed by a University of Illinois team and was LGPL code but changed to the BSD license. They only restrict it to making sure we display their name in the code etc.. as far as I can tell and want it to be integrated into other projects like OFF. C++, open source and all that.

UDT is being used and tested on many big high-speed networks for clustering etc.. It has configurable congestion control algorithms (which I think we should just leave alone, I like simple stuff and it's probably overkill for our situation).

The way I'm doing this is to just include *all* the UDT code as-is in it's own directory (it's not that big) and use their makefiles so that it's easy to upgrade when bugs are found and maintaining the code is super easy for the OFF devs. I think it makes the licensing issue easier too. It links in nicely and needs no special libraries the way I have it set up.

Oh, and it has a mode to punch through NAT firewalls. I'm not sure how to integrate that yet, it may have some other issues but maybe I'll just add another config option and let everyone test it out.

http://udt.sourceforge.net/

da1l6, I don't want to see the current network split because of this, I was hoping to fill a need for special situations, like Nemo's mesh routing problems. Like you said though, a bit more code and it's possible to make it use both TCP (first) then UDP if needed. In the current code it would be easy to have it listen via UDP, the problem is what it will do when originating a outgoing connection and how that's handled. I'll have to think about that more. I2P handles UDP connections nicely and is a good example of how to do it.

OFF-meister and others, please tell me your applications and real world situations for punching through NAT firewalls, and what the advantage is over TCP. Currently there's enough nodes that allow incoming TCP so firewalled nodes are covered fairly well right now I think.

As for security concerns, it's always better to stick with the big group so URLs and searches are hard to track, but if you have rules for outgoing bandwidth at a university for instance, then a small local LAN party is just too fun to pass up. UDP may help them form a network even if they have restricted NAT routers or I think they can use Wifi easier with UDP.

UDT packets may be as hard as TCP to figure out the packet number from outside, so it may be as secure as TCP on that issue (if you can call that security), but someone can easily fake a UDP packet IP and interrupt a current connection (then you just reconnect, ha ha). I'm not sure how much worse this would be than TCP anyway (more study needed) and later when the encryption is working in OFF it should be able to reject fake packets easily.

As for chat, sorry, it's disruptive when concentrating on coding etc.. and you have to sync to other's schedules. This forum works OK for dev things, like I can read and post when I'm letting a test run, and Markus really loves P2P stuff.


Offline OFF-meister

  • Regular
  • **
  • Posts: 98
  • Karma: +3/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #4 on: August 15, 2008, 12:05:48 PM »
Personally, I wouldn't do this - becoming dependent on external packages tends to cause nightmares in cross-platform development and no end of other dev headaches!

We've only just got the headless build working with no other downloads and I don't think anyone is keen on changing that. :)

Obviously, it's unavoidable a lot of the time but we try to keep it to a bare minimum.
« Last Edit: August 15, 2008, 12:08:03 PM by OFF-meister »

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #5 on: August 15, 2008, 05:27:03 PM »
It's about the same amount of code as if I wrote a reliable UDP myself and I don't want to spend a year recreating work that's already done.

UDT tar.gz file is 110K.

The folder for the code will be included in the tar.gz package and checked into CVS etc.. I see no reason to make anyone download it from a outside source. Then OFF is always compiled against a known working compatible version. When the devs feel it's necessary to upgrade UDT they just download the latest version and replace the folder, leaving it as-is.

If for some reason it becomes unsupported you still have the code, same as if I wrote it myself, it works good now so it shouldn't need much but you just fix it like any other part of OFF.

Did I mention that it's windows and osx compatible too? I can't test those but I have it set up to compile properly (I hope).

Speaking about headless clients, they would be a good choice for UDP in strange situations where you just want to run OFF in the background on a remote machine, maybe behind a NAT or put on a USB stick for a Wifi remote station.


Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #6 on: August 16, 2008, 07:05:12 AM »
The new reliable UDP code is working!

I tested it with 3 nodes running and everything just works, including the RSA keys!

I updated the first post in this thread, see the top post for more details about the new patch file and how to compile it in.

Have fun!


Technical stuff...

Makefiles for windows and mac have to be modified, someone else has to do that but you can look at the patch file to see what I did to get it working on linux. The UDT code is made to run on both, you have to get it to set a flag like "WIN32" when doing the make.

See the install docs at http://www.cs.uic.edu/~ygu1/doc/make.htm

Only one small problem I found is that it does a socket level "accept()" twice in some cases on incoming connections, it doesn't hurt anything and I can't see how my mods caused that. It may be a bug that's been there all along, and that might be why there were extra sockets open at program quit. I looked into it a little bit and don't see why it would do that but some thread is running twice or something.

One other bad point, when doing the mode they call "Rendezvous Mode", for punching through NATs, you are a bit limited as far as I can tell from the docs. You can't call "listen()" or "accept()", both sides have to connect at about the same time and it looks like it's limited to one port only, meaning that you could only talk to one other node that way. So more research needs to be done.
« Last Edit: August 16, 2008, 07:20:14 AM by scripter »

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #7 on: August 17, 2008, 02:49:13 AM »
I think that "punching holes" through a NAT firewall will work with the UDP code the way it is.

After some further research I found that the UDP packets from OFF using UDT are being sent from the same port as incoming packets, and that's a good thing (confirmed using tcpdump).

Even firewalled nodes send "from" port 23403 to the other end's 23403, we aren't opening a new port for each outgoing connection like TCP does, no need because the UDT code handles all that.

This is good because if you make just one "outside" connection, your UDP port 23403 is now open to the world *through* your NAT firewall.

But only as long as the NAT you are behind is sort of lame and not checking incoming IP addresses. A lot of simple ones don't check and that's why sometimes it works and sometimes it doesn't as I understand it.

I'm not setup right now to test this so if someone is, please give it a try. You need two nodes that are behind some kind of NAT firewall and have access to at least one "outside" node that isn't firewalled.

This works because a NAT sees that you sent a packet to someone on the outside "from" port 23403, then it sees a return packet come back to that port, from then on it always forwards those packets to your computer, no matter where they come from (with a no-activity timeout I'm sure). If the incoming IP is checked by the NAT then forwarding doesn't happen for just any IP and you can't punch a hole with that type of NAT.

So, for a mesh Wifi network you would need one node, even a "fake" one that accepts incoming UDP packets and sends something back, even random data. Everyone on the network needs to know this IP and have OFF try to connect to it at least once and possibly every 5 minutes to keep your NAT happy.

Another method if there is internet access on the Wifi mesh is to connect to a outside internet node and then disconnect (preventing more internet bandwidth from being used).

There might be a way to make a simple shell script that sits inside a Wifi router, opens a port and accepts UDP packets then sends one back with a few bytes of random data. I think once you *attempted* that connection with OFF and then other nodes started connecting to you (because they already had you listed in their node list from another session and are trying to ping you), then the activity would keep the NAT happy and there would be no need to "hit" the special IP address again.

There's also the possibility since OFF sends pings to nodes on a regular basis that one node on the network will be sending you a ping at about the same time you send one and the NAT will think that's the reply and open the port. It's worth a try to see how well that works.

All you really need is to have at least 2 nodes that are always connected to each other 24/7.

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #8 on: August 19, 2008, 11:36:32 PM »
UDP punching through firewalls works!

I set up firewalls on two computers, setting them to allow *outgoing* UDP only, bidirectional. This is most likely the same setup as most NAT firewalls. The firewall I'm using simply sets up iptables.

I start OFF at one computer (A) and enter the other computer's IP (B) while monitoring packets with tcpdump. The packets go out from A to B across the network but B doesn't see them because they are blocked.

If I enter computer A's IP into OFF on computer B within 30 seconds, they both connect and they both act like they are not firewalled!

After some other tests I found out that 30 seconds is about the limit, 45 is too long. I don't know if that's a standard time limit across all firewalls, but it gives us some idea and it's a reasonable timeout period.

With this data I think it's possible to modify some code to help this process. We already ping nodes on the list, so if we haven't seen a incoming packet yet, ping the other nodes every 30 seconds, and ping more often. Sooner or later two nodes will connect and you will have a open port to the outside. Once your port is open the ping rate can go back to normal. Of course this would be a config option since it wastes a little more bandwidth than normal.

I think that as long as there's OFF network activity that the UDP port will remain open to the outside, but it might be a good idea to monitor incoming packets and if they stop for more than a minute, start the ping process again.

Some code is also needed to figure out the "outside" IP address for internal use and also display it to the user. For now people need to find out their outside IP and the other nodes IP some other way.

« Last Edit: August 19, 2008, 11:38:37 PM by scripter »

Offline Nemo

  • Global Moderator
  • Elite
  • *****
  • Posts: 1303
  • Karma: +27/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #9 on: August 20, 2008, 09:29:01 AM »
If I enter computer A's IP into OFF on computer B within 30 seconds, they both connect and they both act like they are not firewalled!

After some other tests I found out that 30 seconds is about the limit, 45 is too long. I don't know if that's a standard time limit across all firewalls, but it gives us some idea and it's a reasonable timeout period.
I've read on some sites on Internet that timeouts in linux kernel for connection tracking follows values in some RFCs. Some timeouts are adjustable via "/proc/sys/net/ipv4" while others seems to be hardcoded:

From http://lists.netfilter.org/pipermail/netfilter/2006-February/064964.html:
Quote
> it maybe help you.
> /usr/src/linux/net/ipv4/netfilter/ip_conntrack_proto_udp.c
> #define UDP_TIMEOUT (30*HZ)
> #define UDP_STREAM_TIMEOUT (180*HZ)
> To change these values you can set up the udp timeout values.
One single UDP packet stays 30s in the connection tracking table, if the algorithm detected a reply packet then it's called "UDP stream" and the "connection" entry stays 180s in the table. I assume that these values are correct for most Linux/Unix-based systems as they correlate with your observation.
(it's mentioned in more details in german under http://www.informatikserver.at/selflinux/html/iptables08.html)

Greetings,
Nemo.

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #10 on: August 20, 2008, 01:47:05 PM »
Thanks for the info Nemo. That makes sense, I could try connecting and then remove nodes on both sides and wait a minute or two and try reconnecting to see what happens.

When you mentioned this I remembered that this new UDT code sends a packet every so often to see if the other end went away. So once the connection is open I think we are going to be OK without worrying about timeouts.

OFF already pings other nodes on a regular basis so that should keep things going, besides it sending block pushes (or offers) constantly too. 3 minutes is a long time to not get any activity from a OFF node.

Anyone can use this code right now as-is to punch through a NAT, they just have to try to connect within 30 seconds of the other end trying.

Nemo, did you get a chance to try this on your mesh yet? If you're having problems with the compile or anything let me know what OS you and your friend have. It mostly errors on missing libraries which are like openssl, libpng, libjpeg, libtiff, and wxWidgets (2.8). I think on redhat you can find and install them as RPMs. There's some warnings printed about some variables not being used when you first compile but that's normal and doesn't really matter.


Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #11 on: August 22, 2008, 01:23:08 AM »
V 19.25 is out and the *old* test UDP code is included.

Someone please tell me the new UDT code is going to be put in instead of trying to re-write what took the University of Illinois UDT team since 2001 to develop.




Offline OFF-meister

  • Regular
  • **
  • Posts: 98
  • Karma: +3/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #12 on: August 31, 2008, 01:59:01 AM »
Hi scripter,

We seem to be having a lot of trouble getting UDT to build in MSVC6, this is the compiler we use for windows builds. We can build it in VS2005, but for other reasons we can't use VS2005 for releases.

Any help you could provide here, would be much appreciated.

Since you asked about using a Linux box, I *think* that MSVC6 would run in wine, but I'm not sure. :)

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #13 on: August 31, 2008, 05:44:04 AM »
I don't think you are talking about the parts within the OFF code because that should be completely compatible with win32, except that in a few places you had special stuff for win32 in offcnxn, but I tried to cover that. Oh, and no other code should be using a "sockfd" directly to make socket or control calls.

One big question is can you compile OFF using cygwin? And does OFF run in wine? Then you would be back at GCC again and I think you could compile under wine or even linux that way with the right settings. I don't know much about MSVC, sorry. I am heading the cygwin under wine direction.

I would love to have one shell script that built both linux and windows versions at the same time.

As for UDT, I assume you looked at the UDT info about compiling http://udt.sourceforge.net/ , there isn't much.

I started with compiling the UDT example programs first, they have little test programs that do simple stuff. There's some test stuff for MSVC in udt4/win . Maybe that's what you are talking about with visual studio. And they say "It requires Visual C++ 7.0 or above to compile. If other Windows compilers are used, you may need to create your own Makefile or project files.", so that may be a problem.

Nice thing about gcc is that I just had to make libudt.so available to it so it static compiles / links into the code. It may be possible to just use their make file info and put it all in the OFF make file so it handles everything. I thought it was cleaner not to do that and let UDT sort of stand by itself. I probably could have linked to all the *.o files to do it another way.

I would say that since UDT has been around a while, they won't be changing their make files much, and you really don't have to upgrade it unless they make some really great new features available.

I don't think using their dll file is the way to go, but I know a lot of windows programs include them in the same directory as the program and windows finds it and uses it after some sort of "load" call. So that may be a way to go. There may also be a way to statically attach a dll to a .exe file so you just build UDT once and always include it at build time.

Of course during UDT compile you have to #define WIN32 when you see OFF's "_WIN32" (see api.cpp and common.cpp) but I'm sure you know that.

I'm probably not helping much so far, but we know the code works and it's pretty standard C++ code, so there should be a way to get it to compile under any OS.

I hope mac os is a lot more like linux.

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Experimental UDP code is working!
« Reply #14 on: August 31, 2008, 06:20:29 PM »
OFF runs under Wine in Linux!

"HOWTO: Run OFF Under Wine in Linux (windows emulator)"
http://board.planetpeer.de/index.php/topic,5221.0.html

Not sure why "MSVCP60.DLL" isn't included in the installer. And it didn't run under Win98 emulation, it popped up a notice saying it won't run (it wasn't a Wine notice). The installer runs under Win98 OK. OFF wants XP.