Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - scripter

Pages: [1] 2 3 ... 6
1
OFF / Is this the URL that crashed my client?
« on: October 05, 2008, 09:58:40 AM »
I found this link on a web site, clicked it and my client crashed, or stopped, I'm not sure which one.

The link said "people showing things you shouldn't see, click here", so I clicked on it, I couldn't see the underlying actual link, or wasn't paying attention because there were a lot of other OFF links on the page.

I was running the latest version, 0.19.32

The real, actual underlying link was.......

http://localhost:23402/index.htm?shutdown

:)

While we're on the subject, I also clicked on a search result with a filename of

"\Windows\System\some_important_file.dll"

No one could ever fake search returns, right?

"we can avoid the expense of checking the filenames of srch results"

Not a good idea.

2
OFF / Re: Experimental UDP code is working!
« on: September 17, 2008, 08:36:08 AM »
After studying the UDT code more I can see that the difference vs. TCP is that you normally call select() to see if the TCP connection is complete, but UDT doesn't do that.

UDT needs to do a little handshaking, like TCP, it gives the other end a random sequence number to start with which basically helps put out of order packets back together.

That handshaking can be completed with the further calls to select() instead of doing it in connect(). The OFF code at that point can also do a timeout check.

So I'm headed in that direction and created some code that completes the handshake so there's no blocking code. It may not work, but what the heck, I'm headed that way for now.

On the close() call, it shouldn't be hard to check to make sure the UDT (virtual socket) buffer is empty before calling the actual close. With serial communications this is a normal operation, you always check to see if all the data is sent out before closing.

3
OFF / Re: Experimental UDP code is working!
« on: September 12, 2008, 12:37:30 AM »
Update on where I'm at.

If you want to throw the baby out with the bath water, go ahead. I think this code is usable with some changes.

Unlike TCP with UDP we use the same buffer and socket for everything. So the initial setup of that socket is all we get.

If we are expecting to get 200 connections for a normal OFF node, then we need to open a buffer first thing to handle that. Remember there's a UDP socket buffer and also a internal UDT buffer that is used for buffering each "virtual" connection (it's sort of virtual because we are using the same UDP socket and simulating that it's more than one). Once the UDP buffer is set and the socket is open, it's set. But it doesn't have to be that big I think because the other buffer is used.

So some simple math 200 * 50K = 10M. So we should open a 10M buffer so each connection gets it's own 50K buffer.

Some code could be made to stretch the internal UDT buffer as new connections are made, but that code isn't written yet according to one of the UDT developers. It's not a big problem and I may find how to do that later.

Since variables in the code keep track of how much buffer it uses, stretching and reducing it shouldn't be a problem as long as it isn't fragmented.

On top of that, 2GB of ram is now about $20. I don't think people are worried much about a extra 10M being used during P2P operations. Maybe if it was 1985 or something. Lots of people run Java to use a lot of popular P2P programs, that adds 500M on my system.

Bottom line is I'm not going to sweat over a extra 10M, I will make the code set the buffer up at the start at a level we can live with and move on to more important things.

As for blocking. It looks like I can set the variables up so that no "handshake" is needed at connect(). UDT sends a random sequence number to start the connection and waited for the other end to send one back and that's not really necessary. That can be fixed without too much trouble, but the UDT developers probably won't want it that way due to the trouble it will cause etc..

So the main UDT base probably won't change just for us.

For now it times out in 3 seconds, according to a UDT developer.

What I'm still not sure of is why this is such a big problem that we should drop this code completely and go for a year or two of part time development to re-create it from scratch. If there's a better piece of code for reliable UDP out there, let's jump on it. I'm not attached to UDT in any particular way but it looks like the best we can get for this.

If each connection is in it's own thread in the OFF code, then it should be able to block forever and not effect the other code. I thought that was the way things were. I have to spend more time on researching that.

Unfortunately, trying to keep this code as a simple module that we can just plug in and get updates directly from the UDT people and just plug them in is a pipe dream. Sorry.

I think the way to go is to insert this code and call it "ours" from now on. Meaning that we keep this code in OFF and modify it as we need to. When and if security changes are made to the main UDT tree, we copy them line by line and include them in the OFF code. That's just the way it has to go.

I am also of the opinion that the single, large file for the UDT code is the way to go because it will make compiling on other platforms and compilers easier, no special makefile changes needed. If there are really good reasons not to do this, I'm listening.

I need to know what changes were made to UDT if any to make it compile on MSVC6 so I can include that in the single udt.cxx file.

As for page faults, UDT uses "new" and "delete" to open up memory etc.. Is there something being created that isn't being deleted?

I don't mind tracking down the problem, I just need to konw, exactly, in detail please, and under Linux how to set things up so I can see this for myself and hopefully use gdb or something to track this down.


4
OFF / Re: Experimental UDP code is working!
« on: September 10, 2008, 03:06:04 AM »
Wrote a little script so I can easily test more nodes on the local machine.
http://board.planetpeer.de/index.php/topic,5267.0.html

I tested 20 nodes with and without UDP on (via the global config setting), no segfaults, no problems, run was about 30 minutes. On Ubuntu Linux "hardy". CPU usage was just about the same for TCP or UDP but never 100%. Memory usage was higher for UDP.

The setup used 0.19.27 with the single file UDT version (because it was there already), one node with a lot of blocks, the others had none. They all connected together, 20 nodes with each having about 20 nodes on their list, and were pushing blocks around to each other. I could see the printouts for each node in it's own terminal. I used no throttling.

I ran 18 nodes in UDP mode for 1 hour with no segfaults.

I ran 18 nodes again under gdb for 30 or so minutes and looked through each terminal looking for any gdb errors, all I got was new thread notices at startup. If there's a special gdb setting I should use let me know.

The memory usage was up about 30MB over TCP on each node but did not increase much as each new node connected, it looks like it happens when the first server port is open, I'm still trying to figure that out. It's going to take some time.

Blocking wasn't obvious, like the GUI stopping, I'm not sure how to detect that (do tell) but we know it's happening, going through the UDT code it looks like it asks the other node what its settings are and waits for a time then returns. It may be possible to always have the same settings for all OFF nodes and then the handshake won't be necessary and it won't block, hopefully.

The blocking on close may be caused by it wanting to send out the rest of the buffer. It may be possible to control that in other code, like the TCP code that checks to see if the buffer is empty before closing, I didn't tie into that to save some time and get the code out. I'm sure that can be solved.

Each connection should be in it's own thread anyway, from what I saw on on gdb it isn't. Could that be the slow (local) speed problem? I didn't realize it's actually polling each connection one by one, correct me if I'm wrong.

So at this point I'm confused about your tests having segfault problems. Any chance it used up all available memory?

EDIT: On the memory issue, if I open the server port and then close it, then open it again the ram usage vs. TCP is about 10M more instead of the 30M. The reason is that when you make the call to change the buffer size it looks like it really doesn't resize the existing buffer and just changes the variables. If you close the socket the buffer is gone, then next time it uses the new values to open a buffer.

So the solution is that we set the default values to what we want in the UDT code itself so it uses those values first thing. I just have to figure out which ones to set.

5
OFF / HOWTO: Test many OFF nodes on a single Linux system
« on: September 10, 2008, 02:08:34 AM »
The scripts below create new directories for each test node, including all the config options and proper port settings so you can test as many OFF nodes on one system as you can handle.

Communications are over the local interface (lo) of course, which can be monitored with tcpdump if you like.

You must compile OFF with the option RELEASE_ON disabled (see offconstants.h at about line 335 and comment it out) then it doesn't create a lock file for each instance.

Create a new directory and put a default OFF setup in there, including the OFF executable, config, themes and other folders.

Put the two scripts in there and chmod them so they are executable.

Open a terminal and cd to that directory. When you run "./go-test.sh 4" it will create 4 new directories named "1", "2", etc.. and copy the executable and config files over, changing the ports and node name in the config file accordingly. It also adds nodes to connect to within the test range.

When you are done you can use "cleanup.sh 4" to kill OFF and remove the test directories "1", "2" etc..

"go-test.sh"
Code: [Select]
#!/bin/bash

# Script for testing lots of OFF nodes on the same computer
# You must compile OFF with the option RELEASE_ON disabled
# (see offconstants.h at about line 335 and comment it out)
# then it doesn't create a lock file for each instance
# Place a default OFF setup in the working directory, including
# the OFF executable, config and themes folders
# Written by Scripter Sept, 2008

make_nodes=$1

if [ "$make_nodes" == "" ]; then
echo "Usage:  go-test.sh NUMBER (of nodes to make)"
exit
fi

# set this if you only want each node to connect to the next node at
# startup, otherwise each node will try to connect to all other nodes
connect_to_next_node=0

# setup variables we will use
node=1
port=23501

while [ $node -le $make_nodes ]; do

echo "Copying files for Node $node"
if [ ! -d $node ]; then
mkdir $node
fi

cp -a config $node/
cp -a blockcache $node/
cp -a retrieved $node/
cp -a pads $node/
cp -a lang $node/
cp -a keys $node/
cp -a themes $node/
cp -a temp $node/
cp -a offsystem $node/

# remove config lines we want to change, then put them back with what we want
# OFF wants each line to be exactly in order for whatever reason
cat $node/config/offsystem.conf | grep -v "verbose" | grep -v "trusted_search" \
 | grep -v "username" | grep -v "incoming_p" > $node/config/offsystem-temp.conf
echo " verbose: 1" > $node/config/offsystem.conf
echo " username: \"Node-$node\"" >> $node/config/offsystem.conf
echo " trusted_search_in: 0" >> $node/config/offsystem.conf
echo " trusted_search_out: 0" >> $node/config/offsystem.conf
echo " incoming_port_number: $port" >> $node/config/offsystem.conf
cat $node/config/offsystem-temp.conf >> $node/config/offsystem.conf

# connect to the next node, last node gets connected
# to 23403 for a special running node you may have
if [ $connect_to_next_node -ne 0 ]; then
  if [ $node -ne $make_nodes ]; then
    port1=`expr $port + 1`
    echo "localhost $port1" > $node/config/known_nodes.txt
  else
    echo "localhost 23403" > $node/config/known_nodes.txt
  fi
else
  # each node tries to connect to all other nodes at startup
  node1=1
  port2=23501
  echo "localhost $port2" > $node/config/known_nodes.txt
  while [ $node1 -lt $make_nodes ]; do
    port2=`expr $port2 + 1`
    echo "localhost $port2" >> $node/config/known_nodes.txt
    node1=`expr $node1 + 1`
  done
  echo "localhost 23403" >> $node/config/known_nodes.txt
fi

# remove the rsa keys, will be regenerated
rm -f $node/config/off_rsa.pub
rm -f $node/config/off_rsa

node=`expr $node + 1`
port=`expr $port + 1`
done

node=1

while [ $node -le $make_nodes ]; do

echo "Starting Node $node"
cd $node

# my usual run with a terminal for each node
gnome-terminal --title=Node-$node -x ./offsystem &

# just run it
# ./offsystem &

# with gdb, you should killall offsystem; killall gdb before using cleanup.sh
# gnome-terminal --title=Node-$node -x gdb -ex run offsystem

cd ..

node=`expr $node + 1`
done

exit

"cleanup.sh"
Code: [Select]
#!/bin/bash

# Script for cleaning up after using the OFF go-test.sh script
# It will kill OFF and remove the directories
# Written by Scripter Sept, 2008

make_nodes=$1

if [ "$make_nodes" == "" ]; then
echo "Usage:  cleanup.sh NUMBER (of nodes to cleanup)"
exit
fi

node=1

echo "killall offsystem"
killall offsystem
echo "(giving it a little time...)"
sleep 10

while [ $node -le $make_nodes ]; do

if [ -d $node ]; then
echo "Removing folder $node"
rm -fr $node
fi
  node=`expr $node + 1`
done

exit


Note: On Ubuntu you should install "gnome-terminal" so this works properly.

6
OFF / Re: Experimental UDP code is working!
« on: September 09, 2008, 03:26:48 AM »
There are some easily set options in UDT, see http://www.cs.uic.edu/~ygu1/doc/opt.htm

When testing, I found that UDT_SNDBUF was 12058624, UDP_SNDBUF was 12288000 which is a bit much so I set them to 500000 (500KB).

These are *upper limits*, they were set high because UDT is normally used on very high speed networks.

It's really up to you what it's set to, I just wouldn't put it too low.

It didn't make any difference when I tested it because UDT only uses what it needs for the buffer, and we quickly take the data and use it, so it shouldn't reach the limits unless a node is really overworked.

A bigger buffer may be needed for LAN sharing, and this could become a setting that's changed when someone picks LAN mode, at this point I don't know.

The code change in "set_socket_options()" is below. Please include it. (it really doesn't fix the above problems you posted, but it's good to have)

Looking at my system, OFF uses 193m with a 7GB cache loaded and 168m with a 500MB cache. Java, wonderful bloated Java... 501m, firefox with 6 windows open 171m.

So I don't think 27M is much to worry about but I will spend a little more time looking into this.

For the statistics stuff, it's at http://www.cs.uic.edu/~ygu1/doc/structure.htm if you want to see the values, but it's just a struct with a bunch of int's so that's not taking much memory.

Now, the BIG problem... Blocking. It's a problem and it's something I didn't catch at first, sorry.

UDT docs say "UDT does not allow non-blocking operation on connection setup and close", probably not good for P2P situations.
http://www.cs.uic.edu/~ygu1/doc/t-config.htm

It could just be some timeout setting. Is there any chance you know exactly when it blocks? Like at what UDP packet during the first connection?

It could also be the way I tied it into the OFF code, I don't know yet.

I'll just say that worse case it should be possible to call their code in a way to make it non blocking. I'm not sure how TCP does it's thing so it is non blocking at connect, but maybe UDT isn't handling it that way or UDP connections always have to block since you have to wait for the other node to do something.

Again, I'll have to look into it more, if you have any suggestions let me know.

As for the segfaults and page faults, was that on a windows system? I had to ask :)

Did it only start happening at the 15 node limit problem? I haven't seen any on my testing on Linux with three nodes running and transferring blocks for hours on end.

Code: [Select]
  if (G_offcnxn_use_udp) { // UDP set send and rcv to non blocking
    bool block = false;
    int udtbuf = 500000; // 500KB buffer for each UDP connection
    if (UDT::setsockopt(sockfd, 0, UDT_SNDSYN, &block, sizeof(bool)) !=0 ||
        UDT::setsockopt(sockfd, 0, UDT_RCVSYN, &block, sizeof(bool)) !=0) {
      string last_err=UDT::getlasterror().getErrorMessage();
      OFFLog(" OFFCnxnBase::set_socket_options(): ERROR sockfd: %i",sockfd);
      OFFLog(" OFFCnxnBase::set_socket_options(): last error  : %s\n",last_err.c_str());
    }
    if (UDT::setsockopt(sockfd, 0, UDT_SNDBUF, &udtbuf, sizeof(int)) !=0 ||
        UDT::setsockopt(sockfd, 0, UDT_RCVBUF, &udtbuf, sizeof(int)) !=0) {
      string last_err=UDT::getlasterror().getErrorMessage();
      OFFLog(" OFFCnxnBase::set_socket_options(): ERROR sockfd: %i",sockfd);
      OFFLog(" OFFCnxnBase::set_socket_options(): last error  : %s\n",last_err.c_str());
    }
    if (UDT::setsockopt(sockfd, 0, UDP_SNDBUF, &udtbuf, sizeof(int)) !=0 ||
        UDT::setsockopt(sockfd, 0, UDP_RCVBUF, &udtbuf, sizeof(int)) !=0) {
      string last_err=UDT::getlasterror().getErrorMessage();
      OFFLog(" OFFCnxnBase::set_socket_options(): ERROR sockfd: %i",sockfd);
      OFFLog(" OFFCnxnBase::set_socket_options(): last error  : %s\n",last_err.c_str());
    }
  return 1;
  }


7
OFF / Re: Experimental UDP code is working!
« on: September 09, 2008, 12:36:01 AM »
> the download progress bars vanishing when the list items are selected - this also seems to occur when a manifest is used with the current release builds.

It's hard to believe that the compiler has anything to do with this, sounds like a problem in wxwidgets. It's possible that one compiler ignores certain things while the other doesn't, producing different code. Or it may just be some setting for the compiler, like some sort of "strict" thing.

Is it possible that it's a problem with the older version of wxwidgets? If they fixed it, it might be possible to take that code and put it into the code you are using so that it's not a big project.

I'm sorry it's taking more of your time, that wasn't my intention. I didn't realize that different windows compilers had so many differences, and the fact that they are both from MS.

Is there any chance of using a borland compiler? At least that is closer to using a open source compiler, in my opinion. I think we both agree that if you could just run a script and have it compile both versions automatically (without windows anything) that would be the best.

> I am sorry for not being able to reply more often here

The forums worked as they should, you come here when you get a break, it's not a problem.

As for the UDT problems at 15 nodes, thanks for testing it so throughly and it shows that leaving it in a test setting is a good idea for now.

35M? Or what you are saying is 27M more memory to support this. All that it should need is a small list of IPs so it can match it to a particular connection and a little more for thread support etc.

It's possible that there's some sort of statistics code enabled, and it's keeping that data in RAM in case it's called for (and that might become useful for testing, but turn it off in release).

Does this mean it compiled in MSVC? And what incarnation, the single file one or the directory one?

I'm going to do some more research on this so watch for more postings.

I started working on the proposed freenet interface, trying to use the same methods that are used for the loopback server but on port 23401. It's mainly going to be a pass through for http connections, only modifying the http stream when it sees a OFF link go back to the user, making it clickable. It would be good to hear what you think about it. I posted info in the freenet section here.

It would also be good to hear your opinions on putting a date stamp into the OFF URL. I could pull that out and send it to the user when I do the freenet conversion thing. I'm planning on pulling the filename and size from the URL so it makes a nice looking html page using only the info from a simple list of links.

8
OFF / Re: Experimental UDP code is working!
« on: September 07, 2008, 08:00:09 AM »
From a number of things I've read online the problem may be the age of the MSVC6 libraries. It might help to take the VS2005 files for winsock2 and ws2tcpip.

It also looks like the winsock2.h call should come before windows.h, and ws2tcpip.h is suppose to be called from within winsock2.h so I don't understand why it's in the source files.

Are you sure it's a good idea to be using all those other old libraries and a old compiler?

I am still thinking it would be better to use open source tools somehow so that all the code used and the compiler are up to date.

I know you mentioned some bugs, is there a chance that those have been fixed by now?

9
OFF / Re: Experimental UDP code is working!
« on: September 04, 2008, 05:56:00 PM »
Off the top of my head, whenever you get into this sort of problem, go look in the VS2005 libraries or even the Linux ones and steal what you need in little bits.

Make sure you put these additions into the code in some way so that other people can compile from a standard install of MSVC. And whatever settings you had to use for MSVC.

I hope you see that I recently took the whole UDT code and placed it into two files to hopefully make this simpler. The full patch for V19.28 and mushing it all together with your new changes are posted to SF. Just remember that you have to re-compile offcnxn.cxx to get the udt stuff to re-compile. On Linux you just make it think you changed something.

Google says.... (with keywords "ws2tcpip.h addrinfo")

typedef struct addrinfo {
  int ai_flags;
  int ai_family;
  int ai_socktype;
  int ai_protocol;
  size_t ai_addrlen;
  char *ai_canonname;
  struct sockaddr *ai_addr;
  struct addrinfo *ai_next;
} ADDRINFOA,
 *PADDRINFOA;

"The addrinfo structure is used by the getaddrinfo function to hold host address information"
http://msdn.microsoft.com/en-us/library/ms737530(VS.85).aspx

I think the above will just work since they just fill in stuff after zeroing the whole struct.

I think that header is the same for Linux, but I didn't go looking.

But... " 'getaddrinfo' : undeclared identifier" shouldn't happen, it should be included in #include <winsock2.h> file, it should be in there so somethings wrong here.
http://msdn.microsoft.com/en-us/library/ms738519(VS.85).aspx

Are you sure you are defining WIN32 along with your normal define of _WIN32 ?

With the new patch I define it at the top of the file and don't have to worry about it anymore.

Oops.. I see that I forgot to define it in udt.h

You know what, just change the top part of offcnxn.cxx where it defines stuff for windows

#if defined(_WIN32)
  #define WIN32
  #include <winsock2.h>
#else

That should carry into both udt.h and udt.cxx


10
OFF / HOWTO: Post and use OFF URLs on Freenet for even more anonymity
« on: September 04, 2008, 03:43:45 AM »
The complete info is posted to the Freenet section:
http://board.planetpeer.de/index.php/topic,5241.0.html

I am currently working on a simple interface for the OFF client side that will let you just click the OFF links on a Freenet web page to start a download. Right now it's a temporary copy/paste due to the Freenet proxy filtering html links (and it will continue to do so to keep users safe).

If someone wants to write a Freenet plugin for this, please do, I'm only going write the OFF client side one.

11
You will be running OFF and Freenet at the same time.

It's simple, use Freenet anonymous web pages to select from a list of OFF links, use OFF to download the files.

The advantage is the ability of OFF to swarm on popular content and use direct downloads, and Freenet's ability to keep the poster of OFF file links anonymous.

It's a great combination, Freenet web pages (especially small simple ones) load quickly with little bandwidth used, and OFF can swarm popular blocks for fast downloading.

Some may say this is crazy because the downloads won't be anonymous, but the OFF System makes it possible to transfer file blocks in the open, directly from node to node without worries, since OFF blocks are just random data bytes belonging to no one.

To download a file you copy/paste the OFF link into the same browser you are using for Freenet, and the link is sent locally to the running OFF client and starts the download. (it will get even easier soon, see below for more details)

Uploaders use the OFF client to upload their files then create a very basic, simple and fast loading Freenet web page containing the links to the OFF stored content.

There is no real setup needed for anyone to start downloading today, just run both Freenet and OFF then go look for OFF link pages on Freenet. Then copy/paste the OFF link into a your browser's address bar.

I am currently working on a simple interface for the OFF client side that will let you just click the OFF links on a Freenet web page to start a download. The reason for temporary copy/paste is due to the Freenet proxy filtering html links (and it will continue to do so to keep users safe).

To be safe before the new interface is created, make sure the URL you picked starts with http://localhost:23402/ or http://127.0.0.1:23402/ so it only goes to the local OFF client. The OFF client checks the rest or the URL to make sure it's valid before using it.

To create and upload a web page to Freenet (aka "freesite") containing OFF links, you use a program called "Jsite" to make it simple and allow you to later update your anonymous freesite when you add more OFF links.

It will only take a few clicks and a little editing to add links after you first create your freesite.

Once your site is uploaded to Freenet, it's available 24/7 even when you aren't running Freenet, and will continue to exist as long as it's being accessed regularly.

Download Jsite from http://freenetproject.org/jsite.html and place it in the install directory for freenet.

Follow the instructions on that link to create your new site.

Yea, it looks a little complicated at first, but just try it, the Jsite program does most of the work and the process is actually really simple.

You can use the simple example below to create your site. Using a text editor (windows users can use Notepad), create a new "index.html" page and save it somewhere the Jsite program can get to. It's probably best to make a new folder for it.

Code: [Select]
<html>
  <head>
    <title>Joe's OFF Links</title>
    <meta name="description" content="OFF Links, Files, Movies, People Walking"/>
  </head>
 <BODY>
<br>
I have some cool files for you!
<br><br>
Sept 3, 2008 - Size 17K, gpl.txt
<br>
http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)
<br><br>
Sept 2, 2008 - Size 2K, something_else.txt
<br>
http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)
<br><br>
Sept 2, 2008 - Size 1M, people_walking.avi
<br>
http://localhost:23402/offsystem/v1/video/avi/(the_rest_of_the_OFF_URL_here)
<br>
 </BODY>
</html>

You can preview your page at any moment, just double-click the “index.html” file, and it'll show up in your browser. Then just hit refresh after making changes.

The OFF links will not be clickable on this page when you preview it. That is correct and it will remain that way. When the new interface is done they will be detected due to the "offsystem" reference and converted on the fly to be clickable.

Posting your URLs like this may take a bit more effort, but it keeps you safer as a uploader and makes it really easy for others to use.

Once you save your new page, follow the Jsite instructions and "insert" the page to Freenet. Jsite will save the freesite's info and let you update it easily when you want to add more OFF links.

You want to keep your freesite as small as possible, and not include any fancy things or images so it loads as fast as possible over Freenet.

Swarming of new and popular content happens because lots of people are trying to get the same blocks at the same time. So remember to add the file size and date as I did in the example so that people will know what links are older and how big the download will be.

The "<br>" is a line break, you can add those as needed.

Now that your freesite is inserted, it's time to make it known, by announcing it to different places.

There are several ways to get your freesite known by the freenet community, some of them are:

• Posting it's key (the whole “USK@[...]/word/1/”) to Frost or FMS forum boards.
• Posting it to an index within Freenet (a freesite listing other sites, some of them may reside on your Fproxy page).
• ???

You could also reference other OFF link freesites on your page.

You can throttle down your Freenet connection, see the Freenet Configuration page for your node, and small freesites like these will still load in a reasonable time.

Watch for updates about the new OFF interface to Freenet I'm working on. If someone wants to write a Freenet plugin for this, please do, I'm only going write the OFF client side one.

One technical detail, the interface will force the URL to be sent to the OFF client's local port only, so it should be safe for users to just click, even if the link tries to point to a outside address.

Links:
OFF Client http://offsystem.sourceforge.net/
Freenet http://freenetproject.org/
Jsite info http://freenetproject.org/jsite.html


12
OFF / Re: Experimental UDP code is working!
« on: September 01, 2008, 09:34:35 PM »
The new UDT code is ready and tested for a short time on 3 nodes. See the first post of this thread for more info, I just updated it.

It would help if everyone who has a different style network tested this even for a short period to see how well it works. If you have access to a Wifi mesh, NAT routers inside of NAT routers, or other interesting setups, please give it a try.

If two nodes know their NAT IP (what may be called the "outside" IP) and the port that they have opened for incoming connections, if they both try to connect to each other within 30 seconds it may work, "punching a hole" through both NATs. It may take 3 nodes, one simple one that can answer back so the NAT opens up, then the other NATed node may be able to get through and the 3rd node could go away.

Tech details:

It's patched against V0.19.27 and should patch to .28 OK if there haven't been too many changes in those files.

I am using the global variable again so it's still a UDP only or TCP only setting, user selectable in the config file.

I suggest to just leave it for now until the other support code is finished.

The UDT code is in one big file, I suggest to leave it as-is and spend time on something else because it probably won't change for years. No need to spend time breaking it up into little files.

It's a compromise, everyone gets reliable UDP now, the devs get to go on to more important stuff.

I forgot to mention that the other end has to be in UDP mode too so there's really no chance of splitting the network with this test code.

The "#ifdef UDP_ON" should now be used while developing the code to combine and test UDP and TCP together. It shouldn't turn off this test feature if used properly.

The "IsUDP()" thing caused segfaults first thing so I exchanged them for the global. It's headed in the right direction and so is a lot of the other code, but I couldn't easily figure out why. The global is set once so it doesn't create a thread safe problem. It's easy to find/replace it anyway.

UDT is already thread safe and C++ so no need to wrap it further.

I tried to make it as windows ready as possible, there will be problems I'm sure, and it doesn't look like I'm going to be able to help much since I can't cross compile it yet.

Some todo items:

Need a date stamp in the URL :)

Obvious GUI changes and code to support both UDP and TCP together, let the user choose which one(s) to use.

IP address filters that let you block or allow address ranges, for situations where you want to keep OFF from connecting to the internet and stay inside a LAN or connecting to other subnets on the same LAN network. Or become a in-between node that is like a portal to the internet so only one node is seen going outside (think dorm room, fast LAN or Wifi, slow internet).

A setting with speed levels that changes internal delay settings so that a network could be completely saturated with random block pushes.

A button that sets everything to default settings for a LAN and back again.

Improve the ability to punch through a NAT and stay that way.

Go through all the code, new and old, with some tool that looks for buffer overflow problems just to be safe.

13
OFF / Re: Experimental UDP code is working!
« on: September 01, 2008, 01:30:25 PM »
This is now working, compiled and tested on two nodes for a short period, I took all the UDT code and stuffed it into two files

udt.h
udt.cpp

Then included them in offcnxn.cpp and it compiles and works with no modifications to the original makefile so it should compile on windows with MSVC6 or whatever compiler. No DLL's or linking this way!

I never got OFF to compile under cygwin properly so I dropped that idea.

It's using V.19.24 right now, I'm working on patching it into the latest version. Watch for another post soon.

The devs in the future will need to watch the UDT site's changelog once and a while. If there are any *security* bug fixes to UDT they need to be made to that section of these files.

I included references to each code section's old file name so it won't be too hard.

I don't think any further improvements to UDT that the original UDT team does to the code should be included in the OFF code unless it's something really, really important.

The code is now part of the OFF code, just like as if I wrote it myself and provided it to the project. The author's credits are included in the source files as per their requests.

The one thing I would like to see stay is the ability for people to just change the config option so users can test this now on windows or linux (UDP only on/off) until the code is finished that makes this work with both UDP and TCP together.

It's experimental and just for testing in special situations, not to split the network (and it doesn't work with any existing nodes anyway).

We need more testing on this with different network setups so any bugs or tweeking can be worked out before the other code is finished that does both UDP and TCP.

14
OFF / Re: Experimental UDP code is working!
« on: September 01, 2008, 04:31:38 AM »
> I try to minimise my interaction with the MS IDEs

Yea, that's why a script that does it all automatically would be the best. If I have to copy/paste commands into a Wine emulated window that might not be too bad.

I am hoping that cygwin will allow a build without any dll's, but you already need one (MSVCP60.DLL), so adding another to the install script and dumping it into the OFF directory shouldn't be a problem.

It looks like "minggw" lets you cross compile but uses a dll at run time. I'm not 100% up on that so the info I read may be wrong.

One other choice is where you build a cross compiler somehow from GCC source code and it does it directly in Linux. That's the way it should be done, then I use wine only to test the .exe file.

Then of course something has to be figured out for building a installer. If that could be done on a command line...

Thanks for the heads up on the unicode thing, wxwidgets has --enable-unicode as a option and I think it's typically on so I don't know what's going to happen yet.

What happens to OFF if you turned off unicode in wxwidgets? Is it used for something specific?

As for the socket problems, it may be possible to use some code that's not a cygwin supplied thing. The problem would be testing, I'm bet it's one of those random problems that shows up after 50 or more people test it on different systems.

I don't understand the difference between VS2005 and MSVC6 or how it does a makefile type of thing, but I think it would be nice to use all open source and cross compile to build this thing someday, even for the mac.

Would it be possible to build the UDT dll with VS2005 and then link it in using MSVC6? Since the UDT code wouldn't change often the dll would stay the same between versions, that might be a way to go. MSVCP60.DLL is from 2004.

And I agree, UDT should work on all supported platforms. The only other last ditch option we have would be to take the important parts of UDT and try to just stick them within the OFF code file, if the license permits. It's all just C++ code and all the problems we are having right now are because of makefiles.

Maybe I should just do that and be done. Put their little notice at the top of where the code starts and cram it all into offcnxn. Then it has to work on mac too, right? Boy, that would be one long file.

15
OFF / Re: Experimental UDP code is working!
« on: August 31, 2008, 06:20:29 PM »
OFF runs under Wine in Linux!

"HOWTO: Run OFF Under Wine in Linux (windows emulator)"
http://board.planetpeer.de/index.php/topic,5221.0.html

Not sure why "MSVCP60.DLL" isn't included in the installer. And it didn't run under Win98 emulation, it popped up a notice saying it won't run (it wasn't a Wine notice). The installer runs under Win98 OK. OFF wants XP.

Pages: [1] 2 3 ... 6