Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - scripter

Pages: [1] 2
1
OFF / Is this the URL that crashed my client?
« on: October 05, 2008, 09:58:40 AM »
I found this link on a web site, clicked it and my client crashed, or stopped, I'm not sure which one.

The link said "people showing things you shouldn't see, click here", so I clicked on it, I couldn't see the underlying actual link, or wasn't paying attention because there were a lot of other OFF links on the page.

I was running the latest version, 0.19.32

The real, actual underlying link was.......

http://localhost:23402/index.htm?shutdown

:)

While we're on the subject, I also clicked on a search result with a filename of

"\Windows\System\some_important_file.dll"

No one could ever fake search returns, right?

"we can avoid the expense of checking the filenames of srch results"

Not a good idea.

2
OFF / HOWTO: Test many OFF nodes on a single Linux system
« on: September 10, 2008, 02:08:34 AM »
The scripts below create new directories for each test node, including all the config options and proper port settings so you can test as many OFF nodes on one system as you can handle.

Communications are over the local interface (lo) of course, which can be monitored with tcpdump if you like.

You must compile OFF with the option RELEASE_ON disabled (see offconstants.h at about line 335 and comment it out) then it doesn't create a lock file for each instance.

Create a new directory and put a default OFF setup in there, including the OFF executable, config, themes and other folders.

Put the two scripts in there and chmod them so they are executable.

Open a terminal and cd to that directory. When you run "./go-test.sh 4" it will create 4 new directories named "1", "2", etc.. and copy the executable and config files over, changing the ports and node name in the config file accordingly. It also adds nodes to connect to within the test range.

When you are done you can use "cleanup.sh 4" to kill OFF and remove the test directories "1", "2" etc..

"go-test.sh"
Code: [Select]
#!/bin/bash

# Script for testing lots of OFF nodes on the same computer
# You must compile OFF with the option RELEASE_ON disabled
# (see offconstants.h at about line 335 and comment it out)
# then it doesn't create a lock file for each instance
# Place a default OFF setup in the working directory, including
# the OFF executable, config and themes folders
# Written by Scripter Sept, 2008

make_nodes=$1

if [ "$make_nodes" == "" ]; then
echo "Usage:  go-test.sh NUMBER (of nodes to make)"
exit
fi

# set this if you only want each node to connect to the next node at
# startup, otherwise each node will try to connect to all other nodes
connect_to_next_node=0

# setup variables we will use
node=1
port=23501

while [ $node -le $make_nodes ]; do

echo "Copying files for Node $node"
if [ ! -d $node ]; then
mkdir $node
fi

cp -a config $node/
cp -a blockcache $node/
cp -a retrieved $node/
cp -a pads $node/
cp -a lang $node/
cp -a keys $node/
cp -a themes $node/
cp -a temp $node/
cp -a offsystem $node/

# remove config lines we want to change, then put them back with what we want
# OFF wants each line to be exactly in order for whatever reason
cat $node/config/offsystem.conf | grep -v "verbose" | grep -v "trusted_search" \
 | grep -v "username" | grep -v "incoming_p" > $node/config/offsystem-temp.conf
echo " verbose: 1" > $node/config/offsystem.conf
echo " username: \"Node-$node\"" >> $node/config/offsystem.conf
echo " trusted_search_in: 0" >> $node/config/offsystem.conf
echo " trusted_search_out: 0" >> $node/config/offsystem.conf
echo " incoming_port_number: $port" >> $node/config/offsystem.conf
cat $node/config/offsystem-temp.conf >> $node/config/offsystem.conf

# connect to the next node, last node gets connected
# to 23403 for a special running node you may have
if [ $connect_to_next_node -ne 0 ]; then
  if [ $node -ne $make_nodes ]; then
    port1=`expr $port + 1`
    echo "localhost $port1" > $node/config/known_nodes.txt
  else
    echo "localhost 23403" > $node/config/known_nodes.txt
  fi
else
  # each node tries to connect to all other nodes at startup
  node1=1
  port2=23501
  echo "localhost $port2" > $node/config/known_nodes.txt
  while [ $node1 -lt $make_nodes ]; do
    port2=`expr $port2 + 1`
    echo "localhost $port2" >> $node/config/known_nodes.txt
    node1=`expr $node1 + 1`
  done
  echo "localhost 23403" >> $node/config/known_nodes.txt
fi

# remove the rsa keys, will be regenerated
rm -f $node/config/off_rsa.pub
rm -f $node/config/off_rsa

node=`expr $node + 1`
port=`expr $port + 1`
done

node=1

while [ $node -le $make_nodes ]; do

echo "Starting Node $node"
cd $node

# my usual run with a terminal for each node
gnome-terminal --title=Node-$node -x ./offsystem &

# just run it
# ./offsystem &

# with gdb, you should killall offsystem; killall gdb before using cleanup.sh
# gnome-terminal --title=Node-$node -x gdb -ex run offsystem

cd ..

node=`expr $node + 1`
done

exit

"cleanup.sh"
Code: [Select]
#!/bin/bash

# Script for cleaning up after using the OFF go-test.sh script
# It will kill OFF and remove the directories
# Written by Scripter Sept, 2008

make_nodes=$1

if [ "$make_nodes" == "" ]; then
echo "Usage:  cleanup.sh NUMBER (of nodes to cleanup)"
exit
fi

node=1

echo "killall offsystem"
killall offsystem
echo "(giving it a little time...)"
sleep 10

while [ $node -le $make_nodes ]; do

if [ -d $node ]; then
echo "Removing folder $node"
rm -fr $node
fi
  node=`expr $node + 1`
done

exit


Note: On Ubuntu you should install "gnome-terminal" so this works properly.

3
OFF / HOWTO: Post and use OFF URLs on Freenet for even more anonymity
« on: September 04, 2008, 03:43:45 AM »
The complete info is posted to the Freenet section:
http://board.planetpeer.de/index.php/topic,5241.0.html

I am currently working on a simple interface for the OFF client side that will let you just click the OFF links on a Freenet web page to start a download. Right now it's a temporary copy/paste due to the Freenet proxy filtering html links (and it will continue to do so to keep users safe).

If someone wants to write a Freenet plugin for this, please do, I'm only going write the OFF client side one.

4
You will be running OFF and Freenet at the same time.

It's simple, use Freenet anonymous web pages to select from a list of OFF links, use OFF to download the files.

The advantage is the ability of OFF to swarm on popular content and use direct downloads, and Freenet's ability to keep the poster of OFF file links anonymous.

It's a great combination, Freenet web pages (especially small simple ones) load quickly with little bandwidth used, and OFF can swarm popular blocks for fast downloading.

Some may say this is crazy because the downloads won't be anonymous, but the OFF System makes it possible to transfer file blocks in the open, directly from node to node without worries, since OFF blocks are just random data bytes belonging to no one.

To download a file you copy/paste the OFF link into the same browser you are using for Freenet, and the link is sent locally to the running OFF client and starts the download. (it will get even easier soon, see below for more details)

Uploaders use the OFF client to upload their files then create a very basic, simple and fast loading Freenet web page containing the links to the OFF stored content.

There is no real setup needed for anyone to start downloading today, just run both Freenet and OFF then go look for OFF link pages on Freenet. Then copy/paste the OFF link into a your browser's address bar.

I am currently working on a simple interface for the OFF client side that will let you just click the OFF links on a Freenet web page to start a download. The reason for temporary copy/paste is due to the Freenet proxy filtering html links (and it will continue to do so to keep users safe).

To be safe before the new interface is created, make sure the URL you picked starts with http://localhost:23402/ or http://127.0.0.1:23402/ so it only goes to the local OFF client. The OFF client checks the rest or the URL to make sure it's valid before using it.

To create and upload a web page to Freenet (aka "freesite") containing OFF links, you use a program called "Jsite" to make it simple and allow you to later update your anonymous freesite when you add more OFF links.

It will only take a few clicks and a little editing to add links after you first create your freesite.

Once your site is uploaded to Freenet, it's available 24/7 even when you aren't running Freenet, and will continue to exist as long as it's being accessed regularly.

Download Jsite from http://freenetproject.org/jsite.html and place it in the install directory for freenet.

Follow the instructions on that link to create your new site.

Yea, it looks a little complicated at first, but just try it, the Jsite program does most of the work and the process is actually really simple.

You can use the simple example below to create your site. Using a text editor (windows users can use Notepad), create a new "index.html" page and save it somewhere the Jsite program can get to. It's probably best to make a new folder for it.

Code: [Select]
<html>
  <head>
    <title>Joe's OFF Links</title>
    <meta name="description" content="OFF Links, Files, Movies, People Walking"/>
  </head>
 <BODY>
<br>
I have some cool files for you!
<br><br>
Sept 3, 2008 - Size 17K, gpl.txt
<br>
http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)
<br><br>
Sept 2, 2008 - Size 2K, something_else.txt
<br>
http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)
<br><br>
Sept 2, 2008 - Size 1M, people_walking.avi
<br>
http://localhost:23402/offsystem/v1/video/avi/(the_rest_of_the_OFF_URL_here)
<br>
 </BODY>
</html>

You can preview your page at any moment, just double-click the “index.html” file, and it'll show up in your browser. Then just hit refresh after making changes.

The OFF links will not be clickable on this page when you preview it. That is correct and it will remain that way. When the new interface is done they will be detected due to the "offsystem" reference and converted on the fly to be clickable.

Posting your URLs like this may take a bit more effort, but it keeps you safer as a uploader and makes it really easy for others to use.

Once you save your new page, follow the Jsite instructions and "insert" the page to Freenet. Jsite will save the freesite's info and let you update it easily when you want to add more OFF links.

You want to keep your freesite as small as possible, and not include any fancy things or images so it loads as fast as possible over Freenet.

Swarming of new and popular content happens because lots of people are trying to get the same blocks at the same time. So remember to add the file size and date as I did in the example so that people will know what links are older and how big the download will be.

The "<br>" is a line break, you can add those as needed.

Now that your freesite is inserted, it's time to make it known, by announcing it to different places.

There are several ways to get your freesite known by the freenet community, some of them are:

• Posting it's key (the whole “USK@[...]/word/1/”) to Frost or FMS forum boards.
• Posting it to an index within Freenet (a freesite listing other sites, some of them may reside on your Fproxy page).
• ???

You could also reference other OFF link freesites on your page.

You can throttle down your Freenet connection, see the Freenet Configuration page for your node, and small freesites like these will still load in a reasonable time.

Watch for updates about the new OFF interface to Freenet I'm working on. If someone wants to write a Freenet plugin for this, please do, I'm only going write the OFF client side one.

One technical detail, the interface will force the URL to be sent to the OFF client's local port only, so it should be safe for users to just click, even if the link tries to point to a outside address.

Links:
OFF Client http://offsystem.sourceforge.net/
Freenet http://freenetproject.org/
Jsite info http://freenetproject.org/jsite.html


5
OFF / HOWTO: Run OFF Under Wine in Linux (windows emulator)
« on: August 31, 2008, 06:13:25 PM »
Maybe you want to see how the other half lives. Maybe you're like me and want to test the windows version after making modifications.

It's amazingly easy. Install Wine if it isn't already, under Ubuntu use the Synaptic Package Manager.

You should now have a menu item for Wine, go to "Configure Wine" and the tab "Applications", at the bottom under "Windows" select "Windows XP" then click OK.

Download the windows OFFSystem-X.XX.XX-win-installer.exe file from the project
https://sourceforge.net/project/showfiles.php?group_id=96735

And place it into your ".wine/drive_c/Program\ Files/" folder. Select Wine->Browse C:\ Drive and then double click on the installer. Just hit OK for all the defaults. OFF should now show up on the Wine menu under "Programs", and also at ".wine/drive_c/Program\ Files/".

It won't run yet, you need "MSVCP60.DLL". You can get it directly from MS but I found a Source Forge project that has it
http://www.kceasy.com/download/
http://www.kceasy.com/update/msvcp60.zip

You can use "Ark" to unzip the file. Place the file "msvcp60.dll" into the ".wine/drive_c/Program Files/OFFSystem/" folder.

You are now ready to go! Select Wine->Programs->OFFSystem->OFFSystem and it should run. You can also click on it in any Liunx side file browser, like Dolphin or Konqueror, and/or you can right click on it and select "Open With->Wine Windows Emulator".

It's strange to see, but there's OFF running in Linux, but it's a windows version. But it's Linux. But....


6
OFF / Why we need reliable UDP now
« on: August 27, 2008, 05:36:25 AM »
It's just starting and it's going to get worse.

ISU Begins Blocking P2P, Launches 'BirdTrax'
http://www.zeropaid.com/news/9720/ISU+Begins+Blocking+P2P%2C+Launches+%27BirdTrax%27

"Illinois State University" .... "has decided to begin prohibiting the use of P2P and file-sharing applications and services on the campus network." (due to the Higher Education Authorization Act)

The Fall semester is starting soon and reliable UDP is needed along with any needed support for Wifi sharing. This is just one reason why we need this now.

The FCC rules for Wifi won't allow universities or other organizations to stop P2P over Wifi.

Universities have no authority or control over the airwaves owned by the People.

They have no cause or jurisdiction to spy on or regulate speed on Wifi networks since stations are mandated to accept any interference from other stations. No excuse that students are interfering with "campus wireless" since the university doesn't own the airwaves.

OFF V.19.24 has reliable UDP now. Anyone can apply the patch. It's tested and working and includes code that has been working since 2001 (from the University of Illinois no less).

The UDP code has to be switched on, so it doesn't interfere with normal TCP operation and UDP is perfect for a Wifi sharing network.

We don't have time for "not invented here" politics. The reliable UDP code (UDT) is written in C++, which always promised re-usable code. Making C code more like modules you can plug in to add features to software instead of taking programmer's time re-writing code that already functions.

If a C++ module is documented well a programmer should be able to just use that code without worrying about what goes on inside, saving countless hours of programming and allowing the programmer to work on other more important tasks.

We need encryption from node to node to be reliable for Wifi. The UDT code is robust enough to enable this, it's been tested and bugs have already been worked out.

The UDT code can be "frozen" at this point and only critical changes to the original UDT code would be needed in the future. There may be a worry that some sort of change in the UDP packet format may disable older versions but this is a small possibility since the interface is already working and tested on many systems around the world.

OFF is slow on a LAN connection, that needs to be improved if this is going to become popular on campus LAN or Wifi networks. Work needs to be done on other areas of the code too, not re-writing things that already exist and are working today.

If you people out there want UDP and Wifi support, you better start chiming in here.

Here's the patch and the simple instructions on how to patch the code
http://sourceforge.net/tracker/index.php?func=detail&aid=2054407&group_id=96735&atid=615726
(see bottom of that page for file)

7
OFF / Change in V.19.27 and it's effects on swarming
« on: August 27, 2008, 04:46:18 AM »
Changelog V.19.27 "A block is now only requested from one node at a time, it's request timer is frozen for the duration of the request."

This may cause problems with the swarming effect when a popular file is uploaded to the network. To help with swarming of popular content, more nodes need to get the requests so that they contact other nodes for the block and so on down the line.

It may be better to pick a random number of nodes to send requests to the first time and if a block download doesn't start after a time period, pick again from the remaining nodes. You could also use a smaller percentage of nodes on the first try, limiting the max random number to say 30% of the total connected nodes for the first try.

And we still badly need a date stamp in the URL to help with swarming (date only to the day).

"Relayed blocks are now processed before download blocks when requesting from other nodes. Since relayed blocks are generally few, this should not impact download performance too much."

This will help swarming but "relayed" blocks won't be few so download performance will be affected.

For now you may want to simply flip a coin or skew a random number towards the side of picking download blocks over "relayed" blocks.

There's a lot that could be done here, like waiting a little bit to see how many requests you get for one block and then pick the popular blocks to make onward requests for first, then request the other blocks as time permits.

Since two heads are better than one, why not post first about changes like this and see what feedback you get from everyone?

Also, there should be two settings for updates, one is something like "bleeding edge" for people who want to help test the latest versions and "stable" for normal users. The big reason for this is that some changes could slow or shut down the network.

8
OFF / HOWTO: Place and use OFF URLs on I2P for even more anonymity
« on: August 24, 2008, 08:18:24 AM »
You will be running OFF and I2P at the same time.

When you are done with the setup and you want to download something you will use I2P to go anonymously to a webpage which lists a lot of OFF URLs, you simply click on one and that will cause OFF to start the download automatically.

The OFF download will be faster than using I2P to download the file because there's no need to be anonymous anymore during the download of the OFF blocks.

When you want to upload a file to OFF, you do it normally but you place the OFF URL on a I2P webpage so others can access it anonymously via I2P.

The setup:

Install I2P, run it, set it as you like and get used to how it works. http://www.i2p.net/

For the best security you should use a different browser than you normally use, like use Opera instead of Firefox. This way it's easy to tell that you are using I2P and the browser can be setup to only use the I2P proxy for everything, except localhost (127.0.0.1) of course. Make sure you turn off javascript, java and plug-ins (under Tools->Preferences->Advanced->Content, also Network->Proxy). Set everything to proxy 127.0.0.1 port 4444 (see the I2P docs for more info).

I2P calls a webpage a "eepsite".

You can probably figure out how to find a OFF URL eepsite yourself, then you just click on what you want and OFF should start the download. Look on the Downloads page on your OFF client.

To place your uploaded file URLs on I2P you may be able to post URLs in the future to an existing eepsite made to post OFF URLs, use search on I2P to see if there is one.

If you want to make a eepsite yourself on I2P and place your OFF URLs in the page you make a very simple, standard html file (text format) and place it into the I2P directory eepsite/docroot .

To see your current eepsite and some info, go to http://localhost:7658/

The first thing to do is rename eepsite/docroot/index.html to help.html so you can access it later. Don't delete it.

Using a text editor, create a new index.html page, you can use the html code below and edit as you wish. This is a simple page that makes the URLs clickable from your I2P eepsite.

When someone clicks on a URL it will use the OFF loopback server on port 23402 (localhost) to start a download of the file from within the OFF client.

The link would be like this example for "gpl.txt":
gpl.txt

If you are running OFF right now, click the link above and see if a download is started in your OFF client as a test.

Code: [Select]
<html>
  <head>
    <title>OFF Links</title>
  </head>
 <BODY>
<br><a href="http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)">gpl.txt</a> (Size 17K, posted 08/2008)
<br><a href="http://localhost:23402/offsystem/v1/text/plain/(the_rest_of_the_OFF_URL_here)">silly.txt</a> (Size 2K, posted 08/2008)
 </BODY>
</html>

Posting your URLs like this may take a bit more effort, but it makes it really easy for others to use and may attract more people to using the speed of the OFF System..

Once you save your new page, go to http://localhost:7658/ again and you will see your page.

You are part way done, you need to let others access the page anonymously through I2P by turning on the I2P "tunnel" to your page and letting others know it's there.

The easiest way is to go to http://localhost:7658/help.html and read how to do that, instead of me copying and posting that text here.

You can also read some of the howto's at http://forum.i2p/viewforum.php?f=17 (via I2P)

Remember to add the file size and date as I did in the example so that people will know what links are older and how big the download will be.

You can throttle your I2P connection, but not too much if you want people to get your eepsite faster (it's coming from your computer through I2P). The less "cool stuff" you put on your I2P eepsite the better, no graphics, and no fancy stuff will make it load faster for others.

You could also do the same on freenet. http://freenetproject.org/

Links:
Download I2P http://www.i2p.net/
Another method to create a eepsite http://www.i2p.net/i2ptunnel
Another info post about eepsites http://board.planetpeer.de/index.php/topic,687.0.html

9
OFF / V .19.26 UDP code screwed up, it was working and tested
« on: August 24, 2008, 01:35:48 AM »
Don't blame me.

It doesn't even compile. That means it's not tested.

What I did was completely tested and working on Linux as I said before. It's not my fault.

Maybe there was a timing thing between when I posted the patch with the reliable UDP code (UDT) and it will be included later.

I posted the patches to sourceforge and that should generate a e-mail directly to the devs.

I see two new versions with no UDT, is that the answer to the UDT question?

If there's still a concern about including other's code in OFF (meaning UDT), please take out SHA1, Base64, and all the RSA, AES stuff.

The code I wrote didn't interfere with ***ANY*** other code, I was careful about that. When the "use_udp" variable wasn't set it did *exactly* what it did before, using TCP. The default setting was 0 (disabling UDP mode).

I spent a lot of time testing the code I made. That's all in the trash now, why?

With UDT you would make a big leap forward into reliable UDP, today, now, with little time spent on trying to develop the same thing yourself. (why do I have to try to promote this anyway? It just makes sense)

UDT has been tested and in use since at least 2001. Bugs have been fixed. The interface to it is defined in documentation so that shouldn't change. You can standardize on this one version of code and never change it again if you want, leave the code as is forever if you want, it works great. It does everything we want it to do.

You, the users, can still do reliable UDP with V.19.24 and my patch if you want to see if it works on your mesh, local LAN network or whatever. V.19.24 still shares files OK and could probably be used for a local sharing situation for the next year if you wanted to.

It's a little frustrating at this point, I was going to do more work on this and other improvements to OFF but now I've stopped, I hate wasting my time.

10
OFF / Experimental UDP code is working!
« on: August 10, 2008, 07:03:36 PM »
The new experimental UDP code is working and tested on Linux. For windows, let me know how it goes.

EDIT: Sept 01, 2008 - Updated this post for the new reliable UDP code for V0.19.27.

Why UDP? It may solve some problems for people who have different network setups. Like getting around some NAT routing situations, ISP blocking, Wifi Mesh networks or LAN sharing.

Thanks to the team at the University of Illinois, we can now have reliable connections via UDP. They created some C++ code called "UDT".

If you don't want to wait to see if it gets included in the next release you can compile it yourself, I posted a patch file at
http://sourceforge.net/tracker/index.php?func=detail&aid=2087537&group_id=96735&atid=615726
(see bottom of that page for file)

To patch V0.19.27, make a new directory called "test1", download OFFSystem-0.19.27-src.tar.gz and the patch file into it, like this...

cd test1
tar -xzvf OFFSystem-0.19.27-src.tar.gz
patch -p1 < version_diffs.patch

Then download the two other files "udt.cxx" and "udt.h" that are on these patch pages.
http://sourceforge.net/tracker/index.php?func=detail&aid=2087541&group_id=96735&atid=615726
http://sourceforge.net/tracker/index.php?func=detail&aid=2087546&group_id=96735&atid=615726

Download them to off_network/src/ then do the normal compile...

cd off_network/build_gtk
make
(then do whatever you normally do with a new compiled version)

The OFF client will operate normally via TCP unless you change one configuration option.

When the program is not running, in the "config/offsystem.conf" file, near the end, edit (or make) the use_udp option to enable UDP.

 use_udp: 1

You should also rename the "config/known_nodes.txt" file, it contains the IPs of other nodes on the normal OFF network. Renaming it lets you put it back when you want to go back to normal.

You want to start with no "config/known_nodes.txt" or a empty file.

Start the OFF client from a terminal and you will see a lot of debug messages about UDP, showing socket open and a few other messages. UDP is working if you see those messages.

Go to the "Known Nodes" screen and right click then select "Add Node". Type in the other node's IP (and port if different) and a connection should start in about 5 seconds. Both nodes need to be in UDP mode of course.

This code will handle multiple node connections, so have a party and test like mad!

If the other node has no blocks in it's cache then you will see messages and network activity as it pushes randomly selected blocks to the other node.

Have fun and please post on how your test went.


Some things that aren't working yet in the new test code:

No combo TCP and UDP, just one or the other.

Loopback/web server doesn't work since it would be a UDP socket.

WARNING, privacy risk if you are only running two nodes. Don't post URLs that you don't want others to know you have since it's easy to know who has what URL.

Technical notes:

The largest packet I've seen sent is 1040 Bytes so no worries about exceeding the normal MTU so far. However, the IP layer is suppose to break up any packets and handle it if necessary and on Wifi there's more code I'm sure.

TCP and UDP are both a little slow with no throttling set on a local LAN. I'm not sure if it's something I am doing wrong but the CPU usage is low so it isn't processing much. I will be looking into some of the places in the code there could be delays so don't get too worried yet.

Wild ideas:

It may be possible to get around some NAT routing problems by initiating a UDP port scan to some destination from a fixed local port. Wait for one response packet. Keep the UDP hole open by sending UDP packets at a low rate to the remote port that responded. You may now have a open UDP port to the world.

Other clients can do the same (possibly port scan you until they find your open OFF UDP port) and then the connect may go through.

You may be able to forge the sending IP so the other end thinks it's coming from the local or other network. That may take two "connections" to work and both ends would probably have to know what is going on.

You could have "fake" packets that look like they are from other nodes or IP addresses but actually do nothing. With encryption on this would be hard to sort out for anyone "listening in".

It may be possible to add FEC to each packet so that single bit errors are corrected on the fly.

If you have problems with the patch, I posted this file just in case
http://sourceforge.net/tracker/index.php?func=detail&aid=2087553&group_id=96735&atid=615726


11
I've been thinking about how anonymous OFF really is. But before it gets popular, let's shoot holes in the current system and if we find something, see if there's a way to fix it.

I'm concentrating mostly on search and URL anonymity, the system to request transfer of blocks is randomized and that's probably good enough for that side.

The OFF network by nature sends search terms to some directly connected nodes. Those nodes in turn request the same search from some of their directly connected nodes and so on.

When a node receives URLs for a search, it stores them in a temporary buffer, takes out duplicates and combines the results from many nodes together before returning them to the requester. It also keeps that list of URLs for a while and may return them for other searches.

The anonymity of this process happens because you can't tell if a node is returning it's local list of URLs or a list that was received from other nodes.

You also can't tell which node originated the search. In the nature of the OFF system, every node in the path originates the same search.

If you try to look at the timing, saying that the fastest search return is the node that has those URL's locally, that's basically true!! The node does have those URL's in it's local memory, but again, by the nature of the system a node holds URLs sent from other nodes in it's memory for a undermined temporary length of time so those URLs aren't necessarily local.

One attack that is simple to prevent is when a node goes offline, you keep hitting it over and over with a special program until it comes back online, then send it a search and see what URLs come back instantly.

The way to prevent that is to add a simple start up delay for returning local URLs for a search request, or to simply wait until you have some URLs from other nodes in memory.

It's also hard to prove if you are really the first node to connect and if the node hasn't already connected and talked to other nodes before you. Or even better, prove that your connection to that node wasn't simply blocked for a while due to internet network routing problems in your area and in reality the node was up the whole time.

Another attack would be to completely surround a node. Now you know exactly what data goes in and out. With enough resources and possible cooperation by your ISP it could be done. But that doesn't prevent connections via Wifi from your nearby friends, or possible drive by sharers who are "borrowing" your internet. So always have a Wifi router handy and your port open :)

One other topic for discussion could be the random block and multi use principals.

In my personal opinion, the principals are sound. Reuse of blocks is key. The more blocks in a large file that are reused from other files the better.

The principals haven't been tested in a legal setting yet but we know one thing, bittorrent users, trackers and gnutella users are being harassed left and right, those systems give you no anonymity or plausible deniability at all.

Ideas?

12
OFF / OFF can be as fast and popular as bittorrent
« on: August 01, 2008, 03:30:52 AM »
The OFF network can be as fast as bittorrent when downloading popular content. If someone uploads a popular file and enough people know the URL is available, a "swarm" of nodes holding the blocks needed to download will form and speeds will increase as more nodes can provide blocks.

OFF is a multi source downloading protocol, just like bittorrent. When a node downloads a block it's also made directly available to other nodes, just like bittorrent.

Since OFF blocks are just random data there's no need to pass blocks *through* other nodes to gain anonymity, so the transfer speeds can match the outgoing speed of a node (with a small loss for overhead).

OFF is actually better than bittorrent since it doesn't have the single point of failure problem, a tracker.

OFF is more anonymous and protects it's users way better than bittorrent does.

The current problem keeping OFF from becoming popular is that it's hard to tell which files have been recently uploaded and which ones are popular.

On bittorrent you can see which files are popular by the amount of "seeders" and "lechers". The file with the most seeders will probably be the fastest download at that time.

Knowing that a OFF URL has been recently uploaded can be done with a simple date stamp in the URL as I've mentioned before.

This should be done ASAP even if it's not yet supported in search functions so that newer version URLs can start to appear on the network.

Determining which files are popular will probably be left up to the user who sorts search lists by date and can use his knowledge of file names.

The OFF client can help by checking slowly in the background how many local blocks are available in the blockcache for each URL returned in a search. This could be a user selected option since it will create some CPU overhead. The option should include an adjustable speed value.

It's not a prefect answer but if a file is popular your node probably got requests for the blocks and that causes your node to try to get those blocks and store them in the cache.



13
Anonymous Filesharing FAQ / HOWTO: Filesharing via Wifi
« on: July 25, 2008, 02:11:17 AM »
With ISPs throttling and agreeing with big corporations to punish people who use the bandwidth they are paid for, you may want to know more about filesharing over Wifi.

The good news? P2P over Wifi is faster, a lot faster.

Wifi upload speeds are *not* lower than download speeds.

Wifi avoids corporate ownership and other BS. You can do it yourself.

Universities and other organizations don't own the Wifi airwaves and can't regulate them.

You can connect to others up to 16 miles away with an external antenna!

How can I share files over Wifi? Is that a Frequently Asked Question? Let me answer:

In this HOWTO, you need a friend or friends nearby, 2 wireless routers or interfaces, patience, a little persistence, and a P2P program that allows *wired* LAN sharing.

A off the shelf Wifi router or USB device can create a simple network called "adhoc". This kind of network is simply an extension of your home wired network.

You will need to know a few things, like how to get to the settings screen of your router or device.

Most routers, such as the Linksys WRT54G, use your browser to get to the settings screen. You should read the manual that came with it, but most are reached at http://192.168.0.1/ or http://192.168.1.1/ or possibly http://192.168.2.1/ you then enter a default username and password (see the manual).

If you are going the router method, see below for more information on software mods you may need.

A USB device like the Linksys WUSB11 or Netgear has it's own setting screen (see the manual).

At this point you are looking for wireless settings such as SSID and connection type, things like "infrastructure" or "adhoc".

You want "adhoc", and you need to pick a name for the "SSID", it's the wireless network's name, both you and your friend's router/device should have the same name, like "freedom" :)

If you are using a USB device, you need to set a static IP different than the other guy's IP. Keep reading and you will get the idea.

There are several ways to do the next part, but the easiest way with a router is to tell one to start giving out IP's different than the other router.

Look for a setting for "DHCP" or "Network Address Server Settings", look for a setting like "Starting IP Address" which might be set to 192.168.1.100, or 192.168.1.3, something like that.

This is the number where the router gives the connected computer it's IP address when you first turn the computer on. What you want is the other router to start giving out IPs different than the other.

Change one router to something like 192.168.1.110 so now it starts at 110 with the first computer that is connected.

So now one computer will be at 192.168.1.100 and the other at 192.168.1.110 good enough.

Now run your P2P program and give it those two IPs and see what happens.

Simple, fun little network. Tell your ISP to eat dirt!

Want more details? Keep reading and I'll ramble on....

In some cases your local firewall may have to be changed to allow incoming connections from the local network, or on the port that your P2P program uses. Look for posts or documentation on your P2P program about opening firewall ports etc..

If you are still having problems, check the IP address of the computers, check the P2P program settings or try other P2P programs. Not all of them let you connect to 192.168.X.X (local LAN) addresses.

Another thing you may run into is the need for "port forwarding". You can look that up via google if you need the info but in most cases "adhoc" networks aren't using the "NAT translation" routing system and don't need port forwarding.

There are a lot of other things you can try but this kind of network is simple, quick, and uses cheap off the shelf equipment. You can add a number of friends to the network, as long as they can all "see" each other over the wireless. You may be able to add a "range extender" to the network to get more distance or "see" around corners. Be creative.

If you want to use routers you need to find one that supports dd-wrt, see the links below for a list of supported routers. This type of software lets you go "adhoc" with other devices, like a USB device also increase power. Routers let you connect external antennas easily.

USB devices are great because you can put them on USB extender cables and place them in a window, high up or outside. You may be able to go up more than 50 feet!

Don't forget that 2.4 Ghz signals can bounce off of buildings and other objects to get you around obstructions, you may just need a directional antenna to boost the signal.

Bandwidth on Wifi 802.11b is 11M bit/s max. That would be with a strong signal. As you lose signal strength over distance the speed automatically goes down in steps to a 1M bit/s rate.

Start small. It's easier if two friends start this and show two more friends and let them show two more. At some point when things get out of hand you may have to change your network into a "mesh" network or add "bridges" to make the network into sections. If you get to that point, please post and I'm sure there's help out there.

Don't forget there's directional antennas that boost signal, even small ones that don't cost much. Sometimes just moving an antenna to a new position makes the signal stronger. You can avoid interfering signals the same way. Some houses have wire mesh inside the walls, so placing the device in a window can help.

Your friends should all agree to use one P2P program on the network and not use wireless to connect to the internet. One of the friends could agree to run two copies of the P2P program at the same time, one on the *wired* internet to give access to outside content as a sort of "gateway" to the outside world.

On this simple network, anyone trying to use internet over the wireless will not be happy since P2P will flood the thing most of the time. So no wifi internet, sorry, don't ask, get your own. It's up to your group of course, I'm just making suggestions. It's going to take some work to make this ***simple*** adhoc network internet access compatible anyway.

Safety and privacy are concerns for everyone so pick a safe P2P program that gives you safety even on the internet. Why chance it? Planet Peer wiki/forums has a lot of info on your choices.

If you want to test this before you buy wifi routers you can try the same thing with a wired ethernet network, just set static IPs for each computer and set firewalls as needed.

It's possible to use two ethernet cards on one computer so you can have one wired connection to the internet and the other to the wifi router. You may be able to create a "virtual" network interface on the same card and use the same wires but that's another topic for another time.

The reason I picked the Linksys WRT54G is that it has a standard external antenna connections and runs DD-WRT (if you want to increase power output later on), and supports other downloadable programs. Some D-Link models have external connectors too.

Older WRT54G's had more memory and you may be able to find good used ones on the net if you want to run things like DD-WRT. The newer WRT54GL is sold for use with software like DD-WRT. Not all routers support DD-WRT, see below for a list.

Another device is a "Ethernet Bridge", like a Linksys WET200. It's basically a box that extends a wired ethernet network over wireless. Just giving you ideas here.

Please feel free to post your experiences and other useful information I should add to this post or post this to a wiki if you like. (EDIT: Lots of changes since my first post, thanks for the comments)

Answers to other Frequently Asked Questions about Wifi, try these links:
Pics of one type of antenna that can get you 16 miles or more
http://www.radiolabs.com/products/antennas/2.4gig/2.4-aluminum-parabolic.php
ISPs in the UK agree to work with the **AA corporations
http://torrentfreak.com/uk-isps-to-start-sending-mass-080724/
Open source, easy to use replacement software for wifi routers, lets you change output power
http://www.dd-wrt.com/
List of DD-WRT supported devices
http://www.dd-wrt.com/wiki/index.php/Supported_Devices
Creating an ad hoc WiFi network (win XP info)
http://en.kioskea.net/configuration-reseau/creer-reseau-wifi-ad-hoc.php3
WiFi Adhoc Mesh with OLSR (2005 article, look for newer software)
http://www.ccirrus.per.sg/rfc13109/2005/12/wifi-adhoc-mesh-with-olsr/
Freifunk Firmware (for wireless mesh networks and other Wifi info)
http://ff-firmware.sourceforge.net/
Wireless Networking in the Developing World (english, free PDF available)
http://www.lulu.com/content/223168
Mesh - Drahtlose Ad-hoc-Netze (german)
http://www.ixsoft.de/software/products/OS9783937514390BK.html
Google will answer just about anything
http://www.google.com/

14
OFF / Do you have blocks no one needs?
« on: July 24, 2008, 06:40:34 AM »
You can try this test yourself if your blockcache is large enough.

Search for "avi", then "mpg" and even try "txt". Wait for a lot of returns to come back.

Select all the files on one list then right click and select "download". (warning, CPU load goes up)

Now click on the download list button and scroll through the download list and see how many files show that blocks have already been downloaded.

Those are the ones that you have local blocks for. (after that you should select them all and cancel the downloads)

I have accumulated about 5GB of blocks and only a few files showed 1 or 2 blocks.

That's sad. To me it means that most of the URLs returned in search don't have blocks for them out there. Of course, you need the first "tuples" to get the list of the other blocks you need, but don't you think that in 5GB there should be a lot of those for the URLs that other nodes are returning?

I've noticed that the amount of blocks stored locally over time has decreased, which means to me that I have started to reach the point where I have most of the blocks on the network.

Is 5GB about the size of everyone's blockcache?

Is this possibly one of the problems with OFF? We won't have long term storage and so a URL will go "stale" in a short amount of time?

If this is going to be the normal operation, should there be a date stamp on URLs so if it's a few months old we can filter it out, or flag it on the URL list so people can remove them?

15
OFF / What are all those colors in the history graph?
« on: July 17, 2008, 01:39:03 AM »
Guessing from the source code...

"General" - Red
"Downloads" - Blue
"Blocks" - Green
"Search" - Cyan
"Overhead" - White

While we are talking about colors, the bucket gauge is like a gas gauge on a car, if it's near the bottom it's red, empty.

Found in
gui/bw_hist_graph.cxx

  col_line_gen=*wxRED;
  col_line_dls=*wxBLUE;
  col_line_blks=*wxGREEN;
  col_line_srch=*wxCYAN;
  col_line_ovrhd=*wxWHITE;



Pages: [1] 2