Author Topic: Change in V.19.27 and it's effects on swarming  (Read 3433 times)

0 Members and 1 Guest are viewing this topic.

Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Change in V.19.27 and it's effects on swarming
« on: August 27, 2008, 04:46:18 AM »
Changelog V.19.27 "A block is now only requested from one node at a time, it's request timer is frozen for the duration of the request."

This may cause problems with the swarming effect when a popular file is uploaded to the network. To help with swarming of popular content, more nodes need to get the requests so that they contact other nodes for the block and so on down the line.

It may be better to pick a random number of nodes to send requests to the first time and if a block download doesn't start after a time period, pick again from the remaining nodes. You could also use a smaller percentage of nodes on the first try, limiting the max random number to say 30% of the total connected nodes for the first try.

And we still badly need a date stamp in the URL to help with swarming (date only to the day).

"Relayed blocks are now processed before download blocks when requesting from other nodes. Since relayed blocks are generally few, this should not impact download performance too much."

This will help swarming but "relayed" blocks won't be few so download performance will be affected.

For now you may want to simply flip a coin or skew a random number towards the side of picking download blocks over "relayed" blocks.

There's a lot that could be done here, like waiting a little bit to see how many requests you get for one block and then pick the popular blocks to make onward requests for first, then request the other blocks as time permits.

Since two heads are better than one, why not post first about changes like this and see what feedback you get from everyone?

Also, there should be two settings for updates, one is something like "bleeding edge" for people who want to help test the latest versions and "stable" for normal users. The big reason for this is that some changes could slow or shut down the network.

Offline OFF-meister

  • Regular
  • **
  • Posts: 98
  • Karma: +3/-0
    • View Profile
Re: Change in V.19.27 and it's effects on swarming
« Reply #1 on: August 27, 2008, 04:09:30 PM »
>Changelog V.19.27 "A block is now only requested from one node at a time, it's request timer is frozen for the duration of the request."

>This may cause problems with the swarming effect when a popular file is uploaded to the network. To help with swarming of popular content, more nodes need to get the requests so that they contact other nodes for the block and so on down the line.


If a node determines it is a DHT home for a given block, then only one
request is required before it will start to seek that block for itself.
Hence, this change (it is hoped) will reduce the level of network traffic
without impacting swarming too much.


>"Relayed blocks are now processed before download blocks when requesting from other nodes. Since relayed blocks are generally few, this should not impact download performance too much."

>This will help swarming but "relayed" blocks won't be few so download performance will be affected.


Relayed requests are limited to 50 active at a time by default, though this
can be changed in the Behaviour tab. Download blocks depend on which inserts
are being requested, but can number in the many thousands.


Offline scripter

  • Regular
  • **
  • Posts: 88
  • Karma: +2/-0
    • View Profile
Re: Change in V.19.27 and it's effects on swarming
« Reply #2 on: August 27, 2008, 07:50:12 PM »
> If a node determines it is a DHT home for a given block, then only one request is required before it will start to seek that block for itself. Hence, this change (it is hoped) will reduce the level of network traffic without impacting swarming too much.

I am wondering if what you are saying is that it will now wait until it gets a response "yes I have it" or "no I don't" and then move on to the next directly connected node. If that's the case then maybe that won't be so bad.

I assume it won't wait forever, it has some sort of timeout in case a node goes offline or something else happens.

> Relayed requests are limited to 50 active at a time by default, though this can be changed in the Behaviour tab. Download blocks depend on which inserts are being requested, but can number in the many thousands.

I am thinking in some situations like when swarming of a popular file is going on between nodes on a particular path, there's going to be a lot of relay requests flying and may keep download blocks from getting through.

If swarming keeps the active blocks at 50 all the time, won't this prevent any download blocks from coming in?

What I was suggesting is not to assume the situation will always be less relay requests, but rather control the situation through skewing a random number towards the side of picking download blocks over relayed blocks or some other method.