Author Topic: Fragen und Antworten zu OFF  (Read 81960 times)

0 Members and 1 Guest are viewing this topic.

Offline $pO0On

  • Advanced
  • ***
  • Posts: 124
  • Karma: +1/-0
  • von: Coldmirror
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #195 on: July 02, 2008, 07:52:22 PM »
zu 1) Also sobald du zu den meisten Nodes verbunden bist (so 50 - 100) und dann noch 15 min wartest sollte auch reachable dort unten stehen, wenn nicht, dann stimmt etwas mit der Portweiterleitung noch nicht. Wenn du den Fehler nicht findest kannst du OFF ja einfach mal nen anderen Port zuweisen und den dann freigeben. Vielleicht hilft das.
zu 2) Ja das ist korrekt. Die IPs der Teilnehmer sind im OFFsystem kein Geheimnis. Die Sicherheit des Systems steckt woanders drin. Am meisten dazu findet man natürlich auf der Hompage von OFF , aber auch hier im Forum solltest dazu einige Statements finden, wenn du etwas suchst.
Signaturen sind wie Frauen.
Hat man keine, will man eine.
Hat man eine, will man ´ne andere

Offline Peterx3

  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #196 on: July 03, 2008, 09:52:12 AM »
vielen dank für die antwort :D
1) ich versuchs mal mit nem anderen port :]
2) oaky danke ;)

Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #197 on: July 03, 2008, 08:47:00 PM »

Fixed bug in disperses of version 1 URLs where the presence of hashmap
blocks was assumed. This produced incorrect progress display (>100%) and
false errors.

Fixed yet another instance of a socket object that could leak during the
shutdown procedure - hopefully the last one.

Scheduled pings and nodelist requests no longer need a copy of the
relevant node object, instead they keep the rank in the main nodelist
and look up the node details when necessary.

CRITICAL: Added a short delay (5 seconds) between closing an incoming
ping socket and opening the socket for a ping back (firewall check).
This seems to clean up a large number of bizarre errors that regularly
occur on pingback sockets and - no doubt - if I knew anything about TCP,
I would understand why. This is critical since it can cause many nodes
to appear walled, when actually they are not.

Fixed memory leak when an attempt is made to disperse an invalid URL
(not that this would happen very often).

NON_RELEASE_BUILDS ONLY: Fixed shutdown crasher due to static objects
being destroyed in the wrong order.

Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #198 on: July 09, 2008, 11:24:24 PM »

Fixed bug in detection of the system IP address under MSW, this
could cause a node with the local IP and port to appear in the
nodelist and few (hopefully minor) network errors when attempting
to contact it.

If the priority of trim pushes is set higher than that of disperse pushes,
then trim pushes are started before disperse pushes. Previously disperse
pushes were always started first.

Removed the update method from the list of push blocks as all access is
now via the ACH thread.

Refactored the push blocks to be a group of PtrMapH objects instead of
a linked list. This allows for faster lookups and fixes a few small
bugs where pushes of different types for the same block could co-exist.

The push_block class no longer inherits block_name as its name is known
from the map key.

On first startup, the wizard now selects a random port between 23403
and 28403. This affects nothing beyond the initial suggested port.

Modified the handling of nodes with colliding IP/port, this can happen
if two nodes are run behind a NAT using the same listen port. Previously
there was some confusion and the nodes would appear in the list as dupes,
now a pool cnxn is extablished with the first one seen and the second is
rejected. A more permanent solution may or may not appear.

Refactored relayed block requests similarly to push blocks, they are now
stored in a map keyed by the block name.

Implemented a ptr multimap class keyed by a time_t variable (PtrMultimapT)
to be used for keeping objects in chronological order.

Used to PtrMultiMapT to keep track of relayed requests, 6 maps are used
dividing inactive, active and expired blocks into home and non-home blocks.
This saves us from having to examine all requests when looking for the
oldest of a given type an will allow for more varied strategies for
relayed blocks later.

The "Relayed" field of the status bar is now updated from the ACH thread,
it is also now the variable size field.

Refactored lists of active requested blocks to use PtrMapH objects,
this allows for faster lookups and fixes a bug where the same block
could occasionally appear twice in the same request group. This means
that the blocks requested from a single node are now sent in alphabetical
order instead of Ham order, but that shouldn't make too much difference.

Fixed bug where 2way block requests could create a request group larger than
the maximum allowed.

Fixed bug where 2way block requests did not stop on the shutting down flag.

Removed the intermediate list used by block reads to determine which
download they are from. Now the thread structure is simpler, incoming
blocks can look up the info they need directly from the request map.

The download ranks list in SecReq is now built as the block list is
compiled in BlockRequester. It has also been replaced with a set for
faster lookups.

Fixed bug where nodes added with a DNS instead of an IP address would
not detect a pre-existing node with the IP that the DNS resolved to.

The sr_blk_info class (used in block requests) no longer inherits block_name
as its name is known from the map key.

Split the OFFCnxn class into a base class, a general class and a server

Set the SO_REUSEADDR option on the main server port under Linux. This allows
a restart to acquire the listen port immediately instead of waiting for the
system to release it.


Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #199 on: July 15, 2008, 12:21:11 PM »

The offline lifetime no longer applies to blacklisted nodes.

Refactored the list of contactable nodes so that it no longer contains
copies of the main nodelist members but a cut-down class (TNLNode) which
only has the necessary variables, this cuts out most of the need to keep
the contactable nodes updated when variables in the main nodelist change.
(It is still necessary some of the time.) Common variables and functions
between the main and contactable node classes are inherited from a base

Fixed a bug where every ping would trigger an RSA key request. If the
first RSA request fails, another will be made in a couple of minutes
there after.

Refactored management of RSA and AES key requests, they now use a set
of integer references into the main nodelist just as pings and nodelist
requests do, and only need to check for expired keys, etc, every couple
of minutes.

Fixed bug where a pool cnxn would not be disconnected on autoban (not
that we are scoring or autobanning yet).

Fixed bug that broke AES key exchange for non-pool cnxns.

CRITICAL: Fixed memory corruption bug in AES key exchange - array overrun.

Refactored the top level connection handler class into a series of
derived classes that are created from a class factory, the main connection
list still holds base class ptrs and specific msg types are handled with
virtual functions. Many of the existing msg functions are now private
functions of these derived types, however this job is half-finished
at this stage - partly due to poor encapsulation of some msg handling
methods (which must be cleaned up) and partly due to the discovery of the
critical bug above which would seem to force a release.

New src directory:

New src files added:


Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #200 on: July 20, 2008, 09:08:51 PM »

Moved more network functionality from the global space in to the new
derived cnxn types.

Moved the list of contactable nodes out of the DownloadManager class
and into the global namespace, this saves having to pass a ptr to it
down through numerous call trees

Moved the download progress display options from the View menu to
the Download context menu.

The hashmap in the Insert class is now stored in binary as an array of
block_name objects instead of an array of hex hashes.

Implemented a bitfield map class that uses each bit of an unsigned char
array to represent a truth value. This can replace a lot of integer and bool
arrays saving some RAM and providing automatic bounds checking.

Replaced the filemap[] and verified[] arrays in the Download class with
BitfieldMap objects.

Replaced the tuple completion maps in insert and download repair functions
with BitfieldMap objects.

Removed the TempBlockList objects from various insert repair, replace
and localise functions. The use of a TempBlockList to create a copy of
the entire block cache is ancient and unnecessary code that would probably
cause a lot of problems with some of the huge caches we have these days.

Fixed bug where Localising a download did not actually do anything,
looks like a case of bitrot. (Note: localisation, or dynamic descriptors,
is still disabled in release builds.)

Fixed bug where a Localise would not remove the old download save dir,
resulting in the old URL being "rescued" on restart instead of discarded.
Another case of bitrot.

Fixed bug where replacing a block in an insert would not correctly modify
the corresponding entry in the global srch result list. (Block replace
is disabled as well, but this stuff has to be maintained!)

Added a mutex lock to the working dir string after hearing filthy rumours
about std::string thread safety.

Refactored the low-level insert routines to use arrays of blocks instead
of arrays of block pointers. This safes on heap allocations and provides
better allocation failure detection ("better" is a relative term, allocation
failure handling is still an absolute disaster.)

Created a node_id class that inherits block_name and flags if the id has
been set correctly, node IDs represented by block_name objects in NodeBase,
OFFHTTP and OFFSocket have been replaced with this class.

Fixed memory leak when a group of files selected for concatenated store
contains an invalid file and the store is aborted.

Fixed infinite loop bug when key requests find a node in the contactable list
but not in the main list, also added some cleanup code to remove the
entry from the contactable nodes. The real cause of this error remains a

Fixed bug where the progress bar border would cover the first file tuple.

New src files added:


Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #201 on: July 22, 2008, 09:06:58 PM »

When pushing a block we now try to push only within the TBRs of the
target nodes. If this fails, we revert to pushing withing normal
bucket radii as before. This should reduce the amount of blocks
that new nodes obtain that they will only have to get rid of later when
their bucket becomes full.

Converted the list of target nodes for a srch to an integer set.

Made sure that user initiated srches are at the front of the queue of
srches to be performed. This should help with the "dead srch" problem.

Combined the pending search lists for user and relayed srches, they are
all handled by the same code these days.

Added a set of srch ids of those srches which are currently being sent
to other nodes. The timer of a search is frozen while it is in this set,
otherwise the timer can reach the next submit point before the previous
submitted srch has been sent to all target nodes - especially on the
early, shorter, iterations.

Added a mutex lock to the mime database for the same reasons as the
working dir last version.

Offline Andylee

  • MaGOG
  • Elite
  • *****
  • Posts: 757
  • Karma: +17/-13
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #202 on: July 22, 2008, 09:21:51 PM »
Hört sich ja gut an, aber warum wird jetzt wieder an der Suche gearbeitet?

Ich dachte immer, die ist nur so ein Übergangsmodell, das irgendwann durch ein ganz anderes system ersetzt wird?
Ich bin Anti- (-christ, -alk, -kaffe, -tabak, -patent, -firefox)

Das Gewicht des Wortes steigt mit der Zahl der Zuhörer. Gebt Worten ein Gewicht!

Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #203 on: August 21, 2008, 10:56:45 PM »

Lots of minor tidying up of old code.

Fixed bug where the file send in-progress flag was not checked in cmd
line builds.

OFF no longer creates the obsolete directory "./store/".

Removed the host/port check for a node when a new server message
is received. After attempting to add the node, a node with the
ID hash from the header should always exist unless there is some other
issue such as reaching the maximum node count.

Fixed bug where the File Selector dialog would point the autostore to the
wrong directory under MSW when a relative path was used - which just shows
that the selector dialog bug is still present in wxWidgets-2.8.8. :(

Converted most of the remaining hex handling of hashes to binary.

Merged the DisperseMonitor class into the new AsyncDisperser class.

Fixed bug in parsing of cmd line input.

PROTOCOL CHANGE: Added an error message to pad exchange connections to
in form the other end if something tragic (like a pad file getting lost)
has happened - and yes, encryption is still disabled in release builds.

Copied in experimental UDP comms code from a generously submitted patch,
thanks go to the author.

Changed UDP handling in OFFCnxn to be based on a type member instead
of the global variable.

Fixed crasher in cmd line builds when Ctrl-D was pressed, seems that
readline kicks out a NULL pointer in this case.

Sketched out a few classes which will eventually ringfence UDP comms.

Fixed bug where a cache integrity check would not heed the shutting
down flag which caused crashes or leaks if the node was closed while
a check was in progress.

Removed the msg length calculation from a few network msgs since the
length is now already known from the dynamic msg buffer.

Refactored the count of current push blocks to avoid a suspected counting
error due to type conversions that can cause the trim to stall in the last
few blocks.

If a node returns a block error msg saying it is shutting down we now
set a no slot flag for that node to avoid spamming nodes with requests
while they are in the process of closing down.

New src files added:


Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #204 on: August 22, 2008, 10:46:24 PM »

Fixed memory leak if there is an error reading some parts of a download
save file.

CRITICAL: fixed bug where concatenated inserts were not handled correctly
as targets in the targetted store of a new file. This resulted in a broken
URL if concat inserts were selected as targets all concatenated files
were not local. (If all concatenated files were local the target randomisers
would include blocks not selected by the user.)

Converted all remaining uses of malloc()/free() for arrays to new/delete.

Removed the malloc intercept debug code which is no longer needed.
Fixed bug from last version where extra data was sent after the key test

Fixed a false error if a disperse resumes on the last file tuple.

CRITICAL: Fixed a bug where the a disperse would stall before finishing
if the same block was submitted as a disperse push while it was already being
dispersed (due to it occurring more than once in the insert). This caused
an error in the push count and the disperse would not complete - even
cancelling such a stalled disperse would not allow other disperses to start.

Offline ContractSlayer

  • Elite
  • *****
  • Posts: 605
  • Karma: +4/-0
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #205 on: August 27, 2008, 09:47:36 PM »

Replaced the maps of inactive relayed block requests keyed by the
request time with maps keyed by the local hamming distance.

When an inactive relayed block request is displaced by a new one, the
furthest (by hamming) request is now removed instead of the oldest one.

When a new relayed home block is received, and a closer inactive block
exists, that closer block is activated if possible, before trying to
activate the new one.

Download block cycles (see 19.10) are reset if the request list is
emptied - e.g. on completing a descriptor tuple.

Relayed blocks are now processed before download blocks when requesting
from other nodes. Since relayed blocks are generally few, this should
not impact download performance too much.

Lists of candidate nodes for a failed block request no longer include
those nodes for which HD>=TBR.

A block is now only requested from one node at a time, it's request timer
is frozen for the duration of the request.

NOTE: any or all of the above changes may be reversed if they are found
to adversely affect performance.

Fixed bug on browser retrieve, a false error due to the browser not
having a node ID.

Disk free space is now checked on all drives that contain part of the cache,
if the space on any of them drops below the minimum, downloads are suspended.
The txt in the download column should now read:
  "Insufficient disk space on drive '<name>'"

Changed mount point check under linux to use the device ID instead of
the free space.

Included two new themes in windows installer (CrystalProject_1.0 and
Oxygen_1.0), thanks go to the submitters.


Offline Doc Honke

  • Elite
  • *****
  • Posts: 1374
  • Karma: +21/-8
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #206 on: September 07, 2008, 09:04:37 AM »

Ich habe jetzt hunderte Dateien über "Directory store ..." eingefügt. Er zeigt mir auch unter Configuration 60 GB an "Current Block Cache Size" an, aber nur ein paar Dateien unter "Local URL's".

Gestern ist mir allerdings OFF abgeschmiert mitten beim Laden. Ich denke, die Anzeige der gesharten Dateien stammt vom Zeitpunkt des Crashes.

Was kann ich tun? Erneutes "Directory store ..." habe ich schon versucht, ändert nix an der Anzeige der Dateien.   :'(

Offline Andylee

  • MaGOG
  • Elite
  • *****
  • Posts: 757
  • Karma: +17/-13
    • View Profile
Re: Fragen und Antworten zu OFF
« Reply #207 on: September 07, 2008, 09:36:51 AM »
schnelle Antwort aus dem Entwickler-Channel:

Directory store only addes the toplevel OFFDir URL to the list
he will get the remaining URLs when he opens that dir

Ohne jetzt behaupten zu wollen, dass ich dies verstehe :)
Ich bin Anti- (-christ, -alk, -kaffe, -tabak, -patent, -firefox)

Das Gewicht des Wortes steigt mit der Zahl der Zuhörer. Gebt Worten ein Gewicht!