Author Topic: [FEATURE REQUEST] Client/Server separation? Mount into Linux directory tree?  (Read 2221 times)

0 Members and 1 Guest are viewing this topic.

Offline Nemo

  • Global Moderator
  • Elite
  • *****
  • Posts: 1303
  • Karma: +27/-0
    • View Profile
I like the idea behind OFF! So I have two feature requests for a wider usage of OFF:

1) Client/Server separation:
The server is a deamon, which could run on a headless machine (like a Linksys NSLU2 or any other embedded Linux device with attached harddisks. It would run all the time (24/7) silently and without much power consumption for providing a 160GByte OFF blockcache.

For using this OFF-server the existing GUI could be a client for remote access to the server. And the server provides a webinterface for retrieving and inserting content into OFF (similar to a Freenet node).
And a command line interface client would be nice for scripted file inserts or downloads.   8)


2) Mount into Linux directory tree:

OFF means Owner Free Filesystem, right? So why not mount OFF via FUSE directly into the Linux system e.g. for directly watch videos or listen to MP3s from OFF, or browsing OFF as usual like any other filesystem!
Something like FreenetFS is just the beginning (e.g. accessing files by knowing their hashvalue via "/mnt/off/hashes/xyz"), in my opinion it should have a global namespace.
I know that a global namespace (everybody reads and writes to the same directories and files) creates many problems like solving name collisions, handling of file/directory deletion or renames, floodattacks with millions of new directories and files, perhaps a distributed rating system prefering "good" and popular content, ...  ::)



What do you think about these ideas? Are they already on the roadmap?

Greetings,
Nemo.

Offline OFF-meister

  • Regular
  • **
  • Posts: 98
  • Karma: +3/-0
    • View Profile
I like the idea behind OFF! So I have two feature requests for a wider usage of OFF:
These are both great ideas Nemo. We are part way to both of them and could always use more help getting the rest of the way.

Quote
1) Client/Server separation:
The server is a deamon, which could run on a headless machine (like a Linksys NSLU2 or any other embedded Linux device with attached harddisks. It would run all the time (24/7) silently and without much power consumption for providing a 160GByte OFF blockcache.
Where we are: OFF already has a mode to run in the background with the GUI turned off. However, it still requires linking wxWidgets for two small dependencies.
1. We used a wx call to get the mime/type for a given file extension. Wx handles the appropriate system call on each platform.
2. We use a wx call to convert to localized system paths. Wx handles the appropriate system specific parts.

What we need: If we can get cross platform code to replace those two things, we can remove the last wx references from the core OFF System code.

Quote
For using this OFF-server the existing GUI could be a client for remote access to the server. And the server provides a webinterface for retrieving and inserting content into OFF (similar to a Freenet node).
And a command line interface client would be nice for scripted file inserts or downloads.   8)
Where we are: We haven't done anything to use the existing GUI as a separate client app. However, we are planning a web based gui. Currently, you can access OFF URLs through the http interface. If the blocks are present, this works great. However, when the node needs to go searching for blocks, you don't progress feedback yet.

What we need: We could use someone who is crafty with html and perhaps javascript to help us design this implement this interface. Once that code is done, getting OFF to send it to the browser at the appropriate time will be reasonably easy.


Quote
2) Mount into Linux directory tree:

OFF means Owner Free Filesystem, right? So why not mount OFF via FUSE directly into the Linux system e.g. for directly watch videos or listen to MP3s from OFF, or browsing OFF as usual like any other filesystem!
Something like FreenetFS is just the beginning (e.g. accessing files by knowing their hashvalue via "/mnt/off/hashes/xyz"), in my opinion it should have a global namespace.
Where we are: We would really like to do this. Originally we planned to do it using WebDAV, but the FUSE project is another strong contender now. WebDAV is well supported on windows, mac supports both, *nix supports FUSE well, and I'm sure there must be decent support for WebDAV but I've never looked for it.

The main issue is creating the hierarchical structure to house the OFF URLs. We have some plans for how to do this. (see the next section)

What we need: This is just waiting for a developer to step up. WebDAV should be pretty straightforward to implement, given we already have an http interface to build upon. Here are some random nodes from IRC:
Quote
By the way, I can explain to anyone a process for creating a FUSE or WebDAV module for OFF.
It need not be part of the OFF Core.

OFF can just act as an external resource like Flickr.

It is really just a mapping issue. Usually a "file system" is a heirarchical collection of resources.
beyond the "address" and "meta-data" parts of the OFF URL, we currently have a flat space.
meaning there are is no hierarchy to the filenames.ext
OFF Dirs don't really help a lot with this, because their creation is often time based, rather than concept based. Also you can't add anything new to them individually.

However, in most current filesystems, the objects arranged in hierarchies can be some form of "link".  *nix has soft links, OS X has those and aliases, and windows has shortcuts.
So if you maintained the hierarchical structure in an external file, like say an XML file that contained OFF URLs...

You could easily create a FUSE module or WebDAV server that returned directories of soft link objects. Or you could simply make them appear as hard links if you wanted.
It is your fuse module, interpret them as you wish.

Then make an external call to retrieve the resource like the flickr and other FUSE web based modules do.

Quote
I know that a global namespace (everybody reads and writes to the same directories and files) creates many problems like solving name collisions, handling of file/directory deletion or renames, floodattacks with millions of new directories and files, perhaps a distributed rating system prefering "good" and popular content, ...  ::)
Where we are: We are currently designing an OFF System "Catalog Server" to handle this problem. It will parse OFF Directory trees and store all the URLs in a local SQLite database. Along with existing URL information, it can include additional hierarchical path information. This information can be attached to URLs through additional query parameters.

The Catalog Server handles the global namespace problem though anonymous digital signature. Each node can determine its own trust level for each anonymous identity. Then filter the available information accordingly.

What we need: The catalog server becomes a key feature of the three projects mentioned in the other thread. The public domain repository, the library, and publius.

There is some initial experimentation going on at the moment but formation will start in earnest in about a week. Any experience with small SQL databases would be helpful. Currently experimentation is being done in Ruby and PHP. This allows people to get in on ground floor development, without having to grok the entire current C++ code base.

Once an external catalog is functional, it will be ported into the current client using SQLite.

Offline Nemo

  • Global Moderator
  • Elite
  • *****
  • Posts: 1303
  • Karma: +27/-0
    • View Profile
OFF-meister, thank you very much for your detailed answers! It sounds very interesting!  :)

Greetings,
Nemo.