Author Topic: Q/A Block Cache + Block Bucket  (Read 4957 times)

0 Members and 1 Guest are viewing this topic.

Offline ToN-nL

  • Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
    • View Profile
Q/A Block Cache + Block Bucket
« on: November 09, 2007, 02:07:46 PM »

I was having a question about Block Cache + Block Bucket and I asked
OFF-meister if he could explain me.

He answered me with a very clear answer, I will post it over here for other
people that have the same question...

My question:
Quote
Hi,

Can you tell me what will happens when my cache or bucket is 100% full ?
My bucket is at 97% now... do I have to change something ??

Hope you will answer me... thanks.


OFF-meister's answer:
Quote
Sure!

Let's get the definitions first.
-----
Your "Block Cache" is the total amount of your hard disk space allocated of OFF blocks. Once OFF reaches that limit, it will "cull" existing blocks in order to make room for new blocks.

Your "Block Bucket" is the subset of your Block Cache that you are "donating" altruistically to the global storage capacity of the OFF System network. This is a separate allocation number, but does not represent a separate cache. Everything in the bucket is already in the cache.
------
During normally operation of you OFF node, it is expected that both of these levels will reach and remain at near capacity. So that is nothing to worry about by itself. Once the block cache is "full", if it receives another block that it needs to store, it will discard one that it no longer needs to store.

It decides this using a reasonably simple algorithm. First it looks at the URLs you have in your "known urls" list for the ones marked "preserved." (These are the URLs that you as user want to access immediately.) It then goes through all the references from that URL and marks those blocks "preserved." Next, it goes through your block bucket and marks all of those blocks "preserved." (Those are the blocks you have promised to keep them around for others.)

Now it knows every block it wants to keep. So now to make space, the block cache deletes the remaining blocks in "furthest distance from this node ID first" order.

Generally, during normal operation of your OFF node, you need to do nothing at all.

When YOU store a file, the URL in your "known url" list its blocks are marked "preserve = forever" unless you explicitly change that. Once that insert is distributed (meaning the blocks have moved to appropriate homes on the network) you stored file should remain available even if your node goes down. At this point, you can "unprotect" the URL if you don't need immediate access.

When you ACCESS a file, all of those blocks get distributed to appropriate places in the process. So you don't really need to worry about those blocks, since you can always find them again. However, if you want to assure that URL is available immediately to you in the future, you can "protect" that URL and its blocks from being culled from your cache.

If you haven't use "distributed store" when storing your public files, consider right clicking on those URLs and selection "distribute" from the menu. That process which might take a little while, depending on size, will assure that all of your inserted blocks get moved to nodes where others are likely to initially look for them. Speeding up retrieval for everyone.

I hope this helps.

Offline Butterfly

  • Regular
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #1 on: January 15, 2008, 06:20:46 AM »
Hi everybody!
Here is my first post...  :)

I have an other question about the blockcache  :
Is it possible to set an other path to this folder  ???
In fact i wish to set it to an other hard drive which has much more free space.
I've tried to look in the settings or in the offsystem.conf but without success...  :-[

I guess it's me who can't see the obvious way to do it  :-\

Thanks in advance for your time and this nice softawre  ;)

Edit :  Nobody?  :'(
« Last Edit: January 16, 2008, 06:19:20 PM by Butterfly »

Offline Siedlerchr

  • Advanced
  • ***
  • Posts: 160
  • Karma: +0/-0
    • View Profile
    • Siedlerchr.jeak.de
Re: Q/A Block Cache + Block Bucket
« Reply #2 on: January 17, 2008, 03:42:08 PM »
Just install OFF on the other partition, or maybe copy the whole folder of OFF to  the partition..


Offline Butterfly

  • Regular
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #3 on: January 17, 2008, 11:00:05 PM »
I'm a Ubuntu user and not a Windows one where you can do that kind of stuff  ;D
However if you know the trick for Linux, i'll try  ;)

Offline Scrivener

  • Regular
  • **
  • Posts: 36
  • Karma: +1/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #4 on: January 18, 2008, 12:12:51 AM »
I'm a Ubuntu user and not a Windows one where you can do that kind of stuff  ;D
However if you know the trick for Linux, i'll try  ;)

With Linux it should be relatively easy. You just mount the other drive to the root file-system using the blockcache directory as the mount-point.

However, I expect that your other drive is already mounted somewhere on the root filesystem. I assume that you can see the contents of the disk with the file browser. If this is the case then the easy way is to move the blockcache to the other drive and then create a hard link in the .off directory to the blockcache directory in the new location.

Though I haven't done this before, I think it possible.

Offline rb2k

  • Advanced
  • ***
  • Posts: 297
  • Karma: +3/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #5 on: January 18, 2008, 10:08:53 AM »
I'm a Ubuntu user and not a Windows one where you can do that kind of stuff  ;D
However if you know the trick for Linux, i'll try  ;)

With Linux it should be relatively easy. You just mount the other drive to the root file-system using the blockcache directory as the mount-point.

However, I expect that your other drive is already mounted somewhere on the root filesystem. I assume that you can see the contents of the disk with the file browser. If this is the case then the easy way is to move the blockcache to the other drive and then create a hard link in the .off directory to the blockcache directory in the new location.

Though I haven't done this before, I think it possible.


Wouldn't a symbolic link work too? (ln -s)

Offline Butterfly

  • Regular
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #6 on: January 18, 2008, 11:48:09 PM »
Thanks for the tips guys! ;)
I've made a symlink so i could move the block cache directory on my /media/bigfathd, so that's cool.
However Off System always looks at the free space on my /home directory on not on my /media/bigfathd hard rive, and the way Off System will act depends on how much it thinks i've got of free space.

I guess the best would be to add a simple option in the software...  :-X

Offline Scrivener

  • Regular
  • **
  • Posts: 36
  • Karma: +1/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #7 on: January 19, 2008, 12:05:21 AM »
I guess the best would be to add a simple option in the software...  :-X

Yes, a change to the software would be better. This thread will be read by the OFF-meister and can serve as a feature request.

And a symlink using ln -s works as suggested by rb2k. I have just tested it.

Offline Butterfly

  • Regular
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Q/A Block Cache + Block Bucket
« Reply #8 on: January 19, 2008, 02:30:49 AM »
And a symlink using ln -s works as suggested by rb2k.
Yeah, it's worked for me too but this didn't change the fact that Off System always looks at the free space in my /home directory and not on the hard drive where i moved the block cache.
Yes, a change to the software would be better. This thread will be read by the OFF-meister and can serve as a feature request.
I've added a request in the Feature request of Off System's SourceForge forum.