[NCLUG] A *nix DFS alternative?

grant at amadensor.com grant at amadensor.com
Thu Feb 18 16:40:40 MST 2010


> I assume you means servers- just two for now.  Good lord the headache if
> we weren't talking just two!
>
I have an absolutely insane idea that will take care of large file
transfers, recovery from errors, intermittent connections, and everything,
except updating a file (each update would be in essence a new file).

This would also let you expand to an almost limitless number of locations
with minimal pain:

Use a version control system (SVN or CVS) for a central point of control,
but the only files you control are torrent files.   You can remove or add
them as needed.   The remote machines will then update their list of
torrents, and begin downloading.   Everyone gets every file, and fault
tolerance is there.   The bandwidth is only used until the files are
everywhere, then it settles to nothing.   All you need to work out is how
to do access controls on the tracker so that not everyone in the world can
grab your pictures.

In a cron job, stop all of the torrent jobs.   Run a SVn or CVS update.  
The spin through all of the torrents in a shell, starting a download on
each.   The main server it started on will be the seeder, and everything
is already designed to handle the outage and recover.




More information about the NCLUG mailing list