[NCLUG] A *nix DFS alternative?

[Jeremy Hinegardner]

On Thu, Feb 18, 2010 at 04:40:40PM -0700, grant at amadensor.com wrote:
> > I assume you means servers- just two for now.  Good lord the headache if
> > we weren't talking just two!
> >
> I have an absolutely insane idea that will take care of large file
> transfers, recovery from errors, intermittent connections, and everything,
> except updating a file (each update would be in essence a new file).
> 
> This would also let you expand to an almost limitless number of locations
> with minimal pain:
> 
> Use a version control system (SVN or CVS) for a central point of control,
> but the only files you control are torrent files.   You can remove or add
> them as needed.   The remote machines will then update their list of
> torrents, and begin downloading.   Everyone gets every file, and fault
> tolerance is there.   The bandwidth is only used until the files are
> everywhere, then it settles to nothing.   All you need to work out is how
> to do access controls on the tracker so that not everyone in the world can
> grab your pictures.
> 
> In a cron job, stop all of the torrent jobs.   Run a SVn or CVS update.  
> The spin through all of the torrents in a shell, starting a download on
> each.   The main server it started on will be the seeder, and everything
> is already designed to handle the outage and recover.

Not a bad idea, and you can build off a tested deployment system that uses
BitTorrent.  Twitter's internal deployment too, Murder,
http://github.com/lg/murder 

enjoy,

-jeremy

-- 
========================================================================
 Jeremy Hinegardner                              jeremy at hinegardner.org