[NCLUG] Tidbit: /var/log logrotation by date

Bob Proulx bob at proulx.com
Sat Nov 10 12:34:42 MST 2012


Sean Reifschneider wrote:
> Bob Proulx wrote:
> > In theory you could recover a system from backup without recovering
> 
> My philosophy is to back up everything and then exclude what I don't want.
> I know other people who exclude everything and then include what they think
> needs to be backed up, but that seems like a bad idea to me.

That is certainly a good and safe strategy.  Unfortunately I
personally haven't always been able to follow that due to limitations
in the amount of space I have available for backup.  And therefore I
need to take shortcuts and so I do usually always exclude some things
by default and then add them in instead of the other way around.  At
some point I would like to reverse that strategy.  Life is compromise.

For example I always backup from '/' down the file system but I
use -x, --one-file-system to exclude other mount points.  Which means
that if a system has /home, /var, or other on a separate filesystem
then I must add them in explicitly or they would be missed.  And I
add those in also with -x, --one-file-system too.

> Of course, you DO have to exclude /proc.

I always exclude /proc, /dev, /sys.  Accidentally backing up /dev/mem
isn't good.  I usually exclude /media and /mnt since backing up random
cdroms and usb disks that have been mounted isn't useful either.  I
usually exclude /tmp and /var/tmp too since "my most important work"
shouldn't be kept there with any expectation of longevity. ;-)

> > I now always add this to my /etc/logrotate.conf file:
> >   dateext
> 
> FYI: That seems to be the default on RHEL 6 based systems.

Cool!  Since you pointed that out I looked and see that with 6 it was
added.  Looking at RHEL 5 I see that it still wasn't there.  So yay
the world is definitely moving that direction!

> One other thing to consider for backups using rsync, they will typically do
> a full new copy of a file that changes just a little, like log files.  In
> the past we had some systems with big files like 2GB Zope databases, that
> were appended to a little bit every day, and every copy would use 2GB.

Yes.  That would be a problem.  A good heads-up.  So far I have been
able to attack that problem by rotating log files so that individually
they are small.  But that tactic could be difficult for something like
your example or for example with a MySQL database either for the raw
files or for a dump of it.

> So for our current backup system it uses "rsync --inplace" which will not
> make a full new copy.  We make a snapshot after the rsync, so we aren't
> getting hit nearly as hard for big files that grow.

This sounds interesting.  Please say more!  Are you pruning the
previous backup at that time?  So that you are just keeping one
current backup snapshot?  Is this a recipe that you use globally or
just one implemented specially for those large files?

Thanks,
Bob



More information about the NCLUG mailing list