What is Digg?37 Comments
- monkeycat, on 10/12/2007, -0/+17...beacuse some people have no idea what that means.
- demesisx, on 10/12/2007, -0/+5Something tells me that this is one digg story that won't be dugg to death.
- theendlessnow, on 10/12/2007, -0/+3We run a 9-10TB running incremental disk cache backup using rsync. It is very efficient. Author says because it has the ability to put such a high load on systems, it has been used to find many a bug inside of Linux.rnrnHighly recommended for backing up changes rapidly!!rn
- kenelbow, on 10/12/2007, -0/+3Yeah, but what if your website is database driven? Edit: Nevermind, I see he added links for tutorials on this.
- inactive, on 10/12/2007, -0/+2Digg will make this more popular I'm sure.
- Gryffydd, on 10/12/2007, -1/+3@ewithrow
You know, that's way easier than searching through man pages and how tos on this subject as I did recently. I wish there was a collection of summaries along those lines for all commands. Once you're familiar with the Linux command line it's all you need. It seems like the man pages and the how tos just obfuscate things sometimes. - tadelste, on 10/12/2007, -0/+2System administrators have used rsync to setup mirrors for sometime now. I'm surprised it hasn't gotten the notice it deserves. It's efficient because you don't have to completely copy everything. It will just use the delta of changes in the backup or mirror repository. I found this material helpful and something I can pass along.
- XStatic, on 10/12/2007, -0/+2rsync sends diffs rather than whole files, pipelines transfers and works more efficiently even on high-latency
links
Use rsync! - XStatic, on 10/12/2007, -0/+2Use Rsync through ssh as the example shows
- nferrier, on 10/12/2007, -0/+1rsync is cool.
One of the cool things this article doesn't mention is how flexible the -e switch is. The -e switch specifies the transport rsync will use. Most often you do something like:
rsync -e ssh srcdir user@host:/destdir
but -e actually specifies a command so one really cool thing you can do is this:
rsync -e 'ssh user@somehost ssh' srcdir otheruser@secondhost:/destdir
this will have rsync ssh to somehost (as 'user') and then ssh from there to secondhost as 'otheruser'. The files will end up in destdir on secondhost.
Because of the flexibility of ssh as many hops can be added as you need.
Amazingly cool. - inactive, on 10/12/2007, -0/+1I didn't see the part about "a backup server that can take over if the main server fails" WRT web service.
I guess Ultramonkey would be the tool, and I can just look it up on howtoforge, but I figured while he was cutting and pasting URLS from last month, it would be easy to do one more.
http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster - targetX, on 10/12/2007, -0/+1You can have a customized error message in apache (if there is a page not found for example) but you probably allready knew that. The only way I know to do the stuff you're saying is if your servers are behind a load balancer. The loadbalancer will see that one of your servers is down and therefor redirect all traffic to one of the other servers.
- seanharrop, on 10/12/2007, -0/+1Can someone recommend a Windows alternative? As much as I'd like (and suggest and try) to get my company on Linux, it's not going to happen any time soon.
- grendelboogie, on 10/12/2007, -0/+1I believe the command uses rsync over ssh, not the native rsync port.
- wrinkles, on 10/12/2007, -0/+1I've considered having a backup site. But how would I redirect if the server is down? Is there a way to create a custom error page in the case of a server malfunction? The error page could then direct the customer to a backup site.
- neomorphic, on 10/12/2007, -0/+1yes you can:
rsync -avz /path/to/src/ /path/to/data/tmp - nferrier, on 10/12/2007, -0/+1use cygwin:
http://www.gaztronics.net/rsync.php
or use it natively:
http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html
you also need ssh. I seem to recall using the popular putty ssh client to operate rsync on a windows box (in the days when I still had to use them).
- seanharrop, on 10/12/2007, -0/+1Thanks, nferrier. I knew about cygwin, but our admin doesn't want to install it on the server. I did not know about the native version. That might do the trick! :)
- pgouy, on 10/12/2007, -0/+1The linked mysql cluster article was really useful for me now! Thanks :)
dugg! - Bogtha, on 10/12/2007, -0/+1Sigh. That last 'n' should be a backslash n. Digg really should hire people who test their code.
- ralphc, on 10/12/2007, -0/+1Can rsync be used to sync up two directories on the same machine? I can see plugging in the external hard drive on my Mac, opening up a terminal and making an 'intelligent', fast backup of my iTunes instead of the 'wipe and recopy ' I do now.
- wrinkles, on 10/12/2007, -0/+1I admit this isn't something I've heard of, especially in a "shared hosting" account. But it seems to me that Google has a crazy-dependable system, why can't they figure a way for Joe Webmaster to submit a custom error page for his site. Google could present that page when Joe's site is down, with a link to the backup site, or just a message and a admin email address. Google could pay for it with advertising on the error page. The click-thru rate would be high - OK Joe's site is down, Google says Pete can offer the same service.
I'm not sure how it would be done, but it would be a great service. - Bogtha, on 10/12/2007, -0/+1He is hitting enter. Digg has a bug where if you go back and edit a comment, it sometimes changes all newlines into 'n's. Fixing that by going through, deleting the 'n's and hitting enter doesn't work because you just end up with the 'n's again. I suspect it's stripslashes() being called at the wrong time and screwing up the 'n's before nl2br() can be called to convert them into <br>s.
- mscnln, on 10/12/2007, -0/+1yes
- billimek, on 10/12/2007, -0/+0Must like this article talks about, I use rsync to ‘mirror’ my website (and databases) from my webhosting provider over to my local linux machine. It does some post-processing with sed to massage the data in the database dumps in order to translate the domain names from what my hosting provider uses to what my local machine uses.
I have a pretty comprehensive write-up with examples on all of this if you want some supplemental info to go along with the article at: http://billimek.com/jeff/archives/2006/01/16/migration-to-site5/ - blfflix, on 10/12/2007, -0/+0This is the link to the Microsoft page about the Unix subsystem:
http://www.microsoft.com/windowsserver2003/r2/unixinterop/default.mspx
I have installed it but only for the purposes of seeing that it seemed to work and to see if I could trick SBS to run it. There is a version from MS for XP as well. - chener, on 10/12/2007, -0/+0Have you tried robocopy? It has a mirror function which should give what you're looking for.
- nferrier, on 10/12/2007, -0/+0have two apache's, one simple frontend doing not much and the other backending.
The frontend uses mod_proxy or mod_rewrite to hand off requests to the 2nd one. If the 2nd one fails the first one can detect it and move on to another 2nd server.
Note: there's no point in doing this unless you are protecting another point of failure than the network; eg: a flaky database, or a poor app. - ewithrow, on 10/12/2007, -3/+3rsync -azPe ssh username@123.54.67.89:/home/username /path/to/local/drivennrsync = running the rsync command which syncs the data at source to the destination.n-a = archive mode, preserved permissions and recurses directories.n-z = compresses data across the link for faster transfers.n-P = prints the progress to the screen.n-e = specifies data will be transfered via a remote shell (ssh).nssh = lets rsync know we will be connecting to the remote machine using SSH.nusername = the username used to connect to the remote machine via SSH (you will be prompted for the password upon connection).n123.54.67.89 = the IP address of the remote machine.n/home/username = the path of the remote SOURCE files you want to backup.n/path/to/local/drive = the path of the local DESTINATION folder.nn*EDIT* damn the digg edit system is really screwed up, it turns all newlines into "n"
- nferrier, on 10/12/2007, -0/+0they would have to frag dns for that to be done.
name service providers would be able to build such a service. But when network solutions tried to alter the way that nameservers respond to failures there was an outcry. - baagoo, on 10/12/2007, -1/+1Rsync bad idea to have open - especially on a web server.
- blfflix, on 10/12/2007, -0/+0I just got my hands on Windows Server 2003 R2. I was pleased to learn that there was an official Microsoft version of Unix freely downloadable. I was not prepared for the 2 minute window so I will find the link and post it in a minute.
- lexbaby, on 10/12/2007, -1/+1"the digg edit system is really screwed up, it turns all newlines into "n""
Why didn't you just hit enter? - ewithrow, on 10/12/2007, -5/+4Who needs a 3 page howto when you can just run something like:
rsync -azPe ssh username@123.54.67.89:/home/username /path/to/local/drive - maunir, on 10/12/2007, -1/+0Using rdist is easier and worry free. Rdist has the functionality to compare timestamps on files/directories and once you modify it then it copies it to the backup server. Use cron to do this at your time and let it email you with changes that took place. Using it for about couple of weeks and the developers love it.
- axentrix, on 10/12/2007, -3/+1This has been at howtoforge in a month or so..
- TubaTechno, on 10/12/2007, -6/+1Bugs inside Linux?! But I thought there were NO bugs ever?!! (sarcasm for you Linux fanboys)
The Digg Toolbar for Firefox lets you Digg, submit content, and keep track of Digg even when you're not on the Digg site. Download the official 
© Digg Inc. 2009
— Content created and posted by Digg users is