What is Digg?82 Comments
- G33k0ft3chz, on 10/12/2007, -5/+35Here is the list for people who can't get to it:: Below is a list of command line tricks I've found to be very useful in web development that a surprising number of web developers don't know about.
1. ln -s /some/destination name_of_link
This is pretty straightforward: it'll create a symbolic link from name_of_link to /some/destination. If I'm sitting in /www/matthew/ and I entered the above command, a new file will be created inside that directory that moves the user to /some/destination if they CD to name_of_link. (It's a shortcut, basically).
If I'm working on a web server where the docroot is buried deep in a directory structure, I'll often create a symbolic link in something like /www to save myself a few keystrokes when accessing it.
Also, apache won't follow symbolic links unless you tell it to do so using the following directive: Options +FollowSymLinks
2. tail -f /some/file
This tails a file: it'll read what's at the end and output it to your terminal. The -f parameter tells it to keep outputting as the file grows. I use this a lot when examining log files that are constantly being written to.
3. ctrl+z and bg
ctrl+z is the long lost brother of ctrl+c. It's a signal that tells the process to suspend execution, while ctrl+c terminates it. Ctrl+z is useful if you execute a process but you want to get control of your shell; it'll suspend the process and send it to the background. Think of ctrl+z like minimizing a window, except once it's minimized it's not doing anything anymore. If you want the process to stay in the background but continue running, that's where bg (background) comes in: typing bg once a process has been suspended makes the process resume but still keeps it in the background.
I often combine ctrl+z and bg with tail:
shell> perl -e 'while() { print "."; sleep(1); }' > bunch_of_dots.log
[hit ctrl+z, execution is suspended]
shell> bg [process is now in the background and running]
shell>tail -f bunch_of_dots.log [show me the printed dots]
4. fg and jobs
fg is used if you background a process but want to bring it to the foreground. If you have multiple processes in the background, type 'jobs' into your terminal to see them all. This will display a list of processes that are in the background with each process assigned a number (Note: these are not the same as pids). Typing fg [some number] will resume the process you sent to the background where [some number] matches the number reported by 'jobs.'
A good example of using fg is if you're working with an interactive program, such as the mysql command line, but you want to drop back to the shell.
mysql>SELECT foo FROM bar;
mysql> [hit ctrl+z]
[1]+ Stopped
shell>pwd /home/matt/stuff
shell>jobs
[1]+ Stopped mysql -u matt -p matts_db
shell>fg 1
mysql>[hooray, we're back in the mysql command line]
You can omit the job number when running 'fg,' it'll just foreground the first process you sent to the background. So if you only have one process in the background you can just type 'fg' without any extra numbers.
5. Hit the freakin tab key!
Assuming your shell is configured properly, hitting tab will auto-complete with whatever it thinks you need. I've encountered a surprising number of developers who don't know about this and move around servers at glacial speeds. If you aren't familiar with the tab auto-complete, get to a terminal right now and type cd [tap the tab key], you should see a list of available files or commands.
6. scp
This securely copies a file across a network using SSH for data transfer. For example:
scp matt@example.com:~/secret_stuff /some/destination
This would connect via SSH to example.com as the user 'matt' and copy the file 'secure_stuff' from my home directory on example.com to /some/destination on the machine I'm currently on. It's saved me a ton of time when transferring sensitive files from one machine to another.
7. screen
Screen serves two purposes for me:
* It allows things to persist even after I've disconnected
* It allows me to access multiple terminal sessions from inside a single terminal window
Suppose I'm using the wifi connection at a coffee shop and I want to execute a binary that's going to take several hours to run but I'm not going to be able to stay connected for that entire time. Before running the binary, I enter 'screen' into my terminal. This effectively creates a new session that will persist even if I get disconnected. Once this screen has been launched, I execute my binary and exit the screen by pressing ctrl+a+d. If I want to resume the screen later from a different location I simply log into my server and then type 'screen -x,' and my binary should still be running.
Screen is also useful if you want to save some precious real estate and run a bunch of different sessions from the same terminal window. Suppose you have a server at home that you like to SSH into and play /bin/fortune on, and server at work that you want to log into and tail your apache logs.
shell>screen -S homeserver [This would create a screen named 'homeserver' and attach to it]
shell> ssh matt@home [you're in the 'homeserver' screen, now ssh to my box at home]
matt@home>[we're ssh'd in to my box at home, now press ctrl+a+d to detach from this screen]
laptop>screen -x homeserver [this will re-attach to my home server]
The same goes for your work server:
screen -S workserver etc.. screen -x workserver
Running 'screen -list' should show homeserver and workserver listed. Remember that once inside a screen, you type ctrl+a+d to gracefully detach from it. There's an army of other things you can do with screen, I highly recommend reading the man page or the screen user manual. Also, screen is not a built-in command so you might have to install it first.
8. Create a file (the fast way) with touch
shell>touch foobar.log
Bam! You've successfully created foobar.log. It's much faster than running vi, saving, and exiting. Touching an existing file updates the time it was last modified.
9. Gut a file from the inside out
cat /dev/null > foobar.log
This is a nifty trick to clear out the contents of a file but not remove the file itself. Suppose foobar.log has gotten ridiculously huge and you want to remove all the stuff inside it but you don't want to delete the file, simply cat /dev/null and direct the output to your file. In other words, take /dev/null (a black hole of nothingness) and stuff it inside foobar.log.
10. Backup and reload your databases
mysqldump > mydb.sql
mysql my_database.sql
To reload this data, simply direct it back in:
mysql -u someuser -p some_other_database alias mattsdb='mysql -u matt -p mattsdb -h example.com'
Now if I type 'mattsdb' the mysql command line will come up. Unfortunately if I were to log out and then log in again this alias would not persist. If you want your aliases to last beyond your current session, you'll have to tweak the config files for your shell. I primarily use bash and keep my aliases persistent by adding them to the bottom of ~/.bash_profile
Miscellaneous Tricks
Here's a few random gems that I've found to be very useful. As always, I highly recommend reading over all their respective manuals .
* ls -lSr List files (sorted by size)
* df -h Show available disk space in human-readable format
* du -sh /some/dir Show how much space /some/dir is taking up
* ps aux | grep blah List all the running processes but only show ones that contain 'blah'
* wget -spider http://0at.org Fetch pages and behave like a web spider: don't download the pages, just check to see if they are there
* ab - Apache benchmark, use this if you want a quick n' dirty way to benchmark how well your site performs under a heavy load.
Any others you can think of? - jmichaliga, on 10/12/2007, -7/+32...good ones do.
- PleaseBeSerious, on 10/12/2007, -6/+31Web Developers reading the Linux/Unix category into which this story was submitted probably use *nix machines.
- CLucas916, on 10/12/2007, -3/+25Crass:
are you joking? if not, just dont worry about this. (Look at the category this was submitted in) - inactive, on 10/12/2007, -2/+18Um, some people are running Fedora and other Linux server distros and use PuTTY or some other terminal. Not every server is Windows based..
- spliffy, on 10/12/2007, -0/+14i pity the fool who doesn't use the tab auto complete.
- jon3k, on 10/12/2007, -0/+13@ipodsweatshop
I was with you up until the not symlinking a directory. There are so many problems with aliasing.
1. Two user's can't share aliases (easily)
2. If a third user doesn't like that alias (let's assume you made it global) then they have to override it somehow.
It's just a clumsy solution. I'm going to go with the years of conventional wisdom that say to create symlinks to directories. Aliasing directories? That's just ... weird. - JakeX, on 10/12/2007, -3/+16@Crass...
Wow.... just speechless... - inactive, on 10/12/2007, -0/+9No matter what people say, these are still very good tips. you wont always have a GUI app at you disposal and being able to access and efficiently use your web server through the terminal is a great skill to have. Even if you don't use a *nix server/system, chances are that in your life you will come across one and have to use it.
- jtms1200, on 10/12/2007, -2/+10those "fags" with macs have the ability to use most of these commands too (iTerm rocks).
Or if you insist on using Windows... at least get cygwin. - McGrude, on 10/12/2007, -1/+8find is a very powerful command that should be included.
- Dracos, on 10/12/2007, -0/+6To quickly return to the previous dir, pass - as an argument to cd:
/some/dir $ cd /some/otherdir/named/foo
/some/otherdir/named/foo $ cd -
/some/dir $ - malliemcg, on 10/12/2007, -0/+6careful running killall on Solaris for while it will kill the named process, it will also kill all other processes.
http://www.freebsd.org/cgi/man.cgi?query=killall&apropos=0&sektion=0&manpath=SunOS+5.10&format=html
I've made killall one of those commands to just not use - mainly because it does not behave the same everywhere and the risk of running it on the wrong box (as the wrong user ;)) - computerdude33, on 10/12/2007, -0/+6Sorry, but I had to do it.
http://offsite.images.ofacrazedgeek.com/screenshot142.png - greyfade, on 10/12/2007, -1/+7the first and second are very good tips for users new to the Unix-ish shells. `tail -f /var/log/httpd/access.log` is indispensible for both the sysadmin and the web developer. but i do have things to say about the other tips:
3. be careful about what commands you suspend. wget doesn't much like being interrupted, for example. if you need to send a process to the background anyway, don't suspend it, just add & to the end of the command line: `perl -e 'while() { print "."; sleep(1); }' > bunch_of_dots.log &`
6. scp is really picky about the paths you give it. the remote path will need to be *double-quoted*. for example: `scp user@host:"/my file" "/my file"` i've been tripped up by this a few times myself.
7. screen is indispensible, and has far more functionality than the author seems aware. ^A-^C will create new screens for you to work in, and ^A-^A, -^N, -^P, etc. will switch among screens. ^A-^S will split the screen (-^I switches among them) so you can see more at once. i rarely work in screen with fewer than 7 windows open. a good statusline is also helpful. take mine for example:
hardstatus string '%{gk}[ %{G}%H %{g}][%= %{wk}%?%-Lw%?%{=b kR}(%{W}%n*%f %t%?(%u)%?%{=b kR})%{= kw}%?%+Lw%?%?%= %{g}][%{Y}%l%{g}]%{=b C}[ %m/%d %c ]%{W}'
12. be very careful with this command. don't kill anything unless you know exactly what it is and have a very good reason for killing it. especially don't use -9 (SIGKILL) unless you really have no alternative. SIGTERM (the default) will work most of the time, and if it doesn't, -2 (SIGHUP) just might. -9 is a last-ditch action.
13. for security reasons, don't put passwords in your ~/.bash_profile ! it may not have group or world permissions, but there is always the risk that some nefarious person could get their hands on it.
the absences of find(1) and grep(1) on the list are disappointing, as they, with sed(1) and awk(1) are incredibly useful tools for the web developer. (or any developer, for that matter!)
a few people have mentioned grep combined with find to search files - this is good, but there are still other ways to search:
shopt -s extglob; grep -Irns "pattern" !(files|to|not|search) # ignore binaries, recursive search, line numbers, no error messages
(seriously, look up extended globs in the bash man page - some really cool patterns are possible!) - maddmike1959, on 10/12/2007, -0/+6so, why would you use this instead of frgrep -R ?
- babakshirazi, on 10/12/2007, -6/+11
WTF? Dude......these are basic Unix commands. You wonder what happened to software and web development? When people think these are commands you might not know, that's what happened to software development. Complete *****' idiots are in the field.
How would healthcare quality be if doctors were posting articles like: Some useful medications you may not know about: aspirin, penicillin, & anti-biotics.
I'm not sure what amazes me most, the article or the fact that this ***** got dugg. - computerdude33, on 10/12/2007, -2/+7EDIT: Digg down, meant to reply to maddmike1959.
- Markie1006, on 10/12/2007, -1/+6umm, that is the hard way.
pkill procname
to kill by processname
pgrep procname
to give you a list of process id's matching the processname
(add -l option to view the procs it matched - if you lack confidence ;)
or you can go oldschool if you're on a system without pgrep/pkill
ps -ef | grep procname | grep -v grep | awk '{print $2}' | xargs kill - phatvolvo, on 10/12/2007, -0/+5he forgot "svn up" :)
- bflfab, on 10/12/2007, -0/+4killall sounds great. I can't tell you how many times I have used a
ps -ef |grep string | awk
to get process ids to kill - prockcore, on 10/12/2007, -1/+5Also, scp can transfer files between any two points.. even if you're on a 3rd box.
scp bob@server1.com:myfile.txt frank@server2.com:
will transfer myfile.txt from bob's home directory on server1.com to frank's home directory on server2.com - shaitanx, on 10/12/2007, -3/+7Ew windows servers
- acomj, on 10/12/2007, -1/+4The forgot one. when you type jobs it gives you a list of background jobs.
To stop them
kill %[job number]
example
kill %2
to kill job number 2
That UnixPower tools book is usefull for anyone using a shell on unix. - crossmr, on 10/12/2007, -2/+5I've been using Linux for a year. If people are developers using linux, they know these. I just mess around with it and I've seen all but 3.
- dlsspy, on 10/12/2007, -1/+4I'll digg this down specifically for suggesting it's ever appropriate to issue a command that looks like ``killall -9''
SIGKILL is never the first signal to send to a process, especially if you're not even aiming at a specific process. - wyattearp, on 10/12/2007, -3/+6@ipodsweatshop
maybe you should try a decaf next time(edit: damn, bluejet beat me to the caffeine joke). part of the learning process is learning from what other people know, right or wrong. one of the sweet parts of many *nix operating systems is that you can customize, configure, tweak, break, and run your little terminal life anyway you like it. just because someone else does it differently doesn't make it wrong, ctrl+z, bg, &, and fg must pose some benefit if they are still included in your chosen distro. - bigkm, on 10/12/2007, -1/+4heres one i keep in my .profile so that when i need to find a string of text in the hierarchy of the current dir, use it for web development all the time, only prints the files that contain the the text too. good for debugging someone elses code
lookfor() {grep -EiIrl "$*" ./*} - jon3k, on 10/12/2007, -1/+4@bigkm
Explain to me how that's better than a recursive grep, or find'ing and then piping to xargs and grep? - MAdaXe42, on 10/12/2007, -2/+4Yep. F'rinstance, traineo.com was developed entirely in VIM, on linux.
- spudlyo, on 10/12/2007, -0/+2
When the load is 80 on a webserver and the I'm quickly losing control of the machine, you can bet your ass I do:
# killall -9 httpd
You make it sound like processes are soooo fragile you can't kill them. Do I care that the program can't catch the signal? No. Do I care if it doesn't exit cleanly? No. Do I care the program doesn't save whatever it's working on? No. When I want a program to DIE NOW I use -9. - Markie1006, on 10/12/2007, -0/+2I call BS.
The stock grep on my MBP running OS 10.4.8 is gnu grep, so it does have the -R option.
$ grep --version
grep (GNU grep) 2.5.1 - Blapto, on 10/12/2007, -0/+2Screen is a much much better alternative to the bg/fg/jobs, at least for the function it's being used here anyway.
- rspeed, on 10/12/2007, -0/+1I use this to test WebDAV:
curl -i -X propfind -u 'username' http://davurl
"-i" outputs headers
"-X" changes the method (to PROPFIND, in this case)
"-u" makes you authenticate - bluejet, on 10/12/2007, -9/+10@ipodsweatshop
Try some constructive criticism and simply give alternate and/or correct ways to do these tasks without the "this idiot is not a 1337 h4x0r like me" jive.
And put down the coffee...drink some chamomile or something. Geez. - geronimo, on 10/12/2007, -3/+4I use all of those regularly. Here's another goody:
% find / -name "*.blah" -type f -print0 | xargs -0 grep "contents"
...recursively looks for the string "contents" in all files of the form "*.blah".
the -print0 / -0 are key, without it, it won't be able to process files with spaces in the filename. - maddmike1959, on 10/12/2007, -3/+4Because we are tired of seeing digg posts about commands we learned 10 years ago....
- babakshirazi, on 10/12/2007, -1/+2
Yeah, RTFM. - jgtg32a, on 10/12/2007, -1/+2Thank you, the only one I didn't know was touch
- inactive, on 10/12/2007, -0/+1"cat /dev/null > foobar.log" is too much typing. Just "> foobar.log" will do the trick.
- jordan314, on 10/12/2007, -2/+3http://www.duggmirror.com
...load without java or css seems to work.
Where's that handy command to reverse the digg effect? - beeplogic, on 10/12/2007, -0/+1OR (to truncate a file)
: > file - prockcore, on 10/12/2007, -0/+1$ uname -sr
Darwin 7.9.0
$ grep --version
grep (GNU grep) 2.4.2
$ grep test -R *
grep: invalid option -- R - inactive, on 10/12/2007, -0/+1Very handy. I can't believe some people don't know about the tab key though. The one I find most useful is number 9. Another thing that should also be mentioned is the pipe "|", which I use all the time.
- sstidman, on 10/12/2007, -0/+1You might try using "pushd" and "popd" instead of "cd".
- sirmo, on 10/12/2007, -0/+1Don't do the command 12 on a Solaris box.
- crossmr, on 10/12/2007, -0/+1by 3 I mean #3.
- carterman, on 10/12/2007, -0/+1I think it is correct as is.. *nix for web developers.
- samuelcotterall, on 10/12/2007, -0/+1I'd say the majority of them do, seeing as the BASH command line is used in Linux/BSD, UNIX and OSX.
- inactive, on 10/12/2007, -0/+0Yes, but they are too busy sucking off their 'life partner' while doing a podcast on their Mac to learn these commands.
-
Show 51 - 76 of 76 discussions
Digg is coming to a city (and computer) near you! Check out all the details on our 
© Digg Inc. 2009
— Content created and posted by Digg users is 