*nix commands I can’t do without
Unix/Linux/*nix survival 101
Let me start with the obvious: I’m definitely not a unix guru by any means. I do however use it on a daily basis for basic build/development oriented tasks, so I know enough to get by. Since my friend just installed his first ever linux distribution (CentOS, Huzzah!), I thought I’d write something up on some common unix commands that help me get through the day.
grep [command flags] [search text] [filename]
grep (global | regular expression | print) is the file text search command. Give it a regular expression and it will print out what it finds in the file indicated by filename. Here’s an example:
[root@bedrock some_jboss_folder]$ grep html readme.html
<!DOCTYPE html PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN”>
<html><head>
<meta content=”text/html” http-equiv=”content-type”>
<a href=”http://docs.jboss.org/html”>here</a>.</li>
<li><a href=”http://www.jboss.org/index.html?module=bb”>JBoss
Server is licensed under the <a href=”lgpl.html”>LGPL</a>,
</body></html>
Some useful flags include -R (recurse into sub directories), -c (show just the total match count), -m NUM (return the NUM number of results), and -i (ignore upper/lower case).
ps aux | grep [search text]
This is a command you can use to get information about what processes the kernel is currently running. Adding the pipe after the ps command feeds the listing results to the grep search command. This is particularly useful when you want to look for a specific set of procs run by a user or script. Here’s an example:
[root@bedrock ~]$ ps aux | grep jboss
jboss 10910 0.0 0.1 4884 1176 ? S Feb04 0:00 /bin/sh /server/jboss/bin/run.sh -c services -b 192.168.1.253 -Djava.net.preferIPv4Stack=true
jboss 10932 0.2 36.4 1089728 371456 ? Sl Feb04 31:43 java -Dprogram.name=run.sh -Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/server/jboss/lib/endorsed -classpath /server/jboss/bin/run.jar org.jboss.Main -c services -b 192.168.1.253 -Djava.net.preferIPv4Stack=true
500 20300 0.0 0.0 4200 700 pts/0 S+ 20:43 0:00 grep jboss
ps (process status) fetches a list of running pocsses. ax flags the command to return a listing of all procs. u flags to also list the user that the proc is running as. I use the grep to figure out if a jboss server is up and running, and sometimes to see what input parameters it used on startup – like what ip it bound to : “-b 192.168.1.253″. The results above lists first the user and process id, and then information about the proc.
netstat -ntalp | grep [search text]
This command must be run as root, but it lets you get a listing of network ports that are currently in use. This is particularly useful when trying to figure out port conflicts or to see if a particular server is listening on the correct port.
[root@bedrock ~]# netstat -ntalp | grep java
tcp 0 0 192.168.1.253:8009 0.0.0.0:* LISTEN 10932/java
tcp 0 0 192.168.1.253:4713 0.0.0.0:* LISTEN 10932/java
tcp 0 0 192.168.1.253:4457 0.0.0.0:* LISTEN 10932/java
tcp 0 0 192.168.1.253:1098 0.0.0.0:* LISTEN 10932/java
tcp 0 0 192.168.1.253:1099 0.0.0.0:* LISTEN 10932/java
tcp 0 0 192.168.1.253:8080 0.0.0.0:* LISTEN 10932/java
You can grep for port, ip/domain, status etc.
kill [signal flag] [process id]
This is the standard “kill process”, “terminate it dead” command. Usually when a proc refuses to shut down and all hell is breaking loose, and you can’t take no for an answer, signal flag “-9″ will insta kill the proc. You can get the process id from the “ps aux | grep” command.
root@bedrock jboss]$ kill -9 10932
Here I took the process id from the jboss script that was running from the ps aux | grep command example listed above. Use ps aux to figure out which process id you want to terminate.
./run.sh [args]
This is the standard syntax for invoking a script, assuming you have run privileges. In windows you’d just type in the name of the script, but in unix you should prefix the script name with “./”.
As Dave Cheney explains in a comment:
-
The reason you have to put “./” as a prefix to a script in your current working directory is the search path for executable programs does not (generally) include “.”
To the shell, “.” expands to the current directory so ./run.sh is equivalent to /home/kevin/run.sh (for example). As you have provided a full absolute path, the shell will not have to try the prefixes available in your $PATH environment.
So essentially, by adding the “./” before the script name you feed the shell a fully qualified executable path to the script you want to run, that way it doesn’t have to guess where your script is. So if the script is named run.sh and your current working directory is in a folder named bin, you can invoke it like this:
root@bedrock bin]$ run.sh -c services -b 192.168.1.253 -Djava.net.preferIPv4Stack=true
If your script takes parameters, you can pass them into the script after the script name.
tail [-f or -NUM] [path to file]
tail is a command that outputs the contents of a file to the terminal window. If you use the “-f” flag, it’ll continuously read the file as its contents grows. If you feed it a line count like “-1000″ it will output the 1000 most recent line entries of the file. We’ll say something like – “Hey, I’m gonna tail the logs while the server starts up”. This means we’re monitoring the logs using this tail command. And knowing is half the battle.
Musk explains a better alternative that allow you to drop out of follow and search:
-
You do not need tail use less +F or press Shift-F while in less and it will follow the currently choosen file if content is added.
Example: log.txt
less +F log.txt and you will have the same behaviour as when using tail -f log.txt except that you can use CTRL+C to drop out of follow mode and then use the search features available in less.
chown -R [group].[user]
This changes a file or directory’s owners to a new group/owner. The -R flag tell is to recurse the command into sub directories.
root@bedrock jboss]$ chown -R jboss.jboss
This command will work assuming there is a group and user named jboss, and it will change all files and folders in the current directory and lower to jboss.
chmod -R [permissions] [filename/expression]
This will set the permissions for the implied files to the new set of permissions listed. The mode can be indicated as either a string explanation of what each group can do or a 4 octal digit equivalent number.
[root@bedrock some_jboss_folder $ chmod ug=rwx,o=rw readme.html
[root@bedrock some_jboss_folder $ chmod 0775 readme.html
In the first example, we set the file owner (u) and group (g) to allow read (r), write (w) and execute (x). Then we set everyone else's (o) permissions to read and write only, no execute. In the second example, we set it to 0775, which is the octal digit representation of the first command. 0777 will set read/write/exectue permissions allowed to everyone, its the same as ugo=rwx.
vi [filename]
Basic text editor *nix usually ships with. It will open up the indicated file in read mode, and if it doesn’t exist will let you create a new text file without saving to disk. To enter editor mode, hit the Insert key, you can then edit the file. After you make your edits, hit the Escape key to get into command line mode. If you want to save the file, enter “:w”. If you then want to quit, type in “:q”.
[root@bedrock some_jboss_folder ]$ vi readme.txt
<ul>
<li>lib/ – the same
static library jars with a few jars, as most have moved to top level common/lib</li>
</ul≶
:w
“readme.txtl” 718L, 36365C written
ping -c [ip/domain]
Pings an ip or domain with a packet of data. Unlike the windows cousin, you have to either pass in the number of times to ping (-c NUM) or hit control+c to stop pinging.
[root@bedrock some_jboss_folder ]$ ping -c 4 localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=1 ttl=64 time=0.040 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=3 ttl=64 time=0.033 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=4 ttl=64 time=0.048 ms— localhost.localdomain ping statistics —
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.033/0.041/0.048/0.009 ms
top
This command takes over your terminal window and fills it with a listing of all the procs that are currently running, along with instruction crunching information. Hitting the < and > will scroll you through the results. q will quit top, returning you to the linux prompt. This is what it looks like:
top – 23:12:30 up 40 days, 16:38, 1 user, load average: 0.06, 0.02, 0.00
Tasks: 158 total, 1 running, 121 sleeping, 36 stopped, 0 zombie
Cpu(s): 0.0%us, 0.2%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1018232k total, 1003040k used, 15192k free, 138436k buffers
Swap: 2064376k total, 30236k used, 2034140k free, 312912k cachedPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2165 smmsp 20 0 9208 748 640 S 0.0 0.1 0:00.40 sendmail
1359 rpcuser 20 0 2988 560 556 S 0.0 0.1 0:00.03 rpc.statd
1346 rpc 20 0 2404 556 504 S 0.0 0.1 0:02.62 rpcbind
1 root 20 0 2012 624 560 S 0.0 0.1 0:04.71 init
man [command name]
If you need more detail on a specific command, you can get help from the unix manual by invoking man:
[root@bedrock ~]# man top
TOP(1) Linux Userâs Manual TOP(1)NAME
top – display Linux tasksSYNOPSIS
top -hv | -bcHisS -d delay -n iterations -p pid [, pid ...]The traditional switches â-â and whitespace are optional.
DESCRIPTION
The top program provides a dynamic real-time view of a running system. It can display system summary informa-
tion as well as a list of tasks currently being managed by the Linux kernel. The types of system summary
information shown and the types, order and size of information displayed for tasks are all user configurable
and that configuration can be made persistent across restarts.
ls [list flag] [path to directory]
This prints out a listing of the indicted directory’s contents, or the current directory if no path is supplied. -l lists one file/directory per line of output, and -a lists everything including files that start with a period.
[root@bedrock ~]# ls -la
total 168
drwxr-x—. 10 root root 4096 2009-11-20 23:09 .
drwxr-xr-x. 30 root root 4096 2010-02-07 22:32 ..
-rw——-. 1 root root 1675 2009-11-11 18:55 anaconda-ks.cfg
-rw——-. 1 root root 21354 2010-02-10 03:39 .bash_history
-rw-r–r–. 1 root root 18 2009-03-30 07:51 .bash_logout
-rw-r–r–. 1 root root 176 2009-03-30 07:51 .bash_profile
-rw-r–r–. 1 root root 176 2004-09-22 23:59 .bashrc
drwx——. 3 root root 4096 2009-11-12 02:17 .config
-rw-r–r–. 1 root root 100 2004-09-22 23:59 .cshrc
drwx——. 3 root root 4096 2009-11-11 19:06 .dbus
cat [filename1] [filename2] > [outputfile]
cat lets you concatenate and output the contents of a file of multiple files to the terminal window, or write it to a file if you include the “>” operator. Thanks Kevin. Here’s an example:
tests.txt
[root@bedrock ~]# vi test.txt
concatenate me!
this is a test
concatenate.txt
[root@bedrock ~]# vi concatenate.txt
a file that needs to be concatenatedagain
output.txt
[root@bedrock ~]# cat test.txt concatenate.txt > output.txt
The result
[root@bedrock ~]# more output.txt
concatenate me!
this is a test
a file that needs to be concatenatedagain
sed -i ’s/[some_text/other_text]/’ [filename]/
sed – stream editor for filtering and transforming text (blatantly stolen from “man sed”’s documentation). This command will replace “some_text” with “other_text” in the file indicated. One occurrence per line is replaced. Thanks for this one Silvery.
Consider the file “test.txt”:
[root@bedrock jboss]# more test.txt
this is a file
this ia another file
lets faceroll files
ok
And this is what happens when we run sed on it:
[root@bedrock jboss]# sed -i ’s/file/folder/’ test.txt
[root@bedrock jboss]# more test.txt
this is a folder
this ia another folder
lets faceroll folders
ok
[root@bedrock jboss]#
more/less
more/less – enables you to view the contents of a file within a page on the screen. Once you are browsing the contents, you can hit “s” or “f” to scroll multiple lines of text. “v” will fire up an editor at the current line you’re working on. If you have a large list of files and want to check them one page at a time, you could try “ls | less”. Thanks again Kevin.
clear
Clears the visible screen of text, starting your prompt at the top of the window.
mkdir [directory name]
This command simply creates a directory with default permissions and ownership.
cp -R [source] [destination directory]
Copies a file/folder from one location into another. -R flags to copy recursively.
mv [source] [destination directory]
Renames a source directory or folder to a new location/name.
rm -Rf [folder/file]
Deletes a file. -R flags to delete recursively. When invoked on a directory it would normally go line by line asking you if you want to delete such-and-such file, use the “f” flag to force delete and skip the file by file questions.
cd [path to new directory]
cd changes the current directory to the path indicated. A “..” means to move up one directory. If the path begins with “/” it means start from the disk root folder. Anything else implies a relative path to the new folder.
- -color=auto
In his comment, Ryan Fox points out:
- The `- -color=auto` option adds colour to the output of some commands, like ls or grep. In ls, the colours change depending on the file type, permissions, etc. In grep, it will highlight the text that matched your regex.
Here’s an example:
[root@bedrock jboss]# ps aux | grep jboss – -color=auto
root 5215 0.0 0.0 4200 712 pts/1 S+ 05:56 0:00 grep jboss –color=auto
jboss 10910 0.0 0.1 4884 1176 ? S Feb04 0:00 /bin/sh /server/jboss/bin/run.sh -c services -b 192.168.1.253
Open ended
I’m sure there must be other useful commands I have missed. If anyone has any other suggestions to add/edit these entries, please feel free to comment and I’ll update accordingly.
About this blog..
This blog is a collection of java development oriented articles mixed in with some unix, sql, xsl/xml, and other little gems that help make development just a little easier.
Feel free to leave comments, suggestions, qestions and feedback on the posts.