Linux Tips

Simple "nsping"

2008-11-04T03:27:00.000-08:00

there's no need to install a separate utility to run "nsping", you have "dig" already ...

A simple network diagnostic tool to determine the health and reachability of name servers is nsping. as the name suggests you ping a name server, not with an ICMP echo request but with a (random) lookup. the time interval it takes to get a reply back is what you're after. serious lags can indicate network issues. the tool nsping is a standalone binary, and on most UN*X systems it's another package to install. however, almost everyone already has dig installed, part of the BIND package. dig is a complex name server query and diagnostic tool. one useful feature of it is that it reports the amount of time it takes to perform it's query in milliseconds ... exactly what we're after. so, let's use dig to do exactly what nsping does and trim down the answer to look like nsping. all we have to do is a random lookup and report only the query time. this simple shell hack uses the built in random number generator from ksh (i think the tool can work in bash, too) and awk to trim down the query from dig.

#!/bin/ksh
SERVER=$1
dig @${SERVER} -t a ${RANDOM}.${RANDOM}${RANDOM}.${RANDOM} | \
        awk '{if ($0 ~/Query/) print $4" "$5}'

put this in your path and ping a name server:

$ nsping ns.oreilly.com     
97 msec

i have used this in the past to query the root nameservers as a network connectivity monitor (with much success).
See also: man dig
"dns & bind", the book from o'reilly
"sed & awk", another book from o'reilly

[Contributed by: jose nazario]

Introduction to Named Pipes (by Andy Vaught )

2008-09-15T02:05:00.000-07:00

One of the fundamental features that makes Linux and other Unices useful is the “pipe”. Pipes allow separate processes to communicate without having been designed explicitly to work together. This allows tools quite narrow in their function to be combined in complex ways.

A simple example of using a pipe is the command:
ls | grep x

When bash examines the command line, it finds the vertical bar character | that separates the two commands. Bash and other shells run both commands, connecting the output of the first to the input of the second. The ls program produces a list of files in the current directory, while the grep program reads the output of ls and prints only those lines containing the letter x.

The above, familiar to most Unix users, is an example of an “unnamed pipe”. The pipe exists only inside the kernel and cannot be accessed by processes that created it, in this case, the bash shell. For those who don't already know, a parent process is the first process started by a program that in turn creates separate child processes that execute the program.

The other sort of pipe is a “named” pipe, which is sometimes called a FIFO. FIFO stands for “First In, First Out” and refers to the property that the order of bytes going in is the same coming out. The “name” of a named pipe is actually a file name within the file system.
Pipes are shown by ls as any other file with a couple of differences:
% ls -l fifo1
prw-r--r-- 1 andy users 0 Jan 22 23:11 fifo1|

The p in the leftmost column indicates that fifo1 is a pipe. The rest of the permission bits control who can read or write to the pipe just like a regular file. On systems with a modern ls, the | character at the end of the file name is another clue, and on Linux systems with the color option enabled, fifo| is printed in red by default.

On older Linux systems, named pipes are created by the mknod program, usually located in the /etc directory. On more modern systems, mkfifo is a standard utility. The mkfifo program takes one or more file names as arguments for this task and creates pipes with those names. For example, to create a named pipe with the name pipe1 give the command:
mkfifo pipe

The simplest way to show how named pipes work is with an example. Suppose we've created pipe as shown above. In one virtual console1, type: (read more)

ls -l > pipe1

and in another type:
cat <>

Voila! The output of the command run on the first console shows up on the second console. Note that the order in which you run the commands doesn't matter. If you haven't used virtual consoles before, see the article “Keyboards, Consoles and VT Cruising” by John M. Fisk in the November 1996 Linux Journal. If you watch closely, you'll notice that the first command you run appears to hang. This happens because the other end of the pipe is not yet connected, and so the kernel suspends the first process until the second process opens the pipe. In Unix jargon, the process is said to be “blocked”, since it is waiting for something to happen. One very useful application of named pipes is to allow totally unrelated programs to communicate with each other. For example, a program that services requests of some sort (print files, access a database) could open the pipe for reading. Then, another process could make a request by opening the pipe and writing a command. That is, the “server” can perform a task on behalf of the “client”. Blocking can also happen if the client isn't writing, or the server isn't reading. Pipe Madness Create two named pipes, pipe1 and pipe2. Run the commands:
echo -n x | cat - pipe1 > pipe2 & cat pipe1

On screen, it will not appear that anything is happening, but if you run top (a command similar to ps for showing process status), you'll see that both cat programs are running like crazy copying the letter x back and forth in an endless loop.

After you press ctrl-C to get out of the loop, you may receive the message “broken pipe”. This error occurs when a process writing to a pipe when the process reading the pipe closes its end. Since the reader is gone, the data has no place to go. Normally, the writer will finish writing its data and close the pipe. At this point, the reader sees the EOF (end of file) and executes the request.

Whether or not the “broken pipe” message is issued depends on events at the exact instant the ctrl-C is pressed. If the second cat has just read the x, pressing ctrl-C stops the second cat, pipe1 is closed and the first cat stops quietly, i.e., without a message. On the other hand, if the second cat is waiting for the first to write the x, ctrl-C causes pipe2 to close before the first cat can write to it, and the error message is issued. This sort of random behavior is known as a “race condition”.
Command Substitution

Bash uses named pipes in a really neat way. Recall that when you enclose a command in parenthesis, the command is actually run in a “subshell”; that is, the shell clones itself and the clone interprets the command(s) within the parenthesis. Since the outer shell is running only a single “command”, the output of a complete set of commands can be redirected as a unit. For example, the command:
(ls -l; ls -l) >ls.out

writes two copies of the current directory listing to the file ls.out.

Command substitution occurs when you put a <> in front of the left parenthesis. For instance, typing the command:
cat <(ls -l) results in the command ls -l executing in a subshell as usual, but redirects the output to a temporary named pipe, which bash creates, names and later deletes. Therefore, cat has a valid file name to read from, and we see the output of ls -l, taking one more step than usual to do so. Similarly, giving >(commands) results in Bash naming a temporary pipe, which the commands inside the parenthesis read for input.

If you want to see whether two directories contain the same file names, run the single command:
cmp <(ls /dir1) <(ls /dir2) The compare program cmp will see the names of two files which it will read and compare. Command substitution also makes the tee command (used to view and save the output of a command) much more useful in that you can cause a single stream of input to be read by multiple readers without resorting to temporary files—bash does all the work for you. The command: ls | tee >(grep foo | wc >foo.count) \ >(grep bar | wc >bar.count) \ | grep baz | wc >baz.count

counts the number of occurrences of foo, bar and baz in the output of ls and writes this information to three separate files. Command substitutions can even be nested:
cat <(cat <(cat <(ls -l)))) works as a very roundabout way to list the current directory. As you can see, while the unnamed pipes allow simple commands to be strung together, named pipes, with a little help from bash, allow whole trees of pipes to be created. The possibilities are limited only by your imagination.

System Information

2008-09-05T07:32:00.000-07:00

This is a linux command line reference for system information operations

uname -a
Show kernel version and system architecture

head -n1 /etc/issue
Show name and version of distribution

cat /proc/partitions
Show all partitions registered on the system

grep MemTotal /proc/meminfo
Show RAM total seen by the system

grep "model name" /proc/cpuinfo
Show CPU(s) info

lspci -tv
Show PCI info

lsusb -tv
Show USB info

mount | column -t
List mounted filesystems on the system (and align output)

Tcpdump

2008-08-27T06:09:00.000-07:00

Tcpdump is a really great tool for network security analyst, you can dump packets that flows within your networks into file for further analysis. With some filters you can capture only the interested packets, which it reduce the size of saved dump and further reduce loading and processing time of packets analysis.

This post will only covers the fundamental of tcpdump usage, bare in mind tcpdump can do much much more than what I illustrate here.

Lets start with capturing packets based on network interface, ports and protocols. Let assume I wanna capture tcp packets that flow over eth1, port 6881. The dump file with be save as test.pcap.

tcpdump -w test.pcap -i eth1 tcp port 6881

Simple right? What if at the same time I am interested on getting packets on udp port 33210 and 33220?

tcpdump -w test.pcap -i eth1 tcp port 6881 or udp \( 33210 or 33220 \)

‘\’ is an escape symbol for ‘(’ and ‘)’. Logic OR implies PLUS (+). In plain text is I want to capture tcp packets flows over port 6881 plus udp ports 33210 and 33220.

Careful with ‘and’ in tcpdump filter expression, it means intersection. Thats why I put ‘or’ instead of and within udp port 33210 and 33220. The usage of ‘and’ in tcpdump will be illustrate later.

Ok, how about reading pcap that I saved previously?

tcpdump -nnr test.pcap

The -nn is to tell tcpdump not to resolve DNS on IP and Ports, where r is read.

Adding -tttt to makes the timestamp appears more readable format.

tcpdump -ttttnnr test.pcap

How about capture based on IP ?
You need to tell tcpdump which IP you are interested in? Destination IP? or Source IP ? Let say I wanna sniff on destination IP 10.168.28.22 tcp port 22, how should i write?

tcpdump -w test.pcap dst 10.168.28.22 and tcp port 22

So the ‘and’ makes the intersection of destination IP and port.

By default the sniff size of packets is 96 bytes, you somehow can overload that size by specified with -s.

tcpdump -w test.pcap -s 1550 dst 10.168.28.22 and tcp port 22

Some version of tcpdump allows you to define port range. You can as bellow for capturing packets based on a range of tcp port.

tcpdump tcp portrange 20-24

Bare in mind, the line above I didn’t specified -w which it won’t write to a file but i will just print the captured packets on the screen.