How to use the awk command on linux

Print a Text File

Print Specific Field

Use : as the input field separator and print first field only i.e. usernames (will print the the first field. all other fields are ignored):
awk -F’:’ ‘< print $1 >‘ /etc/passwd
Send output to sort command using a shell pipe:
awk -F’:’ ‘< print $1 >‘ /etc/passwd | sort

Pattern Matching

You can only print line of the file if pattern matched. For e.g. display all lines from Apache log file if HTTP error code is 500 (9th field logs status error code for each http request):
awk ‘$9 == 500 < print $0>‘ /var/log/httpd/access.log
The part outside the curly braces is called the “pattern”, and the part inside is the “action”. The comparison operators include the ones from C:

If no pattern is given, then the action applies to all lines. If no action is given, then the entire line is printed. If “print” is used all by itself, the entire line is printed. Thus, the following are equivalent:
awk ‘$9 == 500 ‘ /var/log/httpd/access.log
awk ‘$9 == 500 ‘ /var/log/httpd/access.log
awk ‘$9 == 500 ‘ /var/log/httpd/access.log

Print Lines Containing tom, jerry AND vivek

Print pattern possibly on separate lines:
awk ‘/tom|jerry|vivek/’ /etc/passwd

Print 1st Line From File

awk “NR==1” /etc/resolv.conf
awk “NR==$line” /etc/resolv.conf

Simply Arithmetic

You get the sum of all the numbers in a column:
awk ‘ END ‘ earnings.txt
Shell cannot calculate with floating point numbers, but awk can:
awk ‘BEGIN

Call AWK From Shell Script

A shell script to list all IP addresses that accessing your website. This script use awk for processing log file and verification is done using shell script commands.

AWK and Shell Functions

Here is another example. chrootCpSupportFiles() find out the shared libraries required by each program (such as perl / php-cgi) or shared library specified on the command line and copy them to destination. This code calls awk to print selected fields from the ldd output:

  • No ads and tracking
  • In-depth guides for developers and sysadmins at Opensourceflare✨
  • Join my Patreon to support independent content creators and start reading latest guides:
    • How to set up Redis sentinel cluster on Ubuntu or Debian Linux
    • How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
    • How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
    • A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)

Join Patreon

This function can be called as follows:
chrootCpSupportFiles /lighttpd-jail /usr/local/bin/php-cgi

AWK and Shell Pipes

List your top 10 favorite commands:
history | awk ‘‘ | sort | uniq -c | sort -rn | head
Sample Output:

Another example to find out domain expiry date:
$ whois | awk ‘/Registry Expiry Date:/ < print $4 >‘
Sample outputs:

Awk Program File

You can put all awk commands in a file and call the same from a shell script using the following syntax:
awk -f mypgoram.awk input.txt

Awk in Shell Scripts – Passing Shell Variables TO Awk

You can pass shell variables to awk using the -v option:

I have 2 data files each containing one column. I want to make another data file by merging both the columns. I have the command line in shell but I don’t know how it works.

Please explain elaborately the below command:

I used the command above to write a shell script and got following output:

I want to know how this command line is working on this example to give the output?

How to use the awk command on linux

1 Answer 1

Well, part of learning to use Unix is to figure out what existing scripts are doing. In this case you need to know a bit about how awk works to understand the code. I will focus on describing the awk part, this should get you started in figuring out the rest.

Basically awk is a pattern-driven scripting language, where commands consist of both a (search) pattern/condition and a corresponding code block. During execution, any input files are read line by line and if the pattern/condition is true for a line, the code block is executed. There are special patterns BEGIN and END which are used to trigger code to get executed before the first line or after the last line is read.

In your example you have three pattern/code lines:

NR and FNR are two special variables set by awk . You can look up their meaning with man awk to see that

so basically this condition is true while lines from the first line are read (which means that a[i++]=$0 is executed once for each line from the first file) and false for all additional files. $0 is the current line of input.

This code block has no condition/pattern so it gets executed for every line read (from all files including the first one).

This part runs after the last line of the last file has been read and processed.

With these basics you should be able to figure out the meaning of the different code blocks and variables yourself.