Print a Text File
Print Specific Field
Use : as the input field separator and print first field only i.e. usernames (will print the the first field. all other fields are ignored):
awk -F’:’ ‘< print $1 >‘ /etc/passwd
Send output to sort command using a shell pipe:
awk -F’:’ ‘< print $1 >‘ /etc/passwd | sort
You can only print line of the file if pattern matched. For e.g. display all lines from Apache log file if HTTP error code is 500 (9th field logs status error code for each http request):
awk ‘$9 == 500 < print $0>‘ /var/log/httpd/access.log
The part outside the curly braces is called the “pattern”, and the part inside is the “action”. The comparison operators include the ones from C:
If no pattern is given, then the action applies to all lines. If no action is given, then the entire line is printed. If “print” is used all by itself, the entire line is printed. Thus, the following are equivalent:
awk ‘$9 == 500 ‘ /var/log/httpd/access.log
awk ‘$9 == 500
awk ‘$9 == 500
Print Lines Containing tom, jerry AND vivek
Print pattern possibly on separate lines:
awk ‘/tom|jerry|vivek/’ /etc/passwd
Print 1st Line From File
You get the sum of all the numbers in a column:
Shell cannot calculate with floating point numbers, but awk can:
Call AWK From Shell Script
A shell script to list all IP addresses that accessing your website. This script use awk for processing log file and verification is done using shell script commands.
AWK and Shell Functions
Here is another example. chrootCpSupportFiles() find out the shared libraries required by each program (such as perl / php-cgi) or shared library specified on the command line and copy them to destination. This code calls awk to print selected fields from the ldd output:
- No ads and tracking
- In-depth guides for developers and sysadmins at Opensourceflare✨
- Join my Patreon to support independent content creators and start reading latest guides:
- How to set up Redis sentinel cluster on Ubuntu or Debian Linux
- How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
- How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
- A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)
Join Patreon ➔
This function can be called as follows:
chrootCpSupportFiles /lighttpd-jail /usr/local/bin/php-cgi
AWK and Shell Pipes
List your top 10 favorite commands:
history | awk ‘
Another example to find out domain expiry date:
$ whois cyberciti.com | awk ‘/Registry Expiry Date:/ < print $4 >‘
Awk Program File
You can put all awk commands in a file and call the same from a shell script using the following syntax:
awk -f mypgoram.awk input.txt
Awk in Shell Scripts – Passing Shell Variables TO Awk
You can pass shell variables to awk using the -v option:
I have 2 data files each containing one column. I want to make another data file by merging both the columns. I have the command line in shell but I don’t know how it works.
Please explain elaborately the below command:
I used the command above to write a shell script and got following output:
I want to know how this command line is working on this example to give the output?
1 Answer 1
Well, part of learning to use Unix is to figure out what existing scripts are doing. In this case you need to know a bit about how awk works to understand the code. I will focus on describing the awk part, this should get you started in figuring out the rest.
Basically awk is a pattern-driven scripting language, where commands consist of both a (search) pattern/condition and a corresponding code block. During execution, any input files are read line by line and if the pattern/condition is true for a line, the code block is executed. There are special patterns BEGIN and END which are used to trigger code to get executed before the first line or after the last line is read.
In your example you have three pattern/code lines:
NR and FNR are two special variables set by awk . You can look up their meaning with man awk to see that
so basically this condition is true while lines from the first line are read (which means that a[i++]=$0 is executed once for each line from the first file) and false for all additional files. $0 is the current line of input.
This code block has no condition/pattern so it gets executed for every line read (from all files including the first one).
This part runs after the last line of the last file has been read and processed.
With these basics you should be able to figure out the meaning of the different code blocks and variables yourself.