Mastering the Awk Command in Linux

Introduction

AWK is a powerful text processing and data manipulation tool that’s particularly useful for working with structured data. Named after its creators (Aho, Weinberger, and Kernighan), AWK shines in pattern scanning, field processing, and generating reports.

Why Learn AWK?

• Process column-based data efficiently
• Perform mathematical operations on text data
• Generate formatted reports
• Transform data between different formats
• Combine with other Unix tools in pipelines

Basic AWK Syntax

The basic syntax for AWK is:

awk 'pattern { action }' input-file

AWK Cheat Sheet

Sample Data Setup

Let’s create test files for our examples:

echo "Name,Department,Salary
Alice,Engineering,65000
Bob,Sales,58000
Carol,Marketing,62000
David,Engineering,72000" > employees.csv

echo "1 Apple 2.99
2 Banana 1.50
3 Orange 3.25
4 Grape 4.75" > prices.txt

Basic Examples

Example 1: Print Entire File

awk '{print}' employees.csv

Example 2: Print Specific Fields

awk -F, '{print $1, $3}' employees.csv

Example 3: Filter Rows by Condition

awk -F, '$3 > 60000' employees.csv

Example 4: Add Line Numbers

awk '{print NR, $0}' prices.txt

Intermediate Examples

Example 5: Calculate Total Salary

awk -F, 'NR>1 {sum+=$3} END {print "Total Salary:", sum}' employees.csv

Total Salary: 257000

Example 6: Format Output with Headers

awk -F, 'BEGIN {print "Employee Report\n=============="}
NR==1 {print "Name: " $1; next}
{print "Name: " $1 ", Department: " $2}' employees.csv

Example 7: Field Calculations

awk '{total = $1 * $3; print $2, "total:", total}' prices.txt

Advanced Examples

Example 8: Pattern Matching with Regular Expressions

awk -F, '/Engineer/ {print $1 " is an engineer"}' employees.csv

Example 9: Using Associative Arrays (Count Department Members)

awk -F, 'NR>1 {dept[$2]++} END {for (d in dept) print d ":", dept[d]}' employees.csv

Example 10: Multi-file Processing

awk '{print FILENAME, NR, $0}' employees.csv prices.txt

Built-in Functions

String Functions

awk '{print toupper($2), length($2)}' prices.txt

Math Functions

awk '{print sqrt($3), int($3)}' prices.txt

Pro Tips

1. Combine with other commands:

ps aux | awk '$3 > 9 {print $1, $3}'

karna 10.6

1. Create AWK scripts:

awk -f script.awk data.txt

1. Use for data transformation:

awk -F, 'BEGIN {OFS="|"} {$1=$1; print}' employees.csv