Web Application Deployment - Linux Utilities

Web Application Deployment module A166

Linux Utilities

Exercises, examples and other material relating to training module A166. This topic is presented on public courses Deploying LAMP - Linux, Apache, MySQL, Perl/PHP, Linux Basics, Deploying Java Applications on Linux / Unix

Related technical and longer articles

Korn Shell Scripting - an introduction
Korn Shell - History, storing to file, commands
awk - pattern scanning and formatting
Korn Shell - Basics

Articles and tips on this subject		updated
4682	One line scripts - Awk, Perl and Ruby The art of the one line script, originating in awk and passing through Perl lives on in Ruby! WomanWithCat:4c grahamellis$ awk '/Upper/{c++}; END { print(c,"of",NR)}; BEGIN {c=0}' rstats2015.txt 5 of 2537 WomanWithCat:4c grahamellis$ WomanWithCat:4c grahamellis$ ...	2016-05-20
4586	Extending your bash shell with aliases, functions and extra commands Want a command to do some composite job, but it doesn't exist? You can add new commands into your bash script using functions ... see example [here] from last week's course. aliases ... here are the aliases that I'm using to provide my own extra / modified commands on our web server: -bash-4.1$ ...	2015-11-28
3902	Shell - Grep - Sed - Awk - Perl - Python - which to use when? Last week, I found myself teaching a Multi-vendor Advanced Unix Data Tools and Techniques course as a guest presenter. The tools that the course 'majored' on are grep, sed, awk and Perl ... being an advanced course, some knowledge was assumed ahead of time, and so reference was made to other utilities ...	2012-10-23 (longer)
3764	Shell, Awk, Perl of Python? Very frequently, IT system users find they've got a data file in one format and they need to filter it and transform it. On Linux and Unix systems, utilities such as grep and cut were the tradtions, perhaps with some awk. Perl provided a single-program replacement for shell scripts that bolted together ...	2012-06-16
3446	Awk v Perl "The AWK utility is a data extraction and reporting tool" - Wikipedia "Perl was developed as a language to make report processing easier" - Wikipedia So what's the difference? Looking through all the lines of a file, report the 6th and 1st fields on each line that contains the string "pix" and report ...	2011-09-18
2638	Finding what has changed - Linux / Unix If you want to find out what has changed in a file, use the diff command. And if you want to find out what has changed across a whole directory structure, use diff -r If I want to compare a backup with my current system - to see what has been altered (perhaps when something has gone wrong and I want ...	2010-02-17
2484	Finding text and what surrounds it - contextual grep grep is a very useful tool for finding all the lines in a file (or series of files) that contain a particular string or match some other pattern / criteria. For example, I have a file that contains data for 52 staff members and I want to find who knows Lua ... [trainee@melksham Download]$ grep Lua ...	2009-10-30
2320	Helping new arrivals find out about source code examples On 30th July, nearly 5300 people arrived at source code examples on our web site from Google searches, from 4000 unique IP addresses. How do I know that? From analysing our log files: -bash-3.2$ grep 'GET \/resources\/ex\.php.*\.google\.' ac_20090730 \| grep -v Googlebot \| wc 5382 129055 1832168 -bash-3.2$ ...	2009-08-04
2145	Using the internet to remotely check for power failure at home (PHP) Do you have freezers and other electrical equipment at home and worry about whether they'll be all right while you're away on holiday? Do you have a broadband connection with a rarely changing IP address, and a web site that you host with a web space provider? Then you'll be able to set up a "heartbeat" ...	2009-04-30
1690	Conversion of c/r line ends to l/f line ends Situation - Excel saved file that I want to read into a Lua program I'm writing on Linux. End of line characters on the Excel file are [cr], and Linux used [lf]. Quick solution - sed sed 's/\x0d/\x0a/g' stationusage.txt > st2.txt Literally "substitute globally every hex 0D with a hex 0A" ...	2008-06-28 (short)
1366	awk - a powerful data extraction and manipulation tool Unix and Linux come with a whole toolkit of utility programs for filtering, sorting and manipulating data. As a "one off" job, you can use them from the command line, and for repeated jobs you can save a series of instructions for these tools in a shell script. Yesterday, I ran a private shell scripting ...	2007-09-25
1361	Korn shell course It's been quite a while since I presented a course that majored on the Korn Shell - I know the subject right enough (having written a complete 2 day course that got rave if niche reviews during a previous lifetime), and I jumped at a chance to brush up on my old skills to day, to recall the differences ...	2007-09-22
63	Almost like old times It felt almost like old times yesterday - now that we're doing a LAMP deployment course (with the "L" standing for Linux if anyone's new to that term), customers have started to ask for the on site Linux element tailored / with a few additions. I found myself writing new material to cover Linux Utilities ...	2006-06-05

Examples from our training material

as1	awk - BEGIN, END and patterns
as2	awk - match in whole line or on specific field
as3	Using awk to select and format data, and calculate
aspers.txt	grep example output
demo.awk	awk script to analyse a data file
demo.pl	Sample Perl script to analyse data file
gents.dat	Input data for utilities
ladies.dat	input data for utilities
lsof_extract.awk	awk program to analyse and report on listing of open files
requests.txt	input data for utilities

Background information

Some modules are available for download as a sample of our material or under an Open Training Notes License for free download from [here].

Topics covered in this module

Introduction.
Piping and redirection of utilities.
Regular Expressions.
Options and related commands.
Grep, sort, awk, sed cut and Perl.
Find, man and info.

Complete learning

If you are looking for a complete course and not just a information on a single subject, visit our Listing and schedule page.

Well House Consultants specialise in training courses in Linux and LAMP,Ruby, Lua, Python, Perl, PHP, and MySQL. We run Private Courses throughout the UK (and beyond for longer courses), and Public Courses at our training centre in Melksham, Wiltshire, England. It's surprisingly cost effective to come on our public courses - even if you live in a different country or continent to us.

We have a technical library of over 700 books on the subjects on which we teach. These books are available for reference at our training centre.