Effect on server when memory runs out and swapping starts

For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))

Here's an interesting example from a web server showing what happens when real memory runs out and the system starts swapping ... it can be something of a downwards spiral in terms of performance, as the majority of the resources are essentially tied up by the swapping / keeping going at all once memory is full, and a hard to recover backlog of processes can build up.

First, an example where the system is running in memory:

And this is what can happen with a slightly higher arrival rate, and / or a slightly slower processing rate such that a queue builds up:

You'll notice by comparing these two (look at the purple highlights):

1. The load average has rocketed from 0.23 jobs waiting (average in last minute) to 22.59 jobs.

2. The cpu utilisation remains low - under 10% - for user processes

3. The cpu idle time has dropped from nearly 90% to zero, with all the extra resource being burned up by device wait time - 93.6% no less.

4. Swap space has only increased slightly in its use, but every process in this machine is active and the whole thing hsa started thrashing which means a queue has built up ...

5. There's been a rise from 115 to 191 total tasks (remember that the majority of the initial 115 will be operating system background processes, so that this is a truly massive increase)

6. The process table itself is showing details of just how / where the queue has built up.

See here for some tuning information for httpd if it's the web server that's consuming all the space (there are MySQL notes there too!)
(written 2009-02-26)

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles

W603 - Web and Intranet - Server Side Technologies
  [642] How similar are two words - (2006-03-11)
  [653] Easy feed! - (2006-03-21)
  [732] Where is a web site visitor browsing from - (2006-05-24)
  [1020] Parallel processing in PHP - (2007-01-03)
  [1031] robots.txt - a clue to hidden pages? - (2007-01-13)
  [1355] .php or .html extension? Morally Static Pages - (2007-09-17)
  [1365] Korn Shell scripts on the web - (2007-09-25)
  [1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
  [1615] PHP training courses every month - (2008-04-18)
  [1749] Using server side and client side programming together - (2008-08-11)
  [2282] Checking robots.txt from Python - (2009-07-12)
  [3705] Django Training Courses - UK - (2012-04-23)
  [3915] How does PHP work? - (2012-11-07)
  [4277] Sending a message to the server and changing text on a page when a button is pressed - (2014-05-23)

G913 - Well House Consultants - Hardware and Devices
  [630] Hostnames spring forward - (2006-03-02)
  [1281] Latercomer to the technology - (2007-07-29)
  [1827] What a shock - (2008-10-08)
  [1993] Load Balancing - Hardware or Software? - (2009-01-15)
  [2033] Huawei D100 Wireless Router - Mobile Internet - (2009-02-10)
  [2188] Camera to record where a picture was taken - (2009-05-18)
  [2667] Web page to telephone calls / links using an iPhone - (2010-03-08)
  [3058] MacBook Air - hardware and system review - (2010-11-17)
  [3075] Change of balance, of attitude, and of work method - (2010-11-30)
  [3457] Away to train - but still around by video for Melksham meetings - (2011-09-25)
  [3596] Want to learn iPad and iPhone programming? Come along and learn with me for free. - (2012-01-28)
  [4134] Setting up your MacBook Air as a mobile broadband router - (2013-07-07)

Back to
Tuning httpd / the supermarket checkout comparison

Previous and next
or
Horse's mouth home

Forward to
Web Site Loading - experiences and some solutions shared

Some other Articles

Sharing the load between servers - httpd and Tomcat
Invoker and cgi servlets on Tomcat 6
Train and Coach fares from London (and airports) to Melksham
Web Site Loading - experiences and some solutions shared
Effect on server when memory runs out and swapping starts
Tuning httpd / the supermarket checkout comparison
What a difference a MySQL Index made
How was my web site compromised?
A Presentation about our company - web and PHP
Why the Pony Tail?

4759 posts, page by page

Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

Like this? ??