If ... ..... .... I ..... ..... were ...... ... ... ... to . ...... . write .... .... .... . a ... ...... ... ..... sentence, .. ... but ...... .. drip ...... . ..... ...... ... the ..... ... .. ...... ...... words . . .. out ..... .. ...... ..... . slowly ..... ... ...... with .... ... ..... ... long ... . ... ..... pauses ...... . . .... ... between ..... them, .... ...... I ...... ..... ..... ...... could .... .. ... .... burn ... .... up .. . . ...... a ... ..... .. lot ..... ...... ...... of ..... ... ...... . ..... your ..... .. time, . dear ..... .. .. reader, . .... ..... as ...... .... ...... ... you .... . .. parsed .. .. ... ..... ..... through .... ...... . ...... what ..... .... I'm .. .. ..... saying . ... ... and .... .. come ...... ..... ...... ... ... to ...... the .... the . ..... . end ..... ..... .. ..... .... sentence. .... ...... ....

And if I was to ask you a question. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then I would end up burning up a lot of your time again.

Although we're not rude and inconsiderate of each other in this way face to face, such inconsiderateness is often shown - be it accidental, unthought, or intentional by browsers and browsing programs when they visit web sites, and web site administrators must make consideration of such activity, which can be ongoing 24 hours a day, 7 days a week, when they make their web sites visible.

The text above generated by this Python program - you didn't really expect me to handcode all that did you! ... See our Python courses where you can learn this (and other more conventional) uses of the language!

Our web servers get accessed from time to time by thoughless people such as I've described above (and by thoughtful people who intentionally do this sort of thng to lots of people, looking for security holes), and we need to keep an eye on our loadings and watch how our servers are doing. An old blog (about 4 years ago) tells you how we do the monitoring and graphing - it's here and the techniques are still current and valid, and a very recent discussion / item on our First Great Western Coffee Shop Forum - [here] - shows you how we located and overcame some issues a couple of weeks ago, looking at server log files, using Perl scripts (described here) to analyse the daily logs and find the needle that's causing the pain in the haystack of valid traffic.

From the last 48 hours, and again during the night just gone, I noticed some noise in the standard pattern that I expect to see from the server graphs ... here is the current [16:00 update] graph as I write:



And I pick out from that:

a) A big peak one evening. Not a problem, as that's the time that server backups were processing; I would prefer the peak to be not quite so high during this procedure, and indeed I introduced a recovery delay at a couple of points during the backup procedure recently, whilst making sure that each distinct web site is backed up without long gaps so that any content changes during the procedure will not cause a problem ("syncronisation").

b) A rising level of traffic yesterday, with the orange line being noticably above the lines for previous days all the way through the evening. Using the Perl scripts linked to above, I was rapidly able to take a look at the log files through a filter and see that one single IP address was requesting our hotel guest book that goes into each room ... and requesting it again and again - in total there were over 56,000 requests at one second intervals. You'll notice today's black curve in the graph above distinctly drops from around 03:00 when I told our server it could stop answering these requests for 2 Mbytes per second!

c) Sudden upward loading spikes - including one after I had fixed (b) at about 07:40 this morning. Taking a look at the server status pages (which are available to me as server admin), I notice a rather curious pattern during the spike of busyness:



The diagram is showing all the current threads accessing the web server. (Please ask if you would like me to teach you about these things, and / or take a look at your server!). And each of those lines marked "reading" is a remote browser that is dripping a question in, ever so slowly and with lots of pauses in between just as the first text I started this article with. Result - everyone else who's making normal traffic requests is having to wait until there's a thread available, and / or lots more threads are opening on the server and the machine's getting rather full.

The solution - something we've done before on another server - is to make our server a bit less patient, and to give up more quickly on requests that are dripping in slowly. The resulting server status list looks more like



which I can assure you is much more like what I expect so be seeing. As a reader of this article, you might not appreciate just what is and isn't right for these diagrams - they're things to look and and learn on your server, and learn about Python and / or Perl too so that you can do the extra analyses to look for patterns when things aren't quite as you would expect ... and do so soon, rather than waiting to when you have problems to resolve and can't take a dynamic look at the "when it was working" case.
(written 2014-09-27)

 Y108 - Python - String Handling
  [324] The backtick operator in Python and Perl - (2005-05-25)
  [463] Splitting the difference - (2005-10-13)
  [496] Python printf - (2005-11-15)
  [560] The fencepost problem - (2006-01-10)
  [773] Breaking bread - (2006-06-22)
  [903] Pieces of Python - (2006-10-23)
  [943] Matching within multiline strings, and ignoring case in regular expressions - (2006-11-25)
  [954] Splitting Pythons in Bradford - (2006-11-29)
  [970] String duplication - x in Perl, * in Python and Ruby - (2006-12-07)
  [1110] Python - two different splits - (2007-03-15)
  [1195] Regular Express Primer - (2007-05-20)
  [1517] Python - formatting objects - (2008-01-24)
  [1608] Underlining in Perl and Python - the x and * operator in use - (2008-04-12)
  [1876] Python Regular Expressions - (2008-11-08)
  [2284] Strings as collections in Python - (2009-07-12)
  [2406] Pound Sign in Python Program - (2009-09-15)
  [2692] Flexible search and replace in Python - (2010-03-25)
  [2721] Regular Expressions in Python - (2010-04-14)
  [2765] Running operating system commands from your Python program - (2010-05-14)
  [2780] Formatted Printing in Python - (2010-05-25)
  [2814] Python - splitting and joining strings - (2010-06-16)
  [3090] Matching to a string - what if it matches in many possible ways? - (2010-12-17)
  [3218] Matching a license plate or product code - Regular Expressions - (2011-03-28)
  [3349] Formatting output in Python through str.format - (2011-07-07)
  [3468] Python string formatting - the move from % to str.format - (2011-10-08)
  [3469] Teaching dilemma - old tricks and techniques, or recent enhancements? - (2011-10-08)
  [3796] Backquote, backtic, str and repr in Python - conversion object to string - (2012-07-05)
  [3886] Formatting output - why we need to, and first Python example - (2012-10-09)
  [4027] Collections in Python - list tuple dict and string. - (2013-03-04)
  [4152] Why are bus fares so high? - (2013-08-18)
  [4213] Formatting options in Python - (2013-11-16)
  [4360] Python - comparison of old and new string formatters - (2014-12-22)
  [4593] Command line parameter handling in Python via the argparse module - (2015-12-08)
  [4595] Python formatting update - including named completions - (2015-12-10)
  [4659] Prining a pound sign from Python AND running from the command line at the same time - (2016-03-03)

A603 - Web Application Deployment - Further httpd Configuration
  [345] Spotting a denial of service attack - (2005-06-12)
  [466] Separating 'per instance' data from binaries and web sites - (2005-10-16)
  [526] Apache httpd - serving web documents from different directories - (2005-12-12)
  [550] 2006 - Making business a pleasure - (2006-01-01)
  [631] Apache httpd to Tomcat - jk v proxy - (2006-03-03)
  [649] Denial of Service ''attack'' - (2006-03-17)
  [662] An unhelpful error message from Apache httpd - (2006-03-30)
  [755] Using different URLs to navigate around a single script - (2006-06-11)
  [853] To list a directory under httpd on a web server, or not? - (2006-09-02)
  [934] Clustering, load balancing, mod_rewrite and mod_proxy - (2006-11-21)
  [1009] Passing GET parameters through Apache mod_rewrite - (2006-12-27)
  [1080] httpd.conf or .htaccess? - (2007-02-14)
  [1121] Sharing the load with Apache httpd and perhaps Tomcat - (2007-03-29)
  [1207] Simple but effective use of mod_rewrite (Apache httpd) - (2007-05-27)
  [1351] Compressing web pages sent out from server. Is it worth it? - (2007-09-14)
  [1355] .php or .html extension? Morally Static Pages - (2007-09-17)
  [1377] Load Balancing with Apache mod_jk (httpd/Tomcat) - (2007-10-02)
  [1381] Using a MySQL database to control mod_rewrite via PHP - (2007-10-06)
  [1551] Which modules are loaded in my Apache httpd - (2008-02-23)
  [1554] Online hotel reservations - Melksham, Wiltshire (near Bath) - (2008-02-24)
  [1564] Default file (MiMe types) for Apache httpd and Apache Tomcat - (2008-03-04)
  [1566] Strange behaviour of web directory requests without a trailing slash - (2008-03-06)
  [1619] User and Group settings for Apache httpd web server - (2008-04-22)
  [1636] What to do if the Home Page is missing - (2008-05-08)
  [1707] Configuring Apache httpd - (2008-07-12)
  [1762] WEB-INF (Tomcat) and .htaccess (httpd) - (2008-08-20)
  [1767] mod_proxy and mod_proxy_ajp - httpd - (2008-08-22)
  [1778] Pointing all the web pages in a directory at a database - (2008-08-30)
  [1939] mod_proxy_ajp and mod_proxy_balancer examples - (2008-12-13)
  [1954] mod_rewrite for newcomers - (2008-12-20)
  [1955] How to avoid duplicating web page maintainance - (2008-12-20)
  [1974] Moving a directory on your web site - (2009-01-03)
  [2060] Database connection Pooling, SSL, and command line deployment - httpd and Tomcat - (2009-03-01)
  [2272] Monitoring and loading tools for testing Apache Tomcat - (2009-07-07)
  [2478] How did I do THAT? - (2009-10-26)
  [2900] Redirecting a page - silent, temporary or permanent? - (2010-08-03)
  [3133] An image from a website that occasionally comes out as hyroglyphics - (2011-01-14)
  [3449] Apache Internal Dummy Connection - what is it and what should I do with it? - (2011-09-19)
  [3635] Parse error: parse error, unexpected T_STRING on brand new web site - why? - (2012-03-03)
  [3862] Forwarding a whole domain, except for a few directories - Apache http server - (2012-09-17)
  [3955] Building up from a small PHP setup to an enterprise one - (2012-12-16)
  [4001] Helping search engines with appropriate 400 error codes - (2013-02-11)

A606 - Web Application Deployment - Apache httpd - log files and log tools
  [376] What brings people to my web site? - (2005-07-13)
  [1237] What proportion of our web traffic is robots? - (2007-06-19)
  [1503] Web page (http) error status 405 - (2008-01-12)
  [1598] Every link has two ends - fixing 404s at the recipient - (2008-04-02)
  [1656] Be careful of misreading server statistics - (2008-05-28)
  [1761] Logging Cookies with the Apache httpd web server - (2008-08-20)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [1796] libwww-perl and Indy Library in your server logs? - (2008-09-13)
  [3015] Logging the performance of the Apache httpd web server - (2010-10-25)
  [3019] Apache httpd Server Status - monitoring your server - (2010-10-28)
  [3027] Server logs - drawing a graph of gathered data - (2010-11-03)
  [3087] Making the most of critical emails - reading behind the scene - (2010-12-16)
  [3443] Getting more log information from the Apache http web server - (2011-09-16)
  [3447] Needle in a haystack - finding the web server overload - (2011-09-18)
  [3491] Who is knocking at your web site door? Are you well set up to deal with allcomers? - (2011-10-21)
  [3554] Learning more about our web site - and learning how to learn about yours - (2011-12-17)
  [3670] Reading Google Analytics results, based on the relative populations of countries - (2012-03-24)
  [3974] TV show appearance - how does it effect your web site? - (2013-01-13)
  [3984] 20 minutes in to our 15 minutes of fame - (2013-01-20)
  [4404] Which (virtual) host was visited? Tuning Apache log files, and Python analysis - (2015-01-23)
  [4491] Web Server Admin - some of those things that happen, and solutions - (2015-05-10)

G903 - Well House Consultants - Running and moderating forums and social media sites
  [22] Falling out over the silliest things - (2004-08-21)
  [29] Silence is Golden - (2004-08-26)
  [115] Expiration dates or times on web pages - (2004-11-12)
  [130] Spelling and grammar - (2004-11-25)
  [204] The confidence to allow public comments - (2005-02-06)
  [231] Feedback as lifeblood - (2005-02-28)
  [248] Use me, but use me effectively - (2005-03-16)
  [424] How not to run a forum - (2005-08-24)
  [516] Open source questions? Anyone can ask. - (2005-12-03)
  [651] Please Register with Opentalk - but just once! - (2006-03-19)
  [806] Check your user is human. Have him retype a word in a graphic - (2006-07-17)
  [828] Freedom of speech and freedom to post - (2006-08-10)
  [841] Forum help - a push in the right direction - (2006-08-21)
  [919] Freedom for X is denial of privacy for Y - (2006-11-09)
  [923] Why shouldn't I spam? - (2006-11-13)
  [948] Running an on line campaign - (2006-11-27)
  [1088] Why use BBC code not HTML? - (2007-02-21)
  [1190] Save the Forum - A regular clean sweep - (2007-05-17)
  [1362] No Thank You - (2007-09-23)
  [1472] The Horse goes on and on - (2007-12-15)
  [1485] Copyright and theft of images, bandwidth and members. - (2007-12-26)
  [1523] Ive just received an email from myself. Should I be worried? - (2008-01-29)
  [1532] Comment spam blocked. Please comment via Forums - (2008-02-05)
  [1539] A forum is not always the best vehicle - (2008-02-14)
  [1563] Guidlines for posting on a forum - (2008-03-04)
  [1569] I dont care - goodbye - (2008-03-09)
  [1578] Please don't shout at me! - (2008-03-16)
  [1595] First Great Western Weekend - (2008-03-30)
  [1678] Software - changes and delays. But courses must run on time! - (2008-06-15)
  [1759] While the world sleeps ... - (2008-08-19)
  [1923] Making it all worthwhile - (2008-12-04)
  [1972] Pettifog and forum boards away from public view - (2009-01-03)
  [2103] Ask the Tutor - Open Source forum - (2009-03-25)
  [2116] Why do we delay new forum members through authorisation? - (2009-04-03)
  [2156] Stopping forum spam - control of the signup process - (2009-05-04)
  [2162] Admins thoughts on banning a member from a forum - (2009-05-09)
  [2177] Preventing forum spam - checks at sign up - (2009-05-12)
  [2254] Forum membership - a privilege not a right - (2009-06-22)
  [2386] Computing under the influence of alcohol - (2009-08-29)
  [2526] A reluctance to move from old shoes to new - (2009-12-05)
  [2527] Flying tonight - (2009-12-05)
  [2569] How to run a successful online poll / petition / survey / consultation - (2010-01-10)
  [2781] The 500 pound question to get you started - (2010-05-26)
  [2820] Netiquette for forum newcomers - (2010-06-20)
  [3479] Practical Extraction and Reporting - using Python and Extreme Programming - (2011-10-14)
  [3910] Identifying your real customers and keeping them well informed fast - (2012-11-02)
  [4017] Acceptable User Policy / vexatious interacter - (2013-02-24)
  [4025] Backups, Codebase, Strategy and more - dealing with forum incidents - (2013-03-03)
  [4065] Handling requests to a forum - the background process - (2013-04-17)
  [4234] Change to Libel and Defamation laws from 1st January 2014 - (2013-12-31)
  [4239] Facebook marketing - early experiences - (2014-01-19)
  [4283] Can a legitimate forum post become illegal a year later? - (2014-07-11)
  [4315] Welcoming genuine forum posters quickly - but turning away off topic advertisers - (2014-11-16)
  [4403] The unbalanced relationship between customer and provider - (2015-01-21)
  [4492] Almost so wrong, but perhaps it's right for some? - (2015-05-11)

Back to Four time target - good news. Four time prediction - poor forecasting.

Previous and next or Horse's mouth home

Forward to What can you and I learn from online quizzes?

Melksham Campus - how is it going - October 2014
Problem ... I want to print a series of numbered forms
Even in the dark of night, the train comes bearing passengers
What can you and I learn from online quizzes?
Identifying and clearing denial of service attacks on your Apache server
Four time target - good news. Four time prediction - poor forecasting.
Learning to program in Java - yes, we can help.
Please do not ask me to be the chair!
Libre Office - unable to get past REOPEN WINDOWS? question
Sunday is never quiet at Well House Manor