The Problem
The C programming language is a fixed array size language - you write the size of your variables into your code and that's how big they are. Overrun an array and you corrupt other memory - and it's up to
you the programmer to ensure this doesn't happen. Many past security leaks, such as the codered virus and the Solaris Sendmail hole had as their root causes array overruns that were not checked.
There is a small help in compile time constants -
#defines through the C pre-preprocessor which allow builds with different settings and array lengths (
example here) but that isn't a true solution -
what if you want to read a file with an indeterminate number of lines, all of which are of an unknown length? Surely there much be a better way that putting on an absurd-high limit? Well ... there is ...
The Solution
Malloc, calloc and realloc are standard C functions that let you allocate memory in a different area - know as
the heap - dynamically at run time. Your variable names do NOT point directly at the memory allocated - instead you access the memory through pointers (lots of pointer examples
here).
Taking my example problem above, I want to hold an unknown number of character strings in an array. I start by allocating a pointer to an array of strings which are themselves (you may recall) pointers / arrays. Thus
char ** info = NULL;
That's going to work well as it's a fixed size item, and it's the only memory we allocate in the main program area - the rest will be done through C's memory allocation functions.
We read a line in from the file (into, let's say, a single buffer called
line which has a limit of 4k set on it). And if we have got a line, we then use
realloc to allocate or expand the memory that infor points at:
info = realloc(info,(counter+1) * sizeof(char *));
In other words ... "take the old memory block at the info address, change its size to be one more than the number of lines that you've read so far, and return the new address into info". Oh - and the first time, info will start off as NULL so it will be a new allocation.
Very often, info will come back with the same value as it had before, but if the management algorithms in realloc have had to move it, you'll get a fresh pointer back. It's very clever!
Moving on, we use calloc - a single shot memory allocation function - to allocate the memory for each of the character strings within our array, based on the line length in each case:
llen = strlen(line);
info[counter] = calloc(sizeof(char),llen+1);
If we were storing just a fixed value for each line, we could do the whole thing with just a single realloc - but this is a complete and more realistic example of how, in C, you would slurp a whole file with no limit as to its size into an efficiently structured and allocated area of memory.
The complete working example from which I have taken the source code sample lines above is available
here ... yet another new example, updated last week during our
Public C Programming Course ... next course, June 2008 ... see our
course index for later dates.
(written 2008-03-22)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
C212 - C and C based languages - Memory Management [1497] Training Season Starts again! - (2008-01-07)
[1581] What is an lvalue? (Perl, C) - (2008-03-18)
[1670] Dynamic Memory Allocation in C - (2008-06-09)
[1845] Passing a table from Lua into C - (2008-10-18)
[2669] Efficient use of dynamic memory - C and realloc - (2010-03-10)
[2848] C course - final course example puts it all together - (2010-07-02)
[3118] Arrays of arrays - or 2D arrays. How to program tables. - (2011-01-02)
[3144] Setting up arrays in C - fixed size at compile time, or dynamic - (2011-01-24)
[3386] Adding the pieces together to make a complete language - C - (2011-08-11)
[3416] Storing Tcl source code encoded, and running via your own C program - (2011-09-02)
[4128] Allocating memory dynamically in a static language like C - (2013-06-30)
[4340] Simple C structs - building up to full, dynamic example - (2014-12-03)
[4634] Regression testing - via a very short C testing framework - (2016-01-29)
[4635] Encapsulating logic in functions and structs - the C approach to Object Oriented techniques - (2016-01-30)
Some other Articles
Keep the client experience easy - single server contact pointSetting up a new user - Linux or UnixEaster Sunday at 404, The SpaAll Change, Portsmouth HarbourDynamic Memory Allocation in C - calloc, reallocString interpretation in RubySome Ruby programming examples from our courseVariable types in RubyWell House Consultants / Manor - StaffUsing Structs and Unions together effectively in C