A stack buffer overflow is what happens when a program makes an entry into a computer's memory that is longer than the allocated space. In some circumstances this can corrupt other data in memory, causing problems in the running of the computer. In other cases a stack buffer overflow can be exploited by a malicious application to take control of other parts of the computer.
The name from a stack buffer overflow derives first from the stack, which is effectively an active list in which data is organized: the term comes from the analogy of piling up physical items. One version of this, the call stack, is the list that a computer program uses to keep track of the various parts of the program, the subroutines, that are operating at any particular moment. As the call stack operates on a temporary basis and needs to be accessed quickly, it is located in the computer's memory rather than in permanent storage such as the hard drive.
Because modern operating systems allow for multiple programs to run at once, there is a need to organize the way memory is allocated, including to call stacks. This is done efficiently by assigning buffers, a space in the memory designed to be big enough to cope with the maximum space required by a particular buffer. In most cases some of the space will be unused, so it acts as a buffer between the data from different applications, leaving room to cope if one suddenly requires extra space. One way to visualize it would be to think of a library that allocated a certain amount of space to each subject area, making sure to leave some extra room to cope if, for example, all the books on one subject were in the library at the same time, rather than any of them being checked out.
A stack buffer overflow occurs when a program writes call stack data to the buffer in a way that exceeds the allocated space. This can happen by mistake, usually through a bug in a program. For example, if an application is designed to allow the user to type in a telephone number, but has no limit on the number of characters that can be entered, a hacker may be able to use the limitless field to intentionally cause a stack buffer overflow. Depending on the way the operating system works, this could allow the hacker to indirectly access either another application or the operating system itself.
There are several approaches to mitigating against the effects of a stack buffer overflow. One is known as address space layout randomization. This arranges the most important areas of data on the computer in a random way. The idea is that even if a hacker does cause or exploit a stack buffer overflow, he will not be able to exploit the breach in a reliable manner.