gcc memory allocation issue - buffer overflow attack

Question

Does gcc do memory allocation intelligently to prevent buffer overflow attack?

int function(char *str) {
    int a = 0;                 // See the
    char b[16] = "abcd";       // changes here

    if(!strcmp(b, str))
        a = 1;

    return a;
}

and

int function(char *str) {
    char b[16] = "abcd";       // See the
    int a = 0;                 // changes here

    if(!strcmp(b, str))
        a = 1;

    return a;
}

When I debug it with gdb, it always allocate memory first to integer variables and then character array; no matter what is the order of variable declaration. i.e. In above both cases, compiler allocates memory first to a and then to b.

(higher address)
  Memory
|        |
|        |
+--------+
|        |
|        |
|        |
|        |
+--------+ <----- b (16 bytes)
|        |
+--------+ <----- a (4 bytes)
|        |
(lower address)

So, even if we supply more than 16 character in str, it can not affect value of a. Can anybody help me out here?

Thank you.

The Standard allows such reordering for any purposes, the compiler wants to accomplish. Most common - optimization. It's still "intelligence", but not for preventing "the buffer overflow attack". And it's still an undefined behavior (the buffer overflow) — Kiril Kirov
– Kiril Kirov, Commented Mar 19, 2013 at 9:15

Hasturkun · Accepted Answer · 2013-03-19 09:29:11Z

11

Yes, if run with the -fstack-protector flag.

When run with the flag, GCC adds stack canaries, sorts array variables to the highest part of the stack frame to make it more difficult to overflow them and corrupt other variables, and makes copies of the function arguments to be stored with the other locals.

See the Wikipedia page on Buffer overflow protection and the ProPolice homepage for more information

edited Mar 19, 2013 at 9:29

answered Mar 19, 2013 at 9:20

Hasturkun

36.7k6 gold badges78 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Hasturkun Over a year ago

@Ravi: Some distributions enable it by default. Try compiling with -fno-stack-protector and see if it makes a difference.

Hasturkun Over a year ago

@MM.: Added. Was mostly left out for brevity.

Ravi Over a year ago

You are right. GCC enable it by default. If we compile with -fno-stack-protector, it allocate memory sequentially in order of declaration rather than array variables to the highest part of the stack frame. Thank you

Lundin · Accepted Answer · 2013-03-19 11:43:40Z

Even if GCC has such a feature to protect against buffer overflows, there are many other considerations here that may cause a fixed variable declaration order. Where the declaration is made is not really important, the compiler will take allocation decisions based on when and how the variable is used in runtime.

Most importantly, the compiler will hopefully allocate variables in the stack frame with the best possible alignment in mind. This could be made in entirely different ways depending on CPU and optimizing setting. Optimize for speed may give a completely different allocation, compared to optimize for memory consumption. And most likely, it will put some variables in CPU registers, removing the whole RAM allocation need.

So to answer your question: GCC allocates variables in various ways, depending on compiler port. How it does so, is not something the programmer needs to overly concern themselves about. There may be options to rearrange the stack to protect against buffer overflow attacks, but that only makes sense in some types of applications. There might not even be any input to a particular system, for all we know. So it doesn't make sense for a compiler to have this security feature enabled by default.

Alexey Frunze · Accepted Answer · 2013-03-19 09:51:19Z

-1

Does gcc do memory allocation intelligently to prevent buffer overflow attack?

No, it doesn't. You cannot prevent an attack or a buffer overflow without bounds checking, which isn't always possible. You can only sometimes detect an overflow after the fact.

At best the compiler can include extra information (a so called canary value) near the return address on the stack and before returning from the function check that it's intact and not overwritten as a result of a buffer overflow.

edited Mar 19, 2013 at 9:51

answered Mar 19, 2013 at 9:14

Alexey Frunze

62.4k14 gold badges91 silver badges190 bronze badges

4 Comments

Hasturkun Over a year ago

This is wrong, GCC can and does reorder variables to prevent the effects of buffer overflow

Alexey Frunze Over a year ago

@Hasturkun You cannot prevent an overflow without bounds checking. You can only sometimes detect one.

Hasturkun Over a year ago

That's true. But you can mitigate them, as done by ProPolice, etc.

Alexey Frunze Over a year ago

@Hasturkun Prevention != mitigation.

Collectives™ on Stack Overflow

gcc memory allocation issue - buffer overflow attack

3 Answers 3

3 Comments

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related