The first two examples are both class attributes. The reason they seem different is because you're not doing the same thing in both cases: you're assigning a new value in the first case and modifying the existing value in the second case.
Notice that you are not doing the same thing in the first two examples. In the first example you do a.foo = 5, assigning a new value. In the second example, if you did the analogous thing, assigning, a.foo = [5], you would see the same kind of result as in the first example. But instead you altered the existing list with a.foo.append(5), so the behavior is different. a.foo = 5 changes only the variable (i.e., what value it points to); a.foo.append(5) changes the value itself.
(Notice that there is no way to do the equivalent of the second example in the first example. That is, there's nothing like a.foo.add(1) to add 1 to 5. That's because integers are not mutable but lists are. But what matters is not that lists "are" mutable, but that you mutated one. In other words, it doesn't matter what you can do with a list, it matters what you actually do in the specific code.)
Also, notice that although the foo you defined in the class definition is a class attribute, when you do a.foo = 5, you are creating a new attribute on the instance. It happens to have the same name as the class attribute, but it doesn't change the value of the class attribute, which is what b.foo still sees.
The last example doesn't work because, just like in the first two examples, code inside the class block is at the class scope. There is no self because there are no instances yet at the time the class is defined.
There are many, many other questions about this on StackOverflow and I urge you to search and read a bunch of them to gain a fuller understanding of how this works.