Suppose I want to normalize a matrix A. I came across this code:
A_norm = (A / np.sqrt((A ** 2).sum(-1))[..., np.newaxis]).astype(np.float32)
We're subtracting a mean of 0 here, I assume. My problem is the denominator. We're taking the square root of something we've squared and summed, but I don't understand what.
Specifically, what does this do:
np.sqrt((A ** 2).sum(-1))[..., np.newaxis]
We have
np.sqrt((A ** 2).sum(-1))
which can be decomposed as
B = A**2
C = B.sum(-1)
D = np.sqrt(C)
where C is the row-sum of B, which has been flatten by the operation (the column sum would have been B.sum(0)) and D is the entrywise squareroot of C.
For A to be correctly normalized, we need the denominator to be correctly shaped, so as to divide the kth row of A by the kth term of D. Thus we must explicitly reshape the flatten result of np.sqrt(C) as being a column vector as follows
D = np.sqrt(C)[..., np.newaxis]
[..., np.newaxis]basically keeps the number of dims same that was reduced by the summing. Alternative is using keepdims` with summing. On the ellipsis...to extend the last dim, seehere. More info on ellipsis : stackoverflow.com/questions/772124/…keepdims, with example: stackoverflow.com/questions/40927156/…np.newaxisis something worth figuring out even though it’s confusing. It’s basically Numpy’s answer to the question, “How do we convert rank-1 arrays (vectors) to rank-2 and rank-3 and rank-N arrays (matrix-like arrays, volumes, N-dimensional volumes, respectively)”. Experiment with the following in Python (IPython, Jupyter Notebook, etc.):np.random.rand(3)[:, np.newaxis]andnp.random.rand(3,3)[:, :, np.newaxis], replacing the colors with...and replacingnp.newaxiswithNone.