Incorrect reshaping of 3d array

Question

I am doing some processing with a matrix of shape 1536 x 16 x 48. Here are some details about this dimensions:

1536: data collected in 6 seconds
16: number of collectors (or electrodes in my case)
48: number of samples

So, i have 48 samples of 1536 values (representing 6 seconds of data) from the perspective of 16 different collectors.

My goal is to do some processing with this matrix, but to do so some transformations are needed first.

Since 6 seconds is a large sequence, i want to split each of those samples into smaller sequences, specially 3 or 2 seconds. For instance, 1 sample of 6 seconds (1536) can be split into 2 samples of 3 seconds (768) or 3 samples of 2 seconds (512). The shape of this transformed matrix would go from 1536x16x48 to 768x16x96 (for 3 seconds) or 512x16x144 (for 2 seconds).
Once i have this new matrix, i want to reshape it so i get one 2d matrix per observer and all values organized in columns instead of rows (e.g. for 2 seconds split: 512x16x144 => 144x512x16).
Finally, i can now loop through 3rd dimension (16), do some computations (i.e. fast fourier transform) with each 2d matrix and reduce (sum) them all into a single one to get a final 144 x 512 matrix (in 2 seconds-split scenario).

The following code is what i made with numpy, but it is clearly wrong for me when i plot samples generated from this method.

def generate_fft_data(data,labels, n_seconds_split=3):
    x = 256 * n_seconds_split
    y = 16
    z = 48 * int(6/n_seconds_split)
    data = data.transpose(2,0,1).reshape(x,y,z).transpose(2,0,1)
    fft_data = []
    for electrode in range(data.shape[2]):
        y_t = fft(data[:,:,electrode])
        fft_data.append(np.abs(y_t))
    sum_of_ffts = np.add.reduce(fft_data) 
    return sum_of_ffts

I can provide more details if needed. Thanks in advance.

Thanks for the suggestion, but the hardest part for me now is (believe it or not) the reshaping/splitting part. — heresthebuzz
– heresthebuzz, Commented Feb 21, 2021 at 22:13

bb1 · Accepted Answer · 2021-02-22 01:29:00Z

1

You can try this:

def generate_fft_data(data,n_seconds_split=3):
    x_split = 6//n_seconds_split
    # split along 0 axis, stack pieces along the last axis and transpose
    arr = np.dstack(np.split(data, x_split)).transpose(2,0,1)
    fft_data = []
    for electrode in range(arr.shape[2]):
        y_t = fft(arr[:,:,electrode])
        fft_data.append(np.abs(y_t))
    sum_of_ffts = np.add.reduce(fft_data) 
    return sum_of_ffts

edited Feb 22, 2021 at 1:29

answered Feb 21, 2021 at 22:47

bb1

7,9332 gold badges11 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

heresthebuzz Over a year ago

Thanks. Just a remark: in your case, n_seconds_split doesn't represent the length of the desired sample, but the divisor that will split original samples instead (e.g. n_seconds_split=3 means samples with size 6/3 = 2)

hpaulj · Accepted Answer · 2021-02-22 06:18:11Z

0

In [88]: data = np.ones((1536,16,48))
In [89]:  x = 256 * 3
    ...:  y = 16
    ...:  z = 48 * int(6/3)
In [90]: x,y,z
Out[90]: (768, 16, 96)
In [91]: data.transpose(2,0,1).shape
Out[91]: (48, 1536, 16)
In [92]: data.transpose(2,0,1).reshape(x,y,z).shape
Out[92]: (768, 16, 96)

Reshaping [91] to [92] does not make sense. It works because the numbers 'add' up, but the distribution is messed up.

I think you want to first split the 1536 dimension into (2,768) (or the 3 equivalent), then move that 2 dimension to the end, and recombine it with the 48.

The details could vary, but here's one such sequence:

data1 = data.reshape(2,768,16,48).transpose(1,2,0,3).reshape(768,16,96)

answered Feb 22, 2021 at 6:18

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Collectives™ on Stack Overflow

Incorrect reshaping of 3d array

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related