Numpy array modification in-place

Question

I have a file that looks like this:

row  column  layer value1  value2      
8  454  1  0.000e+0 1.002e+4
8  455  1  0.000e+0 1.001e+4
8  456  1  0.000e+0 1.016e+4
8  457  1  0.000e+0 1.016e+4
.
.
.

I want to do some calculations on the last column (for example multiply by 10) and save it (in-place or as a new file) without changing the format. I know how to load it but I don't know how to continue. I do the following to load the data:

import numpy as np

ic = np.genfromtxt("file.dat",skip_header=1, usecols=(0,1,2,4), 
                    dtype=None, names = ['row', 'column', 'layer', 'value2'])

the file is abound 150M, so fast execution would be helpful.

Did you try printing the result to see what is where is it? That's what Python's interactive console is for. If it's large, you can print just a few rows with ic[:<number>]. — ivan_pozdeev
– ivan_pozdeev, Commented Apr 29, 2017 at 2:28

MassPikeMike · Accepted Answer · 2017-04-29 02:36:51Z

1

Your example has columns indexed only 0 to 4, so usecols=(0,1,2,5) produces an error with the file in your example. Assuming usecols=(0,1,2,4):

You can modify the array in-place with

for i in range(0,len(ic)):
 ic[i]['value2'] *= 10

and save it to tab-delimited text with

np.savetxt("mul.dat", ic, fmt="%d %d %e %e", delimiter="\t", header=" ".join(ic.dtype.names))

producing

# row column layer value2
8 454 1.000000e+00 1.002000e+05
8 455 1.000000e+00 1.001000e+05
8 456 1.000000e+00 1.016000e+05
8 457 1.000000e+00 1.016000e+05

But you won't be able to write out the value1 column if your usecols(0,1,2,4) caused it never to be read in.

answered Apr 29, 2017 at 2:36

MassPikeMike

6823 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bob Over a year ago

It works great. I corrected the index in the main post. I am an old FORTRAN programmer, still Python 0-indexing makes me confessed.

Collectives™ on Stack Overflow

Numpy array modification in-place

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related