Reading complex data into numpy array

Question

I need to read complex numbers from a text file into a numpy array. My question is similar to this one Writing and reading complex numbers using numpy.savetxt and numpy.loadtxt however, the solution here is to alter the format the data is saved in. I don't have this luxury as the text file is generate by other software which I cannot change. A sample of the text file is a follows:

25 + 0i
8.43818 + -4.94194i
4.46817 + -5.08305i
4.55764 + -3.02201i
2.69138 + -5.43104i
-0.151334 + -4.83717i
1.98336 + -1.3339i
3.59002 + -0.932973i
1.42727 + -0.617317i
1.02005 + -1.14214i
-0.15564 + 2.74564i

I have tried the following:

np.loadtxt('file.txt',delimiter='\n',dtype=np.complex128)

However I get the error:

ValueError: complex() arg is a malformed string

The posts I have read suggest this is an issue with the + - notation in some lines, however, I get the same error even if the extra + signs are removed.

abarnert · Accepted Answer · 2014-07-29 19:14:38Z

however, the solution here is to alter the format the data is saved in

Good news, you don't have to!

numpy.loadtxt can take any iterable of lines, not just a file object.

So, you can wrap your file object in a simple generator that transforms the lines on the fly, and feed that to loadtxt, and everyone will be happy.

Like this:

def transform_complex(line):
    # insert your code here

with open('file.txt', 'rb') as f:
    lines = map(transform_complex, f)
    arr = np.loadtxt(lines, dtype=np.complex128)

(If you're using Python 2.x, and the file is large, you probably want to use itertools.imap rather than map.)

The "insert your code here" part, you fill in from the answer that worked, but wasn't an acceptable solution because it required modifying the files. Since I don't see such an answer in your link, I'm not sure what that is, but for example, maybe it's this:

def transform_complex(line):
    return line.replace(b'+ -', b'- ')

Testing things out locally, it looks like there are actually three things wrong with your input.

You can test what the output should look like using savetxt. For example:

>>> arr = np.array([1-2j])
>>> f = io.BytesIO()
>>> np.savetxt(f, arr)
>>> f.getvalue()
b' (1.000000000000000000e+00-2.000000000000000000e+00j)\n'

(In Python 2.x, you won't see the b prefix.)

Not all of those differences turn out to be relevant—you don't have to use exponential notation, you don't need parens, etc.—but it looks like these three are:

No spaces allowed around the + in complex numbers.
The imaginary unit has to be j, not i.
No +- allowed.

So:

def transform_complex(line):
    return line.replace(b' ', b'').replace(b'+-', b'-').replace(b'i', b'j')

Warren Weckesser · Accepted Answer · 2014-07-29 19:48:15Z

@abarnert's answer is fine, but don't forget that loadtxt has an argument, converters, that provides a hook for customizing how a field is processed. Here are a couple examples that show how it can be used to process this file.

In the first version, the default delimiter (whitespace) is retained, so there are three columns. usecols is used to ignore the middle column (the '+'). A converter is used to convert the last column (column 2) to a floating point value. It uses a slice to discard the last character, which is the 'i'. With these arguments, loadtxt returns an array with shape (11, 2) of floating point values. By calling the view method with the type np.complex128, the array is converted to an array of floating point values with shape (11, 1). Finally, indexing with [:,0] gives the 1D array of complex values. (A call to ravel, squeeze, or reshape could have been used instead.)

In [24]: loadtxt('file.txt', converters={2:lambda f: float(f[:-1])}, usecols=(0,2)).view(np.complex128)[:,0]
Out[24]: 
array([ 25.000000+0.j      ,   8.438180-4.94194j ,   4.468170-5.08305j ,
         4.557640-3.02201j ,   2.691380-5.43104j ,  -0.151334-4.83717j ,
         1.983360-1.3339j  ,   3.590020-0.932973j,   1.427270-0.617317j,
         1.020050-1.14214j ,  -0.155640+2.74564j ])

The next version treats the entire line as a single field. The following function converts a string in the format used in the file to a complex number:

In [36]: def to_complex(field):
   ....:     return complex(field.replace(' ', '').replace('+-', '-').replace('i', 'j'))
   ....:

E.g

In [39]: to_complex('8.43818 + -4.94194i')
Out[39]: (8.43818-4.94194j)

This call to loadtxt treats each line as a single field, and uses the converter to convert each field to a complex number:

In [37]: loadtxt('file.txt', converters={0: to_complex}, dtype=np.complex128, delimiter=';')
Out[37]: 
array([ 25.000000+0.j      ,   8.438180-4.94194j ,   4.468170-5.08305j ,
         4.557640-3.02201j ,   2.691380-5.43104j ,  -0.151334-4.83717j ,
         1.983360-1.3339j  ,   3.590020-0.932973j,   1.427270-0.617317j,
         1.020050-1.14214j ,  -0.155640+2.74564j ])

(The delimiter is set to ';', but it could have been any character that doesn't occur in the file.)

Collectives™ on Stack Overflow

Reading complex data into numpy array

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related