Tuple NumPy Data Type

Question

I currently am reading in colors from an SQLite database in the following way:

import numpy as np, apsw
connection = apsw.Connection(db_name)
cursor = connection.cursor()
desc = {'names':('name','R','G','B'),'formats':('a3','float','float','float')}
colorlist = np.array(cursor.execute("SELECT name, R, G, B FROM Colors").fetchall(),desc)

But I was hoping to read in this data in a NumPy array with only two columns, where the second column is a tuple containing (R,G,B), i.e. something like:

desc = {'names':('name','Color'),'formats':('a3','float_tuple')}
colorlist = np.array(cursor.execute("SELECT name, R, G, B FROM Colors").fetchall(),desc)

I want to do this to simplify some of my later statements where I extract the color from the array as a tuple and to eliminate my need to create a dictionary to do this for me:

colorlist[colorlist['name']=='BOS']['Color'][0]

senderle · Accepted Answer · 2011-09-23 12:24:43Z

6

Do you literally need a tuple? Or do you just want the values to be grouped? You can create a numpy record array with arbitrary shapes for each of the fields...

>>> np.array([('ABC', (1, 2, 3)), ('CBA', (3, 2, 1))], dtype='3a, 3i')
array([('ABC', [1, 2, 3]), ('CBA', [3, 2, 1])], 
      dtype=[('f0', '|S3'), ('f1', '<i4', 3)])

This works even for n-dimensional arrays:

>>> np.array([('ABC', ((1, 2, 3), (1, 2, 3))), ('CBA', ((3, 2, 1), (3, 2, 1)))],
             dtype='a3, (2, 3)i')
array([('ABC', [[1, 2, 3], [1, 2, 3]]), ('CBA', [[3, 2, 1], [3, 2, 1]])], 
      dtype=[('f0', '|S3'), ('f1', '<i4', (2, 3))])

Partially applied to your specific problem:

>>> desc = {'names':('name','Color'),'formats':('a3','3f')}
>>> colorlist = np.array([('ABC', (1, 2, 3)), ('CBA', (3, 2, 1))], desc)
>>> colorlist[colorlist['name']=='ABC']['Color'][0]
array([ 1.,  2.,  3.], dtype=float32)

Using rec.fromarrays to generate a record array from two regular arrays:

>>> desc = {'names':('name','Color'),'formats':('a3','3f')}
>>> np.rec.fromarrays([['ABC', 'CBA'], [(1, 2, 3), (3, 2, 1)]], desc)[0][1]
array([ 1.,  2.,  3.], dtype=float32)

A full solution:

color_query = cursor.execute("SELECT R, G, B FROM Colors").fetchall()
name_query = cursor.execute("SELECT name FROM Colors").fetchall()
desc = {'names':('name','Color'),'formats':('a3','3f')}
colorlist = np.rec.fromarrays([color_query, name_query], desc)

If for some reason you can't split the query like that, you'll just have to split the results of the query, perhaps using a list comprehension:

colorlist = np.rec.fromarrays([[row[0]  for row in query], 
                               [row[1:] for row in query]], desc)

edited Sep 23, 2011 at 12:24

answered Sep 22, 2011 at 14:00

senderle

152k36 gold badges218 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

hotshotiguana Over a year ago

An example of the sqlite result from the cursor.execute(QUERY).fetchall() statement looks like this: [('BOS',.96,.85,.80)]. If I can find a way to get sqlite to return R, G, and B grouped, then this is a great solution, but I am not sure if that is possible.

senderle Over a year ago

See above for a couple of possible solutions.

Collectives™ on Stack Overflow

Tuple NumPy Data Type

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related