Two-dimensional arrays in Python

Question

I'm reading in data for three separate cities and I want to keep each set of data in a two-dimensional array, but as I get past a part of my code, loops keep writing over things from my first two cities as I only have a one-dimensional array. Where should I set up these 2-D arrays to keep my files organized and what function and arguments should I use to do so?

Arrays should be 3X54 (3 for each city, 54 for each year of data)

EDIT: All the initial variables in the code below (i.e. precip, tmin, tmax) will have over 19,000 elements in them at the beginning which I end up averaging over each year later in the code.

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

city = ['Lubbock.txt','Erie.txt','Oslo.txt']
years = np.arange(1960,2014,1)
months_summer = range(5,8,1)

for x in range(0,len(city),1):

    data = np.genfromtxt(city[x], skip_header=2, usecols=(1), dtype=('S8'))
    data2 = np.genfromtxt(city[x], skip_header=2, usecols=(2,3,4))

    #ONLY GET 1-D ARRAY WHEN I ASK FOR SHAPE OF VARIABLE AFTER THIS POINT

    dates  = pd.DatetimeIndex(data[:])
    year   = dates.year
    month  = dates.month
    day    = dates.day
    precip = data2[:,0]/10.
    tmax   = data2[:,1]/10.
    tmin   = data2[:,2]/10.

    tmaxF  = (tmax*(9./5.))+32.
    tminF  = (tmin*(9./5.))+32.
    precipinches = precip*0.03937007874

    tmax_avg = []

    JJA3tmax_avg = []

    JJAtmax_avg = []

    DJFtmax_avg = []   

    for yr in years: 
        toavg = np.where(year == yr) 
        tmax_avg.append(np.average(tmax[toavg])) 


        for mo in months_summer:
            sumtoavg = np.where(month == mo)
            JJA3tmax_avg.append(np.average(tmax[sumtoavg]))


        JJAtmax_avg.append(np.average(JJA3tmax_avg))
        JJA3tmax_avg = []

        dec_this_year = (year == yr) & (month == 12)
        jan_next_year = (year == (yr+1)) & (month == 1)
        feb_next_year = (year == (yr+1)) & (month == 2)

        wintoavg = np.where(dec_this_year & jan_next_year & feb_next_year)

        DJFtmax_avg.append(np.average(tmax[wintoavg]))


    tmaxmean30 = np.average(tmax_avg[1:31])
    JJAtmaxmean30 = np.average(JJAtmax_avg[1:31])
    DJFtmaxmean30 = np.average(DJFtmax_avg[1:31])

#THIS IS THE DATA THAT I'M PLOTTING
    tmax_avg_dep = tmax_avg - tmaxmean30
    JJAtmax_avg_dep = JJAtmax_avg - JJAtmaxmean30
    DJFtmax_avg_dep = DJFtmax_avg - DJFtmaxmean30

You're importing pandas, but you don't really seem to be using it-- in particular, you don't seem to be taking advantage of groupby. — DSM
– DSM, Commented Feb 15, 2014 at 0:52
@DSM Do you have any suggestion on how to use the groupby function here? I don't know pandas well and just found that function to deal with making the 8-digit YYYYMMDD in my data into a string. — DJV
– DJV, Commented Feb 16, 2014 at 20:26

Community · Accepted Answer · 2020-06-20 09:12:55Z

5

This line:

data[:]

creates a shallow copy. You need a deep copy:

import copy
copy.deepcopy(data)

From the docs:

The difference between shallow and deep copying is only relevant for compound objects (objects that contain other objects, like lists or class instances):

A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.

A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Feb 15, 2014 at 0:07

mhlester

23.3k10 gold badges55 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

DJV Over a year ago

However, the size of data is not the size of how I need my arrays to be in the end. There are over 19,000 elements in there.

mhlester Over a year ago

Many times it's easier find the problem if you boil the code down to be Minimal, Complete, Tested and Readable. Otherwise it's a needle in a haystack. Pare it down to the smallest amount of code that demonstrates the problem

DJV Over a year ago

Done. All of this is just for one of the variables now.

Collectives™ on Stack Overflow

Two-dimensional arrays in Python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related