Python read multiple .txt and plot in same figure

Question

I have two .txt files of this form with thousands of rows

episode:2840  score:6.0  memory length:700345  epsilon:0.5576491831247304   steps:359    recent reward:5.95
episode:2841  score:4.0  memory length:700629  epsilon:0.5574965123606621   steps:284    recent reward:5.96
episode:2842  score:5.0  memory length:700911  epsilon:0.5573449596383373   steps:282    recent reward:5.93
Max lifes :5
episode:2800  score:7.0  memory length:687331  epsilon:0.5646918805492361   steps:268    recent reward:6.04
episode:2801  score:5.0  memory length:687678  epsilon:0.5645029051613243   steps:347    recent reward:6.03
episode:2802  score:6.0  memory length:688027  epsilon:0.564312906709082   steps:349    recent reward:6.03

I want to plot how the recent_reward column, changes over the episodes. After I remove the Max lifes :5 and some rows where the episode repeats, I get as a result two different plots, one for every file. However, the goal is to plot them together in one figure, to show the difference of the two rewards of the files over episodes. Could somebody help please?

This is my code :

import matplotlib.pyplot as plt
import glob


# Read any .txt output file
path = '/home/plots/*.txt' 
files = glob.glob(path)


for name in files:
    # Read lines from file
    with open(name) as f:
        lines = f.readlines()

    # Extract episode/reward data points
    episodes = []
    rewards = []
    for line in lines:
        columns = line.split()
        if not columns[0].startswith("episode"):
            # Not a line with episode/reward data
            continue
        try:
            episode = int(columns[0].split(":")[1])
            reward = float(columns[7].split(":")[1])
        except:
            # Not a number
            continue
        if episode in episodes:
            # We don't want repeated episodes
            continue
        episodes.append(episode)
        rewards.append(reward)


    print(episodes,rewards)
    
    #plot both in the same figure
    #plt.plot(episodes, rewards, episodes, rewards)
    #plt.show()
    
    # Plot data every 1000 steps
    N = 1000
    for i in range(int(len(l_episodes)/N)+1):
        plt.plot(l_episodes[i*N:i*N+N], l_rewards[i*N:i*N+N])
        plt.xlabel('episodes', fontsize=12)
        plt.ylabel('rewards', fontsize=12)
        plt.savefig("plot{}.png".format(i))
        plt.close()

Output figure

Yi Bao · Accepted Answer · 2018-12-19 21:04:39Z

2

How about creating a list of lists and plotting them one by one? The solution below works for multiple files too.

import matplotlib.pyplot as plt
import glob


# Read any .txt output file
path = '/home/plots/*.txt' 
files = glob.glob(path)

######################################################
# Create a list of episodes and reward from all files
l_episodes = []
l_rewards = []
######################################################
for name in files:
    # Read lines from file
    with open(name) as f:
        lines = f.readlines()

    # Extract episode/reward data points
    episodes = []
    rewards = []
    for line in lines:
        columns = line.split()
        if not columns[0].startswith("episode"):
            # Not a line with episode/reward data
            continue
        try:
            episode = int(columns[0].split(":")[1])
            reward = float(columns[7].split(":")[1])
        except:
            # Not a number
            continue
        if episode in episodes:
            # We don't want repeated episodes
            continue
        episodes.append(episode)
        rewards.append(reward)

    print(episodes,rewards)
    #################################
    # Append to the lists
    l_episodes.append(episodes)
    l_rewards.append(rewards)
    #################################

###########################################    
#plot both in the same figure
for i in range(len(l_episodes)):
    plt.plot(l_episodes[i], l_rewards[i])
###########################################
plt.show()

edited Dec 19, 2018 at 21:04

answered Dec 19, 2018 at 20:50

Yi Bao

1643 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

joasa Over a year ago

Thanks, it works! Because I have a lot of episodes and the difference is not clear overall, I would like to make one plot every 1000 episodes. I have updated my code with what I was doing and was working, but when I tried to incorporate with your solution but I get a ValueError: setting an array element with a sequence. Could you perhaps help me with that too?

Yi Bao Over a year ago

Would you please provide more details on your error message?

joasa Over a year ago

Traceback (most recent call last): File "/home/plots/plots.py", line 96, in <module> plt.plot(l_episodes[i*N:i*N+N], l_rewards[i*N:i*N+N]) plt.close()

ValueError: setting an array element with a sequence.

Yi Bao Over a year ago

Then this has nothing to do with plotting. Adding an if statement before episodes.append(episode) and rewards.append(reward) to decide which data should be kept.

Yi Bao Over a year ago

You are welcome. One final note: it usually will be beneficial to take a macroscopic view on what need to be plotted. Afterwards, plotting may be less confusing.

|

Collectives™ on Stack Overflow

Python read multiple .txt and plot in same figure

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related