Difficult to plot linear regression line on scatter plot with log scale

Question

I have a example dataframe like this:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.DataFrame({'a':[0.05, 0.11, 0.18, 0.20, 0.22, 0.27],
                  'b':[3.14, 1.56, 33.10, 430.00, 239.10, 2600.22]})

I would like to plot these properties as a scatter plot and then show the linear tendency line of these samples. And I need to put the data on the y axis (df['b']) on log scale.

Although, when I try to do that using the aid of np.polyfit, I get a strange line.

# Coefficients for polynomial function (degree 1) 
coefs = np.polyfit(df['a'], df['b'], 1)
fit_coefs = np.poly1d(coefs)

plt.figure()
plt.scatter(df['a'], df['b'], s = 50, edgecolors = 'black') 
plt.plot(df['a'], fit_coefs(df['a']), color='red',linestyle='--')
plt.xlabel('a') 
plt.ylabel('b')
plt.yscale('log')

And if I convert df['b] to log before the plot, I can get the right linear tendency, but I would like to show the y-axis with the values of the last plot and not as converted log values as this one below:

df['b_log'] = np.log10(df['b'])

coefs = np.polyfit(df['a'], df['b_log'], 1)
fit_coefs = np.poly1d(coefs)

plt.figure()
plt.scatter(df['a'], df['b_log'], s = 50, edgecolors = 'black') 
plt.plot(df['a'], fit_coefs(df['a']), color='red', linestyle='--') 
plt.xlabel('a') 
plt.ylabel('b_log')

So basically, I need a plot like the last one, but the values on y-axis should be like the second plot and I still would get the right linear tendency. Anyone could help me?

mcsoini · Accepted Answer · 2021-07-07 19:19:14Z

2

You are doing two different things there: First, you are fitting a linear curve to your exponential data (which is presumably not what you want), then you are fitting a linear curve to your log data, which is ok.

In order to get the linear curve from the linear coefficients in the logarithmic plot, you can just do 10**fit_coefs(df['a']):

df['b_log'] = np.log10(df['b'])

coefs = np.polyfit(df['a'], df['b_log'], 1)
fit_coefs = np.poly1d(coefs)

plt.figure()
plt.scatter(df['a'], df['b'], s = 50, edgecolors = 'black') 
plt.plot(df['a'], 10**fit_coefs(df['a']), color='red', linestyle='--') 
plt.xlabel('a')
plt.ylabel('b_log')
plt.yscale("log")

answered Jul 7, 2021 at 19:19

mcsoini

6,7922 gold badges21 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Difficult to plot linear regression line on scatter plot with log scale

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related