Regression Neural Network with NumPy in Python

Question

I want to make regression neural network with Python and Numpy. I have made one for classification, where I used sigmoid function, but I do not know how to change my function into regression. I know that linear function should look like y=k*x + n, but I do not know how to represent that In python. What is my n and k values, how to pass them to my function?

Ofc, I put simple linear function, but My goal is to create something likeRBF function.

This is the code that I have, but in his code I have sigmoid function, and I want to have regression function.

How can I change sigmoid function with regression function and its derivate function?

import numpy as np
import pandas as pd


df = pd.DataFrame({'input 1':[0.5, 0.3, 0, 0.1, 0.4, -0.4, 0.4, 0.1, -0.6, 0.2, 0.6, 0, 0.2, 0.2, -0.1, -0.1, 0, 0.4, -0.2, -0.4],
                   'input 2':[0.3, 0.6, -0.4, -0.2, 0.9, 0, 0.35, -0.4, -0.9, 0.4, 0.3, -0.1, 0.1, 0.3, 0.1, 0.1, 0.3, 0.1, 0.3, 0.3],
                   'input 3':[0, 0.4, 0, -0.1, 0.4, -0.2, 0.7, -0.3, -0.1, 0.1, 0.3, 0, 0.5, 0.4, -0.31, 0.1, 0.3, 0.1, 0.1, 0.2],
                   'result':[21, 31, 10, 6, 53, 31, 13, 15,16, 15, 26, 36, 52, 41, 14, 8, 14, 13, 11, 4]})

print(df)

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivate(x):
    return x * (1 - x)


features = df.iloc[:,:-1].to_numpy()
results =  df.iloc[:,-1:].to_numpy()

np.random.seed(1)

weights = 2 * np.random.random((3,1)) - 1

print('These are my random weights:\n')
print(weights)

for iteration in range(100000):

    input_layer = features

    outputs = sigmoid(np.dot(input_layer, weights))
    error = results - outputs
    adjustments = error * sigmoid_derivate(outputs)
    weights += np.dot(input_layer.T, adjustments)


df['output prediction'] = outputs.round(0)
print(df)

seralouk · Accepted Answer · 2019-12-15 19:06:22Z

You have implemented a classification system that using the sigmoid will yield values between 0 and 1.

For linear regression type of problem, you can simply create the Output layer without any activation function as we are interested in numerical values without any transformation.

So actually to predict the values, you just need a linear regression model.

Small example for a linear regression model optimized with gradient descend.

X = df.iloc[:,:-1].to_numpy()
y =  df.iloc[:,-1:].to_numpy()

def compute_cost(X, y, params):
    n_samples = len(y)
    h = X @ params  # dot product
    return (1/(2*n_samples))*np.sum((h-y)**2)

def gradient_descent(X, y, params, learning_rate, n_iters):
    n_samples = len(y)
    J_history = np.zeros((n_iters,1))

    for i in range(n_iters):
        params = params - (learning_rate/n_samples) * X.T @ (X @ params - y) 
        J_history[i] = compute_cost(X, y, params)

    return (J_history, params)

n_samples = len(y)
mu = np.mean(X, 0)
sigma = np.std(X, 0)

X = (X-mu) / sigma # normalize

X = np.hstack((np.ones((n_samples,1)),X))  # add bias term
n_features = np.size(X,1)

params = np.zeros((n_features,1))  # initial guess

n_iters = 1500
learning_rate = 0.01

initial_cost = compute_cost(X, y, params)

print("Initial cost is: ", initial_cost, "\n")

(J_history, optimal_params) = gradient_descent(X, y, params, learning_rate, n_iters)

print("Optimal parameters are: \n", optimal_params, "\n")

print("Final cost is: ", J_history[-1])

plt.plot(range(len(J_history)), J_history, 'r')

plt.title("Convergence Graph of Cost Function")
plt.xlabel("Number of Iterations")
plt.ylabel("Cost")
plt.show()

Keep in mind that OLS can be expressed in a matrix close form and thus, the gradient descend will not be needed.

Community · Accepted Answer · 2020-06-20 09:12:55Z

0

I think you should revise your understanding of the function of neural network. Indeed , you shoould initialize your network by a sequential ,then you should add an input layer which should contain the number of features in the data that you will use . Besides you should add the hidden layers that you will use and you finish by add the output layer in which you should mention that the number of output is 1 and the activation function is 'linear'. I give you an exemple

Initialize network

NN=Sequential()

Input layer

NN.add(Dense(N0,input_dim=features.shape[1],activation='relu')

Hidden layers

NN.add(Dense(N1, activation='relu')) NN.add(Dense(N2,activation='relu')) . . . .

Output layer

NN.add(Dense(1,activation='linear'))

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Dec 15, 2019 at 11:53

Taher Haggui

1

1 Comment

taga Over a year ago

I do not understand the code that you posted. Is it possible to implement your solution to my code?

Collectives™ on Stack Overflow

Regression Neural Network with NumPy in Python

2 Answers 2

1 Comment

Initialize network

Input layer

Hidden layers

Output layer

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Initialize network

Input layer

Hidden layers

Output layer

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related