Python Forum
Why does this simple neural network not learn?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Why does this simple neural network not learn?
#1
Hey everyone,
This small perceptron is training with the delta rule and does not really make a great progress when it comes to learning.
The model should solve the simple OR-Problem.
def train(rounds):
    weight_one = 0.5
    weight_two = 0.8
    for i in range(rounds):
        input_one = r.randint(0,1)
        input_two = r.randint(0,1)
        if input_one == 0 and input_two == 0:
            wish_output = 0
        else:
            wish_output = 1
        learning_rate = 0.1
        output = input_one * weight_one + input_two * weight_two
        print(i,' X1: ', input_one, ' X2: ', input_two, ' output = ', output)
        error = wish_output - output
        delta_weight_one = learning_rate * error * input_one
        delta_weight_two = learning_rate * error * input_two
        weight_one = weight_one + delta_weight_one
        weight_two = weight_two + delta_weight_two

train(99999999)
I hope someone is finding my mistake.
Reply
#2
You forgot the bias weight.

import random

LR = 0.1
OR_GATE = {(0,0): 0, (0,1): 1, (1,0): 1, (1,1): 1}

def train(rounds):
    b = random.random()
    w1 = random.random()
    w2 = random.random()
    for i in range(rounds):
        input_one = random.randint(0,1)
        input_two = random.randint(0,1)
        wish_output = OR_GATE[(input_one, input_two)]

        output = b + input_one * w1 + input_two * w2
        error = wish_output - output
        
        delta_b = LR * error
        delta_w1 = LR * error * input_one
        delta_w2 = LR * error * input_two
        
        b = b + delta_b
        w1 = w1 + delta_w1
        w2 = w2 + delta_w2
        
    for x1, x2 in [(0,0), (0,1), (1,0), (1,1)]:
        print(x1, x2, b + x1*w1 + x2*w2)

train(100000)
The output looks good, you need to define a threshold e.g. 0.5 If output is below that means output is 0, if above then output is 1. That´s it.
Output:
0 0 0.18145152675687892 0 1 0.7848461458328603 1 0 0.6773077637591502 1 1 1.2807023828351316
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Neural network and data analysis from clients survey result pthon3 2 1,864 Mar-17-2022, 02:21 AM
Last Post: jefsummers
  Multilayer Perceptron Neural Network erdemath 3 2,282 Aug-09-2021, 11:07 PM
Last Post: jefsummers
  Neural Network importance weights / coefficients jkaustin 1 2,028 Nov-10-2020, 07:44 PM
Last Post: jefsummers
  Get error message in a GAN neural network tutorial jdude50 0 1,648 Oct-22-2020, 11:11 PM
Last Post: jdude50
  Explain Me Neural Network Ai's Harshil 2 1,973 Oct-22-2020, 04:50 AM
Last Post: Harshil
  construction of Neural Network for solving Differential equations arshad 0 1,608 Jun-04-2020, 09:20 AM
Last Post: arshad
  Neural Network mensagem error Dalpi 2 2,826 May-20-2020, 04:03 PM
Last Post: Dalpi
  coding neural network programmer19890620 4 3,391 Feb-27-2020, 04:26 AM
Last Post: programmer19890620
  First neural network: Problem with the weight factos 11112294 0 2,142 Jan-12-2019, 09:11 PM
Last Post: 11112294
  Neural network nesrine 0 2,614 Dec-11-2018, 03:48 PM
Last Post: nesrine

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020