仓库源文，站点原文

layout: post title: "Neural Networks Notes 1 Introduction" date: 2021-2-24 11:00:00 +0800

categories: [Notes, Neural Networks]

Neural Networks Introduction

• Computational paradigm based on biological nervous system

Brain: $10^{10}$ neurons, $10^{4}$ fan in, $10^{14}$ connection strengths

NN learning

NN does not know anything they learnt.

• Initial random paramters (weights)
• Small modifications to weights on each presentation of data
• Simple networks can be set up directly

NN processing

• Input new patterns
• Propagate activations along (weighted) links
• Repeat training

Computational model of a single neuron

McCulloch & Pitts

N binary inputs x1,x2,...,xN
1 binary output y
threshold $\theta$
N weights w1,w2,...,wN
w is 1 or -1
y(x) = 1 iff xi * wi >= threshold for all i

Gating network with memory

Rosenblatt

weights not fixed
random interconnections
learn from experience

Gradient descent

Partial derivative of a function tells us how to change $w$ to minimise $f(w)$.
Gradient descent updates the parameter vector $w$, using the partial derivatives of the error function to minimise the error.

Computational model of Perceptron

weights learned from data