Building a simple neural network from scratch in Python is a great way to understand the inner workings of neural networks. Below is a step-by-step guide to create a basic neural network with one hidden layer. This example will help you understand the foundational concepts without using machine learning libraries like TensorFlow or PyTorch.

1. Import Necessary Libraries

First, you'll need to import libraries such as NumPy for numerical operations.

import numpy as np  

2. Initialize Network Parameters

Define the number of neurons in each layer. For this example, let's create a network with 3 input neurons, 4 hidden neurons, and 1 output neuron.

input_size = 3  
hidden_size = 4  
output_size = 1  
np.random.seed(0)  # For reproducibility  
W1 = np.random.randn(input_size, hidden_size)  # Weight matrix for input to hidden  
b1 = np.zeros((1, hidden_size))                # Bias for hidden layer  
W2 = np.random.randn(hidden_size, output_size) # Weight matrix for hidden to output  
b2 = np.zeros((1, output_size))                # Bias for output layer  

3. Activation Function - Sigmoid

Define the sigmoid activation function and its derivative.

def sigmoid(x):  
    return 1 / (1 + np.exp(-x))  
def sigmoid_derivative(x):  
    return x * (1 - x)  

4. Forward Propagation

Implement the forward propagation function to calculate the output.

def forward_propagation(X):  
    global z1, a1, z2, output  
    z1 = np.dot(X, W1) + b1  
    a1 = sigmoid(z1)  
    z2 = np.dot(a1, W2) + b2  
    output = sigmoid(z2)  
    return output  

5. Backpropagation

Implement the backpropagation algorithm to adjust the weights and biases.

def backpropagation(X, y, learning_rate):  
    global W1, b1, W2, b2  
    m = X.shape[0]  
    # Calculate error  
    output_error = y - output  
    output_delta = output_error * sigmoid_derivative(output)  
    # Calculate hidden layer error  
    a1_error = output_delta.dot(W2.T)  
    a1_delta = a1_error * sigmoid_derivative(a1)  
    # Update weights and biases  
    W2 += a1.T.dot(output_delta) * learning_rate / m  
    b2 += np.sum(output_delta, axis=0, keepdims=True) * learning_rate / m  
    W1 += X.T.dot(a1_delta) * learning_rate / m  
    b1 += np.sum(a1_delta, axis=0, keepdims=True) * learning_rate / m  

6. Training the Network

Create a function to train the neural network by iterating over several epochs.

def train(X, y, epochs, learning_rate):  
    for epoch in range(epochs):  
        forward_propagation(X)  
        backpropagation(X, y, learning_rate)  
        if epoch % 1000 == 0:  
            loss = np.mean(np.square(y - output))  
            print(f'Epoch {epoch}, Loss: {loss}')  

7. Test and Verify

Create test data to ensure the neural network's functionality.

# Example training data (XOR problem)  
X = np.array([[0, 0, 1],  
              [0, 1, 1],  
              [1, 0, 1],  
              [1, 1, 1]])  
y = np.array([[0],  
              [1],  
              [1],  
              [0]])  
# Train the neural network  
train(X, y, epochs=10000, learning_rate=0.1)  
# Test  
print("Predicted Output:")  
print(forward_propagation(X))  

Summary

This neural network is a simple implementation with one hidden layer, using basic operations to show the entire process of forward and backward propagation. You can experiment by adding more layers, changing activation functions, or using different datasets to gain deeper insights into neural networks.