Neural Network Fundamentals: From Activations to Gradients

FAUN.dev() is where engineers from GitHub, Netflix, and Shopify go to stay ahead — fast.

An effortless, straightforward way to keep up with technologies...so you can keep your tabs closed and your mind open!

70,000+ developers already joined our ecosystem ⭐⭐⭐⭐⭐
Trusted by engineers at:

Google • Microsoft • AWS • Netflix

Take the quiz

Your score will show up right here.

Neural Network Fundamentals: From Activations to Gradients

A 10-question quiz on the core mechanics of neural networks, escalating from easy to hard. You will cover activation functions (ReLU, sigmoid, softmax), backpropagation, dropout, the vanishing gradient problem, and training vocabulary like epochs and iterations. Aimed at learners with some grounding who want to test whether they truly understand how these pieces fit together, not just recognize the terms.

Why do neural networks use nonlinear activation functions between layers?

They reduce the number of weights the network has to store, making the model smaller.
They let the network model nonlinear relationships. Without them, a stack of layers collapses into a single linear transformation, no matter how many layers you add.
They guarantee the training loss will eventually reach zero.

What does the ReLU activation function output when its input is a negative number?

0
The input value, unchanged.
The absolute value of the input.

In neural network training, what does one epoch mean?

A single weight update computed from one batch of data.
The processing of one individual training example.
One complete pass through the entire training dataset.

What is the primary job of the backpropagation algorithm?

To initialize the network weights with small random values before training.
To compute the gradient of the loss function with respect to every weight, using the chain rule.
To automatically choose the batch size and learning rate for training.

Why is softmax the standard activation for the output layer of a multiclass classifier?

It converts the raw output scores into a probability distribution across the classes, where all values are between 0 and 1 and sum to exactly 1.
It sets all negative output scores to zero and leaves positive ones unchanged.
It scales each output independently into the range 0 to 1 with no relationship between the outputs.

Which statement about the sigmoid activation function is correct?

It outputs values in the range -1 to 1.
It leaves positive inputs unchanged and zeros out negative inputs.
It maps any real-valued input to a value between 0 and 1.

How does dropout work as a regularization technique?

It permanently removes a set of neurons from the network to shrink the final model.
It randomly deactivates a fraction of neurons on each training step, and is switched off during inference so all neurons are active.
It adds an L2 penalty to the loss function to keep weights from growing too large.

In a deep network, what characterizes the vanishing gradient problem?

Gradients shrink as they are propagated backward, so the early (input-side) layers receive tiny updates and learn very slowly or not at all.
Gradients grow without bound as they propagate backward, causing the weights to overflow.
The loss function has no minimum, so the training process never terminates.

The maximum value of the sigmoid function's derivative is a key reason it causes vanishing gradients. What is that maximum value?