Machine Learning Fundamentals Questions and Answers

Answer: C

The XOR output can be separated with two lines.

Plot the four input pairs: (0, 0) and (1, 1) give 0, while (0, 1) and (1, 0) give 1. The two classes sit on opposite corners of the square, so no single straight line can put both 1s on one side and both 0s on the other. Two lines, one cutting off (0, 0) and one cutting off (1, 1), leave the two 1s in the band between them.

That is why XOR is the classic example of a problem that is not linearly separable, why a single-layer perceptron cannot learn it, and why a hidden layer of two neurons solves it. More than two lines are never needed.

Answer: C

The output is 249.

The neuron first forms the weighted sum of its inputs, then applies the linear transfer function, which multiplies by 3:

\[ \begin{aligned} \text{net} &= 1(5) + 3(10) + 4(2) + 5(8) \\ &= 5 + 30 + 8 + 40 = 83 \\ y &= 3 \times 83 = 249 \end{aligned} \]

5, 30, 8, 40 are the individual weighted inputs before summing, and 83 is the net input before the transfer function is applied. 860 does not come from any step of the calculation.

Answer: A

The estimated fitness of schema 1*** is 0.4870.

Schema 1*** matches genomes whose first bit is 1: 1111 (57 copies, fitness 0.23) and 1010 (201 copies, fitness 0.56). Its estimated fitness is the average fitness of those instances:

\[ \begin{aligned} f &= \frac{57(0.23) + 201(0.56)}{57 + 201} \\ &= \frac{13.11 + 112.56}{258} = \frac{125.67}{258} \\ &\approx 0.4871 \end{aligned} \]

Truncated to four places this is 0.4870. 0.4877 and 0.4880 are rounding slips, and 0.5870 is out by 0.1.

Answer: A

The activation levels lie in [0, 1].

The logistic sigmoid \(f(x) = \frac{1}{1 + e^{-x}}\) maps any real input into the range 0 to 1, approaching 0 for large negative x and 1 for large positive x.

An unrestricted or minus to plus infinity range belongs to linear activations, and the discrete values 0 (or -1) and 1 are produced by step (threshold) functions, not by a sigmoid.

Answer: A

The weight matrix is that of option A, with rows (0, -1, 1, 3, 1), (-1, 0, 1, -1, 1), (1, 1, 0, 1, 3), (3, -1, 1, 0, 1) and (1, 1, 3, 1, 0).

By the outer product rule, \(W = \sum_k x_k x_k^{T} - 3I\) (subtracting 3I sets the diagonal to zero), so each weight is \(w_{ij} = \sum_k x_{k,i}\, x_{k,j}\) for \(i \ne j\).

\[ \begin{aligned} w_{12} &= 1 - 1 - 1 = -1 \\ w_{14} &= 1 + 1 + 1 = 3 \\ w_{24} &= 1 - 1 - 1 = -1 \\ w_{35} &= 1 + 1 + 1 = 3 \end{aligned} \]

The other entries work out the same way to 1, and the matrix is symmetric. Options B, C and D are not even 5 by 5 matrices, while a network of five neurons needs a 5 by 5 weight matrix.

Answer: C

The energy may decrease or remain same.

For a Hopfield network with symmetric weights and no self-connections, the energy is

\[ \begin{aligned} E = &-\tfrac{1}{2} \sum_{i} \sum_{j} w_{ij} s_i s_j \\ &+ \sum_{i} \theta_i s_i \end{aligned} \]

Each asynchronous update of a neuron either leaves its state unchanged or changes it in the direction that lowers E, so \(\Delta E \le 0\). The energy never increases, which is why the network settles into a stable state at a local minimum.

'Increase' and 'decrease or increase' contradict this, and 'decrease' alone ignores updates that leave the state, and so the energy, unchanged.

Answer: B

An epoch is one complete pass of the training data through the network.

In one epoch every example in the training set is presented once, forward through the network and with the error propagated back to update the weights, whether one example at a time or in mini-batches. Training usually runs for many epochs, so the number of epochs is the number of such passes.

Epochs have nothing to do with the number of input-layer nodes or of nodes in the network, and the test data is kept out of training altogether, used only to evaluate the trained network.

Answer: C

Network capacity is the number of patterns that can be stored and recalled in a network.

For associative memories such as the Hopfield network, capacity is how many patterns can be stored and then retrieved correctly; for a Hopfield network of N neurons it is roughly 0.14N patterns before recall breaks down.

It is not network traffic, and it is not simply the number of nodes or of computers, although capacity grows with the number of neurons.

Answer: C

The output is y = 0.73.

The net input is the weighted sum of the inputs plus the bias:

\[ \begin{aligned} net &= 1(0.5) + 3(2) + 2(-1) \\ &\quad + (-1)(2) + 3(-0.5) + 0 \\ &= 0.5 + 6 - 2 - 2 - 1.5 = 1 \end{aligned} \]

With slope parameter 1, the sigmoid gives

\[ y = \frac{1}{1 + e^{-1}} = \frac{1}{1.3679} \approx 0.73 \]

A sigmoid output lies between 0 and 1, so -1 is impossible; 0.52 and 0.92 correspond to net inputs of about 0.08 and 2.4.

Answer: C

This is uniform crossover.

Compare the children with the parents bit by bit. Child A equals Parent A except at positions 3, 5 and 8, where it has Parent B's bits, and Child B is the mirror image. The exchanged positions are isolated and scattered, not one or more continuous segments, which is the mark of uniform crossover: each bit position is independently chosen for exchange, typically by a random mask.

Single site crossover cuts both strings at one point and swaps everything after it. Multipoint crossover swaps continuous segments between several cut points. Neither naturally produces swaps at positions 3, 5 and 8 alone.