Designing a circuit to attempt machine learning with analog electronics

Last time we discussed how combining multiplication based on Ohm’s law with addition based on Kirchhoff’s current law could enable us to represent the neural network’s multiply‑accumulate operation in a physical circuit.

This time, assuming we will eventually build an actual hardware circuit, we first consider the minimal circuit configuration.

In this simulator we will keep the resistor values fixed and dynamically vary the voltage output from the microcontroller.

With this setup, the weight updates and multiplications are performed on the microcontroller side, while the analog circuit handles voltage summation and inversion. We’re not trying to implement all of machine learning with an analog circuit, but this level of setup should be sufficient for first confirming the correspondence between the equations and the circuit.

However, when we actually train the circuit, the question of how to adjust the weights arises.
Using variable resistors could represent the weights, but it’s not feasible for a person to manually turn them each time training occurs.
A mechanism that can automatically update the weights is required.

In a real implementation, the input voltage would be controlled by a DAC, the weights would be set on a tunable weight element that can be electronically adjusted, and the circuit’s output would be read with an ADC.
The DAC controls the input voltage, while the ADC is used to read the output voltage.

However, building a physical circuit right away is difficult.
If a wiring mistake is made or the measured voltage deviates from the theoretical value, it becomes unclear whether the problem is with the circuit or with the calculations.

Therefore, this time we will first create a simulator that directly translates the circuit equations into code, rather than building the actual hardware immediately, and test it.
We will first verify that the calculations match for an ideal circuit, and then proceed to the physical implementation.

Overall configuration of the circuit intended for the real hardware

The final circuit to be deployed on the actual hardware will start with a configuration that processes a single layer.

flowchart LR
    MCU["Microcontroller<br/>learning and weight update"]
    DAC["DAC<br/>using channel 1"]

    MCU -->|"I²C"| DAC

    DAC -->|"V₁"| R1["R₁"]

    R1 --> SUM(("Inverted input"))

    SUM -->|"Negative input"| OP["Inverting amplifier<br/>op‑amp"]
    VREF["Vref = Vcc / 2<br/>voltage divider + buffer"] -->|"Positive input"| OP

    OP --> OUT["Vout"]
    OUT --> ADC["ADC"]
    ADC -.-> MCU

    OUT --> RF["Rf"]
    RF --> SUM

The main components used in the actual hardware are as follows.

The hardware circuit operates on a single power supply. The supply voltage is set to Vcc.

Implementing a single layer first

Instead of implementing the entire neural network in hardware from the start, we will first build a circuit that handles the sum‑of‑products computation for a single layer.

In this case, the input voltage V_i is supplied by the DAC, and the weight w_i is set on a weight element that can be electronically adjusted.
If the resistances or conductances representing the weights can be changed by the microcontroller, a human no longer needs to turn a variable resistor each time learning occurs.

The flow for the single layer we envision is as follows.

flowchart LR
    MCU["Microcontroller<br/>learning and weight update<br/>layer switching"]
    DAC["DAC<br/>input voltage Vi"]
    WEIGHT["Electronically controlled weight element"]
    SUM(("Summing point"))
    OP["Inverting amplifier<br/>sum‑of‑products for one layer"]
    ADC["ADC<br/>layer output"]

    MCU -->|"Weight setting"| WEIGHT
    MCU -->|"Input voltage setting"| DAC
    DAC -->|"Vᵢ"| WEIGHT
    WEIGHT --> SUM
    SUM --> OP
    OP --> ADC
    ADC -->|"Output read"| MCU

After the calculations for one layer are completed, the output is read with an ADC.
The microcontroller applies an activation function as needed, and then outputs the result as the input voltage for the next layer via a DAC.
This process is repeated for each layer.

A circuit that operates all layers simultaneously could enable faster processing, but it would require a separate circuit for each layer.
Initially, reusing a single‑layer circuit and processing one layer at a time through an ADC and DAC seems easier to implement and verify.

Outputting a Single Voltage from the Microcontroller

In the actual hardware, the microcontroller controls the DAC and outputs a voltage using only one of the multiple channels.

The DAC output is

$$ V_1 $$

The DAC output passes through a fixed resistor R_1 and is connected to the inverting input of an op‑amp.

If the microcontroller itself includes a DAC, an external DAC can be omitted.
Although we assume a multi‑channel DAC so that additional inputs can be added later, the first prototype will use only a single channel.

Treat Vcc/2 as the Zero Reference

The circuit in this design operates from a single supply voltage Vcc.

Consequently, negative voltages cannot be handled directly in the circuit.

Therefore, half of the supply voltage, Vcc / 2, is used as the reference voltage.

$$ V_{\mathrm{ref}}=\frac{V_{\mathrm{cc}}}{2} $$

This Vref is treated as the zero reference in the calculations.

For example:

• Vref: 0
• Voltages higher than Vref: positive values
• Voltages lower than Vref: negative values

That’s the idea.

The reference voltage can be generated by dividing Vcc with two equal‑value resistors.

However, connecting the divided voltage directly to the op‑amp may cause the voltage to fluctuate due to current drawn from the circuit.

Therefore, a separate buffer circuit is used to stabilize Vref.

The buffered reference voltage is connected to the non‑inverting input of the inverting amplifier.

First, a Single‑Input Inverting Amplifier

Instead of summing multiple inputs right away, we start with a single input.

Connect R_1 to the inverting input of the op‑amp, and connect R_f from the output back to the inverting input.
Connect the previously created Vref to the non‑inverting input.

Because of the negative feedback, the voltage at the inverting input becomes essentially the same as Vref.
In this case, the circuit’s output can be expressed by the following equation:

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-\frac{R_f}{R_1}(V_1-V_{\mathrm{ref}}) $$

Here, the input resistor and the feedback resistor are set to the same value.

$$ R_1=R_f $$

Since the resistance ratio is 1, the output becomes:

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-(V_1-V_{\mathrm{ref}}) $$

If the DAC output V_1 is Vref,

$$ V_1-V_{\mathrm{ref}}=0 $$

so the output also becomes Vref.

$$ V_{\mathrm{out}}=V_{\mathrm{ref}} $$

If V_1 is made higher than Vref, the output voltage of the inverting amplifier will be lower than Vref.

Conversely, if V_1 is made lower than Vref, the output voltage will be higher than Vref.

First, we will verify this behavior of “how much the input voltage is inverted relative to the reference voltage.”

Converting Weight and Input into a Single Voltage

In this circuit, the resistor values are not changed during training.

The microcontroller multiplies the input x by the weight w, and converts the result into a DAC output voltage.

$$ V_1=V_{\mathrm{ref}}+k w x $$

k is a scaling factor that brings the computed value into the voltage range that the circuit can handle.

Substituting this expression into the equation for a single‑input inverting amplifier yields

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-\frac{R_f}{R_1}k w x $$

When the resistor values are the same,

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-k w x $$

is obtained.

Although the output is inverted, the microcontroller can compute

$$ z=\frac{V_{\mathrm{ref}}-V_{\mathrm{out}}}{k} $$

to obtain a value equivalent to the product for a single input, wx.

At this stage, no addition has been performed yet.
First, we will use the simulator to see whether a value generated from one input and its weight can be passed through the circuit as a voltage change and be recovered as the original value.

Extending to Two Inputs

Once the single‑input calculation has been verified in the simulator, the next step is to add a second resistor, R_2, to the same summing node.

In the actual hardware, this simply means connecting the second DAC output, V_2, to the inverting input via R_2.

flowchart LR
    D1["DAC<br/>V₁"] --> R1["R₁"]
    D2["DAC<br/>V₂"] --> R2["R₂"]

    R1 --> SUM(("Summing point"))
    R2 --> SUM

    SUM -->|"Negative input"| OP["Inverting amplifier<br/>op‑amp"]
    VREF["Vref = Vcc / 2<br/>reference voltage buffer"] -->|"Positive input"| OP

    OP --> OUT["Vout"]
    OUT --> RF["Rf"]
    RF --> SUM

The output at this point is the following expression:

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-\frac{R_f}{R_1}(V_1-V_{\mathrm{ref}})-\frac{R_f}{R_2}(V_2-V_{\mathrm{ref}}) $$

When the input resistor and the feedback resistor are set to the same value,

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-(V_1-V_{\mathrm{ref}})-(V_2-V_{\mathrm{ref}}) $$

is obtained.

With a single channel, only one input voltage is processed, but with two channels two voltages can be fed to the same summing node.
At this point, the currents flowing through the resistors are summed within the circuit for the first time.

Converting the inputs and weights into voltages,

$$ V_1=V_{\mathrm{ref}}+k w_1 x_1 $$

$$ V_2=V_{\mathrm{ref}}+k w_2 x_2 $$

If the resistor values are the same, the output becomes

$$ V_{\mathrm{out}}=V_{\mathrm{ref}}-k(w_1x_1+w_2x_2) $$

.

On the microcontroller side,

$$ z=\frac{V_{\mathrm{ref}}-V_{\mathrm{out}}}{k} $$

If you perform the calculation, you obtain a value equivalent to the sum‑of‑products operation for the two inputs,

$$ z=w_1x_1+w_2x_2 $$

.

The multiplication is still handled by the microcontroller, but the addition is delegated to the circuit.
First, verify the single‑channel equation with a simulator, and then switch to a two‑channel configuration to see whether it behaves as expected when the summing operation is increased.

First, verify with a simulator

Building the circuit described so far directly on a breadboard is still quite cumbersome.
First, we’ll implement the inverting amplifier equation in code and check whether applying an input voltage yields the output predicted by the calculation.

What we are creating this time is not a simulator like SPICE that reproduces transistor characteristics and time‑varying behavior.
It is a highly idealized simulator that computes the equations derived from Ohm’s law and Kirchhoff’s current law.

First, we represent the inverting summing amplifier with the following function.

function simulateInvertingSummer(
  vInputs: number[],
  rInputs: number[],
  rFeedback: number,
  vRef: number,
): number {
  const currents = vInputs.map(
    (voltage, index) => (voltage - vRef) / rInputs[index],
  );
  const totalCurrent = currents.reduce(
    (sum, current) => sum + current,
    0,
  );
  return vRef - rFeedback * totalCurrent;
}

We apply the input voltage to vInputs, the input resistance to rInputs, and the feedback resistance to rFeedback.
The current is calculated from the difference between the input voltage and the reference voltage, summed, and then converted into an output voltage.

When a two‑channel input is applied, the result is as follows.

const vcc = 3.3;
const vRef = vcc / 2;
const rFeedback = 10_000;
const rInputs = [10_000, 10_000];

const k = 0.2;
const x = [2.0, 3.0];
const w = [0.5, 0.2];

const vInputs = w.map(
  (weight, index) => vRef + k * weight * x[index],
);
const vOut = simulateInvertingSummer(
  vInputs,
  rInputs,
  rFeedback,
  vRef,
);
const z = (vRef - vOut) / k;

console.log(vInputs); // [1.85, 1.77]
console.log(vOut);    // 1.33
console.log(z);       // 1.6

w_1 x_1 + w_2 x_2 is 0.5 * 2.0 + 0.2 * 3.0 = 1.6.
Even in the simulator it became z = 1.6, so we can now transfer the 2‑channel circuit formula we devised into code.

We have also made it possible to try the same calculation on the page while varying the input values.

Trying to Learn XOR

Up to this point, the circuit merely computes the weighted sum of the inputs.
Since we have this simulator, let’s also try XOR.

The XOR output is as follows.

However, adding only a weighted‑sum operation and a threshold to a single neuron cannot represent XOR.
This is because the boundary created by a single weighted sum is a straight line, which cannot correctly separate the four points of XOR.

This is not so much a mistake in the circuit’s calculation as a limitation of the model’s expressive power.
To handle XOR, you need to insert a nonlinear activation function between the weighted sums and add additional layers.

This time we will use a network with two inputs, two hidden layers, and one output.
The weighted sum for each neuron is computed by the circuit we have considered so far.
The subsequent sigmoid is calculated in software, and the weights are updated using back‑propagation.

If we denote the hidden‑layer output as h_j and the final output as y, the calculations are as follows.

$$ h_j=\sigma(b_j+w_{j1}x_1+w_{j2}x_2) $$

$$ y=\sigma(b_o+v_1h_1+v_2h_2) $$

In this simulator the initial weights are fixed, so under the same conditions the learning result will be identical each time.
We will first check the output before training and then run the learning process.

The XOR simulator this time computes the entire network in TypeScript.
When implementing it on actual hardware, one hidden layer would be calculated with an analog circuit, read out with an ADC, and passed to the next layer via a DAC from the microcontroller.
In other words, even for multi‑layer processing such as XOR, you first verify that a single‑layer circuit works correctly before combining them.

The overall processing flow can be summarized as follows:

1. The microcontroller multiplies the inputs by the weights.
2. The calculation results are converted into voltages corresponding to each input.
3. The simulator calculates the current flowing through the resistors.
4. The simulator sums the currents and determines the output voltage of the inverting amplifier.
5. The weighted‑sum result is obtained from the output voltage.

In this setup, the multiplication is handled by the microcontroller, while the voltage addition and inversion are reproduced by the simulator.

Although we do not perform the entire weighted‑sum operation with an analog circuit, this should be sufficient for verifying functionality before building the actual hardware.

Pay Attention to Voltage Ranges

In the actual hardware, the op‑amp and ADC are powered from the same single supply, so the output voltage must essentially be kept within the range of 0 to Vcc.
Even in the simulator, if the output exceeds this range, it should be treated as a value that would cause saturation on the real device.

In theory, centered on the reference voltage Vcc / 2, the weighted sum of the inputs and weights must fall within the following range.

When the input resistor and the feedback resistor are equal,

$$ -\frac{V_{\mathrm{cc}}}{2}\leq-k\sum_i w_i x_i\leq\frac{V_{\mathrm{cc}}}{2} $$

applies.
In the case of two channels, w_1 x_1 + w_2 x_2 corresponds to this weighted sum.

However, an actual op‑amp’s output may not be able to reproduce voltages accurately right up to the power‑supply limits.

Initially, it’s advisable to set k so that the output does not swing all the way to the power‑supply rails, leaving some margin.

When moving to the actual hardware, place decoupling capacitors near the power pins of each circuit.

What to Verify with This Simulator

This time, without building a physical circuit, we will first check the following points in the simulator:

• Whether the equation for a one‑input inverting amplifier can be calculated correctly
• Whether the total current can be correctly obtained when using two inputs
• Whether the weighted‑sum result can be recovered from the voltage
• Whether the output stays within the power‑supply voltage range
• Whether the theoretical equations match the simulator’s calculations
• Whether XOR can be learned with a network that includes hidden layers
• Whether the output of one layer can be passed to the next layer

If these points are confirmed, we can test the weighted‑sum operation that combines a microcontroller with an analog circuit in code, and by adding nonlinear layers we can verify that even problems like XOR can be handled.