3.1 Testbenches

Why this matters

A software bug can be found with a print statement. A circuit has no print statement: once it is built, you cannot ask a wire what it is carrying. So hardware designers test the design in a simulator first, where every wire can be watched, and fix the bug while it is still cheap. A testbench is the program that exercises the design in the simulator, and every module in this book comes with one.

Learning goals

Explain what a simulator and a testbench are, and why a testbench has no ports.
Write a testbench for a combinational module that sets every input at time zero and steps through input combinations with delays.
Test every input combination of a small module with a for loop.
Predict a module's output on each line of a testbench, including an output that is unknown (X).
Read a waveform of a testbench run and match each change to the line that caused it.

Intuition first

Think of a unit test in a programming language. You call a function with some arguments and compare what comes back with what you expected. A testbench is the same idea for a circuit, with one difference that matters: a circuit is not called. It is always running, and its outputs are always a function of its inputs right now. So the testbench does not call the module. It plays the part of the rest of the world: it holds the module's inputs at some values for a while, changes them, holds them again, and the simulator records what the outputs did.

A good picture is a test rig in a workshop. The device sits on the bench with its input pins wired to a signal generator and its output pins wired to a probe. The rig contains the device, the generator and the probe, and nothing is wired out of the rig, so the rig itself needs no connectors. That is why a testbench module has an empty port list. The probe is the waveform you look at afterwards.

What a simulator does

Definition 3.1.1(simulator).

A simulator is a program that reads a hardware description and computes, for each moment of simulated time, the value on every signal in the design. It runs on your own computer. It is not the circuit, and it never touches an FPGA; it is a model of what the circuit would do.

Lecture 3 names ModelSim and Icarus Verilog as the simulators.^{F (fact ch03/simulators)} The book's own checks run Icarus; the section slides show a ModelSim workflow and a workflow that writes a waveform file and opens it in the Surfer viewer.^{F (fact ch03/viewer-workflows)} The waveforms in this book are drawn from Icarus runs by the book's own tools, so what you read on a page is what a simulator produced for the listing above it.

Simulation time is separate from your wall clock. The simulator keeps a counter of simulated time, and a delay such as #10 in a testbench tells it to run the design forward by ten time units before executing the next line. Inside one simulated instant the simulator can update several signals and run several processes, one after another, while zero-delay logic settles. A waveform is drawn against time, so it gives those intermediate updates no width: a combination of inputs shows up only if time moves before the next one is applied. Put a delay between input combinations when you want each one to occupy a visible interval.

Definition 3.1.2(testbench and device under test).

A testbench is a module that contains an instance of the circuit being tested, called the device under test or DUT, and a small script that supplies values to every input of the DUT over simulated time. The testbench itself is never turned into hardware: it exists only in the simulator.

A testbench has the same four parts every time.

A module named after the module under test, with _tb added, and no ports.
A signal for every port of the DUT. Inputs are the signals the script drives, and outputs are the signals you watch.
An instance of the DUT named dut. When the signal names match the port names, .* connects every port at once. Naming each port, as in .s(s), or naming a port once when the signal has the same name, as in .s, does the same job.
An initial block with the script. It assigns every input before the first delay, then alternates between changing inputs and waiting, and ends with $stop. The section slides say the $stop is what makes ModelSim pause at the end.^{F (fact ch03/modelsim-stop)}

Every module you write gets a testbench, in its own file, before it goes anywhere near a board.

In SystemVerilog

A first testbench

Example 3.1.1(four steps for a 2-to-1 multiplexer).

A multiplexer, or mux, picks one of its data inputs and passes it to the output. This one, mux2, has the ports of the multiplexer in Definition 2.7.2: it passes d1 when s is 1 and d0 when s is 0.

// A 2-to-1 multiplexer: y follows d1 when s is 1 and follows d0 when s is 0.
module mux2 (
  input  logic s,
  input  logic d0,
  input  logic d1,
  output logic y
);
  assign y = s ? d1 : d0;
endmodule

Here is a testbench with four steps. Each step sets some inputs and waits ten time units.

// Testbench for mux2: four combinations of the inputs, ten time units apart.
module mux2_tb();
  logic s, d0, d1, y;

  mux2 dut(.*);

  initial begin
    s = 1; d1 = 0; d0 = 1; #10;   // expect: y=0
           d1 = 1;         #10;   // expect: y=1
    s = 0;                 #10;   // expect: y=1
                   d0 = 0; #10;   // expect: y=0
    $stop;
  end
endmodule

Walk through it one line at a time, using what the mux does.

The first line sets all three inputs, s, d1 and d0. Because s is 1, y follows d1, which is 0, so y is 0.
The second line changes only d1, to 1. With s still 1, y follows d1 and becomes 1.
The third line sets s to 0, so y now follows d0, which is still 1. y stays 1.
The fourth line changes d0 to 0. y follows it and becomes 0.

The comment on each line starts with expect:. These comments are the book's own device: the book's tools run the testbench and compare each comment with the simulated value of the named signal. A comment is read one time unit before the delay on its line finishes, which is after the inputs of that line have been applied and the logic has settled. So for this circuit a comment names the output that the inputs on its own line produce. Nothing in a simulator reads these comments. In a testbench of your own they are notes to yourself, and the way to make the simulator do the comparing is a self-checking testbench like the one in the aside below. Section 4.8 shows testbenches with a clock, where the same comment reports the result of the previous clock edge instead.

Time in ps; one column is 10 ps.

Figure 3.1.1. The run of the testbench above.

Compare Figure 3.1.1 with the four lines. In the first step s is high, d1 is low, d0 is high and y is low. In the second step d1 rises, and y rises at the same instant. At the start of the third step s falls, and y stays high because d0 is high. In the fourth step d0 falls, and y falls with it. Each edge of y lines up exactly with the edge of the input that caused it.

That last observation is a property of this kind of simulation. By default a simulated gate takes no time at all, so a change at an input reaches the output in the same instant. The section slides say that the simulations in the course give every gate a delay of zero.^{F (fact ch03/zero-delay-setup)} A real gate takes a little time, and Section 3.3 adds that delay to the model. Until then, any delay you see in a waveform is a # delay in the testbench.

In the testbench above, suppose the second line were d1 = 1; s = 0; #10;, changing both inputs at once. With d0 still 1, what is y after that line? 0, because y was 0 before this line and the new value of d1 does not reach it. 1, because s is 0, so the mux selects d0, and d0 is 1. It depends on which assignment the simulator runs first.

With s equal to 0 the mux passes d0, whatever d1 is, and d0 is 1. The two assignments take effect at the same simulated time. The simulator may apply them one after the other inside that instant, but the waveform has no width in which to show a step between them, and the value that remains is the one the mux function gives for the final inputs.

Trying every input

For a combinational module the output depends on the current inputs only, and nothing else. If the module has few inputs, you can test it completely: apply every combination once. A module with $n$ input bits has $2^n$ combinations, so four input bits need sixteen steps. Writing sixteen lines by hand is tedious and error-prone. A for loop counts through the combinations for you.

Example 3.1.2(all sixteen codes of a seven-segment decoder).

A seven-segment display shows a digit by lighting some of its seven segments. The decoder below takes a 4-bit code and produces seven bits, one per segment, where 1 means lit. Segment L0 is the top bar and the others follow the lecture's numbering, so leds[0] drives the top. Only the codes 0 to 9 are digits. For the other six the module sets the output to 7'bX, all unknown, to say that the output does not matter there. Section 3.9 returns to this decoder.

// Seven-segment decoder for the digits 0 to 9. leds[i] drives segment Li and 1 means lit.
// The codes 10 to 15 are not digits, so their output is left unknown.
module seg7_digit (
  input  logic [3:0] bcd,
  output logic [6:0] leds
);
  always_comb begin
    case (bcd)
      //          leftmost bit is segment 6
      4'd0: leds = 7'b0111111;
      4'd1: leds = 7'b0000110;
      4'd2: leds = 7'b1011011;
      4'd3: leds = 7'b1001111;
      4'd4: leds = 7'b1100110;
      4'd5: leds = 7'b1101101;
      4'd6: leds = 7'b1111101;
      4'd7: leds = 7'b0000111;
      4'd8: leds = 7'b1111111;
      4'd9: leds = 7'b1101111;
      default: leds = 7'bX;
    endcase
  end
endmodule

The testbench loops over all sixteen values of the input, twenty time units each.

// Testbench for seg7_digit: each of the sixteen input codes in turn, twenty time units apiece.
module seg7_digit_tb();
  logic [3:0] bcd;
  logic [6:0] leds;

  seg7_digit dut(.*);

  initial begin
    for (int code = 0; code < 16; code++) begin
      bcd = code[3:0]; #20;
    end
    $stop;
  end
endmodule

The loop counter code is an int and bcd is four bits wide, so the testbench assigns code[3:0], the low four bits. The for loop is not hardware. It does not describe sixteen copies of anything. It is a shorthand for sixteen lines of stimulus, run once, in order, by the simulator.

The run is 320 time units long, so it is drawn in three pieces, in Figure 3.1.2, Figure 3.1.3 and Figure 3.1.4. The bcd lane shows the input in hexadecimal and the leds lane shows the seven output bits in binary, bit 6 on the left.

Time in ps; one column is 20 ps.

Figure 3.1.2. The decoder for the input codes 0 to 5.

Time in ps; one column is 20 ps.

Figure 3.1.3. The decoder for the input codes 6 to 9.

Time in ps; one column is 20 ps.

Figure 3.1.4. The decoder for the input codes A to F.

In the first two figures each code gives a different pattern. In the last figure, from code A to code F, the leds lane is hatched: every bit is unknown. That is the default branch of the module at work. The hatching is a message to the reader, not a signal level: the simulator is saying it has no value to report for those inputs.

A loop over every input is practical only while the input is small. Twenty input bits would mean about a million steps, and a 32-bit adder's two operands together have 64 bits, which is far too many to try. Those are tested with chosen cases instead. Combinational modules with a handful of inputs, which is nearly everything in this chapter, are tested exhaustively.

Unknown inputs and the default branch

Example 3.1.3(a gait selector, and what its default does).

Here is a module with a different shape, a case that picks one of four bit patterns. A 2-bit input gait says how fast the horse moves, from standing at 0 up to cantering at 3, and the module drives a row of eight lights called lights: the faster the gait, the more of them are on.

// The faster the gait, the more of the eight lights are on, counting from the right.
module horse_gait (
  input  logic [1:0] gait,
  output logic [7:0] lights
);
  always_comb begin
    case (gait)
      2'd3: lights = 8'b11111111;
      2'd2: lights = 8'b00001111;
      2'd1: lights = 8'b00000011;
      2'd0: lights = 8'b00000001;
      default: lights = 8'bxxxxxxxx;
    endcase
  end
endmodule

Four values of a 2-bit input cover every case, so the default can only be reached when gait contains an X or a Z. A real input pin is never "unknown", but in a simulation an input can be, for example when nothing drives it. The testbench tries that on purpose in its last line. Its five lines each carry an expect: comment, and the last one expects all eight lights unknown.

// Testbench for horse_gait: the four gaits, then a gait with an unknown bit.
module horse_gait_tb();
  logic [1:0] gait;
  logic [7:0] lights;

  horse_gait dut(.*);

  initial begin
    gait = 2'b00; #10;   // expect: lights=8'b00000001
    gait = 2'b01; #10;   // expect: lights=8'b00000011
    gait = 2'b10; #10;   // expect: lights=8'b00001111
    gait = 2'b11; #10;   // expect: lights=8'b11111111
    gait = 2'bx1; #10;   // expect: lights=8'bxxxxxxxx
    $stop;
  end
endmodule

Time in ps; one column is 10 ps.

Figure 3.1.5. The gait selector with the four gaits and then an input with an unknown bit.

In Figure 3.1.5 the first four steps show the row of lights growing from one to eight. In the last step gait is x1, one bit unknown. The case compares the whole value bit by bit, and x1 is not equal to 00, 01, 10 or 11, so none of the four branches is taken and the default assigns X to all eight bits. The output is unknown even though one of the two input bits is known to be 1. The simulator does not guess.

The default line also matters for a second reason, covered in Section 3.9: when a module is turned into hardware, X on the right side of an assignment tells the tools that they may pick whatever output is cheapest for those inputs.

An aside: a testbench that checks itself

ReadingBeyond the course: self-checking testbenches

In everything above, you look at the waveform and decide whether the output is right. A self-checking testbench decides for you: it compares each output with an expected value and reports mismatches, so a long test can be run without anyone reading a waveform. The one below tests mux2 exhaustively and counts the mismatches. It prints the count at the end, and the book's tools read that number: it is 1'd0.

// A self-checking testbench for mux2: it compares y with its own model of the mux and counts the mismatches.
module mux2_selfcheck_tb();
  logic s, d0, d1, y;
  int errors;

  mux2 dut(.*);

  initial begin
    errors = 0;
    for (int i = 0; i < 8; i++) begin
      {s, d1, d0} = i[2:0]; #10;
      if (y !== (s ? d1 : d0)) begin
        errors++;
        $display("mismatch: s=%b d1=%b d0=%b y=%b", s, d1, d0, y);
      end
    end
    $display("DD errors=%0d", errors);
    $stop;
  end
endmodule

A self-checking testbench is only as good as its model of the right answer. Here the model, s ? d1 : d0, is the same expression the design uses, so a misunderstanding of the multiplexer would be copied into both. Real projects write the expected values independently of the design. Nothing else in this book uses one, and you do not need it to follow the rest of the chapter.

Course conventions

How CSE 369 writes this

In CSE 369 we name a testbench after its module with _tb added, give it no ports, name the instance of the module under test dut, and end the script with $stop. Delays are written as #10 and so on, with no units; the simulator counts in its own time units. Every input gets a value at time zero.

The time axis of a simulated waveform counts the simulator's time units, and the book's figures label it ps. Those numbers are arbitrary: they say nothing about how fast a real circuit is, and a label of ps does not mean a gate in a real chip takes that long. The book says this once per chapter, here, and does not repeat it.

Pitfalls

Common pitfall

Giving the testbench ports, or forgetting to instantiate the module under test. The testbench is the whole world of the simulation, so it has an empty port list and the DUT is inside it.
Leaving an input unassigned at time zero. An input nobody drives is unknown, and an output that depends on it can be unknown too. It need not be: an unknown AND 0 is 0, and an unknown OR 1 is 1, so another input can hide the X for a while. Assign every input in the first line of the script rather than relying on that.
Changing inputs on consecutive lines with no delay and then looking for the in-between values. Changes at the same simulated time share one point on the time axis, so only the settled values show. Put a delay after each combination you want to see.
Testing a small combinational module with a few favorite inputs. With few inputs, try them all.
Treating a comment as a check. A comment says what you hoped for, and in your own testbench nothing reads it; only a run, and a look at the waveform or a self-checking test, shows what happened.

Check your understanding

Take the quiz (4 questions)

Exercises

Exercise 3.1.1

The module mux2 from this section is tested by a different testbench, in five steps of ten time units each. The figure shows the inputs. Work out y for every step.

Time in ps; one column is 10 ps.

Figure 3.1.6. The question: the three inputs, with y left empty.

Hint

In each step, look at s first: it says which of d0 and d1 the output follows.

Solution

Take the steps in order, starting from the first line, which sets all three inputs at time zero.

Step 1: s is 0, so y follows d0, which is 0. y is 0.
Step 2: d0 has risen to 1, and s is still 0, so y is 1.
Step 3: s has risen to 1, so y now follows d1, which is 1. y stays 1.
Step 4: d1 has fallen to 0, and y follows d1, so y is 0.
Step 5: s has fallen back to 0, so y follows d0 again, which is 1. y is 1.

Time in ps; one column is 10 ps.

Figure 3.1.7. The answer: every signal, from the simulation of the testbench.

Tags: Warm-upTest benches and simulationWaveforms

Exercise 3.1.2

The module majority3 outputs 1 when at least two of its three inputs are 1.

// Majority of three: F is 1 when at least two of the inputs are 1.
module majority3 (
  input  logic A,
  input  logic B,
  input  logic C,
  output logic F
);
  assign F = (A & B) | (A & C) | (B & C);
endmodule

Write a testbench for it that tries every input combination, ten time units each, and ends with $stop.

Hint

Three inputs have eight combinations. A for loop counting from 0 to 7 produces them in binary order, and the concatenation {A, B, C} on the left of an assignment spreads a number over the three signals, with A taking the top bit.

Solution

The testbench follows the four parts in the order above: a module with no ports, a signal for every port, the instance named dut, and one initial block. The loop counter runs from 0 to 7. In each step, the low three bits of the counter are assigned to A, B and C together, and a delay lets the output settle and appear in the waveform.

// Testbench for majority3: all eight input combinations, counting up in binary.
module majority3_tb();
  logic A, B, C, F;

  majority3 dut(.*);

  initial begin
    for (int i = 0; i < 8; i++) begin
      {A, B, C} = i[2:0]; #10;
    end
    $stop;
  end
endmodule

The module's output is also checked against the majority function by the book's tools, from a specification written separately from this listing, so the listing is known to be a correct majority.

Tags: StandardTest benches and simulationSystemVerilog

Exercise 3.1.3

The module equal2 compares two 2-bit numbers, X1 X0 and Y1 Y0, and outputs 1 when they are equal.

// Compares two 2-bit numbers, X1 X0 and Y1 Y0: Eq is 1 when they are the same.
module equal2 (
  input  logic X1,
  input  logic X0,
  input  logic Y1,
  input  logic Y0,
  output logic Eq
);
  assign Eq = ~(X1 ^ Y1) & ~(X0 ^ Y0);
endmodule

Write a testbench that tries every pair of values. How many steps does it take?

Hint

Two 2-bit numbers are four input bits in all, so a single loop to 15 would work. But there is a tidier way to think about it: one loop for the first number and, inside it, a second loop for the second.

Solution

There are four values of X and four values of Y, so $4 \times 4 = 16$ steps, the same as one loop over the four bits together. Two nested loops say it more clearly: the outer loop picks the value of the number X1 X0, and for each such value the inner loop runs through all four values of Y1 Y0.

// Testbench for equal2: every pair of 2-bit values, with the second number counting fastest.
module equal2_tb();
  logic X1, X0, Y1, Y0, Eq;

  equal2 dut(.*);

  initial begin
    for (int i = 0; i < 4; i++) begin
      for (int j = 0; j < 4; j++) begin
        {X1, X0} = i[1:0]; {Y1, Y0} = j[1:0]; #10;
      end
    end
    $stop;
  end
endmodule

The module is also checked by the book's tools against a specification that states equality of the two numbers directly.

Tags: StandardTest benches and simulationSystemVerilog

Exercise 3.1.4

In the horse-gait run, the first four steps give four different patterns and the fifth, with gait equal to x1, gives an unknown output. Without changing the module, which one input value could you add to the testbench so that the default branch is reached with every bit of gait known? Explain your answer.

Hint

List the values of a 2-bit input that has no unknown bits.

Solution

There is none. A 2-bit input with no unknown or high-impedance bits has four possible values, 00, 01, 10 and 11, and the case has an item for each. Every fully known value matches an item, so the default branch can only be reached when gait contains an x or a z, as in the last step of Figure 3.1.5.

This is also why the default is harmless in hardware for this module: a real input pin is always at 0 or at 1.

Tags: StandardTest benches and simulationWaveforms