Netlist Simulator and Waveform Viewer

The simulator framework enables the reverse engineer to simulate selected parts of the netlist in a cycle-accurate fashion and analyze the results in a graphical viewer. The simulator framework is made out of several plugins, like controller plugin to provide common functionality and API, Verilator simulator plugin and waveform viewer plugin. These plugins are not build automatically. It is recommended to force the build of all available plugins by issuing the flag -DBUILD_ALL_PLUGINS=1 flag when calling cmake.

The simulation procedure (Overview)

The simulation can be controlled from waveform viewer GUI or via Python script or Python command line. In all cases the following steps are necessary

• Instantiate a simulation controller
• Select parts of the current netlist to be simulated
• Prepare simulation
  • Select the clock(s) driving the simulation
  • Select the simulation engine
  • Load or create data for the inputs of the selected partial netlisst.
• Run the simulation
• Analyze the results

Instantiate a simulation controller

Python:

from hal_plugins import netlist_simulator_controller
pl_sim = hal_py.plugin_manager.get_plugin_instance("netlist_simulator_controller")
sim_ctrl = pl_sim.create_simulator_controller(<optional_name>)

Immediately when creating a new controller instance a working directory for the waveform database gets created as well. The optional controller name and a random string will become part of the working directory name. The user can choose any variable name for the python controller instance. In this tutorial we name it sim_ctrl which will be used in subsequent python command lines.

When creating a new controller instance a log message will be written on console revealing the autogenerated controller-ID.

GUI:

[Create simulation controller] from toolbar

Select parts of the current netlist to be simulated

Python:

sim_ctrl.add_gates(<list_of_gates>)

The list of gates defining the partial netlist for simulation can be prepared using the well documented functions get_gates from hal netlist or module.

GUI:

[Simulation settings] from toolbar -> 'Select gates for simulation'

The user has the choice to pick the gates for the partial netlist individually, to select the entire netlist, or to use a selection previously made in the graph view from the netlist.

All subsequent preparation steps depend on the selected partial netlist for simulation. For this reason, the selection of gates can only be done. If the user changes the mind concerning selected gates a new controller can be set up with the correct selection.

Prepare simulation

The sequence of the following steps is not important, they can be done in any order. However, GUI will only allow to do the actual simulation if all steps below have been completed.

Select the clock(s) driving the simulation

Setting the clock is a convenient method to define a clock with a given period for a duration of time. All times are given as integer values. In most cases the unit 'picosecond' is assumed but that is not important for the 'ideal' simulation which does not simulate time jitter or delay.

It is however possible to provide user generated input data for the clock like for every other input net. In this case it must be announced that no clock will be used to skip this setup step.

Python:

sim_ctrl.add_clock_period(<clock_net>, <period>, <start_at_zero>, <duration>)
#---or---
sim_ctrl.set_no_clock_used()

GUI:

[Simulation settings] from toolbar -> 'Select clock net'

Setup parameter for selected clock as above or check 'Do not use clock in simulation'.

Select the simulation engine

While the menu offers three different engines the user should always select the external Verilator engine for most accurate simulation. The built-in HAL-simulator has known issues and is kept only for testing and comparison with simulation results from early HAL days. The dummy engine is for debugging purpose only.

Python:

eng = sim_ctrl.create_simulation_engine("verilator")

GUI:

[Simulation settings] from toolbar -> 'Select engine ...' -> 'verilator'

The Verilator engine will be started as a separate process. For special cases (number of threads, additional input, remote execution) the engine might need additional parameter.

Python:

eng.set_engine_property("num_of_threads", "8")

GUI:

[Simulation settings] from toolbar -> 'Settings ...' -> 'Engine properties'-Tab

The output and error messages the engine generates while performing the simulation will be dumped into console. GUI users can get a slightly enhanced view of the engines output. GUI:

[Simulation settings] from toolbar -> 'Show output of engine'

Load or create data for the inputs of the selected partial netlist.

Python:

sim_ctrl.import_vcd(<filename>,<filter>)                            # import .vcd input
sim_ctrl.import_csv(<filename>,<filter>,<timescale>)                # import .csv input
sim_ctrl.import_saleae(<directory>,<lookupDictionary>,<timescale>)  # import .csv input

GUI:

[Open input file] from toolbar -> select one of (*.vcd) (*.csv) (saleae.json)

The regular way providing input data to drive the simulation is by importing a waveform data file. Supported file formats are .VCD, .CSV, and Saleae. GUI user can double click on input data to open an editor and modify individual values. In python individual values can be set by 'sim_ctrl.set_input()' command but performance would be lousy since every input requires synchronization with database.

While the name (or ID) of the assigned net from netlist is expected to be found in header data of VCD or CSV files this information is missing in binary Saleae data format. Therefore in hal a file named saleae.json must reside in the same directory where the binary files are located providing the link to nets from netlist. GUI users have to pick that file from file selection dialog when importing Saleae data. In python a lookup dictionary providing that information must be given as parameter.

Run the simulation

Python:

sim_ctrl.run_simulation()
while eng.get_state() > 0 and eng.get_state() != 2:
    print ("eng state running %d" % (eng.get_state())) # Output: 1
    time.sleep(1)    
if not eng.get_state()==0:
    # ... Error handling when simulation engine failed to complete
if not sim_ctrl.get_results():
    # ... Error handling when simulation results cannot be parsed

Simulation will be started in a separate thread. Therefore a loop is mandatory to check the engines state (1=Simulation running, 0=Ended successfully, -1=Some error). Once the engine completed successfully the sim_ctrl.get_results() method must be called to parse engines output into waveform database. Waveform results are not loaded into the viewer immediately (See 'Analyze the results').

GUI:

[Run simulation] from toolbar

A status message at the bottom of the waveform viewer widget informs the user whether the simulation is still running, has ended successfully or crashed with some error. On success the controller will try to parse the results and create the waveform database. Dependent on the size of the simulated network this may take a while.

Analyze the results

Waveform viewer elements

Tree view controls order of waveform shown. Reorder or assign to groups by drag'n drop. Invoke context menu to remove items, rename, or create groups.

Waveform diagram shows graphical representation of waveform. Use mouse wheel to zoom in or out. Can also drag over region to zoom in.

Cursor reports waveform value for selected cursor time. Cursor can be moved by drag or double click to new position.

1 Example for 'regular waveform' : CLK, START, DONE.

2 Example for 'waveform group' : OUTPUT (combining 16 bit OUTPUT_0 ... OUTPUT_15 to value displayed as hex number).

3 Example for 'boolean waveform' : A^B (performing XOR operation on waveform A and B).

4 Example for 'trigger time set' : trigger1 (time stamps whenever waveform plusOp_0 has a raising edge).

All results parsed from simulation engine are 'regular' waveforms which can be considered as a function of values 0,1,X,Z over simulated time. Typical steps in analysis are to form groups out of signals representing a single bit, to combine waveform by logical operations, or to extract a set of 'trigger' timestamps at interesting events.

While the commands and options to analyze waveform results from simulation are different in Python and GUI the user can combine the most convenient tools from both 'worlds'. Whenever a waveform gets loaded or assigned to a group its graphical representation gets visible in the waveform viewer and the user can interact with it through GUI. When the user wants to work with elements produced by GUI commands he can access them through netlist controller using the command pl_sim.get_controller_by_id(<id>).

Python commands for waveform analysis

Python: Get a waveform object and load waveform to viewer:

waveform = sim_ctrl.get_waveform_by_net(net)

Get value at time t as integer value. The undecided state 'X' will be represented by -1.

value = waveform.get_value_at(t)

It is possible to retrieve all transition events from a waveform using the waveform.get_events(t0) method. However, a single call must not consume all of the computers memory by generating an extremely big list. Therefore list size is limited and for waveforms with a very high number of transitions multiple calls must be issued according to the following scheme:

waveform = s.get_waveform_by_net(n)
t0 = 0         # start time for requested event data buffer (list of tuples)
while (evts := waveform.get_events(t0)):   # as long as there are events …
    t0 = evts[-1][0] + 1                   # next start time := maxtime + 1
    for tuple in evts:                     # dump events in console
        print ("%8d %4d" % (tuple))

Sometimes the user want to retrieve waveform values whenever an other waveform signal toggles or reaches a certain value. For this special case a set of trigger times needs to be generated first (see below). Once it has been created it can be used in the following command:

evts = waveform.get_triggered_events(trigger,t0)

The user can customize the name of a waveform. Note that the rename method belongs to the controller since the controller is allowed to send signals to the GUI thus making the new name known by the viewer.

sim_ctrl.rename_waveform(waveform,<new name>)

GUI interactions for waveform analysis

Waveforms to be analyzed must be loaded into viewer first.

GUI:

[Load waveform] from toolbar -> select waveforms to analyze or ...
 - [All waveform items] button
 - [Current GUI selection] button

Restore Results from previous Simulation

When a HAL project is saved to disk and gets reopened the simulation results are restored as well. Note that the core waveform data is implemented as database so modifications are stored without explicit ''save'' command. It is also possible to open manually results from previous simulations if the simulation has been performed with the same netlist.

Python:

# based on plugin instance pl_sim (see above)
sim_ctrl = pl_sim.restore_simulator(netlist,"<path_to_external_simulation_workdir>/netlist_simulator_controller.json")

GUI:

[Open input file] from toolbar
Select file 'netlist_simulator_controller.json' in working directory of external simulation

User settings for waveform viewer

[Simulation settings] from toolbar -> 'settings ...'

The waveform viewer plugin has its own user settings file. Choices made are stored and restored in next session. Settings items are:

Global settings

• Load waveform to memory if number transitions <
• Maximum number of values to load into editor
• Custom base directory instead of project dir:

Enging properties

• Engine property name and value

Color settings

• Color for regular waveform in waveform diagram
• Color for selected waveform in waveform diagram
• Color for waveform with undefined value
• Value (0,1,X) specific color both for background of bullet icon and net in graphical netlist.

Sample GUI session to simulate cipher network

In hal import netlist <hal-directory>/plugins/simulator/netlist_simulator_controller/test/netlists/toycipher/cipher_flat.vhd into project. It is Ok to unfold the top_module.

The module has a total of 34 inputs: the clock CLK, START, 16 KEY pins, and 16 PLAINTEXT pins. The following Python script generates sample input for simulation.

def toBinary(hex):
    retval = []
    for cc in hex[::-1]:
        hexdigit = int(cc,16)
        mask = 1
        for j in range(4):
            bit = 0 if (hexdigit&mask==0) else 1
            retval.append(bit)
            mask <<= 1
    return retval

class Waveform:
    def __init__(self,name,trans):
        self.tr = trans
        self.name = name

    def value(self,t):
        count = 0
        while count < len(self.tr):
            if self.tr[count] > t:
                return count % 2
            count += 1
        return count % 2

#Input values for cipher converted to LSB list of binary values
key = "ABCD"
txt = "1337"
print ("key..............<%s>" % (key))
print ("plaintext....... <%s>" % (txt))
key_bits = toBinary(key)
txt_bits = toBinary(txt)

#Transition times (unit=100ps) for input waveform
strt = Waveform("START",[10,20,130,140,250,260])
inputs = [strt]
for i in range(16):
    inputs.append(Waveform("KEY_%d"%(i), [] if key_bits[i] == 0 else [5,240]))
    inputs.append(Waveform("PLAINTEXT_%d"%(i), [] if txt_bits[i] == 0 else [120,240]))

#Generate CSV header, net names must be enclosed in quotes
of=open("cipher_simul_input.csv","w")
of.write("time,\"CLK\"")
for inp in inputs:
    of.write (",\"" + inp.name + "\"")
of.write("\n")

#Generate values for nets, first column is time (unit ms)
t = 0
clk = 0
while t < 290:
    of.write("%8.7f,%d" % (t/10000000.0,clk))
    for inp in inputs:
        of.write(",%d" % (inp.value(t)))
    t += 5
    clk = 1 if clk==0 else 0
    of.write("\n")

of.close()

Running the simple python script above (with hard-coded key and plaintext values and build-in output file name) will produce the workbench file 'cipher_simul_input.csv'.

[Create simulation controller] from toolbar.

[Simulation settings] from toolbar -> 'Select gates for simulation' -> 'All gates' -> 'Ok'

[Simulation settings] from toolbar -> 'Select clock net'  -> 'Do not use clock in simulation' -> 'Ok'

[Simulation settings] from toolbar -> 'Select engine ...'  -> 'verilator'

[Open input file] from toolbar -> files of type : CSV files (*.csv) -> locate the generated workbench file 'cipher_simul_input.csv'

[Run simulation] from toolbar

[Add waveform by net] from toolbar -> 'All waveform items' -> 'Ok'

Might be a good time to detach waveform viewer widget and enlarge its viewport.

[Toggle waveform resolution]

Moving cursor by drag'n drop or double click reveals simulated value at any time

[Simulation settings] from toolbar -> Visualize net state by color

Nets are colored according to simulated value (0=blue, 1=red, X=grey)

Running the built-in hal simulator

This part describes the hal build-in simulator which is should not be used any more. However, for the moment it is kept for testing and prove of backward compatibility. For this purpose the documentation behind this point is kept ... but will be removed eventually.

The user must specify the subset of the netlist he wants to simulate. For this purpose, a list of gates must be specified and passed to the simulator instance sim such that it can automatically determine input and output nets of the subgraph. This can be done using the add_gates function as shown below.

gate_list = [g1, g2, g3, g4]   # collect gates to simulate in a list
sim_ctrl.add_gates(gate_list)        # add the gates to the simulation instance

As stated before, the simulator will automatically detect all inputs and outputs to the simulated subgraph. Using get_input_nets and get_output_nets, the user can retrieve those nets and may use them later on to, e.g., set input values during simulation.

sim_ctrl.get_input_nets()    # get all input nets
sim_ctrl.get_output_nets()   # get all output nets

Next, the clock signal needs to be specified. This can be done using either add_clock_frequency or add_clock_period, which take the net connected to the clock and a clock frequency or period as input. Hence, the function assigns a specific clock frequency to the given net and thus allows for different clock domains.

sim_ctrl.add_clock_frequency(clk, 1000000)   # assign a clock frequency of 1 MHz to input net 'clk'

To avoid getting stuck in infinite loops, it may be helpful to set a iteration timeout that stops the simulation after a pre-defined number of iterations. The timeout is given in number of iterations and can be controlled using set_iteration_timeout and get_iteration_timeout. It defaults to 10000000 iterations and assigning a value of 0 disables the timeout.

sim_ctrl.set_iteration_timeout(1000)   # set the timeout to 1000 iterations

In some cases, sequential gates may come with an initial value that is assigned to their internal state during device startup. For example, this is the case for flip-flops within FPGA netlists, which may retrieve their initial state from the netlist or rather bitstream file. To accommodate this during simulation, these initial values can be loaded into the internal state of such sequential elements using load_initial_values_from_netlist. Further, load_initial_values may be used to specify an initial value to load into all sequential elements before running the simulation.

sim_ctrl.load_initial_values_from_netlist()                        # load initial values into sequential gates if available
sim_ctrl.load_initial_values(hal_py.BooleanFunction.Value.ZERO)   # load logical 0 into all FFs before running simulation

Running the Hal-builtin Simulator

To finally run the simulation, logical values should be assigned to all of the input nets of the simulated subgraph. The function set_input provides exactly this functionality and takes a net as well as a value as input. It may also be used during simulation to change input values between simulation cycles. The simulation itself is started using simulate and proceeds for the specified number of picoseconds. An example is given below.

sim_ctrl.set_input(in1, hal_py.BooleanFunction.Value.ONE)    # assign a logical 1 to input net in1
sim_ctrl.set_input(in2, hal_py.BooleanFunction.Value.ZERO)   # assign a logical 0 to input net in2
sim_ctrl.simulate(3000)                                       # simulate for 3000 ps
sim_ctrl.set_input(in1, hal_py.BooleanFunction.Value.ZERO)   # change the value of in1 to a logical 0
sim_ctrl.simulate(2000)                                       # simulate for another 2000 ps

Retrieving Simulation Results

To get the results of the simulation, the function get_simulation_state may be called to retrieve the entire simulation result. It returns a simulation state containing detailed information of the simulation including all incurred events. By calling get_net_value on that instance, the user can query the value of each net included in the subgraph at every point in time during simulation.

state = sim_ctrl.get_simulation_state()     # retrieve the simulation state
val = state.get_net_value(net, 1000)   # get the value of net 'net' after 1000 ps

Furthermore, the simulation state holds a list of all events incurred during simulation that can be obtained using get_events. This list may also be expanded by the user by creating and adding a new event using add_event. The current simulation state of the simulator instance may be overwritten using set_simulation_state. More on these operations can be found in the official Python API documentation.

Generating a Waveform File

To assist the simulation process, HAL can generate waveform files using the VCD file format. For this purpose, the function generate_vcd can be called to write the current simulation instance to a waveform file. The user may specify a start and an end time (in ps) to avoid the generation of unnecessarily large traces. The VCD file can then be viewed using popular open-source tools like gtkwave. An example looks like this:

sim_ctrl.generate_vcd("trace.vcd", 0, 300000)

Example: Simulating a simple counter

In our tests we provide a netlist for a simple counter. The following code simulates the netlist:

from hal_plugins import netlist_simulator
pl_sim = hal_py.plugin_manager.get_plugin_instance("netlist_simulator")

sim_ctrl = pl_sim.create_simulator_controller()

sim_ctrl.add_gates(netlist.get_gates())
sim_ctrl.load_initial_values(hal_py.BooleanFunction.Value.ZERO)

clock_enable_B = netlist.get_net_by_id(8)
clock = netlist.get_net_by_id(9)
reset = netlist.get_net_by_id(7)
output_0 = netlist.get_net_by_id(6)
output_1 = netlist.get_net_by_id(5)
output_2 = netlist.get_net_by_id(4)
output_3 = netlist.get_net_by_id(3)

sim_ctrl.add_clock_period(clock, 10000)

sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(40 * 1000)

sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(110 * 1000)

sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(20 * 1000)

sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(70 * 1000)

sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(23 * 1000)

sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(20 * 1000)

sim_ctrl.simulate(20 * 1000)

sim_ctrl.simulate(5 * 1000)

sim_ctrl.generate_vcd("counter.vcd", 0, 300000)

The generated vcd can be viewed using gtkwave.