Added support for dynamic seed generation.

docs/news.d/1089.feature.rst

@@ -0,0 +1,4 @@
+Provided a unique base seed to every simulation. From that base seed any number of unique seed can derived
+to initialize the random number generators in the testbench. Unique seeds increase the test coverage for
+randomized tests over time as they are executed repeatedly by the user and the CI. The seed can be overrideen
+from the command line with the --seed option in order to reproduce failing tests.

docs/run/img/get_seed_and_runner_cfg.html

@@ -0,0 +1,2 @@
+<div class="highlight"><pre><span></span><span class="k">constant</span><span class="w"> </span><span class="n">global_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="kt">string</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="n">get_seed</span><span class="p">(</span><span class="n">runner_cfg</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;Optional salt&quot;</span><span class="p">);</span>
+</pre></div>

docs/run/img/get_seed_wo_runner_cfg.html

@@ -0,0 +1,5 @@
+<div class="highlight"><pre><span></span><span class="n">randomizing_process</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">process</span><span class="w"> </span><span class="k">is</span>
+<span class="w">  </span><span class="k">variable</span><span class="w"> </span><span class="n">local_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="n">string_seed_t</span><span class="p">;</span><span class="w"> </span><span class="c1">-- string_seed_t = string(1 to 16)</span>
+<span class="k">begin</span>
+<span class="w">  </span><span class="n">get_seed</span><span class="p">(</span><span class="n">local_seed</span><span class="p">,</span><span class="w"> </span><span class="n">salt</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="n">randomizing_process</span><span class="na">&#39;path_name</span><span class="p">);</span>
+</pre></div>

docs/run/img/get_uniform_seed.html

@@ -0,0 +1,5 @@
+<div class="highlight"><pre><span></span><span class="n">get_uniform_seed</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;Optional salt&quot;</span><span class="p">);</span>
+
+<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">a_random_value</span><span class="p">);</span>
+<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">another_random_value</span><span class="p">);</span>
+</pre></div>

docs/run/img/list.html

@@ -1,16 +1,22 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py --list
+lib.tb_fail_on_warning.Test that fails on an assert
+lib.tb_fail_on_warning.Test that crashes on boundary problems
+lib.tb_fail_on_warning.Test that fails on VUnit check procedure
+lib.tb_fail_on_warning.Test that a warning passes
 lib.tb_magic_paths.all
 lib.tb_minimal.all
 lib.tb_running_test_case.Test scenario A
 lib.tb_running_test_case.Test scenario B
 lib.tb_running_test_case.Test something else
 lib.tb_run_all_in_same_sim.Test to_string for integer again
 lib.tb_run_all_in_same_sim.Test to_string for boolean again
+lib.tb_seed.all
 lib.tb_standalone.Test that fails on VUnit check procedure
 lib.tb_standalone.Test to_string for boolean
 lib.tb_stopping_failure.Test that fails on an assert
 lib.tb_stopping_failure.Test that crashes on boundary problems
 lib.tb_stopping_failure.Test that fails on VUnit check procedure
+lib.tb_stopping_failure.Test that a warning passes
 lib.tb_stop_level.Test that fails multiple times but doesn't stop
 lib.tb_with_lower_level_control.Test something
 lib.tb_with_lower_level_control.Test something else
@@ -19,5 +25,5 @@
 lib.tb_with_watchdog.Test that stalls
 lib.tb_with_watchdog.Test to_string for boolean
 lib.tb_with_watchdog.Test that needs longer timeout
-Listed 20 tests
+Listed 26 tests
 </pre></div>

docs/run/img/seed_option.html

@@ -0,0 +1 @@
+<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py "lib.tb_seed.Test that fails" --seed fb19f3cca859d69c</pre></div>

docs/run/img/tb_magic_paths_stdout.html

@@ -1,18 +1,19 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py -v
 Starting lib.tb_magic_paths.all
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94\output.txt
+Seed for lib.tb_magic_paths.all: 3680f2e2321cdac3
                0 fs - default              -    INFO - Directory containing testbench: C:/github/vunit/docs/run/src/
                0 fs - default              -    INFO - Test output directory: C:/github/vunit/docs/run/src/vunit_out/test_output/lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94/
 simulation stopped @0ms with status 0
-<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.5 seconds)
+<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.6 s)
 
 ==== Summary ==================================
-<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.5 seconds)
+<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.6 s)
 ===============================================
 <span style="color: #008000; ">pass</span> 1 of 1
 ===============================================
-Total time was 0.5 seconds
-Elapsed time was 0.5 seconds
+Total time was 0.6 s
+Elapsed time was 0.6 s
 ===============================================
 <span style="color: #008000; ">All passed!</span>
 </pre></div>

docs/run/img/tb_minimal_stdout.html

@@ -1,15 +1,15 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
 Starting lib.tb_minimal.all
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_minimal.all_42aa262c7c96c708ab3f3960f033f2328c642136\output.txt
-<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.5 seconds)
+<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.6 s)
 
 ==== Summary ==============================
-<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.5 seconds)
+<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.6 s)
 ===========================================
 <span style="color: #008000; ">pass</span> 1 of 1
 ===========================================
-Total time was 0.5 seconds
-Elapsed time was 0.5 seconds
+Total time was 0.6 s
+Elapsed time was 0.6 s
 ===========================================
 <span style="color: #008000; ">All passed!</span>
 </pre></div>

docs/run/img/tb_running_test_case_stdout.html

@@ -1,25 +1,25 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
 Starting lib.tb_running_test_case.Test scenario A
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_A_b118fcdf82c6ba5772e038ce7455962692f50c2a\output.txt
-<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.5 seconds)
+<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.6 s)
 
-Starting lib.tb_running_test_case.Test scenario B
+(22:55:49) Starting lib.tb_running_test_case.Test scenario B
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_B_8cd407ec92589901f9b4594c0f27835478242f2c\output.txt
-<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.5 seconds)
+<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.6 s)
 
-Starting lib.tb_running_test_case.Test something else
+(22:55:50) Starting lib.tb_running_test_case.Test something else
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_something_else_27dcc1aa8d44993b6b2d0b0a017fa6001b4c2aa7\output.txt
-<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.5 seconds)
+<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.6 s)
 
 ==== Summary ========================================================
-<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A     (0.5 seconds)
-<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B     (0.5 seconds)
-<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.5 seconds)
+<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A     (0.6 s)
+<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B     (0.6 s)
+<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.6 s)
 =====================================================================
 <span style="color: #008000; ">pass</span> 3 of 3
 =====================================================================
-Total time was 1.6 seconds
-Elapsed time was 1.6 seconds
+Total time was 1.8 s
+Elapsed time was 1.8 s
 =====================================================================
 <span style="color: #008000; ">All passed!</span>
 </pre></div>

docs/run/img/tb_seed_stdout.html

@@ -0,0 +1,28 @@
+<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
+Starting lib.tb_seed.Test that passes
+Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_seed.Test_that_passes_4125d67fe52dadd934f892b1209f41e7a94a39bd\output.txt
+Seed for lib.tb_seed.Test that passes: 8c5c0ea80b58a8ee
+simulation stopped @0ms with status 0
+<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=2) lib.tb_seed.Test that passes (0.5 s)
+
+(09:37:45) Starting lib.tb_seed.Test that fails
+Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_seed.Test_that_fails_0f173e63967af845f06d4a86c622bba76f3ffb3d\output.txt
+Seed for lib.tb_seed.Test that fails: fb19f3cca859d69c
+               0 fs - default              -   ERROR - Something bad happened
+C:\repos\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error
+ghdl:error: report failed
+ghdl:error: simulation failed
+<span style="color: #cb4b16; ">fail</span> (P=1 S=0 F=1 T=2) lib.tb_seed.Test that fails (0.5 s)
+
+==== Summary ========================================
+<span style="color: #008000; ">pass</span> lib.tb_seed.Test that passes (0.5 s)
+<span style="color: #cb4b16; ">fail</span> lib.tb_seed.Test that fails  (0.5 s)
+=====================================================
+<span style="color: #008000; ">pass</span> 1 of 2
+<span style="color: #cb4b16; ">fail</span> 1 of 2
+=====================================================
+Total time was 1.0 s
+Elapsed time was 1.0 s
+=====================================================
+<span style="color: #cb4b16; ">Some failed!</span>
+</pre></div>

docs/run/img/tb_stop_level_stdout.html

@@ -1,26 +1,23 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
 Starting lib.tb_stop_level.Test that fails multiple times but doesn't stop
 Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stop_level.Test_that_fails_multiple_times_but_doesn't_stop_d08f48d859442d0bc71e2bcdd8b429119f7cc17c\output.txt
+Seed for lib.tb_stop_level.Test that fails multiple times but doesn't stop: 7a0e979de335b966
                0 fs - check                -   ERROR - Equality check failed - Got 17. Expected 18.
                0 fs - check                -   ERROR - Equality check failed - Got 17. Expected 19.
 FAILURE - Logger check has 2 errors
-C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:84:7:@0ms:(report failure): Final log check failed
-C:\ghdl\bin\ghdl.exe:error: report failed
-in process .tb_stop_level(tb).test_runner
-  from: vunit_lib.logger_pkg.final_log_check at logger_pkg-body.vhd:1249
-  from: vunit_lib.run_pkg.test_runner_cleanup at run.vhd:114
-  from: process lib.tb_stop_level(tb).test_runner at tb_stop_level.vhd:29
-C:\ghdl\bin\ghdl.exe:error: simulation failed
-<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds)
+C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Final log check failed
+ghdl:error: report failed
+ghdl:error: simulation failed
+<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s)
 
 ==== Summary =============================================================================
-<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds)
+<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s)
 ==========================================================================================
 <span style="color: #008000; ">pass</span> 0 of 1
 <span style="color: #cb4b16; ">fail</span> 1 of 1
 ==========================================================================================
-Total time was 0.5 seconds
-Elapsed time was 0.5 seconds
+Total time was 0.6 s
+Elapsed time was 0.6 s
 ==========================================================================================
 <span style="color: #cb4b16; ">Some failed!</span>
 </pre></div>

docs/run/img/tb_stopping_failure_stdout.html

@@ -1,40 +1,43 @@
 <div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
 Starting lib.tb_stopping_failure.Test that fails on an assert
-Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt
-C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation
+Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt
+Seed for lib.tb_stopping_failure.Test that fails on an assert: a866d0986a663e6b
+C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation
 ghdl:error: assertion failed
 ghdl:error: simulation failed
-<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.5 s)
+<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.6 s)
 
-(11:35:31) Starting lib.tb_stopping_failure.Test that crashes on boundary problems
-Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt
-ghdl:error: index (314) out of bounds (1 to 17) at C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:26
+(22:55:55) Starting lib.tb_stopping_failure.Test that crashes on boundary problems
+Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt
+Seed for lib.tb_stopping_failure.Test that crashes on boundary problems: cbcb543764149e78
+ghdl:error: index (340) out of bounds (1 to 17) at C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:26
 ghdl:error: simulation failed
-<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.5 s)
+<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.6 s)
 
-(11:35:31) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure
-Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt
+(22:55:55) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure
+Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt
+Seed for lib.tb_stopping_failure.Test that fails on VUnit check procedure: 6581894e9ea614b1
                0 fs - check                -   ERROR - Equality check failed - Got 17. Expected 18.
-C:\repos\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error
+C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error
 ghdl:error: report failed
 ghdl:error: simulation failed
-<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s)
+<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s)
 
-(11:35:32) Starting lib.tb_stopping_failure.Test that a warning passes
-Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt
-<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.4 s)
+(22:55:56) Starting lib.tb_stopping_failure.Test that a warning passes
+Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt
+<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.6 s)
 
 ==== Summary ============================================================================
-<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes               (0.4 s)
-<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert             (0.5 s)
-<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems   (0.5 s)
-<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s)
+<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes               (0.6 s)
+<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert             (0.6 s)
+<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems   (0.6 s)
+<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s)
 =========================================================================================
 <span style="color: #008000; ">pass</span> 1 of 4
 <span style="color: #cb4b16; ">fail</span> 3 of 4
 =========================================================================================
-Total time was 1.8 s
-Elapsed time was 1.8 s
+Total time was 2.4 s
+Elapsed time was 2.4 s
 =========================================================================================
 <span style="color: #cb4b16; ">Some failed!</span>
 </pre></div>