[rv_dm] V1 Signoff

[msfschaffner]

Description

Ensure V1 signoff criteria are fulfilled after focus area changes have landed.

[rswarbrick]

I've gone through this reasonably carefully. I'm not completely sure
whether we can follow the light-weight signoff here, but hopefully
this information will be useful either way.

Commits since Earlgrey-ES tapeout

Checked with version c31cc32 (the head of master on 23/2/24).
There are lots of commits since the Earlgrey ES tapeout, found by
git log Earlgrey-M2.5.2-RC0..HEAD --oneline hw/ip/rv_dm.

• d6b5a48 [RV_DM]rv_dm_abstractcmd_status_vseq
• 39ce8f4 [RV_DM]rv_dm_abstractcmd_testentry
• 882c862 [RV_DM]rv_dm_progbuf_busy_vseq
• b2d81a3 [rv_dm,dv] Shuffle tasks in smoke vseq
• 0fa0d60 [rv_dm,dv] Factor out cases in smoke vseq
• 4fff7d2 [rv_dm,dv] Add a short wait in rv_dm_smoke_vseq
• 71af5e7 [rv_dm,dv] fix typo in seq
• 7c3cb2c [RV_DM]rv_dm_mem_tl_access_resuming_vseq
• df543eb [rv_dm,dv] Remove TODOs when setting reset values in env cfgs
• c690338 [RV_DM]rv_dm_mem_tl_access_halted_vseq
• baf6555 [rv_dm,dv] Add checks to randomization in rv_dm_tap_fsm_vseq
• 253a688 [rv_dm,dv] Add checks to randomization in ..._debug_disabled_vseq
• 091f292 [rv_dm,dv] Add a trivial clocking block to rv_dm_if
• c88d925 [RV_DM]rv_dm_scoreboard
• 85a7350 [RV_DM]rv_dm_stress_all_vseq
• 74c1b11 [rv_dm,dv] Remove a TODO in rv_dm_env.core
• ab4b36f [ipgen,rstmgr] Fix paths to rstmgr in md files
• fc84846 [reggen,hw] Create index parameter for registers windows
• 31433a4 [RV_DM]rv_dm_stress_all_vseq
• 7849658 [RV_DM]rv_dm_mem_tl_access_resuming_vseq
• 18f1692 [rv_dm,dv] Slightly strengthen dmi_debug_disabled test
• 331ca9e [rv_dm,dv] Add comments to rv_dm_jtag_dmi_debug_disabled_vseq
• 48cb2e2 [RV_DM]rv_dm_mem_tl_access_halted_vseq
• de31bdf [reggen] Remove the devmode input
• c535a58 [RV_DM] rv_dm_jtag_dtm_hard_reset_vseq
• 637d1b8 [doc,reset] Fix references to reset behavior
• 5a92acb [RV_DM] rv_dm_jtag_dtm_hard_reset_vseq
• 0dc1bca [rv_dm,dv] update test description for jtag_dtm_hard_reset
• fde2e0c [RV_DM]JTAG_dtm_idle_hint_vseq
• 358d838 [rv_dm,dv] fix typo in the testplan
• 51130d7 [RV_DM] rv_dm_jtag_dmi_debug_disabled_vseq
• 17d386c [rv_dm,dv] fix regression failure
• ee62ddb [rv_dm/doc] Add some missing CSR documentation
• ca2b62b [dv, testplan] Replace descr by desc for consistency
• 6329668 [rv_dm,sival] RV_DM post silicon validation testplan
• f259192 [RV_DM]jtag_dmi_dm_inactive
• 4143382 [RV_DM]jtag_dtm_idle_hint
• 5d8bab5 [RV_DM]ndmreset_req sequence
• aa2af96 [RV_DM]halt_resume_whereto
• efea9f8 [rv_dm,dv] Add whereto check to halt_resume testplan
• 467dfb2 [rv_dm,dv] update testplan for halt/going/resume
• 1b16ca2 [reggen] Add mubi support SWAccess that sets/clears a reg
• bb5f3b6 [rv_dm] update register description
• 7aeb4c2 [RV_DM]Dataaddr_rw_access Test
• 6585d93 [RV_DM] Updated HALTED and RESUMING test
• 9cf0ef2 [RV_DM] test_rename
• 21b8733 [RV_DM]cmderr_halt_resume_test
• 806815a [RV_DM] Hart unavailable test
• 59f8142 [doc] Moved badges over to using hosted images
• 0773985 [RV_DM] Hart resuming test
• a23f5d6 [rv_dm] included seq to seq list
• 0eaa70f [rv_dm] remove extra displays
• 50d9dbb [rv_dm] Halted test
• 7d5f36f [RV_DM] Command error exception test
• e9ba8b3 [RV_DM]cmderr_not_supported_test
• ee6f00f [doc] rv_dm registers and interfaces now use CMDGEN
• a0f8509 [RV_DM]cmderr_busy_test
• 420c29a [RV_DM]cmderr_busy test
• 7688e71 [reggen] Add initial support for version and cip_id hjson fields
• fbd888e Revert "[reggen] Add CIP_IDs and bump all major versions"
• 0ba10b3 [reggen] Add CIP_IDs and bump all major versions

This is a long list! But most of it is DV implementation work done by
Imparé. Looking at the other changes first (and ignoring the reverted
CIP_ID change), I find:

• ab4b36f [ipgen,rstmgr] Fix paths to rstmgr in md files
  • Fixing doc typo
• fc84846 [reggen,hw] Create index parameter for registers windows
  • Adds (unused) parameter
• de31bdf [reggen] Remove the devmode input
  • No change in behaviour
• 637d1b8 [doc,reset] Fix references to reset behavior
  • Documentation change (not affecting things that are tracked in V1)
• ee62ddb [rv_dm/doc] Add some missing CSR documentation
  • Documentation change, documenting some undocumented CSRs.
• ca2b62b [dv, testplan] Replace descr by desc for consistency
  • Minor tooling tidyup
• 6329668 [rv_dm,sival] RV_DM post silicon validation testplan
  • SiVal testplan, which won't affect V1
• 1b16ca2 [reggen] Add mubi support SWAccess that sets/clears a reg
  • No change in behaviour
• 59f8142 [doc] Moved badges over to using hosted images
  • Documentation infrastructure change.
• ee6f00f [doc] rv_dm registers and interfaces now use CMDGEN
  • Documentation infrastructure change.
• 7688e71 [reggen] Add initial support for version and cip_id hjson fields
  • Only changes hjson file. No change to design or DV code.

None of those changes should affect the V1 status.

There are quite a lot of DV changes, driven by the work that Imparé
has been doing. I've tried to narrow down a bit to things that might
affect V1 status.

I think the V1 checklist items that might be affected are:

• TESTPLAN_COMPLETED

  This is actually the closest to being affected by the changes that
  have gone in. Looking at the history of
  hw/ip/rv_dm/data/rv_dm_testplan.hjson find lots of changes.
  These all add testpoints.

  The background is that RV_DM was signed off as V1 status back in
  2022, but we never did much DV work after that. Imparé have
  started trying to improve coverage numbers (as reported by nightly
  regressions), which has required adding extra tests/testpoints.
  This explains the testplan changes that have happened.

• SIM_ALT_TOOL_SETUP

  I've checked, and this seems to work on both Xcelium and VCS.

• TESTPLAN_REVIEWED

  I'm not sure how this should work. An earlier version of the
  testplan was reviewed, and it's not completely obvious whether
  changes to the testplan should invalidate this checklist item.

  Given how this has been driven by coverage closure concerns, it
  might make sense to consider this to still be complete.

Issues closed since the Earlgrey-ES tapeout

Using the filter is:issue is:closed closed:>2023-06-27 rv_dm finds
quite a lot of issues. I've dropped the issues that were closed as
not-planned (after checking that they aren't really relevant here).

• [dv,csr] Stop transaction generation for random resets #21531
  Unrelated DV change (I mentioned rv_dm in one of the comments)
• [ip,doc] Fix inaccurate references to reset behavior #20700
  Documentation bug. Fixed by 637d1b8 above.
• [rtl] Are design guidlines for use of prim_lc_sync correct? #19044
  All action items have been spun into separate issues.
• [opentitanlib,testing] add integrity check mechanism on SRAM programs #18635
  Unrelated SW bug (which mentions rv_dm in description)
• [clkmgr] Check whether switch to external clock should be disallowed in DEV for production silicon devices #18291
  Chip-level discussion (mentioning rv_dm in one comment)
• [dv/cdc] Enable CDC instrumentation in all ips #16689
  CDC check tracking issue
• [dv,chip] Wrong register alignment when using process_tl_access  #13648
  Chip-level test. Closed as obsolete.
• [lc_ctrl] Add debug register for observing critical system signals #10462
  lc_ctrl issue which mentions rv_dm as another way of getting some
  information out.
• Issues tracking implementation of tests:
  • [rv_dm,dv] Verify debugger can read status of unavailable hart (hart_unavail BLT) #17036
  • [rv_dm,dv] Verify that writes to DMI registers are ignored while DM is not active (jtag_dmi_dm_inactive BLT) #17035
  • [rv_dm,dv] Verify DTM idle behavior (jtag_dtm_idle_hint BLT) #17033
  • [rv_dm,dv] Verify that the debugger can cancel an on-going DMI transaction with a hard reset (jtag_dtm_hard_reset BLT) #15670
  • [rv_dm,dv] Verify that the JTAG DMI register space cannot be accessed if lc_hw_debug_en is a non strict true value (jtag_dmi_debug_disabled BLT) #15667

Currently open issues

Here is a list of tests based on the filter is:issue is:open rv_dm NOT sival NOT chip-test (to exclude sival tracking issues). I've
skipped the V1/D2 signoff issues.

• [rv_dm,dv] Smoke vseq is broken for some seeds #21517
  • Minor DV testbench problem. Fix pending in a PR.
• [rv_dm] Restricting JTAG-based Debug in DEV  #20829
  • This tracks implementation of an RFC (approved).
  • The implementation will be in rv_dm and this is not yet done (23/2/24)
• [rv_dm,rtl] Potential issue with hartsel WARL behaviour #19750
  • Potential design change which is not yet implemented (23/2/24)
  • An implementation will only have a small DV impact (tweak
    scoreboard; plan and implement a vseq to misuse the register).
• [bazel] Add QEMU support for opentitan_functest #19375
  • SW only
• [rv_dm] V2.5 TODOs  #18395
  • TODO tracking. This needn't gate V1 signoff (and some items have
    actually been implemented already).
• [rv_dm] V2 + V2S tests back to 90% pass rate #18310
  • A task for V2 tracking. This needn't gate V1 signoff.
• [rv_dm,dv] Verify that a failed DMI op is reflected properly in dtmcs register (jtag_dmi_failed_op BLT) #17034
  • Tracking implementation of a test (which might well turn out to be
    a situation that can't occur and doesn't need testing).
• [dv/rstmgr] Apply different reset kinds where applicable #16148
  • An issue in a different block (which mentions rv_dm as an example
    of how to do it!)
• [dv] Knobs for ignoring TL errors in top-level sims #15803
  • Chip-level DV problem
• [rv_dm,dv] Verify that access to SBA interface is disabled if lc_hw_debug_en is set to not strict true value (sba_debug_disabled BLT) #15668
  • Unimplemented test. This is tracked in the testplan, so shouldn't
    gate V1.
• [rv_dm,rtl] Fix acknowledgment of reset in dmstatus.allhavereset/dmstatus.anyhavereset #15123
  • Low(ish) priority potential design change.
  • Our (currently incorrect) behaviour isn't actually tracked in models.
  • We'd probably want to improve the model/scoreboard side if we
    implement the correct behaviour, but it won't need explicit
    testplanning.
• [top] "rom" interface of "rv_dm" is not a memory in top_earlgrey.hjson #14345
  • Tooling issue (mentioning rv_dm as somewhere that worked around
    the problem).

Throwing away the items that can probably be ignored for V1, I believe
these are the only two tasks that might affect V1 signoff:

• [rv_dm] Restricting JTAG-based Debug in DEV  #20829
• [rv_dm,rtl] Potential issue with hartsel WARL behaviour #19750

Coverage report from 21/02/2024

All V1 tests are implemented. Looking more closely at the report, they also all pass for most seeds: it's not like there's one test which is completely broken.

Summary

• I don't believe that the lightweight sign-off flow really makes
  sense for rv_dm to get to V1.
• Fortunately, I think we are ready to do a full sign-off meeting.
• I propose we schedule one soon.

[rswarbrick]

We had a V1 review meeting this afternoon. Notes here.

The meeting was happy that we have satisfied the items on the V1 checklist, except for the fact that we hadn't actually addressed V2_CHECKLIST_SCOPED (my fault!). Below are some rough estimates for the work I expect V2 signoff to entail.

@andreaskurth, @hcallahan-lowrisc: Would you mind taking a look at this? If you agree with the rough estimates below, I think we're ready to sign off rv_dm with a V1 status.

---

The V1 checklist contains an item, V2_CHECKLIST_SCOPED. This ensures
that we've thought through any pending DV work that we know is going
to be needed for V2 signoff.

Some of the V2 checklist items are necessarily dependent on what
happens between now and then(!) but here are the items that we
currently know will require work, together with rough estimates for
how many days' work they might entail.

• DV_DOC_COMPLETED

  The current DV document still has some placeholder items! I think
  they are probably just examples that can be deleted, but there's
  some tidying required.

  Estimate: 1 day

• FUNCTIONAL_COVERAGE_IMPLEMENTED

  The existing coverage plan lies at the bottom of
  rv_dm_testplan.hjson. It's pretty barebones, and there's not
  currently much implemented (e.g. rv_dm_env_cov.sv is empty)

  Estimate: 4 days

• SIM_ALL_TESTS_PASSING

  Obviously this is pretty hard to estimate! But most existing tests
  do currently pass, so we might be hopeful this isn't actually
  too much work.

  Estimate: 5 days

• SIM_FUNCTIONAL_COVERAGE_V2

  This is also pretty hard to estimate. Practically speaking, we're
  not expecting to have an enormous list of coverpoints and the
  state space for a debug module isn't enormous.

  Estimate: 5 days.

Summing the numbers above: ~15 days work.

[hcallahan-lowrisc]

I think your estimates seem reasonable to me @rswarbrick. Based on that, I think we should be okay to sign it off. Thanks for keeping it moving!

[andreaskurth]

Thanks @rswarbrick, effort estimates LGTM

[rswarbrick]

Great, thanks guys.