fix(MainPipe): fix error report valid when Atomics and SBuffer request miss (#4572)

* Sbuffer write and Atomics should not report errors, but refill from L2
should report ecc error, but requests in

fix(MainPipe): fix error report valid when Atomics and SBuffer request miss (#4572)

* Sbuffer write and Atomics should not report errors, but refill from L2
should report ecc error, but requests in MissQueue carry `isAmo` or
`isStore` and `req.miss` in a request, hence `(s2_req.isAMO ||
s2_req.isStore)` includes the refill, so the missing request of `isAmo`
or `isStore` will not report an error

show more ...

fix(MainPipe): fix `s1_way_en` logic when pseudo tag error inject (#4573)

* `s1_way_en` should use `io.pseudo_error.valid` on the same stage, not
`RegNext(io.pseudo_error.valid)`. Otherwise, `MainPi

fix(MainPipe): fix `s1_way_en` logic when pseudo tag error inject (#4573)

* `s1_way_en` should use `io.pseudo_error.valid` on the same stage, not
`RegNext(io.pseudo_error.valid)`. Otherwise, `MainPipe` may use wrong
way enable to write DCache, it will result in two identical tags.

show more ...

fix(MainPipe): fix error valid generate logic (#4480)

* Atomics and Sbuffer write request should not report error

fix(MainPipe): fix tag match logic when ecc inject occur (#4479)

* When ecc injection occurs, it is necessary to determine whether there
is a hit cacheline in the `TagArray`. For example, `prefetch.

fix(MainPipe): fix tag match logic when ecc inject occur (#4479)

* When ecc injection occurs, it is necessary to determine whether there
is a hit cacheline in the `TagArray`. For example, `prefetch.w` will be
refilled due to permission mismatch. At this time, there is a hit
cacheline in the `TagArray`. If there is a hit way, it is necessary to
select the way, otherwise select the way determined by replacement.

* It's necessary to determine whether there is a hit with a non-toggle
tag.

show more ...

feat(dfx): reuse dcache data sram read data register as mbist pipeline (#4371)

Co-authored-by: sfencevma <[email protected]>

fix(MainPipe): `error` and `writeback` addr generate logic (#4394)

There were errors in the previous design

* `writeback` generate wrong addr
* `writeback`'s addr use `s3_tag` to generate , no

fix(MainPipe): `error` and `writeback` addr generate logic (#4394)

There were errors in the previous design

* `writeback` generate wrong addr
* `writeback`'s addr use `s3_tag` to generate , no need to use
`s3_tag_error` to select.

* `error` generate wrong addr
* `error` must use `s3_tag` to generate, not use `s3_req.addr`,
* because the enable condition of `s3_req.addr` is different from that
of `s3_error`,
* should use access cacheline corresponding address

show more ...

fix(MainPipe): `DCache` meta is not changed when sc/cas fails (#4217)

`sc` miss are treated as failures directly and are no longer sent to the
`missqueue`.
The meta is no longer updated when `sc` fa

fix(MainPipe): `DCache` meta is not changed when sc/cas fails (#4217)

`sc` miss are treated as failures directly and are no longer sent to the
`missqueue`.
The meta is no longer updated when `sc` fail.

show more ...

fix(MainPipe): fix `s3_l2_error` and `s3_error` enable signal (#4345)

* `s2_fire` and `s2_can_to_s3` are different
* `io.error.valid` uses `s2_fire`, but `s3_l2_error` uses
`s2_can_to_s3`, causing

fix(MainPipe): fix `s3_l2_error` and `s3_error` enable signal (#4345)

* `s2_fire` and `s2_can_to_s3` are different
* `io.error.valid` uses `s2_fire`, but `s3_l2_error` uses
`s2_can_to_s3`, causing `io.error.valid` to be updated, but
`s3_l2_error` not to be updated.

show more ...

fix(DCache): fix wrong condition for blocking lr (#4337)

Following lr should be blocked when previous lr's resv_set is still
valid, which means `lrsc_count > 0`.

In previous PR #3017 and #4117, `lr

fix(DCache): fix wrong condition for blocking lr (#4337)

Following lr should be blocked when previous lr's resv_set is still
valid, which means `lrsc_count > 0`.

In previous PR #3017 and #4117, `lrsc_count > 8` is used as block
condition, and stop update `lrsc_count` when it reaches 8, fix it now.

show more ...

timing(MainPipe): remove `set_conflict` for tag/meta read (#4295)

* reduce meta/tag read enable generate logic
* when `set_conflict == 1`, request can not go to `s1`, and `write` has
higher priori

timing(MainPipe): remove `set_conflict` for tag/meta read (#4295)

* reduce meta/tag read enable generate logic
* when `set_conflict == 1`, request can not go to `s1`, and `write` has
higher priority to `read`

show more ...

fix(MainPipe): fix `s1_way_en` generate logic when ecc inject occur (#4285)

* When inject tag ecc error `io.pseudo_error.valid == 1` and mainpipe
request valid (for example `io.miss_req.valid == 1`

fix(MainPipe): fix `s1_way_en` generate logic when ecc inject occur (#4285)

* When inject tag ecc error `io.pseudo_error.valid == 1` and mainpipe
request valid (for example `io.miss_req.valid == 1`
```
val s1_need_replacement = s1_req.miss && !s1_tag_match
val s1_need_eviction = s1_req.miss && !s1_tag_match && s1_repl_coh.state =/= ClientStates.Nothing
val s1_way_en = Mux(s1_need_replacement, s1_repl_way_en, s1_tag_match_way)
```
when generate `s1_way_en`, `s1_tag_match == 0` cause inject tag ecc
error, so `s1_need_replacement == 1`, but may be target cacheline
already in dcache (for example `prefetch.w` miss), finally there is 2
same paddr cacheline in dcache, obviously this is illegal.

* Determine whether it is a pseudo error. If it's pseudo error, use
no-toggled-tag (which no toggled by cacheCtrl) for generate `s1_way_en`,
otherwise use toggled-tag

show more ...

fix(DCache): pass `amo_cmp` to MSHR when cas req miss (#4272)

fix(DCache): remove error wakeup signal for atomic resp (#4224)

The existence of `s2/s3_req.miss` in `atomic_resp_valid` will cause an
error resp to AtomicsUnit by a normal refill_req from MissQueue

fix(DCache): remove error wakeup signal for atomic resp (#4224)

The existence of `s2/s3_req.miss` in `atomic_resp_valid` will cause an
error resp to AtomicsUnit by a normal refill_req from MissQueue. Remove
it now to fix the problem.

show more ...

fix(MainPipe): fix bug in data select of AMOCAS.Q (#4213)

Data selection logic should use signals in `s3` instead of `s1`, which
is introduced in a former pr #4117.

fix(MainPipe): use s3_tag_error to generate error report signal (#4208)

area(MainPipe): remove duplicate signals (#4117)

feat(XSLog): move all XSLog outside WhenContext for collection

As data in WhenContext is not acessible in another module. To support
XSLog collection, we move all XSLog and related signal outside
Wh

feat(XSLog): move all XSLog outside WhenContext for collection

As data in WhenContext is not acessible in another module. To support
XSLog collection, we move all XSLog and related signal outside
WhenContext. For example, when(cond1){XSDebug(cond2, pable)} to
XSDebug(cond1 && cond2, pable)

show more ...

feat(CtrlUnit, DCache): support L1 DCache RAS (#4009)

# L1 DCache RAS extension support

The L1 DCache supports the part of Reliability, Availability, and
Serviceability (RAS) Extension.
* L1 DCache

feat(CtrlUnit, DCache): support L1 DCache RAS (#4009)

# L1 DCache RAS extension support

The L1 DCache supports the part of Reliability, Availability, and
Serviceability (RAS) Extension.
* L1 DCache protection with Single Error Correct Double Error Detect
(SECDED) ECC on the RAMs. This includes the L1 DChace tag and data RAMs.
Not recovery error tag or data.
* Fault Handling Interrupt (Bus Error Unit Interrupt,BEU, 65)
* Error inject

## ECC Error Detect
An error might be triggered, when access L1 DCache.
* **Error Report**:
* Tag ECC Error: As long as an ECC error occurs on a certain path, it
is judged that an ECC error has occurred.
* Data ECC Error: If an ECC error occurs in the hit line, it is
considered
that an ECC error has occurred. If it does not hit, it will not be
processed.
* If an instruction access triggers an ECC error, a Hardware error is
considered and an exception is reported.
* Whenever there is an error in starting, an error message needs to
be sent to BEU.
* When the hardware detects an error, it reports it to the BEU and
triggers the NMI external interrupt(65).

* **Load instruction**:
* Only ECC errors of tags or data will be triggered during execution,
and the errors will be reported to the BEU and a `Hardware Error`
will be reported.

* **Probe/Snoop**:
* If a tag ecc error occurs, there is no need to change the cache
status,
and a `ProbeAck` with `corrupt=1` needs to be returned to l2.
* If a data ecc error occurs, change the cache status according to
the rules. If data needs to be returned, `ProbeAckData` with `corrupt=1`
needs to be returned to l2.

* **Replace/Evict**:
* `ReleaseData` with `corrupt=1` needs to be returned to l2.

* **Store to L1 DCache**:
* If a tag ecc error occurs, the cacheline is released according to the
`Repalce/Evict` process and the data is written to L1 DCache without
reporting errors to l2.
* If a data ecc error occurs, the data is written directly without
reporting
the error to l2.

* **Atomics**:
* report `Hardware Error`, do not report errors to l2.

## Error Inject
Each core's L1 DCache is configured with a memory map
register-controlled
controller, and each hardware unit that supports ECC is configured with
a
control bank. After the Bank register configuration is completed, L1
DCache
will trigger an ecc error for the first access L1 DCache.
<div style="text-align: center;">
<img
src="https://github.com/user-attachments/assets/8c4d23c5-0324-4e52-bcf4-29b47a282d72"
alt="err_inject" width="200" />
</div>

### Address Space
Address space `0x38022000`-`0x3802207F`, a total of 128 bytes of space,
this space is the local space of each hart.
<div style="text-align: center;">
<img width="292" alt="ctl_bank"
src="https://github.com/user-attachments/assets/89f88b24-37a4-4786-a192-401759eb95cf">
</div>

### L1 DCache Control Bank
Each Control Bank contains registers: `ECCCTL`, `ECCEID`, `ECCMASK`,
each register is 8 bytes.
<img width="414" alt="eccctl"
src="https://github.com/user-attachments/assets/b22ff437-d05d-4b3c-a353-dbea1afdc156">
* ECCCTL(ECC Control): ECC injection control register.
* `ese(error signaling enable)`: Indicates that the injection is valid
and is initialized to 0. When the injection is successful and `pst==0`,
ese will be clean.
* `pst(persist)`: Continuously inject signals. When `pst==1`,
the `ECCEID`
counter decreases to 0 and after successful injection, the
injection timer will be restored to the last set `ECCEID` and
re-injected;
when `pst==0`, it will be injected only once.
* `ede(error delay enable)`: Indicates that counter is valid and
initialized to 0. If
* `ese==1` and `ede==0`, error injection is effective immediately.
* `ese==1` and `ede==1`, you need to wait until `ECCEID`
decrements to 0 before the injection is effective.
* `cmp(component)`: Injection target, initialized to 0.
* 1'b0: The injection object is tag.
* 1'b1: The injection object is data.
* `bank`: The bank valid signal is initialized to 0. When the bit in
the `bank` is set, the corresponding mask is valid.

<img width="414" alt="ecceid"
src="https://github.com/user-attachments/assets/8cea0d8d-2540-44b1-b1f9-c1ed6ec5341e">

* ECCEID(ECC Error Inject Delay): ECC injection delay controller.
* When `ese==1` and `ede==1`, it
starts to decrease until it reaches 0. Currently, the same clock as
the core frequency is used, which can also be divided. Since ECC
injection relies on L1 DCache access, the time of the `EID` and the
time when the ECC error is triggered may not be consistent.

<img width="414" alt="eccmask"
src="https://github.com/user-attachments/assets/b1be83fd-17a6-4324-8aa6-45858249c476">

* ECCMASK(ECC Mask): ECC injection mask register.
* 0 means no inversion, 1 means flip.
Tag injection only uses the bits in `ECCMASK0` corresponding to
the tag length.

### Error Inject Example
```
1 # set control bank base address
2 mv x3, $(BASEADDR)
3
4 # set eid
5 mv x5, 500 # delay 500 cycles
6 sd x5, 8(x3) # mmio store
7
8 # set mask
9 mv x5, 0x1 # flip bit 0
10 sd x5, 16(x3) # mmio store
11
12 # set ctl
13 mv x5, 0x7 # comp = 0, ede = 1, pst = 1, ese = 1
14 sd x5, 0(x3) # mmio store
```

show more ...

fix(MainPipe): fix bug in data used for atomics

feat(MemBlock): add support for Zacas extension

fix(AtomicsUnit, MemBlock): fix loss of multiple stds

In the previous design, AtomicsUnit receives stds from StdExeUnit and
arbitrate at most one std

feat(MemBlock): add support for Zacas extension

fix(AtomicsUnit, MemBlock): fix loss of multiple stds

In the previous design, AtomicsUnit receives stds from StdExeUnit and
arbitrate at most one std uop for one cycle. This works fine on most of
the AMOs and LR/SC because they require only one std uop. However AMOCAS
requires at least two std uops, which may be issued from two separate
issue queues at the same time, leading to the loss of std uops.

This commit fixes this by taking all the outputs of the StdExeUnits into
account with arbitration logics.

fix(AtomicsUnit): DCache req can only be sent at `s_cache_req`

fix(AtomicsUnit, difftest): fix difftest io for atomic events

fix(MainPipe): fix precedence of `&` and `=/=` operator

fix(MainPipe): AMOCAS should not wait for AMOALU

fix(MemBlock): remove unnecessary assertion

fix(MainPipe): only CAS instruction can assert `s3_cas_fail`

fix(AtomicsUnit): fix bug in data select logic

submodule(difftest): bump difftest

show more ...

area(DCache ECC): combine ecc with tag/data (#3902)

timing(MainPipe): move replace select from s1 to s2

timing(MainPepe): remove s1_ready for meta_read, tag_read

timing(MainPipe): adjust s1_hit_tag generate logic

timing(MemBlock): optimize MemBlock timing (#3467)

This PR optimizes the timing of MemBlock. Specific optimizations include
but are not limited to:
+ TLB use the redirect for the next cycle
+ Opt

timing(MemBlock): optimize MemBlock timing (#3467)

This PR optimizes the timing of MemBlock. Specific optimizations include
but are not limited to:
+ TLB use the redirect for the next cycle
+ Optimize VLSU feedback and redirect
+ Optimise ldCancel and writeback signal generation
+ Optimise TLB Query Vaddr/hlv/hlvx/valid etc
+ Delay MMIO Store writeback for 1 Cycle
+ Fix tlbNoQuery and pmp logic
+ Remove clock gating for s3_fast_rep
+ Remove wbq conflict check to LoadPipe/MainPipe
+ Remove Mux in dcache resp data
+ Optimise data generation logic of LoadUnit
+ Duplicate Register in LoadUnit for data writeback
+ Duplicate Register in loadPipe for missQueue enq
+ Add skid buffer in VLSU
+ Select data from metaArray at S1
+ Simplify the enqueuing logic of missQueue
+ Separately generate the ready logic of miss Queue
+ Relax the conditions valid for bankdataArray reads
+ Add Reg between Dcache Mainpipe with sms prefetcher
+ Optimise store exceptionBuffer pipeline

---------

Co-authored-by: weiding liu <[email protected]>
Co-authored-by: Charlie Liu <[email protected]>
Co-authored-by: good-circle <[email protected]>

show more ...