IFU: mmio wait until last instruction retiring

* add 1 stage for mmio_state before sending request to MMIO bus
* check whether the last fetch packet commit all its intructions (the
result of executi

IFU: mmio wait until last instruction retiring

* add 1 stage for mmio_state before sending request to MMIO bus
* check whether the last fetch packet commit all its intructions (the
result of execution path has been decided)
* avoid speculative execution to MMIO bus

show more ...

<bug-fix> IFU: fix f3_lastHalf_disable flush bug

* when f3_flush is enabled, f3_lastHalf_disable is still set and
influence the next packet

<bug-fix> IFU: update pd for mmio instuction

<bug-fix> IFU: fix f3_lastHalf cancel condition

* Under the circumstance that 2 continuous ftq reqs both have last half
RVI, but the f3_lastHalf.valid cancel condition in wb-stage is set by
!f3_last

<bug-fix> IFU: fix f3_lastHalf cancel condition

* Under the circumstance that 2 continuous ftq reqs both have last half
RVI, but the f3_lastHalf.valid cancel condition in wb-stage is set by
!f3_lastHalf.valid， which makes the miss pred f3_lastHalf req has not
been flushed.

show more ...

<bug-fix> cancel f3_lastHalf_valid (#1737)

<bug-fix> IFU: fix last half register bug

if a req with a last half but miss predicted enters in wb stage, and this cycle f3 stalls, we set a flag to notify f3 that the last half flag need not to be

<bug-fix> IFU: fix last half register bug

if a req with a last half but miss predicted enters in wb stage, and this cycle f3 stalls, we set a flag to notify f3 that the last half flag need not to be set.

show more ...

<timing-opt> IFU: move expander from f2 to f3

Nanhu frontend debug (#1696)

* <bug-fix> IFU: cancel lastHalf for miss prediction

* <bug-fix> ICacheMainPipe: latch tlb resp for stall

* <bug-fix> only tlb_slot.valid can raise has_latch

Optimize ICache s2_hit_reg and Ftq timing

* copy Ftq to ICache read valid signal

* move sram read data and miss data selection to IFU (after predecode)

IFU: add ICache ready

IFU/IPrefetch/ReplacePipe: adjust meta/data access

* IFU: ignore ICache access bundle

* ICacheMainPipe: expand meta/data access output to 4 identical vector
output, each output is connected to a co

IFU/IPrefetch/ReplacePipe: adjust meta/data access

* IFU: ignore ICache access bundle

* ICacheMainPipe: expand meta/data access output to 4 identical vector
output, each output is connected to a copied register trigger by FTQ
requests

* IPrefetch/ReplacePipe: expand meta/data access outpu to 4 indentical
vector output, and each output is triggered by the same signal group

show more ...

[WIP]FTQ: add icache req port

* separate ifu req and icache req for timing optimization

* both ifu ftq_req_ready and icache ftq_req_ready depend on each other

* ifu and icache has pc_mem register

[WIP]FTQ: add icache req port

* separate ifu req and icache req for timing optimization

* both ifu ftq_req_ready and icache ftq_req_ready depend on each other

* ifu and icache has pc_mem register

[WIP]ICacheMainPipe: add copied registers

[WIP]ftq: read ftq_pc_mem one cycle ahead, reqs to be copied

[WIP] FTQ: delete outside bypass

show more ...

<timing>: move targetFault to wb stage

<timng>: optimize IFU f2 cut function

IFU <bug-fix>: deal with itlb miss for resend (#1488)

* IFU <bug-fix>: deal with itlb miss for resend

* IFU <bug fix>: enable crossPageFault for resend-pf

Co-authored-by: DeltaZero <lacrosseelis@g

IFU <bug-fix>: deal with itlb miss for resend (#1488)

* IFU <bug-fix>: deal with itlb miss for resend

* IFU <bug fix>: enable crossPageFault for resend-pf

Co-authored-by: DeltaZero <[email protected]>

show more ...

Sync timing modification of #1681 and #1793 (#1793)

* ldu: optimize dcache hitvec wiring

In previous design, hitvec is generated in load s1, then send to dcache
and lsu (rs) side separately. As

Sync timing modification of #1681 and #1793 (#1793)

* ldu: optimize dcache hitvec wiring

In previous design, hitvec is generated in load s1, then send to dcache
and lsu (rs) side separately. As dcache and lsu (rs side) is far in real
chip, it caused severe wiring problem.

Now we generate 2 hitvec in parallel:

* hitvec 1 is generated near dcache.
To generate that signal, paddr from dtlb is sent to dcache in load_s1
to geerate hitvec. The hitvec is then sent to dcache to generate
data array read_way_en.

* hitvec 2 is generated near lsu and rs in load_s2, tag read result
from dcache, as well as coh_state, is sent to lsu in load_s1,
then it is used to calcuate hitvec in load_s2. hitvec 2 is used
to generate hit/miss signal used by lsu.

It should fix the wiring problem caused by hitvec

* ldu: opt loadViolationQuery.resp.ready timing

An extra release addr register is added near lsu to speed up the
generation of loadViolationQuery.resp.ready

* l1tlb: replace NormalPage data module and add duplicate resp result

data module:
add BankedSyncDataMoudleWithDup data module:
divided the data array into banks and read as Async, bypass write data.
RegNext the data result * #banks. choose from the chosen data.

duplicate:
duplicate the chosen data and return to outside(tlb).
tlb return (ppn+perm) * #DUP to outside (for load unit only)

TODO: load unit use different tlb resp result to different module.
one for lsq, one for dcache.

* l1tlb: Fix wrong vidx_bypass logic after using duplicate data module

We use BankedSyncDataMoudleWithDup instead of SyncDataModuleTemplate,
whose write ports are not Vec.

Co-authored-by: William Wang <[email protected]>
Co-authored-by: ZhangZifei <[email protected]>
Co-authored-by: good-circle <[email protected]>

show more ...

frontend: Add ChiselDB records

IFU: Add toIBuffer and toFtq record
Ftq: Add branch trace datebase framework

rs,mem: optimize load-load forwarding timing (#1742)

This commit optimizes the timing of load-load forwarding by making
it speculatively issue requests to TLB/dcache.

When load_s0 does not have

rs,mem: optimize load-load forwarding timing (#1742)

This commit optimizes the timing of load-load forwarding by making
it speculatively issue requests to TLB/dcache.

When load_s0 does not have a valid instruction and load_s3 writes
a valid instruction back, we speculatively bypass the writeback
data to load_s0 and assume there will be a pointer chasing instruction
following it. A pointer chasing instruction has a base address that
comes from a previous instruction with a small offset. To avoid timing
issues, now only when the offset does not change the cache set index,
we reduce its latency by speculatively issuing it.

show more ...

l1tlb: tlb's req port can be configured to be block or non-blocked (#1656)

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tl

l1tlb: tlb's req port can be configured to be block or non-blocked (#1656)

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tlb, but belong to
core pipeline, which means only core pipeline flush will invalid them.

For another, itlb also use PTW Filter but with only 4 entries.
Last, keep svinval extension as usual, still work.


* tlb: add blocked-tlb support, miss frontend changes

* tlb: remove tlb's sameCycle support, result will return at next cycle

* tlb: remove param ShouldBlock, move block method into TLB module

* tlb: fix handle_block's miss_req logic

* mmu.filter: change filter's req.ready to canEnqueue

when filter can't let all the req enqueue, set the req.ready to false.
canEnqueue after filtering has long latency, so we use **_fake
without filtering, but the filter will still receive the reqs if
it can(after filtering).

* mmu.tlb: change name from BTlbPtwIO to VectorTlbPtwIO

* mmu: replace itlb's repeater to filter&repeaternb

* mmu.tlb: add TlbStorageWrapper to make TLB cleaner

more: BlockTlbRequestorIO is same with TlbRequestorIO, rm it

* mmu.tlb: rm unused param in function r_req_apply, fix syntax bug

* [WIP]icache: itlb usage from non-blocked to blocked

* mmu.tlb: change parameter NBWidth to Seq of boolean

* icache.mainpipe: fix itlb's resp.ready, not always true

* mmu.tlb: add kill sigal to blocked req that needs sync but fail

in frontend, icache,itlb,next pipe may not able to sync.
blocked tlb will store miss req ang blocks req, which makes itlb
couldn't work. So add kill logic to let itlb not to store reqs.

One more thing: fix icache's blocked tlb handling logic

* icache.mainpipe: fix tlb's ready_recv logic

icache mainpipe has two ports, but these two ports may not valid
all the same time. So add new signals tlb_need_recv to record whether
stage s1 should wait for the tlb.

* tlb: when flush, just set resp.valid and pf, pf for don't use it

* tlb: flush should concern satp.changed(for blocked io now)

* mmu.tlb: add new flush that doesn't flush reqs

Sfence.vma will flush inflight reqs and flushPipe
But some other sfence(svinval...) will not. So add new flush to
distinguish these two kinds of sfence signal

morw: forget to assign resp result when ptw back, fix it

* mmu.tlb: beautify miss_req_v and miss_v relative logic

* mmu.tlb: fix bug, when ptw back and bypass, concern level to genPPN

bug: when ptw back and bypass, forgot to concern level(1GB/2MB/4KB)
when genPPN.

by the way: some funtions need ": Unit = ", add it.

* mmu.filter: fix bug of canEnqueue, mixed with tlb_req and tlb.req

* icache.mainpipe: fix bug of tlbExcp's usage, & with tlb_need_back

Icache's mainpipe has two ports, but may only port 0 is valid.
When a port is invalid, the tlbexcp should be false.(Actually, should
be ignored).
So & tlb_need_back to fix this bug.

* sfence: instr in svinval ext will also flush pipe

A difficult problem to handle:
Sfence and Svinval will flush MMU, but only Sfence(some svinval)
will flush pipe. For itlb that some requestors are blocked and
icache doesn't recv flush for simplicity, itlb's blocked ptw req
should not be flushed.
It's a huge problem for MMU to handle for good or bad solutions. But
svinval is seldom used, so disable it's effiency.

* mmu: add parameter to control mmu's sfence delay latency

Difficult problem:
itlb's blocked req should not be abandoned, but sfence will flush
all infight reqs. when itlb and itlb repeater's delay is not same(itlb
is flushed, two cycles later, itlb repeater is flushed, then itlb's
ptw req after flushing will be also flushed sliently.
So add one parameter to control the flush delay to be the same.

* mmu.tlb: fix bug of csr.priv's delay & sfence valid when req fire

1. csr.priv's delay
csr.priv should not be delayed, csr.satp should be delayed.
for excep/intr will change csr.priv, which will be changed at one
instruction's (commit?). but csrrw satp will not, so satp has more
cycles to delay.
2. sfence
when sfence valid but blocked req fire, resp should still fire.
3. satp in TlbCsrBundle
let high bits of satp.ppn to be 0.U

* tlb&icache.mainpipe: rm commented codes

* mmu: move method genPPN to entry bundle

* l1tlb: divide l1tlb flush into flush_mmu and flush_pipe

Problem:
For l1tlb, there are blocked and non-blocked req ports.
For blocked ports, there are req slots to store missed reqs.
Some mmu flush like Sfence should not flush miss slots for outside
may still need get tlb resp, no matter wrong and correct resp.
For example. sfence will flush mmu and flush pipe, but won't flush
reqs inside icache, which waiting for tlb resp.
For example, svinval instr will flush mmu, but not flush pipe. so
tlb should return correct resp, althrough the ptw req is flushed
when tlb miss.

Solution:
divide l1tlb flush into flush_mmu and flush_pipe.
The req slot is considered to be a part of core pipeline and should
only be flushed by flush_pipe.
flush_mmu will flush mmu entries and inflight ptw reqs.
When miss but sfence flushed its ptw req, re-send.

* l1tlb: code clean, correct comments and rm unused codes

* l2tlb: divide filterSize into ifiterSize and dfilterSize

* l2tlb: prefetch req won't enter miss queue. Rename MSHR to missqueue

* l1tlb: when disable vm, ptw back should not bypass tlb and should let miss req go ahead

show more ...

fix bugs in IFU and delete 500-cycle ready

* fix mmio_resend_af wrong assignment
* fix wb_half_flush missOffset(using wb_lastIdx instead of PredictWidth
-1)
* change pipeline ready condition (this_r

fix bugs in IFU and delete 500-cycle ready

* fix mmio_resend_af wrong assignment
* fix wb_half_flush missOffset(using wb_lastIdx instead of PredictWidth
-1)
* change pipeline ready condition (this_ready = this_stage_fire || this_stage_empty)
* delete 500-cycle ready condition (toICache(*).ready means the SRAM has
been reset and ready for read)

show more ...

<bug-fix>: fix f3 mmio write back override bug (#1567)

fix for chipsalliance/chisel3#2496 (#1563)

IFU <bug-fix>: deal with itlb miss for resend (#1488)

* IFU <bug-fix>: deal with itlb miss for resend

* IFU <bug fix>: enable crossPageFault for resend-pf

Co-authored-by: DeltaZero <lacrosseel

IFU <bug-fix>: deal with itlb miss for resend (#1488)

* IFU <bug-fix>: deal with itlb miss for resend

* IFU <bug fix>: enable crossPageFault for resend-pf

Co-authored-by: DeltaZero <[email protected]>

show more ...

ICache : fix 2 potential rule violations according to TL specification (#1444)

* ReplacePipe: block miss until get ReleaseAck

* IPrefetch: cancle prefetch req when meet MSHR

* Fetch <perf>: ad

ICache : fix 2 potential rule violations according to TL specification (#1444)

* ReplacePipe: block miss until get ReleaseAck

* IPrefetch: cancle prefetch req when meet MSHR

* Fetch <perf>: add fetch bubble performance counters

show more ...

IFU <info>: add debug info for predecode redirect