VLSU: support of vector store/load exception

VLSU: add support that replay vector load mask

rv64v: implement lsu & lsq vector datapath

rv64v: rewrite VLSU

uop split in V*SplitImp, flow merge in V*MergeBufferImp, uop issued out of order

Merge remote-tracking branch 'upstream/master' into tmp-master

TODO: add gpaddr data path from frontend to backend

RVH: fix the errors after git rebase

RVH: finish the desigh of H extention

Scheduler: implement wakeup from LoadUnit

stIn: connect missing wire

MemBlock: fix exceptionVec

* Renaming exp to vecActive
* TODO: don't let not active element entry the load pipeline

MemBlock,VLSU: fix vector st-ld violation detect

When load issue before vector store (younger than load), load will not enqueue LoadRAWQueue, because the addrReady of vector store which in StoreQueu

MemBlock,VLSU: fix vector st-ld violation detect

When load issue before vector store (younger than load), load will not enqueue LoadRAWQueue, because the addrReady of vector store which in StoreQueue was set vector store's addr is 'Ready' when vector store dispatch. When load issue, the LoadQueue will think that data can forward from stu (think vector store's addr is ready). It will lead to st-ld violation. We add a flag named vecAddrvalid in StoreQueue, when vector store's last element was issue to stu, vecAddrvalid will be set

show more ...

Merge remote-tracking branch 'upstream/master' into backendq

Rebase Timing Fix of Memblock from fix-timing branch (#2501)

* fix LQ timing

* l1pf: fix pf queue to ldu timing

* disable ecc path for timing analysis

* TODO: remove this

* fix pipeline

Rebase Timing Fix of Memblock from fix-timing branch (#2501)

* fix LQ timing

* l1pf: fix pf queue to ldu timing

* disable ecc path for timing analysis

* TODO: remove this

* fix pipeline

* memblock: add a Reg between inner/outer reset_vec

* missqueue: make mem_grant always ready

* Enable ECC path again

* remove fast replay reorder logic

* l1pf: use chosen of arbiter to improve timing

* remove reorder remain logic

* mq: use ParallelORR instead of orR

* Strengthen the conditions for load to load path for timing

* fix load to load data select for timing

* refactoring lq replay valid logic

* fix replay port

* fix load unit s0 arbitor logic

* add topdown wiring

* fix ldu ecc path

* remove lateKill

* ecc: physically remove ecc in DataArray

* loadpipe: use ParallelORR and ParallelMux for timing

* mainpipe: use ParallelMux and ParallelorR for timing

* fix fast replay is killed at s1

* fix replay cancel logic

* fix mq nack feedback logic

* sms: fix pf queue tlb req logic for timing

* kill load at s1

* fix loadqueuereplay enq logic

* opt raw rollback arbiter logic

* fix ecc_delayed writeback logic

* train all l1 pf and sms at load s3 for better timing

* disable load to load forward

* Revert "kill load at s1"

This reverts commit 56d47582ad4dd9c83373fb2db2a0709075485d4d.

* fix s0 kill logic

* ITLBRepeater: Add one more buffer when PTW resp

* remove trigger

* fix feedback_slow logic

* add latch in uncachebuffer rollback

* remove trigger in port

* fast replay: use dcache ready

* fix replay logic at s1

* uncache: fix uncache writeback

* fix delay kill logic

* fix clean exception loigc at s3

* fix ldu rollback logic

* fix ldu rollback valid logic

---------

Co-authored-by: sfencevma <[email protected]>
Co-authored-by: XiChen <[email protected]>
Co-authored-by: Lyn <[email protected]>
Co-authored-by: good-circle <[email protected]>

show more ...

VLSU: clean up fqIdx and flowIdx

Set VsFlowSize to 128 to avoid vector store deadlock

Add VLSU

* miscs: optimize code style

* vector: add VLSU param system and redefine vector lq io

* VLUopQueue: add flow split and address generation logic

* VLUopQueue: add flow issue and writebac

Add VLSU

* miscs: optimize code style

* vector: add VLSU param system and redefine vector lq io

* VLUopQueue: add flow split and address generation logic

* VLUopQueue: add flow issue and writeback logic

* VLUopQueue: set vstart for elements with exception

* VLUopQueue: handle unit-stride fof loads

* VLUopQueue: implement vector masking according to vm

* vector: rewrite vector store io

* VlFlowQueue: add enqueue and dequeue logic

* VLFlowQueue: fix some coding problem

* VlFlowQueue: add issue, replay and result logic

* VLFlowQueue: add redirect logic

* Rob: fix compilation error

* vector: remove stale codes

* vector: add VSUopQueue and fix bugs for vector load

* backbone: add vector load/store execution paths

* VSFlowQueue: Basic function

* VLUopQueue: add redirect logic for load-load violation

* VSFlowQueue: fix some compile problems

* VSUopQueue: add signal to indicate whether a flow is the last one

* VSFlowQueue: inform scala sq when vector store finished

* StoreQueue: maintain sequential retirement between scalar & vector stores

* LoadQueueRAW: handle violation between vector stores & scalar loads

* LDU: add vector store to scalar load forwarding

* XSCore: fix writeback width of MemBlock

* vector: fix load/store whole register and masked unit-stride load/store emul, evl, flownum (#2383)

* VSFlowQueue: Support STLF

* VLFlowQueue: fix compile bug

* VSFlowQueue: fix compile problem

---------

Co-authored-by: xuzefan <[email protected]>
Co-authored-by: good-circle <[email protected]>
Co-authored-by: weidingliu <[email protected]>

show more ...

fix merge error

Merge upstream/master into tmp-backend-merge-master

MemCommon: only use robIdx and sqIdx in MemWaitUpdateReq

Bump rocket-chip (#2353)

chore: remove deprecated brackets, APIs, etc. (#2321)

backend: fix rebase bugs

Scheduler: implement load speculative wakeup and cancel

chore: code cleanup

L1 Prefetch (#2261)

* dcache: optimize the ready signal of missqueue

Add a custom arbiter. In the case of multiple sources with the same
cache block address, the arbiter will assign only one ent

L1 Prefetch (#2261)

* dcache: optimize the ready signal of missqueue

Add a custom arbiter. In the case of multiple sources with the same
cache block address, the arbiter will assign only one entry in
misssqueue but ready for all same cache block address requests.

This will reduce the number of replays of the load instruction which cannot
enter the missqueue

* sta, dcache: add A StorePipe in dcache

When the store command passes through the sta pipeline, access the tag
and meta of dcache to determine whether it hits, if it hits, update the
replacement algorithm, and if miss, send a write intent to missqueue

* sta prefetch: add a queue

Enter this queue when the Store Address pipeline sends a request,
determines that it has a cache miss, and the contention for MSHR fails.

The miss request in this queue will be sent to the Store pipeline later.

* sbuffer, dcache: store prefetch burst

A basic implementation of "Boosting Store Buffer Efficiency with
Store-Prefetch Bursts".

Store prefetch at exe is disabled.
Now, when store goes from sq to sbuffer, it will trigger a store
prefetch; when 48 stores fall into 6 cache lines, trigger a store burst
perfetch, which will bring a whole page back into dcache.

* dcache: restric mshr alloc for prefetch req

* restric the max number of entries which can be used by prefetch
* merge two same cache line address prefetch write req
* dynamically detect memset pattern, all mshr can accept prefetch when
pattern is detected

* spb: constantin support

* dcache: fix missqueue prefetch ready

* make prefetch req goes mshr with bigger id

* Revert "spb: constantin support"

This reverts commit 4ee50b89ba4a62cd28fa22d7fbcb2338ad4b1849.

* spb: fix bug in burst generator

* spb: add load prefetch burst support

* topdown: add defines of topdown counters enum

* redirect: add redirect type for perf

* top-down: add stallReason IOs

frontend -> ctrlBlock -> decode -> rename -> dispatch

* top-down: add dummy connections

* top-down: update TopdownCounters

* top-down: imp backend analysis and counter dump

* top-down: add HartId in `addSource`

* top-down: broadcast lqIdx of ROB head

* top-down: frontend signal done

* top-down: add memblock topdown interface

* Bump HuanCun: add TopDownMonitor

* top-down: receive and handle reasons in dispatch

* top-down: remove previous top-down code

* TopDown: add MemReqSource enum

* TopDown: extend mshr_latency range

* TopDown: add basic Req Source

TODO: distinguish prefetch

* store prefetch: refactor parameters and fix bug

* change some parameters
* fix store pipe bug
* fix load prefetch burst

* dcache: distinguish L1DataPrefetch and CPUData

* top-down: comment out debugging perf counters in ibuffer

* TopDown: add path to pass MemReqSource to HuanCun

* TopDown: use simpler logic to count reqSource and update Probe count

* frontend: update topdown counters

* Update HuanCun Topdown for MemReqSource

* top-down: fix load stalls

* top-down: Change the priority of different stall reasons

* store prefetch: add stride and l2 prefetch

* add a stride prefetcher
* spb and stride will issue prefetch to l2
* when store commits, issue a prefetch to l1

* sbuffer: fix eviction

* when valid count reaches StoreBufferSize, do eviction

* spf: change store prefetch structure

* prefetch @ exe -> l2 cache
* stride -> l2 cache

* sbuffer: fix replaceIdx

* If the way selected by the replacement algorithm cannot be written into dcache, its result is not used.

* Revert "sbuffer: fix replaceIdx"

This reverts commit 40c16aca956af9fb32554a0f12d18db41c22eecd.

* spf: find best interval in stamissqueue

* Revert "spf: find best interval in stamissqueue"

This reverts commit d179f0ce15a5ab989a822de7fe48cc5e2cd96914.

* sms: port store to sms

Miss store will train sms like load.

Now, sms will recieve 4 train sources, 2 for miss load, 2 for miss
store, but prefetcher consume 1 train req per cycle, PrefetchTrainFilter
is added to deal with this case.

* bump huancun

* spf: refactor structure

* miss stores will train sms, and send prefetch to l2
* miss stores will send prefetch to l1 on issue or commit
* spb will send prefetch to l1

* memset: fix memset detection

use lqEmpty to check this

* constantin: storepf constantin support

cherry-pick this to use constantin in storepf

* Revert "constantin: storepf constantin support"

This reverts commit 2b97767b9fa757d920cac3d80d4893a1380592c7.

* storepf: add EnableAtCommitMissTrigger

* trigger prefetch at commit only when the store misses with
EnableAtCommitMissTrigger

* bump coupledl2

* prefetch req from L1 to L2 will Acquire T

* fix merge conflict

* storepf: do not read meta&tag when pf is disabled

* storepf: do not read pcMem when sms store is disabled

* fix verilog check

* fix verilog

* missqueue: support merging prefetch

* prefetch req can be merged to pipeline reg
* merging prefetch write will update cmd
* delay sending out acquire when a prefetch write is about to merge

* missqueue: fix bug of merging prefetch write

* delay sending out acquire when a pipeline reg is about to merging a
prefetch write

* temp: disable store pf

* missqueue: disable merging prefetch

* late prefetch will be ignored
* check alias when merging
* enable store pf at issue

* add L1StreamPrefetcher

* fix assert

* let prefetch req prefer loadunit1 more than 0

* stream prefetcher

* disable stream component in SMS, SMS is only trained on real miss
* add a prefetcher monitor to adjust depth & confidence ..
* add L1 L2 stream prefetch

* add gene support

* Revert "add gene support"

This reverts commit 59ae15640ff3d1cc96347f4d3567d48c740a03bb.

* add miss db

* l1pf: add stride & store source info in cache meta

* add a Stride prefetcher and disable Stride component in sms
* prefetch bit in meta is expanded into 3 bits to store source info of
prefetcher

* prefetch: support sending prefetch req to l3

* l1pf: add FDP & refactor

* add basic FDP counters
* change stride from Block addr to Byte addr
* refactor the code

* bump submodules

* disable load related chiseldb to reduce db size

* fix compile

* fix minimalConfig & enable stream

* fix stride pc problem

* fix minimalconfig compile

* bump submodules

* refactor stream stride helper

* fix compile

* bump huancun

* disable db to save size

* fix l2 assert

* bump submodules

---------

Co-authored-by: tastynoob <[email protected]>
Co-authored-by: Haojin Tang <[email protected]>
Co-authored-by: Guokai Chen <[email protected]>
Co-authored-by: XiChen <[email protected]>
Co-authored-by: Zhou Yaoyang <[email protected]>

show more ...