Commit Graph

902 Commits (0d86580ac701f3ff614ac44ca3ded1c86b304a2e)
 

Author SHA1 Message Date
Anton Blanchard 0d86580ac7 Reformat writeback
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard f67b143165 Reformat plru
Also fix a typo

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard c76e638a77 Reformat rotator
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard 601f3211be Reformat divider
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard bf96279ff1 Reformat countzero
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard af7e330d69 Reformat cr_file
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard 9208276aa2 Reformat register_file
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Anton Blanchard 74254bf11a Reformat cache_ram
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years ago
Michael Neuling 6523acc743
Merge pull request #274 from mikey/read-sprs
Fix reading DSISR/DAR before writing and add a test to read from all SPRs
3 years ago
Anton Blanchard d26a157cd7 Add a test to read from all SPRs
Make sure the SPRs are initialized and we can't read X state.

(Mikey: rebased and added console/bin file for testing)

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
Signed-off-by: Michael Neuling <mikey@neuling.org>
4 years ago
Michael Neuling 4c21587c4d Fix DAR/DSISR reading before they are written
If the DAR and DSISR are read before they are written, we assert with:

  register_file.vhdl:55:25:@60195ns:(report note): Writing GPR 09 00000000XXXXXXXX
  register_file.vhdl:61:17:@60195ns:(assertion failure): Assertion violation

This initialises DAR/DSISR to avoid this.

Signed-off-by: Michael Neuling <mikey@neuling.org>
4 years ago
Michael Neuling 6c7689052d
Merge pull request #269 from paulusmack/pipeline
Rework load/store pipeline to achieve one load/store per cycle throughput
4 years ago
Michael Neuling 9a6a7e9fe5
Merge pull request #268 from paulusmack/btc
Implement branch target cache
4 years ago
Michael Neuling 7652452367
Merge pull request #273 from antonblanchard/wishbone-checking
Add some wishbone checking
4 years ago
Michael Neuling c4e3ade4ed
Merge pull request #267 from paulusmack/master
Improve architecture compliance and other miscellaneous changes
4 years ago
Anton Blanchard 481f3cdfea Add some wishbone checking
Check that stb, cyc and ack are never undefined. While not really needed
here, this also tests if --pragma synthesis_off/--pragma synthesis_on
works on all the tools we use.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
4 years ago
Paul Mackerras 17fd069640 core: Allow multiple loadstore instructions to be in flight
The idea here is that we can have multiple instructions in progress at
the same time as long as they all go to the same unit, because that
unit will keep them in order.  If we get an instruction for a
different unit, we wait for all the previous instructions to finish
before executing it.  Since the loadstore unit is the only one that is
currently pipelined, this boils down to saying that loadstore
instructions can go ahead while l_in.in_progress = 1 but other
instructions have to wait until it is 0.

This gives a 2% increase on coremark performance on the Arty A7-100
(from ~190 to ~194).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras f583d088b7 loadstore: Convert to 3-stage pipeline
This makes loadstore use a 3-stage pipeline.  For now, only one
instruction goes through the pipe at a time.  Completion and writeback
are still combinatorial off the valid signal back from the dcache, so
performance should be the same as before.  In future it should be able
to sustain one load or store per cycle provided they hit in the
dcache.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras f636bb7c39 dcache: Fix bugs in pipelined operation
This fixes two bugs which show up when multiple operations are in
flight in the dcache, and adds a 'hold' input which will be needed
when loadstore1 is pipelined.

The first bug is that dcache needs to sample the data for a store on
the cycle after the store request comes in even if the store request
is held up because of a previous request (e.g. if the previous request
is a load miss or a dcbz).

The second bug is that a load request coming in for a cache line being
refilled needs to be handled immediately in the case where it is for
the row whose data arrives on the same cycle.  If it is not, then it
will be handled as a separate cache miss and the cache line will be
refilled again into a different way, leading to two ways both being
valid for the same tag.  This can lead to data corruption, in the
scenario where subsequent writes go to one of the ways and then that
way gets displaced but the other way doesn't.  This bug could in
principle show up even without having multiple operations in flight in
the dcache.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras acb3d2d745 core: Send FPU interrupts to writeback rather than execute1
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 29221315e9 core: Send loadstore1 interrupts to writeback rather than execute1
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 3cd3449b4b core: Move redirect and interrupt delivery logic to writeback
This moves the logic for redirecting fetching and writing SRR0 and
SRR1 to writeback.  The aim is that ultimately units other than
execute1 can send their interrupts to writeback along with their
instruction completions, so that there can be multiple instructions
in flight without needing execute1 to keep track of the address
of each outstanding instruction.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4fd8d9509c execute1: Move CR result to data path process
Also work out in decode2 whether the instruction sets the XER common
bits.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras d6ac43251a execute1: Move data-path logic out to a separate process
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras ae2afeca5c core: Track CR hazards and bypasses using tags
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras d290d2a9bb core: Restore bypass path from execute1
This changes the bypass path.  Previously it went from after
execute1's output to after decode2's output.  Now it goes from before
execute1's output register to before decode2's output register.  The
reason is that the new path will be simpler to manage when there are
possibly multiple instructions in flight.  This means that the
bypassing can be managed inside decode2 and control.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras c0b45e153b core: Track GPR hazards using tags that propagate through the pipelines
This changes the way GPR hazards are detected and tracked.  Instead of
having a model of the pipeline in gpr_hazard.vhdl, which has to mirror
the behaviour of the real pipeline exactly, we now assign a 2-bit tag
to each instruction and record which GSPR the instruction writes.
Subsequent instructions that need to use the GSPR get the tag number
and stall until the value with that tag is being written back to the
register file.

For now, the forwarding paths are disabled.  That gives about a 8%
reduction in coremark performance.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras a1d7b54f76 core: Crack branches that update both CTR and LR
This uses the instruction doubling machinery to convert conditional
branch instructions that update both CTR and LR (e.g., bdnzl, bdnzlrl)
into two instructions, of which the first updates CTR and determines
whether the branch is taken, and the second updates LR and does the
redirect if necessary.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4c61a71a62 core: Crack update-form loads into two internal ops
This uses the instruction-doubling machinery to send load with update
instructions down to loadstore1 as two separate ops, rather than
one op with two destinations.  This will help to simplify the value
tracking mechanisms.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 0fb207be60 fetch1: Implement a simple branch target cache
This implements a cache in fetch1, where each entry stores the address
of a simple branch instruction (b or bc) and the target of the branch.
When fetching sequentially, if the address being fetched matches the
cache entry, then fetching will be redirected to the branch target.
The cache has 1024 entries and is direct-mapped, i.e. indexed by bits
11..2 of the NIA.

The bus from execute1 now carries information about taken and
not-taken simple branches, which fetch1 uses to update the cache.
The cache entry is updated for both taken and not-taken branches, with
the valid bit being set if the branch was taken and cleared if the
branch was not taken.

If fetching is redirected to the branch target then that goes down the
pipe as a predicted-taken branch, and decode1 does not do any static
branch prediction.  If fetching is not redirected, then the next
instruction goes down the pipe as normal and decode1 does its static
branch prediction.

In order to make timing, the lookup of the cache is pipelined, so on
each cycle the cache entry for the current NIA + 8 is read.  This
means that after a redirect (from decode1 or execute1), only the third
and subsequent sequentially-fetched instructions will be able to be
predicted.

This improves the coremark value on the Arty A7-100 from about 180 to
about 190 (more than 5%).

The BTC is optional.  Builds for the Artix 7 35-T part have it off by
default because the extra ~1420 LUTs it takes mean that the design
doesn't fit on the Arty A7-35 board.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras f7b855dfc3 execute1: Improve timing on comparisons
Using the main adder for comparisons has the disadvantage of creating
a long path from the CA/OV bit forwarding to v.busy via the carry
input of the adder, the comparison result, and determining whether a
trap instruction would trap.  Instead we now have dedicated
comparators for the high and low words of a_in vs. b_in, and combine
their results to get the signed and unsigned comparison results.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras b0510fd1bb core: Reorganize execute1
This breaks up the enormous if .. elsif .. case .. elsif statement in
execute1 in order to try to make it simpler and more understandable.
We now have decode2 deciding whether the instruction has a value to be
written back to a register (GPR, GSPR, FPR, etc.) rather than
individual cases in execute1 setting result_en.  The computation of
the data to be written back is now independent of detection of various
exception conditions.  We now have an if block determining if any
exception condition exists which prevents the next instruction from
being executed, then the case statement which performs actions such as
setting carry/overflow bits, determining if a trap exception exists,
doing branches, etc., then an if statement for all the r.busy = 1
cases (continuing execution of an instruction which was started in a
previous cycle, or writing SRR1 for an interrupt).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras c0f282b691 decode1: Implement tlbsync as a no-op
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 658feabfd4 core: Make result multiplexing explicit
This adds an explicit multiplexer feeding v.e.write_data in execute1,
with the select lines determined in the previous cycle based on the
insn_type.  Similarly, for multiply and divide instructions, there is
now an explicit multiplexer.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras d6134babc0 decode1: Implement obsolete dst, dstst, dss instructions as no-ops
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 9ea1ab0215 execute1: Move branch adder after register
This does the addition of the instruction NIA and the branch offset
after the register at the output of execute1 rather than before.
The propagation through the adder was showing up as a critical path
on the A7-100.  Performance is unaffected and now it makes timing.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 89a67a18d0 decode: Add a facility field to the instruction decode tables
This makes it simpler to work out when to deliver a FPU unavailable
interrupt.  This also means we can get rid of the OP_FPLOAD and
OP_FPSTORE insn_type values.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras cb1e3f6d70 decode1: Take an extra cycle for predicted branch redirects
This does the addition of NIA plus the branch offset from the
instruction after a clock edge, in order to ease timing, as the path
from the icache RAM through the adder in decode1 to the NIA register
in fetch1 was showing up as a critical path.

This adds one extra cycle of latency when redirecting fetch because of
a predicted-taken branch.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras ec5730a75a tests: Add tests for lq/stq and lqarx/stqcx.
Lq and stq are tested in both BE and LE modes (though only 64-bit
mode) by the 'modes' test.

Lqarx and stqcx. are tested by the 'reservation' test in LE mode mode
(64-bit).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 6427cab46f loadstore1/dcache: Send store data one cycle later
This makes timing easier and also means that store floating-point
single precision instructions no longer need to take an extra cycle.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4b2c23703c core: Implement quadword loads and stores
This implements the lq, stq, lqarx and stqcx. instructions.

These instructions all access two consecutive GPRs; for example the
"lq %r6,0(%r3)" instruction will load the doubleword at the address
in R3 into R7 and the doubleword at address R3 + 8 into R6.  To cope
with having two GPR sources or destinations, the instruction gets
repeated at the decode2 stage, that is, for each lq/stq/lqarx/stqcx.
coming in from decode1, two instructions get sent out to execute1.

For these instructions, the RS or RT register gets modified on one
of the iterations by setting the LSB of the register number.  In LE
mode, the first iteration uses RS|1 or RT|1 and the second iteration
uses RS or RT.  In BE mode, this is done the other way around.  In
order for decode2 to know what endianness is currently in use, we
pass the big_endian flag down from icache through decode1 to decode2.
This is always in sync with what execute1 is using because only rfid
or an interrupt can change MSR[LE], and those operations all cause
a flush and redirect.

There is now an extra column in the decode tables in decode1 to
indicate whether the instruction needs to be repeated.  Decode1 also
enforces the rule that lq with RT = RT and lqarx with RA = RT or
RB = RT are illegal.

Decode2 now passes a 'repeat' flag and a 'second' flag to execute1,
and execute1 passes them on to loadstore1.  The 'repeat' flag is set
for both iterations of a repeated instruction, and 'second' is set
on the second iteration.  Execute1 does not take asynchronous or
trace interrupts on the second iteration of a repeated instruction.

Loadstore1 uses 'next_addr' for the second iteration of a repeated
load/store so that we access the second doubleword of the memory
operand.  Thus loadstore1 accesses the doublewords in increasing
memory order.  For 16-byte loads this means that the first iteration
writes GPR RT|1.  It is possible that RA = RT|1 (this is a legal
but non-preferred form), meaning that if the memory operand was
misaligned, the first iteration would overwrite RA but then the
second iteration might take a page fault, leading to corrupted state.
To avoid that possibility, 16-byte loads in LE mode take an
alignment interrupt if the operand is not 16-byte aligned.  (This
is the case anyway for lqarx, and we enforce it for lq as well.)

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras d1f35705c0 loadstore1: Improve timing of data path from cache RAM to writeback
Work out select inputs for writeback mux a cycle earlier.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 784d409999 dcache: Add more commentary, no code change
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 54f89afab7 loadstore1: Decide on load formatting controls a cycle earlier
This helps timing.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 55f7d99376 decode1: Fix decoding of recommended NOP instruction
We were decoding nop with the wrong major opcode.  Fix it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 3361c460b8 core_debug: Stop logging 256 cycles after trigger
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 470f1b2140 core_debug: Add an address trigger to stop logging at a given address
This compares the address being fetched with the contents of a
register that can be set via DMI, and if they match, stops the
logging.  Since this works on the address being fetched rather than
executed, it is subject to false positives.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 5535257c71 FPU: Don't use mask generator for rounding
Instead of using the mask generator in the rounding process, this uses
simpler logic to add in a 1 at the appropriate position (bit 2 or bit
31, depending on precision) and mask off the low-order bits.  Since
there are only two positions at which the masking and incrementing
need to be done, we don't need the full generality of the mask
generator.  This reduces the amount of logic and improves timing.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 45c5236700 FPU: Relax timing around multiplier output
At present there is a state transition in the handling of the fmadd
instructions where the next state depends on the sign bit of the
multiplier result.  This creates a critical path which doesn't make
timing on the A7-100.  To fix this, we make the state transition
independent of the sign of the multiplier result, which improves
timing, but means we take one more cycle to do a fmadd-family
instruction in some cases.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras f14e731ec6 mw_debug: Display terminated status when stopping
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago