Commit Graph

144 Commits (047be5c0c3b2f12c9321412518e17b7267fe14ea)

Author SHA1 Message Date
Paul Mackerras 047be5c0c3 loadstore1: Do SPR reading in stage 2 rather than stage 3
This eliminates one leg of the output value multiplexer, and seems
to improve timing slightly on the A7-100.

Since SPR values are written in stage 3 and read in stage 2, an mfspr
immediately following an mtspr to the same SPR won't give the correct
value.  To avoid this, we make mtspr to the load/store SPRs single
issue in decode1.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras fdb3ef6874 Finish off taking SPRs out of register file
With this, the register file now contains 64 entries, for 32 GPRs and
32 FPRs, rather than the 128 it had previously.  Several things get
simplified - decode1 no longer has to work out the ispr{1,2,o} values,
decode_input_reg_{a,b,c} no longer have the t = SPR case, etc.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras 337b104250 Move LR, CTR and TAR out of the register file
By putting CTR on the odd side and LR and TAR on the even side, we can
read and write CTR for bdnz-style instructions in parallel with
reading LR or TAR for indirect branches and writing LR for branches
with LK=1.  Thus we don't need to double up any of these instructions,
giving a simplification in decode2.

We now have logic for printing LR and CTR at the end of a simulation
in execute1, in addition to the similar logic in register_file and
cr_file.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras bc4d02cb0d Start removing SPRs from register file
This starts the process of removing SPRs from the register file by
moving SRR0/1, SPRG0-3, HSRR0/1 and HSPRG0/1 out of the register file
and putting them into execute1.  They are stored in a pair of small
RAM arrays, referred to as "even" and "odd".  The reason for having
two arrays is so that two values can be read and written in each
cycle.  For example, SRR0 and SRR1 can be written in parallel by an
interrupt and read in parallel by the rfid instruction.

The addresses in the RAM which will be accessed are determined in the
decode2 stage.  We have one write address for both sides, but two read
addresses, since in future we will want to be able to read CTR at the
same time as either LR or TAR.

We now have a connection from writeback to execute1 which carries the
partial SRR1 value for an interrupt.  SRR0 comes from the execute
pipeline; we no longer need to carry instruction addresses along the
LSU and FPU pipelines.  Since SRR0 and SRR1 can be written in the same
cycle now, we don't need the little state machine in writeback any
more.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras 73cc5167ec Use FPU for division instructions if we have an FPU
- Arrange for XER to be written for OE=1 forms
- Arrange for condition codes to be set for RC=1 forms
  (including correct handling for 32-bit mode)
- Don't instantiate the divider if we have an FPU.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras 2da08bcf2e decode1: Remove stash buffer
Now that the timing of the busy signal from decode2 doesn't depend on
register numbers or downstream instruction completion, we no longer
need the stash buffer on the output of decode1.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras c9e838b656 Remove support for lq, stq, lqarx and stqcx.
They are optional in SFFS (scalar fixed-point and floating-point
subset), are not needed for running Linux, and add complexity, so
remove them.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras ebe1caab85 decode1: Reduce number of single-issue instructions
This reduces the set of instructions marked as single-issue to just
attn and mtspr to "slow" SPRs (those that are not stored in the
register file).

The instructions that were previously single-issue are: isync, dcbf,
dcbst, dcbt, dcbtst, eieio, icbi, mfmsr, mtmsr, mtmsrd, mfspr to slow
SPRS, sync, tlbsync and wait.  The synchronization instructions are
mostly no-ops anyway due to the in-order nature of the core, and the
cache-management instructions are unimplemented (except for icbi).
The MSR ops don't need to be single-issue due to the in-order core and
the fact that MSR updates are effective on the following instruction.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras 813e2317bf execute1: Restructure to separate out execution of side effects
We now have a record that represents the actions taken in executing an
instruction, and a process that computes that for the incoming
instruction.  We no longer have 'current' or 'r.cur_instr', instead
things like the destination register are put into r.e in the first
cycle of an instruction and not reinitialized in subsequent busy
cycles.

For mfspr and mtspr, we now decode "slow" SPR numbers (those SPRs that
are not stored in the register file) to a new "spr_selector" record
in decode1 (excluding those in the loadstore unit).  With this, the
result for mfspr is determined in the data path.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years ago
Paul Mackerras 83dea94793 decode1: Conditional trap instructions don't need to be single-issue
They can generate interrupts, but that doesn't mean they have to
single-issue.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Michael Neuling 2224b28c2c
Merge pull request #324 from paulusmack/master
Performance and timing improvements
3 years ago
Paul Mackerras 54b0e8b8c8 core: Predict not-taken conditional branches using BTC
This adds a bit to the BTC to store whether the corresponding branch
instruction was taken last time it was encountered.  That lets us pass
a not-taken prediction down to decode1, which for backwards direct
branches inhibits it from redirecting fetch to the target of the
branch.  This increases coremark by about 2%.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Paul Mackerras d4cfdb1bfe decode1: Fix form of isel marked as single-issue
The row in the decode table for isel with BC=0 was inadvertently left
marked as single-issue by commit 813f834012 ("Add CR hazard
detection", 2019-10-15).  Fix it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Paul Mackerras 06e07c69a8 decode1: Fix maddld and maddhdu to not set CR0
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Paul Mackerras a68921edca core: Fix mcrxrx, addpcis and bpermd
- mcrxrx put the bits in the wrong order

- addpcis was setting CR0 if the instruction bit 0 = 1, which it
  shouldn't

- bpermd was producing 0 always and additionally had the wrong bit
  numbering

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Paul Mackerras 18120f153d MMU: Implement a vestigial partition table
This implements a 1-entry partition table, so that instead of getting
the process table base address from the PRTBL SPR, the MMU now reads
the doubleword pointed to by the PTCR register plus 8 to get the
process table base address.  The partition table entry is cached.

Having the PTCR and the vestigial partition table reduces the amount
of software change required in Linux for Microwatt support.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years ago
Paul Mackerras ae2afeca5c core: Track CR hazards and bypasses using tags
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras a1d7b54f76 core: Crack branches that update both CTR and LR
This uses the instruction doubling machinery to convert conditional
branch instructions that update both CTR and LR (e.g., bdnzl, bdnzlrl)
into two instructions, of which the first updates CTR and determines
whether the branch is taken, and the second updates LR and does the
redirect if necessary.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4c61a71a62 core: Crack update-form loads into two internal ops
This uses the instruction-doubling machinery to send load with update
instructions down to loadstore1 as two separate ops, rather than
one op with two destinations.  This will help to simplify the value
tracking mechanisms.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 0fb207be60 fetch1: Implement a simple branch target cache
This implements a cache in fetch1, where each entry stores the address
of a simple branch instruction (b or bc) and the target of the branch.
When fetching sequentially, if the address being fetched matches the
cache entry, then fetching will be redirected to the branch target.
The cache has 1024 entries and is direct-mapped, i.e. indexed by bits
11..2 of the NIA.

The bus from execute1 now carries information about taken and
not-taken simple branches, which fetch1 uses to update the cache.
The cache entry is updated for both taken and not-taken branches, with
the valid bit being set if the branch was taken and cleared if the
branch was not taken.

If fetching is redirected to the branch target then that goes down the
pipe as a predicted-taken branch, and decode1 does not do any static
branch prediction.  If fetching is not redirected, then the next
instruction goes down the pipe as normal and decode1 does its static
branch prediction.

In order to make timing, the lookup of the cache is pipelined, so on
each cycle the cache entry for the current NIA + 8 is read.  This
means that after a redirect (from decode1 or execute1), only the third
and subsequent sequentially-fetched instructions will be able to be
predicted.

This improves the coremark value on the Arty A7-100 from about 180 to
about 190 (more than 5%).

The BTC is optional.  Builds for the Artix 7 35-T part have it off by
default because the extra ~1420 LUTs it takes mean that the design
doesn't fit on the Arty A7-35 board.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras cb1e3f6d70 decode1: Take an extra cycle for predicted branch redirects
This does the addition of NIA plus the branch offset from the
instruction after a clock edge, in order to ease timing, as the path
from the icache RAM through the adder in decode1 to the NIA register
in fetch1 was showing up as a critical path.

This adds one extra cycle of latency when redirecting fetch because of
a predicted-taken branch.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras c0f282b691 decode1: Implement tlbsync as a no-op
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras d6134babc0 decode1: Implement obsolete dst, dstst, dss instructions as no-ops
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 89a67a18d0 decode: Add a facility field to the instruction decode tables
This makes it simpler to work out when to deliver a FPU unavailable
interrupt.  This also means we can get rid of the OP_FPLOAD and
OP_FPSTORE insn_type values.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4b2c23703c core: Implement quadword loads and stores
This implements the lq, stq, lqarx and stqcx. instructions.

These instructions all access two consecutive GPRs; for example the
"lq %r6,0(%r3)" instruction will load the doubleword at the address
in R3 into R7 and the doubleword at address R3 + 8 into R6.  To cope
with having two GPR sources or destinations, the instruction gets
repeated at the decode2 stage, that is, for each lq/stq/lqarx/stqcx.
coming in from decode1, two instructions get sent out to execute1.

For these instructions, the RS or RT register gets modified on one
of the iterations by setting the LSB of the register number.  In LE
mode, the first iteration uses RS|1 or RT|1 and the second iteration
uses RS or RT.  In BE mode, this is done the other way around.  In
order for decode2 to know what endianness is currently in use, we
pass the big_endian flag down from icache through decode1 to decode2.
This is always in sync with what execute1 is using because only rfid
or an interrupt can change MSR[LE], and those operations all cause
a flush and redirect.

There is now an extra column in the decode tables in decode1 to
indicate whether the instruction needs to be repeated.  Decode1 also
enforces the rule that lq with RT = RT and lqarx with RA = RT or
RB = RT are illegal.

Decode2 now passes a 'repeat' flag and a 'second' flag to execute1,
and execute1 passes them on to loadstore1.  The 'repeat' flag is set
for both iterations of a repeated instruction, and 'second' is set
on the second iteration.  Execute1 does not take asynchronous or
trace interrupts on the second iteration of a repeated instruction.

Loadstore1 uses 'next_addr' for the second iteration of a repeated
load/store so that we access the second doubleword of the memory
operand.  Thus loadstore1 accesses the doublewords in increasing
memory order.  For 16-byte loads this means that the first iteration
writes GPR RT|1.  It is possible that RA = RT|1 (this is a legal
but non-preferred form), meaning that if the memory operand was
misaligned, the first iteration would overwrite RA but then the
second iteration might take a page fault, leading to corrupted state.
To avoid that possibility, 16-byte loads in LE mode take an
alignment interrupt if the operand is not 16-byte aligned.  (This
is the case anyway for lqarx, and we enforce it for lq as well.)

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 55f7d99376 decode1: Fix decoding of recommended NOP instruction
We were decoding nop with the wrong major opcode.  Fix it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 1037c6aa2e core: Implement mtmsr instruction
This is like mtmsrd except it only alters the lower 32 bits of the MSR.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras dc1544db69 FPU: Implement floating multiply-add instructions
This implements fmadd, fmsub, fnmadd, fnmsub and their
single-precision counterparts.  The single-precision versions operate
the same as the double-precision versions until the final rounding and
overflow/underflow steps.

This adds an S register to store the low bits of the product.  S
shifts into R on left shifts, and can be negated, but doesn't do any
other arithmetic.

This adds a test for the double-precision versions of these
instructions.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras c083b9507d FPU: Implement ftdiv and ftsqrt
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras c350bc1f25 FPU: Implement fsqrt[s] and add a test for fsqrt
This implements the floating square-root calculation using a table
lookup of the inverse square root approximation, followed by three
iterations of Goldschmidt's algorithm, which gives estimates of both
sqrt(FRB) and 1/sqrt(FRB).  Then the residual is calculated as
FRB - R * R and that is multiplied by the 1/sqrt(FRB) estimate to get
an adjustment to R.  The residual and the adjustment can be negative,
and since we have an unsigned multiplier, the upper bits can be wrong.
In practice the adjustment fits into an 8-bit signed value, and the
bottom 8 bits of the adjustment product are correct, so we sign-extend
them, divide by 4 (because R is in 10.54 format) and add them to R.

Finally the residual is calculated again and compared to 2*R+1 to see
if a final increment is needed.  Then the result is rounded and
written back.

This implements fsqrts as fsqrt, but with rounding to single precision
and underflow/overflow calculation using the single-precision exponent
range.  This could be optimized later.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 394f993e75 FPU: Implement frsqrte[s] and a test for frsqrte
This implements frsqrte by table lookup.  We first normalize the input
if necessary and adjust so that the exponent is even, giving us a
mantissa value in the range [1.0, 4.0), which is then used to look up
an entry in a 768-entry table.  The 768 entries are appended to the
table for reciprocal estimates, giving a table of 1024 entries in
total.  frsqrtes is implemented identically to frsqrte.

The estimate supplied is accurate to 1 part in 1024 or better.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 49f3d1e77a FPU: Implement fcmpu and fcmpo
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4cd9301da6 FPU: Implement fsel
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4ad5ab9203 FPU: Implement fre[s]
This just returns the value from the inverse lookup table.  The result
is accurate to better than one part in 512 (the architecture requires
1/256).

This also adds a simple test, which relies on the particular values in
the inverse lookup table, so it is not a general test.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 9cce936251 FPU: Implement fdiv[s]
This implements floating-point division A/B by a process that starts
with normalizing both inputs if necessary.  Then an estimate of 1/B
from a lookup table is refined by 3 Newton-Raphson iterations and then
multiplied by A to get a quotient.  The remainder is calculated as
A - R * B (where R is the result, i.e. the quotient) and the remainder
is compared to 0 and to B to see whether the quotient needs to be
incremented by 1.  The calculations of 1 / B are done with 56 fraction
bits and intermediate results are truncated rather than rounded,
meaning that the final estimate of 1 / B is always correct or a little
bit low, never too high, and thus the calculated quotient is correct
or 1 unit too low.  Doing the estimate of 1 / B with sufficient
precision that the quotient is always correct to the last bit without
needing any adjustment would require many more bits of precision.

This implements fdivs by computing a double-precision quotient and
then rounding it to single precision.  It would be possible to
optimize this by e.g. doing only 2 iterations of Newton-Raphson and
then doing the remainder calculation and adjustment at single
precision rather than double precision.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras e6a5f237bc FPU: Implement fmul[s]
This implements the fmul and fmuls instructions.

For fmul[s] with denormalized operands we normalize the inputs
before doing the multiplication, to eliminate the need for doing
count-leading-zeroes on P.  This adds 3 or 5 cycles to the
execution time when one or both operands are denormalized.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 86b826cd7e FPU: Implement fadd[s] and fsub[s] and add tests for them
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 4807d0bdb6 FPU: Implement fmrgew and fmrgow and add tests for them
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 0ad2aa3014 FPU: Implement floating round-to-integer instructions
This implements frin, friz, frip and frim, and adds tests for them.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 03d1aa968a FPU: Implement floating convert to integer instructions
This implements fctiw, fctiwz, fctiwu, fctiwuz, fctid, fctidz, fctidu
and fctiduz, and adds tests for them.

There are some subtleties around the setting of the inexact (XX) and
invalid conversion (VXCVI) flags in the FPSCR.  If the rounded value
ends up being out of range, we need to set VXCVI and not XX.  For a
conversion to unsigned word or doubleword of a negative value that
rounds to zero, we need to set XX and not VXCVI.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 34b5d4a7b5 FPU: Implement the frsp instruction
This brings in the invalid exception for the case of frsp with a
signalling NaN as input, and the need to be able to convert a
signalling NaN to a quiet NaN.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 9e8fb293ed FPU: Implement floating convert from integer instructions
This implements fcfid, fcfidu, fcfids and fcfidus, which convert
64-bit integer values in an FPR into a floating-point value.
This brings in a lot of the datapath that will be needed in
future, including the shifter, adder, mask generator and
count-leading-zeroes logic, along with the machinery for rounding
to single-precision or double-precision, detecting inexact results,
signalling inexact-result exceptions, and updating result flags
in the FPSCR.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras b628af6176 FPU: Implement fmr and related instructions
This implements fmr, fneg, fabs, fnabs and fcpsgn and adds tests
for them.

This adds logic to unpack and repack floating-point data from the
64-bit packed form (as stored in memory and the register file) into
the unpacked form in the fpr_reg_type record.  This is not strictly
necessary for fmr et al., but will be useful for when we do actual
arithmetic.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras fc2968f132 FPU: Implement remaining FPSCR-related instructions
This implements mcrfs, mtfsfi, mtfsb0/1, mffscr, mffscrn, mffscrni and
mffsl.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 856e9e955f core: Add framework for an FPU
This adds the skeleton of a floating-point unit and implements the
mffs and mtfsf instructions.

Execute1 sends FP instructions to the FPU and receives busy,
exception, FP interrupt and illegal interrupt signals from it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 9d285a265c core: Add support for single-precision FP loads and stores
This adds code to loadstore1 to convert between single-precision and
double-precision formats, and implements the lfs* and stfs*
instructions.  The conversion processes are described in Power ISA
v3.1 Book 1 sections 4.6.2 and 4.6.3.

These conversions take one cycle, so lfs* and stfs* are one cycle
slower than lfd* and stfd*.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 45cd8f4fc3 core: Add support for floating-point loads and stores
This extends the register file so it can hold FPR values, and
implements the FP loads and stores that do not require conversion
between single and double precision.

We now have the FP, FE0 and FE1 bits in MSR.  FP loads and stores
cause a FP unavailable interrupt if MSR[FP] = 0.

The FPU facilities are optional and their presence is controlled by
the HAS_FPU generic passed down from the top-level board file.  It
defaults to true for all except the A7-35 boards.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras b589d2d472 execute1: Implement trace interrupts
Trace interrupts occur when the MSR[TE] field is non-zero and an
instruction other than rfid has been successfully completed.  A trace
interrupt occurs before the next instruction is executed or any
asynchronous interrupt is taken.

Since the trace interrupt is defined to set SRR1 bits depending on
whether the traced instruction is a load or an instruction treated as
a load, or a store or an instruction treated as a store, we need to
make sure the treated-as-a-load instructions (icbi, icbt, dcbt, dcbst,
dcbf) and the treated-as-a-store instructions (dcbtst, dcbz) have the
correct opcodes in decode1.  Several of them were previously marked as
OP_NOP.

We don't yet implement the SIAR or SDAR registers, which should be set
by trace interrupts.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 6a80825e70 decode1: Avoid overriding fields of v.decode in decode1
In the cases where we need to override the values from the decode ROMs,
we now do that overriding after the clock edge (eating into decode2's
cycle) rather than before.  This helps timing a little.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago
Paul Mackerras 83816cb9e3 core: Implement BCD Assist instructions addg6s, cdtbcd, cbcdtod
To avoid adding too much logic, this moves the adder used by OP_ADD
out of the case statement in execute1.vhdl so that the result can
be used by OP_ADDG6S as well.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
4 years ago