This uses the instruction-doubling machinery to send load with update
instructions down to loadstore1 as two separate ops, rather than
one op with two destinations. This will help to simplify the value
tracking mechanisms.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements a cache in fetch1, where each entry stores the address
of a simple branch instruction (b or bc) and the target of the branch.
When fetching sequentially, if the address being fetched matches the
cache entry, then fetching will be redirected to the branch target.
The cache has 1024 entries and is direct-mapped, i.e. indexed by bits
11..2 of the NIA.
The bus from execute1 now carries information about taken and
not-taken simple branches, which fetch1 uses to update the cache.
The cache entry is updated for both taken and not-taken branches, with
the valid bit being set if the branch was taken and cleared if the
branch was not taken.
If fetching is redirected to the branch target then that goes down the
pipe as a predicted-taken branch, and decode1 does not do any static
branch prediction. If fetching is not redirected, then the next
instruction goes down the pipe as normal and decode1 does its static
branch prediction.
In order to make timing, the lookup of the cache is pipelined, so on
each cycle the cache entry for the current NIA + 8 is read. This
means that after a redirect (from decode1 or execute1), only the third
and subsequent sequentially-fetched instructions will be able to be
predicted.
This improves the coremark value on the Arty A7-100 from about 180 to
about 190 (more than 5%).
The BTC is optional. Builds for the Artix 7 35-T part have it off by
default because the extra ~1420 LUTs it takes mean that the design
doesn't fit on the Arty A7-35 board.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Using the main adder for comparisons has the disadvantage of creating
a long path from the CA/OV bit forwarding to v.busy via the carry
input of the adder, the comparison result, and determining whether a
trap instruction would trap. Instead we now have dedicated
comparators for the high and low words of a_in vs. b_in, and combine
their results to get the signed and unsigned comparison results.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This breaks up the enormous if .. elsif .. case .. elsif statement in
execute1 in order to try to make it simpler and more understandable.
We now have decode2 deciding whether the instruction has a value to be
written back to a register (GPR, GSPR, FPR, etc.) rather than
individual cases in execute1 setting result_en. The computation of
the data to be written back is now independent of detection of various
exception conditions. We now have an if block determining if any
exception condition exists which prevents the next instruction from
being executed, then the case statement which performs actions such as
setting carry/overflow bits, determining if a trap exception exists,
doing branches, etc., then an if statement for all the r.busy = 1
cases (continuing execution of an instruction which was started in a
previous cycle, or writing SRR1 for an interrupt).
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds an explicit multiplexer feeding v.e.write_data in execute1,
with the select lines determined in the previous cycle based on the
insn_type. Similarly, for multiply and divide instructions, there is
now an explicit multiplexer.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This does the addition of the instruction NIA and the branch offset
after the register at the output of execute1 rather than before.
The propagation through the adder was showing up as a critical path
on the A7-100. Performance is unaffected and now it makes timing.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This makes it simpler to work out when to deliver a FPU unavailable
interrupt. This also means we can get rid of the OP_FPLOAD and
OP_FPSTORE insn_type values.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements the lq, stq, lqarx and stqcx. instructions.
These instructions all access two consecutive GPRs; for example the
"lq %r6,0(%r3)" instruction will load the doubleword at the address
in R3 into R7 and the doubleword at address R3 + 8 into R6. To cope
with having two GPR sources or destinations, the instruction gets
repeated at the decode2 stage, that is, for each lq/stq/lqarx/stqcx.
coming in from decode1, two instructions get sent out to execute1.
For these instructions, the RS or RT register gets modified on one
of the iterations by setting the LSB of the register number. In LE
mode, the first iteration uses RS|1 or RT|1 and the second iteration
uses RS or RT. In BE mode, this is done the other way around. In
order for decode2 to know what endianness is currently in use, we
pass the big_endian flag down from icache through decode1 to decode2.
This is always in sync with what execute1 is using because only rfid
or an interrupt can change MSR[LE], and those operations all cause
a flush and redirect.
There is now an extra column in the decode tables in decode1 to
indicate whether the instruction needs to be repeated. Decode1 also
enforces the rule that lq with RT = RT and lqarx with RA = RT or
RB = RT are illegal.
Decode2 now passes a 'repeat' flag and a 'second' flag to execute1,
and execute1 passes them on to loadstore1. The 'repeat' flag is set
for both iterations of a repeated instruction, and 'second' is set
on the second iteration. Execute1 does not take asynchronous or
trace interrupts on the second iteration of a repeated instruction.
Loadstore1 uses 'next_addr' for the second iteration of a repeated
load/store so that we access the second doubleword of the memory
operand. Thus loadstore1 accesses the doublewords in increasing
memory order. For 16-byte loads this means that the first iteration
writes GPR RT|1. It is possible that RA = RT|1 (this is a legal
but non-preferred form), meaning that if the memory operand was
misaligned, the first iteration would overwrite RA but then the
second iteration might take a page fault, leading to corrupted state.
To avoid that possibility, 16-byte loads in LE mode take an
alignment interrupt if the operand is not 16-byte aligned. (This
is the case anyway for lqarx, and we enforce it for lq as well.)
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This deletes some commentary that is now out of date and replaces it
with a simple statement about the XER common bits being forwarded from
the output of execute1 to the input.
The comment being deleted talked about a hazard if an instruction that
modifies XER[SO] is immediately followed by a store conditional. That
is no longer a problem because the operands for loadstore1 are sent
from execute1 (and therefore have the forwarded value) rather than
decode2. This was in fact fixed in 5422007f83 ("Plumb loadstore1
input from execute1 not decode2", 2020-01-14).
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
We don't care what the values of TB and DECR are after reset, but we
don't want the X state to propagate to other parts of the chip.
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
Random execution testcases showed that a bdnzl which doesn't branch,
followed immediately by a bdnz, uses the wrong value for CTR for the
bdnz. Decode2 detects the read-after-write hazard on CTR and tells
execute1 to use the bypass path. However, the bdnzl takes two cycles
because it has to write back both CTR and LR, meaning that by the time
the bdnz starts to execute, r.e.write_data no longer contains the CTR
value, but instead contains zero.
To fix this, we make execute1 maintain the written-back value of CTR
in r.e.write_data across the cycle where LR is written back (this is
possible because the LR writeback uses the exc_write_data path).
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Branch instructions which do a redirect and write both CTR and LR were
not doing the write to LR due to a logic error. This fixes it.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
If an instruction fetch results in an instruction TLB miss, an
OP_FETCH_FAILED instruction is sent down the pipe. If the MSR[TE]
field is set for instruction tracing, the core currently considers
that executing the OP_FETCH_FAILED counts as having executed one
instruction and so generates a trace interrupt on the next valid
instruction, meaning that the trace interrupt happens before the
desired instruction rather than after it.
Fix this by not tracing OP_FETCH_FAILED instructions.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds the skeleton of a floating-point unit and implements the
mffs and mtfsf instructions.
Execute1 sends FP instructions to the FPU and receives busy,
exception, FP interrupt and illegal interrupt signals from it.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds code to loadstore1 to convert between single-precision and
double-precision formats, and implements the lfs* and stfs*
instructions. The conversion processes are described in Power ISA
v3.1 Book 1 sections 4.6.2 and 4.6.3.
These conversions take one cycle, so lfs* and stfs* are one cycle
slower than lfd* and stfd*.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This extends the register file so it can hold FPR values, and
implements the FP loads and stores that do not require conversion
between single and double precision.
We now have the FP, FE0 and FE1 bits in MSR. FP loads and stores
cause a FP unavailable interrupt if MSR[FP] = 0.
The FPU facilities are optional and their presence is controlled by
the HAS_FPU generic passed down from the top-level board file. It
defaults to true for all except the A7-35 boards.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Trace interrupts occur when the MSR[TE] field is non-zero and an
instruction other than rfid has been successfully completed. A trace
interrupt occurs before the next instruction is executed or any
asynchronous interrupt is taken.
Since the trace interrupt is defined to set SRR1 bits depending on
whether the traced instruction is a load or an instruction treated as
a load, or a store or an instruction treated as a store, we need to
make sure the treated-as-a-load instructions (icbi, icbt, dcbt, dcbst,
dcbf) and the treated-as-a-store instructions (dcbtst, dcbz) have the
correct opcodes in decode1. Several of them were previously marked as
OP_NOP.
We don't yet implement the SIAR or SDAR registers, which should be set
by trace interrupts.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
In the cases where we need to override the values from the decode ROMs,
we now do that overriding after the clock edge (eating into decode2's
cycle) rather than before. This helps timing a little.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Load-and-reserve and store-conditional instructions are required to
generate an alignment interrupt (0x600 vector) if their EA is not
aligned. Implement this.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
In 32-bit mode, effective addresses are truncated to 32 bits, both for
instruction fetches and data accesses, and CR0 is set for Rc=1 (record
form) instructions based on the lower 32 bits of the result rather
than all 64 bits.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Big-endian mode affects both instruction fetches and data accesses.
For instruction fetches, we byte-swap each word read from memory when
writing it into the icache data RAM, and use a tag bit to indicate
whether each cache line contains instructions in BE or LE form.
For data accesses, we simply need to invert the existing byte_reverse
signal in BE mode. The only thing to be careful of is to get the sign
bit from the correct place when doing a sign-extending load that
crosses two doublewords of memory.
For now, interrupts unconditionally set MSR[LE]. We will need some
sort of interrupt-little-endian bit somewhere, perhaps in LPCR.
This also fixes a debug report statement in fetch1.vhdl.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
To avoid adding too much logic, this moves the adder used by OP_ADD
out of the case statement in execute1.vhdl so that the result can
be used by OP_ADDG6S as well.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
The addex instruction is like adde but uses the XER[OV] bit for the
carry in and out rather than XER[CA].
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a true random number generator for the Xilinx FPGAs which
uses a set of chaotic ring oscillators to generate random bits and
then passes them through a Linear Hybrid Cellular Automaton (LHCA) to
remove bias, as described in "High Speed True Random Number Generators
in Xilinx FPGAs" by Catalin Baetoniu of Xilinx Inc., in:
https://pdfs.semanticscholar.org/83ac/9e9c1bb3dad5180654984604c8d5d8137412.pdf
This requires adding a .xdc file to tell vivado that the combinatorial
loops that form the ring oscillators are intentional. The same
code should work on other FPGAs as well if their tools can be told to
accept the combinatorial loops.
For simulation, the random.vhdl module gets compiled in, which uses
the pseudorand() function to generate random numbers.
Synthesis using yosys uses nonrandom.vhdl, which always signals an
error, causing darn to return 0xffff_ffff_ffff_ffff.
This adds an implementation of the darn instruction. Darn can return
either raw or conditioned random numbers. On Xilinx FPGAs, reading a
raw random number gives the output of the ring oscillators, and
reading a conditioned random number gives the output of the LHCA.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
These instructions use major opcode 4 and have a third GPR input
operand, so we need a decode table for major opcode 4 and some
plumbing to get the RC register operand read.
The multiply-add instructions use the same insn_type_t values as the
regular multiply instructions, and we distinguish in execute1 by
looking at the major opcode. This turns out to be convenient because
we don't have to add any cases in the code that handles the output of
the multiplier, and it frees up some insn_type_t values.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This also removes OP_MCRXR, as the mcrxr instruction was removed in
version 3.0B of the Power ISA, having been phased-out for the server
architecture since v2.02.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
We now expect the overflow signal from the multiplier to come along
one cycle later than the product.
This breaks up a long combinatorial path and improves timing.
This also changes some uses of v.<field> to r.<field> in the slow
op logic, which should help timing as well.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This makes the interface to the multiplier more general so an instance
of it can be used in the FPU. It now has a 128-bit addend that is
added on to the product. Instead of an input to negate the output,
it now has a "not_result" input to complement the output. Execute1
uses not_result=1 and addend=-1 to get the effect of negating the
output. The interface is defined this way because this is what can
be done easily with the Xilinx DSP slices in xilinx-mult.vhdl.
This also adds clock enable signals to the DSP slices, mostly for the
sake of reducing power consumption.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds "if LOG_LENGTH > 0 generate" to the places in the core
where log output data is latched, so that when LOG_LENGTH = 0 we
don't create the logic to collect the data which won't be stored.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This eliminates a dependency of r.f.redirect_nia on the carry out
from the main adder in the case of a conditional trap instruction.
We can set r.f.redirect_nia unconditionally, even if no interrupt
is generated.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a path to allow the CR result of one instruction to be
forwarded to the next instruction, so that sequences such as
cmp; bc can avoid having a 1-cycle bubble.
Forwarding is not available for dot-form (Rc=1) instructions,
since the CR result for them is calculated in writeback. The
decode.output_cr field is used to identify those instructions
that compute the CR result in execute1.
For some reason, the multiply instructions incorrectly had
output_cr = 1 in the decode tables. This fixes that.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This latches the redirect signal inside execute1, so that it is sent
a cycle later to fetch1 (and to decode/icache as flush). This breaks
a long combinatorial chain from the branch and interrupt detection
in execute1 through the redirect/flush signals all the way back to
fetch1, icache and decode.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements the CFAR SPR as a slow SPR stored in 'ctrl'. Taken
branches and rfid update it to the address of the branch or rfid
instruction.
To simplify the logic, this makes rfid use the branch logic to
generate its redirect (requiring SRR0 to come in to execute1 on
the B input and SRR1 on the A input), and the masking of the bottom
2 bits of NIA is moved to fetch1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This reduces the number of different things that are assigned to
the result variable.
- The computations for the popcnt, prty, cmpb and exts instruction
families are moved into the logical unit.
- The result of mfspr from the slow SPRs is computed in 'spr_val'
before being assigned to 'result'.
- Writes to LR as a result of a blr or bclr instruction are done
through the exc_write path to writeback.
This eases timing considerably.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements a simple branch predictor in the decode1 stage. If it
sees that the instruction is b or bc and the branch is predicted to be
taken, it sends a flush and redirect upstream (to icache and fetch1)
to redirect fetching to the branch target. The prediction is sent
downstream with the branch instruction, and execute1 now only sends
a flush/redirect upstream if the prediction was wrong. Unconditional
branches are always predicted to be taken, and conditional branches
are predicted to be taken if and only if the offset is negative.
Branches that take the branch address from a register (bclr, bcctr)
are predicted not taken, as we don't have any way to predict the
branch address.
Since we can now have a mflr being executed immediately after a bl
or bcl, we now track the update to LR in the hazard tracker, using
the second write register field that is used to track RA updates for
update-form loads and stores.
For those branches that update LR but don't write any other result
(i.e. that don't decrementer CTR), we now write back LR in the same
cycle as the instruction rather than taking a second cycle for the
LR writeback.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This reduces the number of cycles where loadstore1 asserts its busy
output, leading to increased throughput of loads and stores. Loads
that hit in the cache can now be executed at the rate of one every two
cycles. Stores take 4 cycles assuming the wishbone slave responds
with an ack the cycle after we assert strobe.
To achieve this, the state machine code is split into two parts, one
for when we have an existing instruction in progress, and one for
starting a new instruction. We can now combinatorially clear busy and
start a new instruction in the same cycle that we get a done signal
from the dcache; in other words we are completing one instruction and
potentially writing back results in the same cycle that we start a new
instruction and send its address and data to the dcache.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This changes the instruction dependency tracking so that we can
generate a "busy" signal from execute1 and loadstore1 which comes
along one cycle later than the current "stall" signal. This will
enable us to signal busy cycles only when we need to from loadstore1.
The "busy" signal from execute1/loadstore1 indicates "I didn't take
the thing you gave me on this cycle", as distinct from the previous
stall signal which meant "I took that but don't give me anything
next cycle". That means that decode2 proactively gives execute1
a new instruction as soon as it has taken the previous one (assuming
there is a valid instruction available from decode1), and that then
sits in decode2's output until execute1 can take it. So instructions
are issued by decode2 somewhat earlier than they used to be.
Decode2 now only signals a stall upstream when its output buffer is
full, meaning that we can fill up bubbles in the upstream pipe while a
long instruction is executing. This gives a small boost in
performance.
This also adds dependency tracking for rA updates by update-form
load/store instructions.
The GPR and CR hazard detection machinery now has one extra stage,
which may not be strictly necessary. Some of the code now really
only applies to PIPELINE_DEPTH=1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This puts the logic that selects which bits of the multiplier result
get written into the destination GPR into execute1, moved out from
multiply.
The multiplier is now expected to do an unsigned multiplication of
64-bit operands, optionally negate the result, detect 32-bit
or 64-bit signed overflow of the result, and return a full 128-bit
result.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This logs 256 bits of data per cycle to a ring buffer in BRAM. The
data collected can be read out through 2 new SPRs or through the
debug interface.
The new SPRs are LOG_ADDR (724) and LOG_DATA (725). LOG_ADDR contains
the buffer write pointer in the upper 32 bits (in units of entries,
i.e. 32 bytes) and the read pointer in the lower 32 bits (in units of
doublewords, i.e. 8 bytes). Reading LOG_DATA gives the doubleword
from the buffer at the read pointer and increments the read pointer.
Setting bit 31 of LOG_ADDR inhibits the trace log system from writing
to the log buffer, so the contents are stable and can be read.
There are two new debug addresses which function similarly to the
LOG_ADDR and LOG_DATA SPRs. The log is frozen while either or both of
the LOG_ADDR SPR bit 31 or the debug LOG_ADDR register bit 31 are set.
The buffer defaults to 2048 entries, i.e. 64kB. The size is set by
the LOG_LENGTH generic on the core_debug module. Software can
determine the length of the buffer because the length is ORed into the
buffer write pointer in the upper 32 bits of LOG_ADDR. Hence the
length of the buffer can be calculated as 1 << (31 - clz(LOG_ADDR)).
There is a program to format the log entries in a somewhat readable
fashion in scripts/fmt_log/fmt_log.c. The log_entry struct in that
file describes the layout of the bits in the log entries.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
By adding logic to decode2 to be able to send the instruction address
down the A input, and making CONST_DX_HI (renamed to CONST_DXHI4) add
4 to the immediate value (easy since the bottom 16 bits were zero),
we can do addpcis using the main adder. This reduces the width of the
result mux and frees up one value in insn_type_t, since we can now use
OP_ADD for addpcis.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This commit adds support for the addpcis instruction from ISA 3.0.
A new input_reg_b_t type, CONST_DX_HI, was added to support the
shifted immediate value used in DX-Form instructions.
Signed-off-by: Shawn Anastasio <shawn@anastas.io>
Use a simple wire. common.vhdl types are better kept for things
local to the core. We can add more wires later if we need to for
HV irqs etc...
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
- Changing use of others in core files to satisfy VCS
- Adding workaround for VCS subtype constraint inconsistencies in common.vhdl
Signed-off-by: Jonathan Balkind <jbalkind@princeton.edu>
Slbia (with IH=7) is used in the Linux kernel to flush the ERATs
(our iTLB/dTLB), so make it do that.
This moves the logic to work out whether to flush a single entry
or the whole TLB from dcache and icache into mmu. We now invalidate
all dTLB and iTLB entries when the AP (actual pagesize) field of
RB is non-zero on a tlbie[l], as well as when IS is non-zero.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This hooks up the connections so that an OP_FETCH_FAILED coming down
to loadstore1 will get sent to the MMU for it to do a radix tree walk
for the instruction address. The MMU then sends the resulting PTE to
the icache module to be installed in the iTLB. If no valid PTE can
be found, the MMU sends an error signal back to loadstore1 which sends
it on to execute1 to generate an ISI.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a direct-mapped TLB to the icache, with 64 entries by default.
Execute1 now sends a "virt_mode" signal from MSR[IR] to fetch1 along
with redirects to indicate whether instruction addresses should be
translated through the TLB, and fetch1 sends that on to icache.
Similarly a "priv_mode" signal is sent to indicate the privilege
mode for instruction fetches. This means that changes to MSR[IR]
or MSR[PR] don't take effect until the next redirect, meaning an
isync, rfid, branch, etc.
The icache uses a hash of the effective address (i.e. next instruction
address) to index the TLB. The hash is an XOR of three fields of the
address; with a 64-entry TLB, the fields are bits 12--17, 18--23 and
24--29 of the address. TLB invalidations simply invalidate the
indexed TLB entry without checking the contents.
If the icache detects a TLB miss with virt_mode=1, it will send a
fetch_failed indication through fetch2 to decode1, which will turn it
into a special OP_FETCH_FAILED opcode with unit=LDST. That will get
sent down to loadstore1 which will currently just raise a Instruction
Storage Interrupt (0x400) exception.
One bit in the PTE obtained from the TLB is used to check whether an
instruction access is allowed -- the privilege bit (bit 3). If bit 3
is 1 and priv_mode=0, then a fetch_failed indication is sent down to
fetch2 and to decode1, which generates an OP_FETCH_FAILED. Any PTEs
with PTE bit 0 (EAA[3]) clear or bit 8 (R) clear should not be put
into the iTLB since such PTEs would not allow execution by any
context.
Tlbie operations get sent from mmu to icache over a new connection.
Unfortunately the privileged instruction tests are broken for now.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
A data segment interrupt (DSegI) occurs when an address to be
translated by the MMU is outside the range of the radix tree
or the top two bits of the address (the quadrant) are 01 or 10.
This is detected in a new state of the MMU state machine, and
is sent back to loadstore1 as an error, which sends it on to
execute1 to generate an interrupt to the 0x380 vector.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds logic to the dcache to check the permissions encoded in
the PTE that it gets from the dTLB. The bits that are checked are:
R must be 1
C must be 1 for a store
EAA(0) - if this is 1, MSR[PR] must be 0
EAA(2) must be 1 for a store
EAA(1) | EAA(2) must be 1 for a load
In addition, ATT(0) is used to indicate a cache-inhibited access.
This now implements DSISR bits 36, 38 and 45.
(Bit numbers above correspond to the ISA, i.e. using big-endian
numbering.)
MSR[PR] is now conveyed to loadstore1 for use in permission checking.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a path from loadstore1 back to execute1 for reporting
errors, and machinery in execute1 for generating data storage
interrupts at vector 0x300.
If dcache is given two requests in successive cycles and the
first encounters an error (e.g. a TLB miss), it will now cancel
the second request.
Loadstore1 now responds to errors reported by dcache by sending
an exception signal to execute1 and returning to the idle state.
Execute1 then writes SRR0 and SRR1 and jumps to the 0x300 Data
Storage Interrupt vector. DAR and DSISR are held in loadstore1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a TLB to dcache, providing the ability to translate
addresses for loads and stores. No protection mechanism has been
implemented yet. The MSR_DR bit controls whether addresses are
translated through the TLB.
The TLB is a fixed-pagesize, set-associative cache. Currently
the page size is 4kB and the TLB is 2-way set associative with 64
entries per set.
This implements the tlbie instruction. RB bits 10 and 11 control
whether the whole TLB is invalidated (if either bit is 1) or just
a single entry corresponding to the effective page number in bits
12-63 of RB.
As an extension until we get a hardware page table walk, a tlbie
instruction with RB bits 9-11 set to 001 will load an entry into
the TLB. The TLB entry value is in RS in the format of a radix PTE.
Currently there is no proper handling of TLB misses. The load or
store will not be performed but no interrupt is generated.
In order to make timing at 100MHz on the Arty A7-100, we compare
the real address from each way of the TLB with the tag from each way
of the cache in parallel (requiring # TLB ways * # cache ways
comparators). Then the result is selected based on which way hit in
the TLB. That avoids a timing path going through the TLB EA
comparators, the multiplexer that selects the RA, and the cache tag
comparators.
The hack where addresses of the form 0xc------- are marked as
cache-inhibited is kept for now but restricted to real-mode accesses.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This arranges for some mfspr and mtspr to get sent to loadstore1
instead of being handled in execute1. In particular, DAR and DSISR
are handled this way. They are therefore "slow" SPRs.
While we're at it, fix the spelling of HEIR and remove mention of
DAR and DSISR from the comments in execute1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This provides commands on the debug interface to read the value of
the MSR or any of the 64 GSPR register file entries. The GSPR values
are read using the B port of the register file in a cycle when
decode2 is not using it.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Mfspr from an unimplemented SPR should be a no-op in privileged state,
so in this case we need to write back whatever was previously in the
destination register. For problem state, both mtspr and mfspr to
unimplemented SPRs should cause a program interrupt.
There are special cases in the architecture for SPRs 0, 4 5 and 6
which we still don't implement.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This mainly required the addition of an entry to the opcode 31 decode
table and a 32-bit sign-extender in the rotator.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
During slow instructions such as multiply or divide, if a decrementer
(or other asynchronous) interrupt becomes pending, it disrupts the
logic that keeps stall asserted until the end of the slow
instruction, and the interrupt logic starts trying to deliver the
interrupt before the slow instruction has finished.
To fix that, make the interrupt logic wait until it sees e_in.valid
set before setting exception to 1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
In preparation for adding a TLB to the dcache, this plumbs the
insn_type from execute1 through to loadstore1, so that we can have
other operations besides loads and stores (e.g. tlbie) going to
loadstore1 and thence to the dcache. This also plumbs the unit field
of the decode ROM from decode2 through to execute1 to simplify the
logic around which ops need to go to loadstore1.
The load and store data formatting are now not conditional on the
op being OP_LOAD or OP_STORE. This eliminates the inferred latches
clocked by each of the bits of r.op that we were getting previously.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds logic to execute1 to check, when MSR[PR] = 1, whether each
instruction arriving to be executed is a privileged instruction.
If it is, a privileged-instruction type program interrupt is generated.
For the mtspr and mfspr instructions, we need to look at bit 20 of the
instruction (bit 4 of the SPR number) to determine if the SPR is
privileged.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This makes our treatment of the MSR conform better with the ISA.
- On reset, initialize the MSR to have the SF and LE bits set and
all the others reset. For good measure initialize r properly too.
- Fix the bit numbering in msr_copy (the code was using big-endian
bit numbers, not little-endian).
- Use constants like MSR_EE to index MSR bits instead of expressions
like '63 - 48', for readability.
- Set MSR[SF, LE] and clear MSR[PR, IR, DR, RI] on interrupts.
- Copy the relevant fields for rfid instead of using msr_copy, because
the partial function fields of the MSR should be left unchanged,
not zeroed. Our implementation of rfid is like the architecture
description of hrfid, because we don't implement hypervisor mode.
- Return the whole MSR for mfmsr.
- Implement the L field for mtmsrd (L=1 copies just EE and RI).
- For mtmsrd with L=0, leave out the HV, ME and LE bits as per the arch.
- For mtmsrd and rfid, if PR ends up set, then also set EE, IR and DR
as per the arch.
- A few other minor tidyups (no semantic change).
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
New unified ICP and ICS XICS compliant interrupt controller.
Configurable number of hardware sources.
Fixed hardware source number based on hardware line taken. All
hardware interrupts are a fixed priority. Level interrupts supported
only.
Hardwired to 0xc0004000 in SOC (UART is kept at 0xc0002000).
Signed-off-by: Michael Neuling <mikey@neuling.org>
This fixes a bug in the logic where we would still send a load
or store instruction to loadstore1 even though we have decided
to take an asynchronous interrupt.
Reported-by: Michael Neuling <mikey@neuling.org>
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This decodes attn using entry 0 of the major_decode_rom_array table
instead of a special case in the decode1_1 process. This means that
only the major opcode (the top 6 bits) is checked at decode time.
To make sure the instruction is attn not some random illegal pattern,
we now check bits 1-10 of the instruction at execute time and
generate an illegal instruction interrupt if those bits are not
0100000000.
This reduces LUT consumption by 42 LUTs on the Arty A7-100.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This decodes sc using entry 17 of the major_decode_rom_array table
instead of a special case in the decode1_1 process. This means that
only the major opcode (the top 6 bits) is checked at decode time.
To make sure that the instruction is sc not scv, we now check bit
1 of the instruction at execute time and generate an illegal
instruction interrupt if it is 0 (indicating scv). The level field
of the sc instruction is now ignored.
This reduces LUT consumption by 31 LUTs on the Arty A7-100.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements the trap instructions (tw, twi, td, tdi) using
much of the same code as is used for the cmp/cmpl instructions.
A 5-bit comparison value is generated, and for cmp/cmpl, the
appropriate 3 bits are used to update the destination CR, and for
trap instructions, the comparison value is ANDed with the TO
field, and an exception is generated if any bit of the result
is 1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This replaces OP_TD, OP_TDI, OP_TW and OP_TWI with a single OP_TRAP,
distinguishing the cases by the input_reg_b and is_32bit fields of
the decode ROM. This adds the twi and td cases to the decode tables.
For now we make all of the trap instructions unconditionally generate
a trap-type program interrupt if the TO field of the instruction is
all ones, and do nothing otherwise.
This reduces the number of values in insn_type_t from 65 to 62,
meaning that an insn_type_t can now be encoded in 6 bits rather
than 7.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This makes some simplifications to the interrupt logic which will
help with later commits.
- When irq_valid is set, don't set exception to 1 until we have a
valid instruction. That means we can remove the if e_in.valid = '1'
test from the exception = '1' block.
- Don't assert stall_out on the first cycle of delivering an
interrupt. If we do get another instruction in the next cycle,
nothing will happen because we have ctrl.irq_state set and we
will just continue writing the interrupt registers.
- Make sure we deliver as many completions as we got instructions,
otherwise the outstanding instruction count in control.vhdl gets
out of sync.
- In writeback, make sure all of the other write enables are ignored
when e_in.exc_write_enable is set.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
OP_MCRF covers the CR logical ops as well as mcrf since commit
c05441bf47 ("Implement CRNOR and friends"), so this renames
OP_MCRF to OP_CROP. The OP_* values for the individual CR logical
ops (OP_CRAND, etc.) are not used, so remove them from insn_type_t.
No functional change.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds separate fields in Execute1ToWritebackType for use in
writing SRR0/1 (and in future other SPRs) on an interrupt. With
this, we make timing once again on the Arty A7-100 -- previously
we were missing by 0.2ns, presumably due to the result mux being
wider than before.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds the following exceptions:
- 0x700 program check (for illegal instructions)
- 0x900 decrementer
- 0xc00 system call
This also adds some supervisor state:
- decremeter
- msr
(SPRG0/1 and SRR0/1 already exist as fast SPRs)
It also adds some supporting instructions:
- rfid
- mtmsrd
- mfmsr
- sc
MSR state is added but only EE is used in this patch set. Other bits
are read/written but are not used at all.
This adds a 2 stage state machine to execute1.vhdl. This state machine
allows fast SPRS SRR0/1 to be written in different cycles. This state
machine can be extended later to add DAR and DSISR SPR writing for
more complex exceptions like page faults.
Signed-off-by: Michael Neuling <mikey@neuling.org>
Currently we decode attn but we just mark it as an illegal.
This adds a separate case statement in execute 1 for attn to terminate
the core. Illegals also do this currently but we are soon implementing
a 0x700 execption for them.
Signed-off-by: Michael Neuling <mikey@neuling.org>
This adds support for lbzcix, lhzcix, lwzcix, ldcix, stbcix, sthcix,
stwcix and stdcix. The temporary hack where accesses to addresses of
the form 0xc??????? are made non-cacheable is left in for now to avoid
making existing programs non-functional.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
So that the dcache could in future be used by an MMU, this moves
logic to do with data formatting, rA updates for update-form
instructions, and handling of unaligned loads and stores out of
dcache and into loadstore1. For now, dcache connects only to
loadstore1, and loadstore1 now has the connection to writeback.
Dcache generates a stall signal to loadstore1 which indicates that
the request presented in the current cycle was not accepted and
should be presented again. However, loadstore1 doesn't currently
use it because we know that we can never hit the circumstances
where it might be set.
For unaligned transfers, loadstore1 generates two requests to
dcache back-to-back, and then waits to see two acks back from
dcache (cycles where d_in.valid is true).
Loadstore1 now has a FSM for tracking how many acks we are
expecting from dcache and for doing the rA update cycles when
necessary. Handling for reservations and conditional stores is
still in dcache.
Loadstore1 now generates its own stall signal back to decode2,
so we no longer need the logic in execute1 that generated the stall
for the first two cycles.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This involves plumbing the (existing) 'reserve' and 'rc' bits in
the decode tables down to dcache, and 'rc' and 'store_done' bits
from dcache to writeback.
It turns out that we had 'RC' set in the 'rc' column for several
ordinary stores and for the attn instruction. This corrects them
to 'NONE', and sets the 'rc' column to 'ONE' for the conditional
stores.
In writeback we now have logic to set CR0 when the input from dcache
has rc = 1.
In dcache we have the reservation itself, which has a valid bit
and the address down to cache line granularity. We don't currently
store the reservation length. For a store conditional which fails,
we set a 'cancel_store' signal which inhibits the write to the
cache and prevents the state machine from starting a bus cycle or
going to the STORE_WAIT_ACK state. Instead we set r1.stcx_fail
which causes the instruction to complete in the next cycle with
rc=1 and store_done=0.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This removes the constraint that loads and stores are single-issue,
at the expense of a stall of at least 2 cycles for every load and
store.
To do this, we plumb the existing stall signal that was generated
in dcache to core, where it gets ORed with the stall signal from
execute1. Execute1 generates a stall signal for the first two
cycles of each load and store, and dcache generates the stall
signal in the 3rd and subsequent cycles if it needs to.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
It turns out that CR logical instructions have the truth table of
the operation embedded in the instruction word. This means that we
can collect the two input operand bits into a 2-bit value and use
that as the index to select the appropriate bit from the instruction
word.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This adds a register in the middle of the countzero computation,
so that we now have two cycles to count leading or trailing zeroes
instead of just one. Execute1 now outputs a one-cycle stall signal
when it encounters a cntlz* or cnttz* instruction. With this,
the countzero path no longer fails timing on the Artix-7 at 100MHz.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This allows us to use the bypass at the input of execute1 for the
address and data operands for loadstore1.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This enables back-to-back execution of integer instructions where
the first instruction writes a GPR and the second reads the same
GPR. This is done with a set of multiplexers at the start of
execute1 which enable any of the three input operands to be taken
from the output of execute1 (i.e. r.e.write_data) rather than the
input from decode2 (i.e. e_in.read_data[123]).
This also requires changes to the hazard detection and handling.
Decode2 generates a signal indicating that the GPR being written
is available for bypass, which is true for instructions that are
executed in execute1 (rather than loadstore1/dcache). The
gpr_hazard module stores this "bypassable" bit, and if the same
GPR needs to be read by a subsequent instruction, it outputs a
"use_bypass" signal rather than generating a stall. The
use_bypass signal is then latched at the output of decode2 and
passed down to execute1 to control the input multiplexer.
At the moment there is no bypass on the inputs to loadstore1, but that
is OK because all load and store instructions are marked as
single-issue.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This implements logic in the logical entity to calculate the results
of the popcnt* and prty* instructions. We now have one insn_type_t
value for the 3 popcnt variants and one for the two prty variants,
using the length field of the decode_rom_t to distinguish between
them. The implementations in logical.vhdl using recursive
algorithms rather than the simple functions in ppc_fx_insns.vhdl.
This gives a saving of about 140 slice LUTs on the A7-100 and
improves timing slightly.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This handles OP_CMP like a subtraction; the main adder computes
~RA + RB + 1, and the condition codes are computed from the results.
A direct comparison of the two input operands is used to calculate the
EQ bit of the condition result. The LT and GT bits are computed from
the MSB of the subtraction result, the carry out from the subtraction,
and the MSBs of the operands. For a 32-bit comparison, the 32-bit
carry and bit 31 of the result and input operands are used; for a
64-bit comparison, the 64-bit carry and bit 63 of the operands and
result are used.
It turns out to be more convenient to use the 'signed' field of
the decode table to distinguish signed from unsigned comparisons,
rather than the insn_type. Therefore this uses OP_CMP for both
cmp and cmpl, which also has the benefit of reducing the number
of values in insn_type_t.
Doing this saves over 200 slice LUTs on the Arty A7-100 and improves
timing slightly as well.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
This moves the sign extension done by the extsb, extsh and extsw
instructions back into execute1. This means that we no longer need
any data formatting in writeback for results coming from execute1,
so this modifies writeback so the data formatter inputs come
directly from the loadstore unit output. The condition code
updates for RC=1 form instructions are now done on the value from
execute1 rather than the output of the data formatter, which should
help timing.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
For multiply and divide operations, execute1 now records the
destination GPR number, RC and OE from the instruction, and the
XER value. This means that the multiply and divide units don't
need to record those values and then send them back to execute1.
This makes the interface to those units a bit simpler. They
simply report an overflow signal along with the result value, and
execute1 takes care of updating XER if necessary.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
With this, the divider is a unit that execute1 sends operands to and
which sends its results back to execute1, which then send them to
writeback. Execute1 now sends a stall signal when it gets a divide
or modulus instruction until it gets a valid signal back from the
divider. Divide and modulus instructions are no longer marked as
single-issue.
The data formatting step that used to be done in decode2 for div
and mod instructions is now done in execute1. We also do the
absolute value operation in that same cycle instead of taking an
extra cycle inside the divider for signed operations with a
negative operand.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
With this, the multiplier isn't a separate pipe that decode2 issues
instructions to, but rather is a unit that execute1 sends operands
to and which sends the result back to execute1, which then sends it
to writeback. Execute1 now sends a stall signal when it gets a
multiply instruction until it gets a valid signal back from the
multiplier.
This all means that we no longer need to mark the multiply
instructions as single-issue.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
It should never happen in practise, but ghdlsynth is complaining about
an inferred latch here. Fix it
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
From the architecture:
bits 0:31 and 35:43 are treated as reserved and return 0s when read
using mfxer
Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
This stores the most common SPRs in the register file.
This includes CTR and LR and a not yet final list of others.
The register file is set to 64 entries for now. Specific types
are defined that can represent a GPR index (gpr_index_t) or
a GPR/SPR index (gspr_index_t) along with conversion functions
between the two.
On order to deal with some forms of branch updating both LR and
CTR, we introduced a delayed update of LR after a branch link.
Note: We currently stall the pipeline on such a delayed branch,
but we could avoid stalling fetch in that specific case as we
know we have a branch delay. We could also limit that to the
specific case where we need to update both CTR and LR.
This allows us to make bcreg, mtspr and mfspr pipelined. decode1
will automatically force the single issue flag on mfspr/mtspr to
a "slow" SPR.
[paulus@ozlabs.org - fix direction of decode2.stall_in]
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
We were copying in XER[SO] for the dot-form instructions but not the
explicit compare instructions. Fix this.
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
The carry is currently internal to execute1. We don't handle any of
the other XER fields.
This creates type called "xer_common_t" that contains the commonly
used XER bits (CA, CA32, SO, OV, OV32).
The value is stored in the CR file (though it could be a separate
module). The rest of the bits will be implemented as a separate
SPR and the two parts reconciled in mfspr/mtspr in latter commits.
We always read XER in decode2 (there is little point not to)
and send it down all pipeline branches as it will be needed in
writeback for all type of instructions when CR0:SO needs to be
updated (such forms exist for all pipeline branches even if we don't
yet implement them).
To avoid having to track XER hazards, we forward it back in EX1. This
assumes that other pipeline branches that can modify it (mult and div)
are running single issue for now.
One additional hazard to beware of is an XER:SO modifying instruction
in EX1 followed immediately by a store conditional. Due to our writeback
latency, the store will go down the LSU with the previous XER value,
thus the stcx. will set CR0:SO using an obsolete SO value.
I doubt there exist any code relying on this behaviour being correct
but we should account for it regardless, possibly by ensuring that
stcx. remain single issue initially, or later by adding some minimal
tracking or moving the LSU into the same pipeline as execute.
Missing some obscure XER affecting instructions like addex or mcrxrx.
[paulus@ozlabs.org - fix CA32 and OV32 for OP_ADD, fix order of
arguments to set_ov]
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
Use a function to obtain the integer number and use constants
with the architected numbers. Replace std_match with a case
statement.
This also has the side effect of returning 0 instead of some
random previous result on mfspr of an unknown SPR.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Anton Blanchard
