You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Linux-Architecture-Reference/LoPAR/ch_address_map.xml

1152 lines
54 KiB
XML

<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2016, 2020 OpenPOWER Foundation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xl="http://www.w3.org/1999/xlink"
version="5.0"
xml:lang="en"
xml:id="dbdoclet.50569328_Address-Map">
<title>Address Map</title>
<para>The address map of an LoPAR platform is made up of several distinct
areas. These areas are one of five basic types. Each of these types has its own
general characteristics such as coherency, alignment, size restrictions,
variability of starting address and size, the system action on access of the
area, and so on. This chapter gives details on some of those characteristics,
and other chapters define the other characteristics. The variable
characteristics of these areas are reported to the OS via properties in the OF
device tree.</para>
<section xml:id="dbdoclet.50569328_42362">
<title>Address Areas</title>
<para>The following is a definition of the five areas and some of their
characteristics:</para>
<itemizedlist>
<listitem>
<para><emphasis>System Memory</emphasis> refers to memory
which forms a coherency domain with respect
to the PA processor(s) that execute application software on a system. See
<xref linkend="dbdoclet.50569329_37207"/> for details on aspects of coherence.
<emphasis>System Memory Spaces</emphasis> refer to one or more pieces that
together form the System Memory. System Memory areas may be marked with a
special value of the <emphasis role="bold"><literal>&#x201C;status&#x201D;</literal></emphasis> property of
&#x201C;reserved&#x201D; which means that this memory is not for general use by
the base OS, but may be reserved for use by OS extensions (see
<xref linkend="dbdoclet.50569329_41706"/>). Some System Memory areas may be
preservable across boots (see <xref linkend="dbdoclet.50569327_70628"/>).</para>
</listitem>
<listitem>
<para>Peripheral Memory Space refers to a range of real addresses which
are assigned to the Memory Space of a Host Bridge (HB) or System Bus attached
IOA, and which are sufficient to contain all of the Load and Store address
space requirements of all IOAs in the Memory Space of the I/O bus that is
generated by the HB or which are encompassed by the System Bus attached IOA.
The frame buffer of a graphics IOA is an example of a device which may reside
in the Peripheral Memory Space. Due to space limitations in the address space
below 4 GB, the HBs of platforms may split this space into two pieces; one to
support the IOAs that need to have their addresses below 4 GB (because they
only support 32-bit addresses) and another to support the IOAs that can have
their addresses above 4 GB (because they support 64-bit addresses). In addition
to a Memory Space, many types of I/O buses have a separate address space called
the I/O Space. An HB which generates such I/O buses must decode another address
range, the Peripheral I/O Space.<footnote xml:id="pgfId-128932"><para>A
peripheral space may also include a &#x201C;configuration&#x201D; address
space. The configuration space is abstracted by a Run-Time Abstraction Service
(for example, see <xref linkend="dbdoclet.50569332_14852"/>).</para></footnote></para>
</listitem>
<listitem>
<para>Peripheral I/O Space refers to a range of real addresses which
are assigned to the I/O Space of an HB or System Bus attached IOA and which are
sufficient to contain all of the Load and Store address space requirements of
all the IOAs in the I/O Space of the I/O bus that is generated by the HB or
which are encompassed by the System Bus IOA. A keyboard controller is an
example of an IOA which may require Peripheral I/O Space addresses. </para>
</listitem>
<listitem>
<para>System Control Area (SCA) refers to a range of addresses which
contains all reserved addresses (architected or unarchitected) which are not
part of one of the other defined address spaces. For example, the system ROM(s),
unarchitected platform-dependent addresses used by firmware and Run-Time
Abstraction Services for control of the platform, and architected entities like
interrupt controller addresses when those addresses are not in another defined
address space. </para>
</listitem>
<listitem>
<para>Undefined refers to areas that are not one of the above four
areas. The result of accessing one of these areas is defined in <xref
linkend="dbdoclet.50569337_52952"/> as an invalid address error.</para>
</listitem>
</itemizedlist>
<para>In addition to the above definitions, it is convenient, relative to
I/O op erations, to define a Partitionable Endpoint. A <emphasis>Partitionable
Endpoint</emphasis> (PE) is an I/O subtree that can be treated as a unit for
the purposes of partitioning and error recovery. A PE may be a single or
multi-function IOA, a function of a multi-function IOA, or multiple IOAs
(possibly including switch and bridge structures above the multiple IOAs). See
<xref linkend="dbdoclet.50569330_34831"/> for more information about PEs.</para>
<para>In describing the characteristics of these various areas, it is
convenient to have a nomenclature for the various boundary addresses.
<xref linkend="dbdoclet.50569328_34186"/> defines the labels which are used in this
document when describing the various address ranges. Note that
&#x201C;bottom&#x201D; refers to the smallest address of the range and
&#x201C;top&#x201D; refers to the largest address.</para>
<table frame="all" pgwide="1" xml:id="dbdoclet.50569328_34186">
<title>Map Legend </title>
<tgroup cols="2">
<colspec colname="c1" colwidth="20*" align="center"/>
<colspec colname="c2" colwidth="80*"/>
<thead valign="middle">
<row>
<entry>
<para>
<emphasis role="bold">Label</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Description</emphasis>
</para>
</entry>
</row>
</thead>
<tbody valign="middle">
<row>
<entry>
<para> BIOn</para>
</entry>
<entry>
<para> Bottom of Peripheral I/O Space for HBn (n=0, 1, 2,...).
The OF property <emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis>
in the OF device tree for HBn contains the value of BIOn.</para>
</entry>
</row>
<row>
<entry>
<para> TIOn</para>
</entry>
<entry>
<para> Top of Peripheral I/O Space for HBn (n=0, 1, 2,...). The
value of TIOn can be determined by adding the size of the area as found in the
OF property <emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis> in the OF device tree
for HBn to the value of BIOn found in that same property and then subtracting
1.</para>
<para>This architecture allows at most one Peripheral I/O area
per HB which may be above or below 4 GB. For any given n, BIOn to TIOn cannot
span from the first 4 GB of address space to the second.</para>
</entry>
</row>
<row>
<entry>
<para> BPMn,m</para>
</entry>
<entry>
<para> Bottom of Peripheral Memory Space m (m=0,1) for HBn (n=0,
1, 2,...), as viewed from the system side of HBn. The OF property
<emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis> in the OF device tree for HBn contains the
value of BPMn,m.</para>
</entry>
</row>
<row>
<entry>
<para> BPM&#x2019;n,m</para>
</entry>
<entry>
<para> Bottom of Peripheral Memory Space m (m=0,1) for HBn (n=0,
1, 2,...), as viewed from the I/O side of the HBn. That is, this is the value
to which BPMn,m gets translated to as it passes through the HB. The OF property
<emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis> in the OF device tree for HBn
contains the value of BPM&#x2019;n,m. BPM&#x2019;n,m may be equal to BPMn,m or
may not be.</para>
</entry>
</row>
<row>
<entry>
<para> TPMn,m</para>
</entry>
<entry>
<para> Top of Peripheral Memory Space m (m=0,1) for HBn (n=0, 1,
2,...) as viewed from the system side of HBn. The Peripheral Memory Space
address range is in the OF device tree, as indicated by the
<emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis> property in the node in the OF device tree
for HBn; BPMn,m to TPMn,m. The value of TPMn,m can be determined by adding the
size of the area as found in the OF property
<emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis> in the OF device tree for HBn to the value of
BPMn,m found in that same property and then subtracting 1.</para>
<para>This architecture allows for one or two Peripheral Memory
areas per HB (hence, m=0,1). A Peripheral Memory area may be above 4 GB or
below. For any given n, BPMn,m to TPMn,m cannot span from the first 4 GB of
address space to the second.</para>
</entry>
</row>
<row>
<entry>
<para> TPM&#x2019;n,m</para>
</entry>
<entry>
<para> Top of Peripheral Memory Space m (m=0,1) for HBn (n=0, 1,
2,...) as viewed from the I/O side of HBn. The value of TPM&#x2019;n,m can be
calculated from the values in the<emphasis role="bold"><literal>&#x201C;ranges&#x201D;</literal></emphasis>
property as was TPMn,m. In some cases TPM&#x2019;n,m is required to be equal to
TPMn,m and in some cases it is not required to be equal. For any given n,
BPM&#x2019;n,m to TPM&#x2019;n,m cannot span from the first 4 GB of address
space to the second.</para>
</entry>
</row>
<row>
<entry>
<para> BSCAn</para>
</entry>
<entry>
<para> Bottom of System Control Area. Corresponding top of the
System Control Area is TSCAn. This architecture allows for one or two SCAs per
platform. The SCA below 4 GB is at the top (largest addresses) of the lower 4
GB range. </para>
</entry>
</row>
<row>
<entry>
<para> TSCAn</para>
</entry>
<entry>
<para> Top of System Control Area. For any given n, BSCAn to
TSCAn cannot span from the first 4 GB of address space to the second.</para>
</entry>
</row>
<row>
<entry>
<para> BSMn</para>
</entry>
<entry>
<para> Bottom of System Memory Space n (n=0, 1, 2,...); BSM0 = 0.
The OF property <emphasis role="bold"><literal>&#x201C;reg&#x201D;</literal></emphasis> in the OF device tree
for the Memory Controller&#x2019;s node contains the value of BSMn.</para>
</entry>
</row>
<row>
<entry>
<para> TSMn</para>
</entry>
<entry>
<para> Top of System Memory Space n (n=0, 1, 2,...). The value of
TSMn can be determined by adding the value of BSMn as found in the Memory
Controller&#x2019;s node of the OF device tree to the value of the size of that
area as found in the same property, and then subtracting 1.</para>
</entry>
</row>
<row>
<entry>
<para> BTTAn,m</para>
</entry>
<entry>
<para> Bottom of TCE Translatable Address space m (m=0, 1, 2,...)
for HBn (n=0, 1, 2,...) as viewed from the I/O side of HBn. This is the bottom
of an address range that is translatable by a Translation Control Entry (TCE)
table. The value of BTTAn,m is obtained from the
<emphasis role="bold"><literal>&#x201C;ibm,dma-window&#x201D;</literal></emphasis> or
<emphasis role="bold"><literal>&#x201C;ibm,my-dma-window&#x201D;</literal></emphasis> property in the OF device
tree.</para>
</entry>
</row>
<row>
<entry>
<para> TTTAn,m</para>
</entry>
<entry>
<para> Top of TCE Translatable Address space m (m=0, 1, 2,...)
for HBn (n=0, 1, 2,...) as viewed from the I/O side of HBn. This is the top of
an address range that is translatable by a TCE table. The range BTTAn,m to
TTTAn,m is not accessible by more than one PE for any given &#x201C;n&#x201D;.
The value of TTTAn,m can be determined by adding the size of the area as found
in the OF property <emphasis role="bold"><literal>&#x201C;ibm,dma-window&#x201D;</literal></emphasis> or
<emphasis role="bold"><literal>&#x201C;ibm,my-dma-window&#x201D;</literal></emphasis> in the OF device tree
for HBn to the value of BTTAn,m found in that same property and then
subtracting 1.</para>
</entry>
</row>
</tbody>
</tgroup>
</table>
<para>The figures found in <xref linkend="dbdoclet.50569328_70508"/>, show
examples of the areas referenced by the labels in <xref
linkend="dbdoclet.50569328_34186"/>.</para>
<para>The OS and other software should not use fixed addresses for these
various areas. A given platform may, however, make some of these addresses
unchangeable. Each of these areas is defined in the OF device tree in the node
of the appropriate controller. This gives platforms the most flexibility in
implementing the System Address Map to meet their market requirements. </para>
<variablelist>
<varlistentry xml:id="dbdoclet.50569328_66354">
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_42362"
xrefstyle="select: labelnumber nopage"/>-1.</emphasis></term>
<listitem>
<para>All unavailable addresses in the Peripheral Memory and Peripheral
I/O Spaces must be conveyed in the OF device tree.</para>
<orderedlist numeration="loweralpha">
<listitem>
<para>A <emphasis role="bold"><literal>&#x201C;device_type&#x201D;</literal></emphasis> of
<emphasis role="bold"><literal>&#x201C;reserved&#x201D;</literal></emphasis> must be used to specify areas which are not
to be used by software and not otherwise reported by OF.</para>
</listitem>
<listitem>
<para>Shadow aliases must be communicated as specified by the
appropriate OF bus binding.</para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_42362"
xrefstyle="select: labelnumber nopage"/>-2.</emphasis></term>
<listitem>
<para>There must not be any address generated by
the system which causes the system to hang.</para>
</listitem>
</varlistentry>
</variablelist>
<para><emphasis role="bold">Hardware Implementation Note:</emphasis> The reason for Requirement
<xref linkend="dbdoclet.50569328_66354"/> is to reserve address space for registers
used only by the firmware or addresses which are used only by the
hardware.</para>
</section>
<section xml:id="dbdoclet.50569328_28737">
<title> Address Decoding (or Validating) and Translation</title>
<para>In general, different components in the hardware are going to decode
the address ranges for the various areas. In some cases the component may be
required to translate the address to a new address as it passes through the
component. The requirements, below, describe the various system address decodes
(or validating) and, where appropriate, what address transforms take place
outside of the processor. </para>
<para>The HB requirements in this section refer to HBs which are defined by
this architecture. Currently, there is only one HB defined by this
architecture, and that is the PHB. HBs which implement I/O buses other than
those defined by this architecture may or may not require changes to this
addressing model. </para>
<para>The reader may want to reference the example address maps found in
<xref linkend="dbdoclet.50569328_70508"/>, while reading through the
requirements of this section.</para>
<section xml:id="sec_address_map_address_xlate">
<title> <emphasis>Load</emphasis> and <emphasis>Store</emphasis>
Address Decoding and Translation</title>
<para><emphasis>Load</emphasis> and <emphasis>Store</emphasis>
operations may be targeted at System Memory or I/O. The latter is called Memory
Mapped I/O (MMIO). </para>
<variablelist>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-1.</emphasis></term>
<listitem>
<para>Processor Load and Store
operations must be routed and translated as shown in <xref
linkend="dbdoclet.50569328_29870"/>.</para>
</listitem>
</varlistentry>
</variablelist>
<table frame="all" pgwide="1" xml:id="dbdoclet.50569328_29870">
<title>Processor Bus Address Space Decoding and Translation</title>
<tgroup cols="3">
<colspec colname="c1" colwidth="20*" align="center"/>
<colspec colname="c2" colwidth="50*"/>
<colspec colname="c3" colwidth="30*"/>
<thead valign="middle">
<row>
<entry>
<para>
<emphasis role="bold">Address Range at Processor
Bus</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Route and Translation
Requirements</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Other Requirements and
Comments</emphasis>
</para>
</entry>
</row>
</thead>
<tbody valign="middle">
<row>
<entry>
<para>BSCAn to TSCAn<?linebreak?>
(n=0, 1)</para>
</entry>
<entry>
<para> To ROM controller or to a platform dependent area.
Translation dependent on implementation.</para>
</entry>
<entry>
<para> Areas other than ROM are reserved for firmware use, or
have their address passed by the OF device tree.</para>
</entry>
</row>
<row>
<entry>
<para>BIOn to TIOn<?linebreak?>
(n=0,1, 2,...)</para>
</entry>
<entry>
<para> Send through the HB to the I/O space of the I/O bus,
translating by subtracting the value of BIO from each address in this range
(that is, translate BIO to TIO to be at 0 to (TIO - BIO) on the I/O
side).</para>
</entry>
<entry>
<para> &#xA0;</para>
</entry>
</row>
<row>
<entry>
<para>BPMn,m to TPMn,m<?linebreak?>
(n=0, 1, 2,...)<?linebreak?>
(m=0, 1)</para>
</entry>
<entry>
<para> Send through HBn to the Memory Space of the I/O bus.</para>
<itemizedlist>
<listitem>
<para>If BPMn,m &lt; 4 GB, do not translate an address in the
BPMn,m to TPMn,m range as the transaction passes through the bridge (that is,
BPM&#x2019;n,m = BPMn,m and TPM&#x2019;n,m = TPMn,m).</para>
</listitem>
<listitem>
<para>If BPMn,m is at or above 4 GB then if BPM&#x2019;n,m is
to be below 4 GB (for 32-bit IOAs) then translate addresses in the BPMn,m to
TPMn,m range so that this address range becomes BPM&#x2019;n,m to
TPM&#x2019;n,m (where BPM&#x2019;n,m and TPM&#x2019;n,m are less than 4 GB) as
the transaction passes through the bridge, otherwise do not translate an
address in the BPMn,m to TPMn,m range as the transaction passes through the
bridge (for 64-bit IOAs which are configured at or above 4 GB).</para>
</listitem>
</itemizedlist>
</entry>
<entry>
<para> Platforms that need to support both 32-bit capable and
64-bit capable IOAs and do not want to configure the 64-bit capable IOAs below
4 GB need to support two Peripheral Memory spaces per HB.</para>
</entry>
</row>
<row>
<entry>
<para>BSMm to TSMm<?linebreak?>
(m&gt;0)</para>
</entry>
<entry>
<para> To System Memory Space m, no translation.</para>
</entry>
<entry>
<para> Can be at or above 4 GB, or below BSCA0. </para>
</entry>
</row>
<row>
<entry>
<para> 0 to TSM0</para>
</entry>
<entry>
<para> To System Memory Space 0, no translation</para>
</entry>
<entry>
<para> &#xA0;</para>
</entry>
</row>
<row>
<entry>
<para> All other addresses</para>
</entry>
<entry>
<para> See <xref linkend="dbdoclet.50569337_52952"/>.</para>
</entry>
<entry>
<para> Access is to undefined space.</para>
</entry>
</row>
</tbody>
</tgroup>
</table>
<variablelist>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-2.</emphasis></term>
<listitem>
<para>There must be no architected address
spaces (Peripheral Memory, Peripheral I/O, SCA, or System Memory) which span
the (4GB - 1) to 4 GB boundary.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-3.</emphasis></term>
<listitem>
<para>The following are the System Control Area requirements:</para>
<orderedlist numeration="loweralpha">
<listitem xml:id="dbdoclet.50569328_15915">
<para>The platform must have at most one System Control Area below 4 GB
and at most one per platform or per NUMA node at or above 4 GB.</para>
</listitem>
<listitem>
<para>The System Control Area must not overlap with the System Memory
Space(s), Peripheral Memory Space(s), or the Peripheral I/O Space(s) in the
platform.</para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
<varlistentry xml:id="dbdoclet.50569328_18841">
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-4.</emphasis></term>
<listitem>
<para>The following are the System Memory Space requirements:</para>
<orderedlist numeration="loweralpha">
<listitem>
<para>Each platform must have at least one System Memory Space.</para>
</listitem>
<listitem>
<para>The System Memory Space(s) must not overlap with the Peripheral I/O
Space(s), Peripheral Memory Space(s), the System Control Area, or other System
Memory Space(s) in the platform.</para>
</listitem>
<listitem>
<para>The first System Memory Space must start at address 0 (BSM0 = 0),
must be at least 128 MB before a second System Memory Space is added and must
be contiguous.</para>
</listitem>
<listitem>
<para>Each of the additional (optional) System Memory Space(s)
must start on a 4 KB boundary.</para>
</listitem>
<listitem>
<para>Each of the additional (optional) System Memory Space(s)
must be contiguous within itself.</para>
</listitem>
<listitem>
<para>There must be at most eight System Memory Spaces below BSCA0 and
at most eight at or above 4 GB.</para>
</listitem>
<listitem>
<para>If multiple System Memory Spaces exist below 4 GB, then they
must not have any Peripheral Memory or Peripheral I/O Spaces interspersed
between them and if multiple System Memory Spaces exist above 4 GB, then they
must not have any Peripheral Memory or Peripheral I/O Spaces interspersed
between them. </para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
<varlistentry xml:id="dbdoclet.50569328_81773">
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-5.</emphasis></term>
<listitem>
<para>The following are the Peripheral Memory Space requirements:</para>
<orderedlist numeration="loweralpha">
<listitem>
<para>The Peripheral Memory Space(s) must not overlap with the System
Memory Space(s), Peripheral I/O Space(s), the System Control Area, or other
Peripheral Memory Space(s) in the platform.</para>
</listitem>
<listitem xml:id="dbdoclet.50569328_50885">
<para>The size of each
Peripheral Memory Space (TPMn,m - BPMn,m + 1) must be a power of two for sizes
up to and including 256 MB, with the minimum size being 1 MB, and an integer
multiple of 256 MB plus a power of two which is greater than or equal to 1 MB
for sizes greater than 256 MB (for example, 1 MB, 2 MB, 4 MB, 8 MB, 16 MB, 32
MB, 64 MB, 128 MB, 256 MB, (256 + 1) MB, (256 + 2) MB,..., (512 + 1)
MB,...).</para>
</listitem>
<listitem xml:id="dbdoclet.50569328_88434">
<para>The boundary alignment for each Peripheral Memory Space must be an integer multiple
of the size of the space up to and including 256 MB and must be an integer
multiple of 256 MB for sizes greater than 256 MB.</para>
</listitem>
<listitem>
<para>There must be at most two Peripheral Memory Spaces per HB.
</para>
</listitem>
<listitem xml:id="dbdoclet.50569328_33398">
<para>If the Peripheral Memory Space for a HB is below 4 GB, then the
address must not be translated as it passes through the HB from the system side
to the I/O side of the HB (see <xref
linkend="dbdoclet.50569328_29870"/>).</para>
</listitem>
<listitem xml:id="dbdoclet.50569328_44965">
<para>If the Peripheral Memory Space for a HB is above 4 GB, then the
address may or may not be translated as it passes through the HB from the
system side to the I/O side of the HB, but if it is translated, then the
translated address range must be aligned on a boundary which is an integer
multiple of the size of the Peripheral Memory Space.</para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
</variablelist>
<para><emphasis role="bold">Implementation Note:</emphasis> Relative to Requirement
<xref linkend="dbdoclet.50569328_81773"/><xref
linkend="dbdoclet.50569328_44965"/>, not all OSs can support BPM&#x2019; to
TPM&#x2019; being above 4 GB.</para>
<variablelist>
<varlistentry xml:id="dbdoclet.50569328_14048">
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-6.</emphasis></term>
<listitem>
<para>The following are the Peripheral I/O Space requirements:</para>
<orderedlist numeration="loweralpha">
<listitem>
<para>The Peripheral I/O Space(s) must not overlap with the System
Memory Space(s), Peripheral Memory Space(s), the System Control Area, or other
Peripheral I/O Space(s) in the platform.</para>
</listitem>
<listitem>
<para>The size of each Peripheral I/O Space (TIOn - BIOn + 1) must be a
power of two with the minimum size being 64 KB (that is, sizes of 64 KB, 128
KB, 256 KB, 512 KB, 1 MB, 2 MB, 4 MB, 8 MB, 16 MB, 32 MB, 64 MB, and so on, are
acceptable).</para>
</listitem>
<listitem xml:id="dbdoclet.50569328_47269">
<para>The boundary alignment for each Peripheral I/O Space must be an integer multiple of the size of
the space.</para>
</listitem>
<listitem>
<para>There must be at most one Peripheral I/O Space per
HB.</para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
<varlistentry xml:id="dbdoclet.50569328_63622">
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_address_xlate"
xrefstyle="select: labelnumber nopage"/>-7.</emphasis></term>
<listitem>
<para>All System Memory must be
accessible via DMA operation from all IOAs in the system, except where LPAR
requirements limit accessibility of an IOA belonging to one partition to the
System Memory of another partition.</para>
</listitem>
</varlistentry>
</variablelist>
<para><emphasis role="bold">Hardware Implementation Notes:</emphasis> Memory controller and memory card
designers who are designing for 64-bit platforms should be careful to consider
that the amount of I/O space below 4 GB is reduced by the amount of System
Memory space below 4 GB. Therefore it may be prudent to design the hardware to
allow minimization of the amount of System Memory below 4 GB, in order to allow
maximization of the space for 32-bit Peripheral Memory and Peripheral I/O
spaces below 4 GB.</para>
<para>The beginning addresses and sizes of the Peripheral I/O Space(s)
and Peripheral
Memory Space(s), are controlled by firmware. Information about the address map
is reported by the OF Device Tree or, for items that can change, through RTAS
calls (for example, for Dynamic Reconfiguration, through the
<emphasis>ibm,configure-connector</emphasis> RTAS call). </para>
<para>Certain System Memory addresses must be reserved in all systems for
specific uses (see <xref linkend="dbdoclet.50569329_27369"/> and <xref
linkend="dbdoclet.50569329_41706"/> for more information).</para>
</section>
<section xml:id="dbdoclet.50569328_17364">
<title> DMA Address Validation and Translation</title>
<para><xref linkend="dbdoclet.50569328_18795"/> is a representation of
how the validation and translation mechanism works, along with a description of
the steps which are involved. At the core of the translation mechanism is the
Translation and Control Entry (TCE) table. </para>
<figure xml:id="dbdoclet.50569328_18795">
<title>PE DMA Address Validation and Translation in the Platform</title>
<mediaobject>
<imageobject role="html">
<imagedata fileref="figures/PAPR-8.gif" format="GIF" scalefit="1"/>
</imageobject>
<imageobject role="fo">
<imagedata contentdepth="100%" fileref="figures/PAPR-8.gif"
format="GIF" scalefit="1" width="100%"/>
</imageobject>
</mediaobject>
</figure>
<section xml:id="sec_address_map_dma_req">
<title>DMA Addressing Requirements</title>
<variablelist>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_dma_req"
xrefstyle="select: labelnumber nopage"/>-1.</emphasis></term>
<listitem>
<para>Upon receiving a DMA
transaction to the Memory Space of an I/O bus, the HB must perform the
validation and translation steps, as indicated in
<xref linkend="dbdoclet.50569328_18795"/> and in
<xref linkend="dbdoclet.50569328_74684"/>.</para>
</listitem>
</varlistentry>
</variablelist>
<table frame="all" pgwide="1" xml:id="dbdoclet.50569328_74684">
<title>DMA Address Decoding and Translation (I/O Bus Memory
Space) </title>
<tgroup cols="3">
<colspec colname="c1" colwidth="20*" align="center"/>
<colspec colname="c2" colwidth="50*"/>
<colspec colname="c3" colwidth="30*"/>
<thead valign="middle">
<row>
<entry>
<para>
<emphasis role="bold">Address Range at I/O Side of
HBn</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Route and Translation
Requirements</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Other Requirements and
Comments</emphasis>
</para>
</entry>
</row>
</thead>
<tbody valign="middle">
<row>
<entry>
<para>BPM&#x2019;n,m to TPM&#x2019;n,m<?linebreak?>
(n=0, 1, 2,...)<?linebreak?>
(m=0, 1)<?linebreak?>
(note 1)</para>
</entry>
<entry>
<para> HB does not respond or responds and signals an invalid
address error (See <xref linkend="dbdoclet.50569337_52952"/>).</para>
</entry>
<entry>
<para> &#xA0;</para>
</entry>
</row>
<row>
<entry>
<para> BTTAn,m to TTTAn,m<?linebreak?>
(n=0, 1, 2,...)<?linebreak?>
(m=0, 1, 2,...)<?linebreak?>
(note 1)</para>
</entry>
<entry>
<para> If the PE that is trying to access this space is allowed
to access this space, then translate via the TCE table (as specified in <xref
linkend="dbdoclet.50569328_76588"/>) and pass the translated address through
the HB, otherwise generate an invalid address or TCE extent error, as
appropriate (See <xref linkend="dbdoclet.50569337_52952"/>). </para>
</entry>
<entry>
<para> See Notes 2, 3</para>
</entry>
</row>
<row>
<entry>
<para> All other addresses</para>
</entry>
<entry>
<para> Generate an invalid address error (See <xref linkend="dbdoclet.50569337_52952"/>).</para>
</entry>
<entry>
<para> See Note 3</para>
</entry>
</row>
<row>
<entry nameend="c3" namest="c1" align="left">
<para><emphasis role="bold">Notes:</emphasis>
<orderedlist spacing="compact">
<listitem>
<para>n = # of HB Viewing or Receiving the Operation, m = #
of instance within the HB.</para>
</listitem>
<listitem>
<para>After translation of the address, if the translated
address would re-access the same HB or another HB (for example, is in the
Peripheral Memory Space or Peripheral I/O Space of that HB or another HB), then
the HB generates an invalid address error (See <xref
linkend="dbdoclet.50569337_52952"/>).</para>
</listitem>
<listitem>
<para>If the Enhanced I/O Error Handling (EEH) option is
implemented and enabled, then on an error, the PE will enter the DMA Stopped
State (See <xref linkend="dbdoclet.50569330_17337"/>).</para>
</listitem>
</orderedlist>
</para>
</entry>
</row>
</tbody>
</tgroup>
</table>
<variablelist>
<varlistentry xml:id="dbdoclet.50569328_12543">
<term><emphasis role="bold">R2-<xref linkend="sec_address_map_dma_req"
xrefstyle="select: labelnumber nopage"/>-2.</emphasis></term>
<listitem>
<para>An HB must not act as a target
for operations in the I/O Space of an I/O bus.</para>
</listitem>
</varlistentry>
</variablelist>
</section>
<section xml:id="dbdoclet.50569328_76588">
<title> DMA Address Translation and Control via the TCE
Mechanism</title>
<para>This architecture defines a Translation and Control Entry (TCE)
mechanism for translating and controlling DMA addresses. There are several
reasons for doing such translations, including:</para>
<itemizedlist>
<listitem>
<para>To provide a mechanism for increasing the number of addressing
bits for some IOAs. For example, IOAs which are only capable of accessing up to
4 GB via DMA need a way to access above that limit when used in 64-bit
addressing systems and the addressing requirements go beyond 4 GB. </para>
</listitem>
<listitem>
<para>To provide a redirection mechanism. A redirection mechanism is
needed, even for 64-bit addressing capable IOAs, in order to provide the
protection and indirection benefits provided by such a translation.</para>
</listitem>
</itemizedlist>
<para>The description of how the access to the TCE table occurs, for the
translation of a 32-bit address and using a 4 KB I/O page size, follows. The
most significant 20 bits of the address (for example, AD[31:12], for PCI) is
used as an offset into the TCE table for the PE to select the TCE. Thus, the
first TCE maps the addresses BTTAn to BTTAn + 0x00000FFF of the Memory Space of
the I/O bus; the second entry controls translation of addresses BTTAn +
0x00001000 to BTTAn + 0x00001FFF, and so on. The translated real system address
is generated as follows. The Real Page Number (RPN) from the TCE replaces the
20 most significant bits of the address from the I/O bus. The least significant
12 bits from the I/O bus address are used as-is for the least significant 12
bits of the new address.</para>
<para>Thus, the TCE table entries have a one-to-one correspondence with
the first n pages of the Memory Space of the I/O bus starting at BTTAn that
corresponds to the TCE table. The size of the Memory address space of the I/O
bus that can be mapped to the system address space for a particular HB depends
on how much System Memory is allocated to the TCE table(s) and on how much
mappable I/O bus Memory Space is unavailable due to IOAs which are mapped
there. </para>
<para>Each TCE also contains two control bits. These are used to identify
whether that page is mapped to the system address space, and if the page is
mapped, whether it is mapped read/write, read only, or write only. See the
<xref linkend="dbdoclet.50569328_91476"/> for a definition of these control
bits. </para>
<para>The TCE table is the analogue of the system translation tables.
However, unlike the system translation tables, the dynamic page faulting of
memory during an I/O operation is not required (the page fault value, 0b00, in
the TCE Page Mapping and Control field is used for error detection; that is,
access to an invalid TCE by the I/O creates an error indication to the
software).</para>
<para>The size and location of the HB&#x2019;s TCE table is set up and
changed only by the firmware. </para>
<variablelist>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-1.</emphasis></term>
<listitem>
<para>The platform must provide the
<emphasis role="bold"><literal>&#x201C;64-bit-addressing&#x201D;</literal></emphasis> and
<emphasis role="bold"><literal>&#x201C;ibm,extended-address&#x201D;</literal></emphasis> OF properties in all HB nodes
of the device tree and the <emphasis role="bold"><literal>&#x201C;ibm,extended-address&#x201D;</literal></emphasis>
OF property in the <emphasis role="bold"><literal>root</literal></emphasis> node of the OF device tree.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-2.</emphasis></term>
<listitem>
<para>The bits of the TCE must be implemented
as defined in <xref linkend="dbdoclet.50569328_91476"/>.</para>
</listitem>
</varlistentry>
</variablelist>
<table frame="all" pgwide="1" xml:id="dbdoclet.50569328_91476">
<?dbhtml table-width="75%" ?><?dbfo table-width="75%" ?>
<title>TCE Definition </title>
<tgroup cols="2">
<colspec colname="c1" colwidth="10*" align="center"/>
<colspec colname="c2" colwidth="90*"/>
<thead valign="middle">
<row>
<entry>
<para>
<emphasis role="bold">Bits</emphasis>
</para>
</entry>
<entry align="center">
<para>
<emphasis role="bold">Description</emphasis>
</para>
</entry>
</row>
</thead>
<tbody valign="middle">
<row>
<entry>
<para> 0 to 51</para>
</entry>
<entry>
<para> RPN: If the page mapping and control field of the TCE
indicate anything other than page fault, then these bits contain the Real Page
Number (RPN) to which the bus address is mapped in the system address space. In
certain HB implementations, all of these bits may not be required, however
enough bits must be implemented to match the largest real address in the
platform.</para>
</entry>
</row>
<row>
<entry>
<para> 52 to 61</para>
</entry>
<entry>
<para> Reserved for future use.</para>
</entry>
</row>
<row>
<entry>
<para> 62 to 63</para>
</entry>
<entry>
<para> Page Mapping and Control: These bits define page mapping
and read-write authority. They are coded as follows:</para>
<para> 00 Page fault (no access)</para>
<para> 01 System address space (read only)</para>
<para> 10 System address space (write only)</para>
<para> 11 System address space (read/write)</para>
<para> Code point 0b00 signifies that the page is not mapped.
It must be used to indicate a page fault error. Hardware must not change its
state based on the value in the remaining bits of a TCE when code point 0b00 is
set in this field of the TCE.</para>
<para> For accesses to system address space with an invalid
operation (write to a read-only page or read to a write-only page), the HB
generates an error. See <xref linkend="dbdoclet.50569337_52952"/> for more information
about error handling.</para>
</entry>
</row>
</tbody>
</tgroup>
</table>
<variablelist>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-3.</emphasis></term>
<listitem>
<para>If the address that the HB would use to
access the TCE table (in order to get the TCE) would access outside of the TCE
table, then the HB must create a TCE extent error (See <xref
linkend="dbdoclet.50569337_52952"/>).</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-4.</emphasis></term>
<listitem>
<para>Enough bits must be implemented in the
TCE so that DMA IOAs are able to access all System Memory addresses.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-5.</emphasis></term>
<listitem>
<para>Each PE must have its own independent
TCE table.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-6.</emphasis></term>
<listitem>
<para>Any non-recoverable error while an HB
is accessing its TCE table must result in a TCE access error; the action to be
taken by the HB being defined under the TCE access error in <xref linkend="dbdoclet.50569337_52952"/>.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-7.</emphasis></term>
<listitem>
<para>In implementations which cache TCEs, if
software changes a TCE, then the platform must perform the following steps:
First, if any data associated with the page represented by that TCE is in an
I/O bridge cache or buffer, the hardware must write the data, if modified, to
System Memory. Secondly, it must invalidate the data in the cache. Finally, it
must invalidate the TCE in the cache. </para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-8.</emphasis></term>
<listitem>
<para>Neither an IOA nor an HB must ever
modify a TCE.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-9.</emphasis></term>
<listitem>
<para>If the page mapping and control bits in
the TCE are set to 0b00, the hardware must not change its state based on the
values of the remaining bits of the TCE.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><emphasis role="bold">R2-<xref linkend="dbdoclet.50569328_76588"
xrefstyle="select: labelnumber nopage"/>-10.</emphasis></term>
<listitem>
<para>The OS must initialize all its TCEs
upon receiving control from the platform.</para>
</listitem>
</varlistentry>
</variablelist>
</section>
</section>
<section xml:id="dbdoclet.50569328_70508">
<title> Example Address Maps</title>
<para><xref linkend="dbdoclet.50569328_42040"/> shows how to construct a
simple address map with one PHB and with Peripheral Memory, Peripheral I/O, and
SCA spaces below 4 GB. </para>
<para><xref linkend="dbdoclet.50569328_17694"/> shows how to construct
the address map with Peripheral Memory, Peripheral I/O, and SCA spaces above 4
GB. This configuration allows some overlap of the System Memory space and
32-bit I/O bus memory space (with the resulting loss of the TCE table in the
overlap), while moving some of the SCA spaces above 4 GB. Several things can be
noted from this configuration:</para>
<itemizedlist>
<listitem>
<para>I/O bus memory areas can overlap System Memory addresses (see
memory space of PHB0). However, significant overlap of these I/O bus memory
areas and the TCE table may significantly reduce the amount of TCE table space
that is available for mapping I/O memory space to system address space (a
potential performance impact).</para>
</listitem>
<listitem>
<para>The System Memory which is above 4GB is shown starting at 4GB.
This architecture also allows this to be pushed further up, with Peripheral
Memory, Peripheral I/O, and SCAs existing above 4 GB and below the System
Memory areas. </para>
</listitem>
<listitem>
<para>BPM&#x2019;n,m to TPM&#x2019;n,m spaces for different PHBs
(different &#x201C;n&#x201D;) are allowed to occur at the same memory addresses
in the various memory spaces of different I/O buses, but are not required to do
so (and are not shown as the same in the figure). Implementations are likely
have BPM&#x2019;n,m to TPM&#x2019;n,m at the same address range for all
&#x201C;n&#x201D; when the BPM&#x2019;n,m to TPM&#x2019;n,m ranges are below 4
GB.</para>
</listitem>
</itemizedlist>
<figure xml:id="dbdoclet.50569328_42040">
<title>Example Address Map: One PHB, Peripheral Memory and Peripheral
I/O Spaces below 4 GB</title>
<mediaobject>
<imageobject role="html">
<imagedata fileref="figures/PAPR-10.gif" format="GIF" scalefit="1"/>
</imageobject>
<imageobject role="fo">
<imagedata contentdepth="100%" fileref="figures/PAPR-10.gif" format="GIF" scalefit="1" width="100%"/>
</imageobject>
</mediaobject>
</figure>
<figure xml:id="dbdoclet.50569328_17694">
<title>Example Address Map: Four PHBs, all Peripheral Memory and
Peripheral I/O Spaces above 4GB</title>
<mediaobject>
<imageobject role="html">
<imagedata fileref="figures/PAPR-11.gif" format="GIF" scalefit="1"/>
</imageobject>
<imageobject role="fo">
<imagedata contentdepth="100%" fileref="figures/PAPR-11.gif" format="GIF" scalefit="1" width="100%"/>
</imageobject>
</mediaobject>
</figure>
</section>
</section>
</chapter>