diff options
Diffstat (limited to 'Documentation/PCI/endpoint')
| -rw-r--r-- | Documentation/PCI/endpoint/function/binding/pci-ntb.rst | 38 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/function/binding/pci-test.rst | 26 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/function/binding/pci-test.txt | 19 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/index.rst | 7 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-endpoint-cfs.rst | 26 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-endpoint.rst | 18 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-ntb-function.rst | 348 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-ntb-howto.rst | 158 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-vntb-function.rst | 129 | ||||
| -rw-r--r-- | Documentation/PCI/endpoint/pci-vntb-howto.rst | 164 |
10 files changed, 902 insertions, 31 deletions
diff --git a/Documentation/PCI/endpoint/function/binding/pci-ntb.rst b/Documentation/PCI/endpoint/function/binding/pci-ntb.rst new file mode 100644 index 000000000000..40253d3d5163 --- /dev/null +++ b/Documentation/PCI/endpoint/function/binding/pci-ntb.rst @@ -0,0 +1,38 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +PCI NTB Endpoint Function +========================== + +1) Create a subdirectory to pci_epf_ntb directory in configfs. + +Standard EPF Configurable Fields: + +================ =========================================================== +vendorid should be 0x104c +deviceid should be 0xb00d for TI's J721E SoC +revid don't care +progif_code don't care +subclass_code should be 0x00 +baseclass_code should be 0x5 +cache_line_size don't care +subsys_vendor_id don't care +subsys_id don't care +interrupt_pin don't care +msi_interrupts don't care +msix_interrupts don't care +================ =========================================================== + +2) Create a subdirectory to directory created in 1 + +NTB EPF specific configurable fields: + +================ =========================================================== +db_count Number of doorbells; default = 4 +mw1 size of memory window1 +mw2 size of memory window2 +mw3 size of memory window3 +mw4 size of memory window4 +num_mws Number of memory windows; max = 4 +spad_count Number of scratchpad registers; default = 64 +================ =========================================================== diff --git a/Documentation/PCI/endpoint/function/binding/pci-test.rst b/Documentation/PCI/endpoint/function/binding/pci-test.rst new file mode 100644 index 000000000000..57ee866fb165 --- /dev/null +++ b/Documentation/PCI/endpoint/function/binding/pci-test.rst @@ -0,0 +1,26 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +PCI Test Endpoint Function +========================== + +name: Should be "pci_epf_test" to bind to the pci_epf_test driver. + +Configurable Fields: + +================ =========================================================== +vendorid should be 0x104c +deviceid should be 0xb500 for DRA74x and 0xb501 for DRA72x +revid don't care +progif_code don't care +subclass_code don't care +baseclass_code should be 0xff +cache_line_size don't care +subsys_vendor_id don't care +subsys_id don't care +interrupt_pin Should be 1 - INTA, 2 - INTB, 3 - INTC, 4 -INTD +msi_interrupts Should be 1 to 32 depending on the number of MSI interrupts + to test +msix_interrupts Should be 1 to 2048 depending on the number of MSI-X + interrupts to test +================ =========================================================== diff --git a/Documentation/PCI/endpoint/function/binding/pci-test.txt b/Documentation/PCI/endpoint/function/binding/pci-test.txt deleted file mode 100644 index cd76ba47394b..000000000000 --- a/Documentation/PCI/endpoint/function/binding/pci-test.txt +++ /dev/null @@ -1,19 +0,0 @@ -PCI TEST ENDPOINT FUNCTION - -name: Should be "pci_epf_test" to bind to the pci_epf_test driver. - -Configurable Fields: -vendorid : should be 0x104c -deviceid : should be 0xb500 for DRA74x and 0xb501 for DRA72x -revid : don't care -progif_code : don't care -subclass_code : don't care -baseclass_code : should be 0xff -cache_line_size : don't care -subsys_vendor_id : don't care -subsys_id : don't care -interrupt_pin : Should be 1 - INTA, 2 - INTB, 3 - INTC, 4 -INTD -msi_interrupts : Should be 1 to 32 depending on the number of MSI interrupts - to test -msix_interrupts : Should be 1 to 2048 depending on the number of MSI-X - interrupts to test diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpoint/index.rst index d114ea74b444..4d2333e7ae06 100644 --- a/Documentation/PCI/endpoint/index.rst +++ b/Documentation/PCI/endpoint/index.rst @@ -11,3 +11,10 @@ PCI Endpoint Framework pci-endpoint-cfs pci-test-function pci-test-howto + pci-ntb-function + pci-ntb-howto + pci-vntb-function + pci-vntb-howto + + function/binding/pci-test + function/binding/pci-ntb diff --git a/Documentation/PCI/endpoint/pci-endpoint-cfs.rst b/Documentation/PCI/endpoint/pci-endpoint-cfs.rst index b6d39cdec56e..fb73345cfb8a 100644 --- a/Documentation/PCI/endpoint/pci-endpoint-cfs.rst +++ b/Documentation/PCI/endpoint/pci-endpoint-cfs.rst @@ -24,7 +24,7 @@ Directory Structure The pci_ep configfs has two directories at its root: controllers and functions. Every EPC device present in the system will have an entry in -the *controllers* directory and and every EPF driver present in the system +the *controllers* directory and every EPF driver present in the system will have an entry in the *functions* directory. :: @@ -43,6 +43,7 @@ entries corresponding to EPF driver will be created by the EPF core. .. <EPF Driver1>/ ... <EPF Device 11>/ ... <EPF Device 21>/ + ... <EPF Device 31>/ .. <EPF Driver2>/ ... <EPF Device 12>/ ... <EPF Device 22>/ @@ -68,6 +69,24 @@ created) ... subsys_vendor_id ... subsys_id ... interrupt_pin + ... <Symlink EPF Device 31>/ + ... primary/ + ... <Symlink EPC Device1>/ + ... secondary/ + ... <Symlink EPC Device2>/ + +If an EPF device has to be associated with 2 EPCs (like in the case of +Non-transparent bridge), symlink of endpoint controller connected to primary +interface should be added in 'primary' directory and symlink of endpoint +controller connected to secondary interface should be added in 'secondary' +directory. + +The <EPF Device> directory can have a list of symbolic links +(<Symlink EPF Device 31>) to other <EPF Device>. These symbolic links should +be created by the user to represent the virtual functions that are bound to +the physical function. In the above directory structure <EPF Device 11> is a +physical function and <EPF Device 31> is a virtual function. An EPF device once +it's linked to another EPF device, cannot be linked to a EPC device. EPC Device ========== @@ -88,7 +107,8 @@ entries corresponding to EPC device will be created by the EPC core. The <EPC Device> directory will have a list of symbolic links to <EPF Device>. These symbolic links should be created by the user to -represent the functions present in the endpoint device. +represent the functions present in the endpoint device. Only <EPF Device> +that represents a physical function can be linked to a EPC device. The <EPC Device> directory will also have a *start* field. Once "1" is written to this field, the endpoint device will be ready to @@ -115,4 +135,4 @@ all the EPF devices are created and linked with the EPC device. | interrupt_pin | function -[1] :doc:`pci-endpoint` +[1] Documentation/PCI/endpoint/pci-endpoint.rst diff --git a/Documentation/PCI/endpoint/pci-endpoint.rst b/Documentation/PCI/endpoint/pci-endpoint.rst index 0e2311b5617b..4f5622a65555 100644 --- a/Documentation/PCI/endpoint/pci-endpoint.rst +++ b/Documentation/PCI/endpoint/pci-endpoint.rst @@ -78,8 +78,8 @@ by the PCI controller driver. Cleanup the pci_epc_mem structure allocated during pci_epc_mem_init(). -APIs for the PCI Endpoint Function Driver -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +EPC APIs for the PCI Endpoint Function Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This section lists the APIs that the PCI Endpoint core provides to be used by the PCI endpoint function driver. @@ -117,8 +117,8 @@ by the PCI endpoint function driver. The PCI endpoint function driver should use pci_epc_mem_free_addr() to free the memory space allocated using pci_epc_mem_alloc_addr(). -Other APIs -~~~~~~~~~~ +Other EPC APIs +~~~~~~~~~~~~~~ There are other APIs provided by the EPC library. These are used for binding the EPF device with EPC device. pci-ep-cfs.c can be used as reference for @@ -160,8 +160,8 @@ PCI Endpoint Function(EPF) Library The EPF library provides APIs to be used by the function driver and the EPC library to provide endpoint mode functionality. -APIs for the PCI Endpoint Function Driver -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +EPF APIs for the PCI Endpoint Function Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This section lists the APIs that the PCI Endpoint core provides to be used by the PCI endpoint function driver. @@ -204,8 +204,8 @@ by the PCI endpoint controller library. The PCI endpoint controller library invokes pci_epf_linkup() when the EPC device has established the connection to the host. -Other APIs -~~~~~~~~~~ +Other EPF APIs +~~~~~~~~~~~~~~ There are other APIs provided by the EPF library. These are used to notify the function driver when the EPF device is bound to the EPC device. @@ -214,7 +214,7 @@ pci-ep-cfs.c can be used as reference for using these APIs. * pci_epf_create() Create a new PCI EPF device by passing the name of the PCI EPF device. - This name will be used to bind the the EPF device to a EPF driver. + This name will be used to bind the EPF device to a EPF driver. * pci_epf_destroy() diff --git a/Documentation/PCI/endpoint/pci-ntb-function.rst b/Documentation/PCI/endpoint/pci-ntb-function.rst new file mode 100644 index 000000000000..3b9d836a4924 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-ntb-function.rst @@ -0,0 +1,348 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +PCI NTB Function +================= + +:Author: Kishon Vijay Abraham I <kishon@ti.com> + +PCI Non-Transparent Bridges (NTB) allow two host systems to communicate +with each other by exposing each host as a device to the other host. +NTBs typically support the ability to generate interrupts on the remote +machine, expose memory ranges as BARs, and perform DMA. They also support +scratchpads, which are areas of memory within the NTB that are accessible +from both machines. + +PCI NTB Function allows two different systems (or hosts) to communicate +with each other by configuring the endpoint instances in such a way that +transactions from one system are routed to the other system. + +In the below diagram, PCI NTB function configures the SoC with multiple +PCI Endpoint (EP) instances in such a way that transactions from one EP +controller are routed to the other EP controller. Once PCI NTB function +configures the SoC with multiple EP instances, HOST1 and HOST2 can +communicate with each other using SoC as a bridge. + +.. code-block:: text + + +-------------+ +-------------+ + | | | | + | HOST1 | | HOST2 | + | | | | + +------^------+ +------^------+ + | | + | | + +---------|-------------------------------------------------|---------+ + | +------v------+ +------v------+ | + | | | | | | + | | EP | | EP | | + | | CONTROLLER1 | | CONTROLLER2 | | + | | <-----------------------------------> | | + | | | | | | + | | | | | | + | | | SoC With Multiple EP Instances | | | + | | | (Configured using NTB Function) | | | + | +-------------+ +-------------+ | + +---------------------------------------------------------------------+ + +Constructs used for Implementing NTB +==================================== + + 1) Config Region + 2) Self Scratchpad Registers + 3) Peer Scratchpad Registers + 4) Doorbell (DB) Registers + 5) Memory Window (MW) + + +Config Region: +-------------- + +Config Region is a construct that is specific to NTB implemented using NTB +Endpoint Function Driver. The host and endpoint side NTB function driver will +exchange information with each other using this region. Config Region has +Control/Status Registers for configuring the Endpoint Controller. Host can +write into this region for configuring the outbound Address Translation Unit +(ATU) and to indicate the link status. Endpoint can indicate the status of +commands issued by host in this region. Endpoint can also indicate the +scratchpad offset and number of memory windows to the host using this region. + +The format of Config Region is given below. All the fields here are 32 bits. + +.. code-block:: text + + +------------------------+ + | COMMAND | + +------------------------+ + | ARGUMENT | + +------------------------+ + | STATUS | + +------------------------+ + | TOPOLOGY | + +------------------------+ + | ADDRESS (LOWER 32) | + +------------------------+ + | ADDRESS (UPPER 32) | + +------------------------+ + | SIZE | + +------------------------+ + | NO OF MEMORY WINDOW | + +------------------------+ + | MEMORY WINDOW1 OFFSET | + +------------------------+ + | SPAD OFFSET | + +------------------------+ + | SPAD COUNT | + +------------------------+ + | DB ENTRY SIZE | + +------------------------+ + | DB DATA | + +------------------------+ + | : | + +------------------------+ + | : | + +------------------------+ + | DB DATA | + +------------------------+ + + + COMMAND: + + NTB function supports three commands: + + CMD_CONFIGURE_DOORBELL (0x1): Command to configure doorbell. Before + invoking this command, the host should allocate and initialize + MSI/MSI-X vectors (i.e., initialize the MSI/MSI-X Capability in the + Endpoint). The endpoint on receiving this command will configure + the outbound ATU such that transactions to Doorbell BAR will be routed + to the MSI/MSI-X address programmed by the host. The ARGUMENT + register should be populated with number of DBs to configure (in the + lower 16 bits) and if MSI or MSI-X should be configured (BIT 16). + + CMD_CONFIGURE_MW (0x2): Command to configure memory window (MW). The + host invokes this command after allocating a buffer that can be + accessed by remote host. The allocated address should be programmed + in the ADDRESS register (64 bit), the size should be programmed in + the SIZE register and the memory window index should be programmed + in the ARGUMENT register. The endpoint on receiving this command + will configure the outbound ATU such that transactions to MW BAR + are routed to the address provided by the host. + + CMD_LINK_UP (0x3): Command to indicate an NTB application is + bound to the EP device on the host side. Once the endpoint + receives this command from both the hosts, the endpoint will + raise a LINK_UP event to both the hosts to indicate the host + NTB applications can start communicating with each other. + + ARGUMENT: + + The value of this register is based on the commands issued in + command register. See COMMAND section for more information. + + TOPOLOGY: + + Set to NTB_TOPO_B2B_USD for Primary interface + Set to NTB_TOPO_B2B_DSD for Secondary interface + + ADDRESS/SIZE: + + Address and Size to be used while configuring the memory window. + See "CMD_CONFIGURE_MW" for more info. + + MEMORY WINDOW1 OFFSET: + + Memory Window 1 and Doorbell registers are packed together in the + same BAR. The initial portion of the region will have doorbell + registers and the latter portion of the region is for memory window 1. + This register will specify the offset of the memory window 1. + + NO OF MEMORY WINDOW: + + Specifies the number of memory windows supported by the NTB device. + + SPAD OFFSET: + + Self scratchpad region and config region are packed together in the + same BAR. The initial portion of the region will have config region + and the latter portion of the region is for self scratchpad. This + register will specify the offset of the self scratchpad registers. + + SPAD COUNT: + + Specifies the number of scratchpad registers supported by the NTB + device. + + DB ENTRY SIZE: + + Used to determine the offset within the DB BAR that should be written + in order to raise doorbell. EPF NTB can use either MSI or MSI-X to + ring doorbell (MSI-X support will be added later). MSI uses same + address for all the interrupts and MSI-X can provide different + addresses for different interrupts. The MSI/MSI-X address is provided + by the host and the address it gives is based on the MSI/MSI-X + implementation supported by the host. For instance, ARM platform + using GIC ITS will have the same MSI-X address for all the interrupts. + In order to support all the combinations and use the same mechanism + for both MSI and MSI-X, EPF NTB allocates a separate region in the + Outbound Address Space for each of the interrupts. This region will + be mapped to the MSI/MSI-X address provided by the host. If a host + provides the same address for all the interrupts, all the regions + will be translated to the same address. If a host provides different + addresses, the regions will be translated to different addresses. This + will ensure there is no difference while raising the doorbell. + + DB DATA: + + EPF NTB supports 32 interrupts, so there are 32 DB DATA registers. + This holds the MSI/MSI-X data that has to be written to MSI address + for raising doorbell interrupt. This will be populated by EPF NTB + while invoking CMD_CONFIGURE_DOORBELL. + +Scratchpad Registers: +--------------------- + + Each host has its own register space allocated in the memory of NTB endpoint + controller. They are both readable and writable from both sides of the bridge. + They are used by applications built over NTB and can be used to pass control + and status information between both sides of a device. + + Scratchpad registers has 2 parts + 1) Self Scratchpad: Host's own register space + 2) Peer Scratchpad: Remote host's register space. + +Doorbell Registers: +------------------- + + Doorbell Registers are used by the hosts to interrupt each other. + +Memory Window: +-------------- + + Actual transfer of data between the two hosts will happen using the + memory window. + +Modeling Constructs: +==================== + +There are 5 or more distinct regions (config, self scratchpad, peer +scratchpad, doorbell, one or more memory windows) to be modeled to achieve +NTB functionality. At least one memory window is required while more than +one is permitted. All these regions should be mapped to BARs for hosts to +access these regions. + +If one 32-bit BAR is allocated for each of these regions, the scheme would +look like this: + +====== =============== +BAR NO CONSTRUCTS USED +====== =============== +BAR0 Config Region +BAR1 Self Scratchpad +BAR2 Peer Scratchpad +BAR3 Doorbell +BAR4 Memory Window 1 +BAR5 Memory Window 2 +====== =============== + +However if we allocate a separate BAR for each of the regions, there would not +be enough BARs for all the regions in a platform that supports only 64-bit +BARs. + +In order to be supported by most of the platforms, the regions should be +packed and mapped to BARs in a way that provides NTB functionality and +also makes sure the host doesn't access any region that it is not supposed +to. + +The following scheme is used in EPF NTB Function: + +====== =============================== +BAR NO CONSTRUCTS USED +====== =============================== +BAR0 Config Region + Self Scratchpad +BAR1 Peer Scratchpad +BAR2 Doorbell + Memory Window 1 +BAR3 Memory Window 2 +BAR4 Memory Window 3 +BAR5 Memory Window 4 +====== =============================== + +With this scheme, for the basic NTB functionality 3 BARs should be sufficient. + +Modeling Config/Scratchpad Region: +---------------------------------- + +.. code-block:: text + + +-----------------+------->+------------------+ +-----------------+ + | BAR0 | | CONFIG REGION | | BAR0 | + +-----------------+----+ +------------------+<-------+-----------------+ + | BAR1 | | |SCRATCHPAD REGION | | BAR1 | + +-----------------+ +-->+------------------+<-------+-----------------+ + | BAR2 | Local Memory | BAR2 | + +-----------------+ +-----------------+ + | BAR3 | | BAR3 | + +-----------------+ +-----------------+ + | BAR4 | | BAR4 | + +-----------------+ +-----------------+ + | BAR5 | | BAR5 | + +-----------------+ +-----------------+ + EP CONTROLLER 1 EP CONTROLLER 2 + +Above diagram shows Config region + Scratchpad region for HOST1 (connected to +EP controller 1) allocated in local memory. The HOST1 can access the config +region and scratchpad region (self scratchpad) using BAR0 of EP controller 1. +The peer host (HOST2 connected to EP controller 2) can also access this +scratchpad region (peer scratchpad) using BAR1 of EP controller 2. This +diagram shows the case where Config region and Scratchpad regions are allocated +for HOST1, however the same is applicable for HOST2. + +Modeling Doorbell/Memory Window 1: +---------------------------------- + +.. code-block:: text + + +-----------------+ +----->+----------------+-----------+-----------------+ + | BAR0 | | | Doorbell 1 +-----------> MSI-X ADDRESS 1 | + +-----------------+ | +----------------+ +-----------------+ + | BAR1 | | | Doorbell 2 +---------+ | | + +-----------------+----+ +----------------+ | | | + | BAR2 | | Doorbell 3 +-------+ | +-----------------+ + +-----------------+----+ +----------------+ | +-> MSI-X ADDRESS 2 | + | BAR3 | | | Doorbell 4 +-----+ | +-----------------+ + +-----------------+ | |----------------+ | | | | + | BAR4 | | | | | | +-----------------+ + +-----------------+ | | MW1 +---+ | +-->+ MSI-X ADDRESS 3|| + | BAR5 | | | | | | +-----------------+ + +-----------------+ +----->-----------------+ | | | | + EP CONTROLLER 1 | | | | +-----------------+ + | | | +---->+ MSI-X ADDRESS 4 | + +----------------+ | +-----------------+ + EP CONTROLLER 2 | | | + (OB SPACE) | | | + +-------> MW1 | + | | + | | + +-----------------+ + | | + | | + | | + | | + | | + +-----------------+ + PCI Address Space + (Managed by HOST2) + +Above diagram shows how the doorbell and memory window 1 is mapped so that +HOST1 can raise doorbell interrupt on HOST2 and also how HOST1 can access +buffers exposed by HOST2 using memory window1 (MW1). Here doorbell and +memory window 1 regions are allocated in EP controller 2 outbound (OB) address +space. Allocating and configuring BARs for doorbell and memory window1 +is done during the initialization phase of NTB endpoint function driver. +Mapping from EP controller 2 OB space to PCI address space is done when HOST2 +sends CMD_CONFIGURE_MW/CMD_CONFIGURE_DOORBELL. + +Modeling Optional Memory Windows: +--------------------------------- + +This is modeled the same was as MW1 but each of the additional memory windows +is mapped to separate BARs. diff --git a/Documentation/PCI/endpoint/pci-ntb-howto.rst b/Documentation/PCI/endpoint/pci-ntb-howto.rst new file mode 100644 index 000000000000..4261e7157ef1 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-ntb-howto.rst @@ -0,0 +1,158 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================================================== +PCI Non-Transparent Bridge (NTB) Endpoint Function (EPF) User Guide +=================================================================== + +:Author: Kishon Vijay Abraham I <kishon@ti.com> + +This document is a guide to help users use pci-epf-ntb function driver +and ntb_hw_epf host driver for NTB functionality. The list of steps to +be followed in the host side and EP side is given below. For the hardware +configuration and internals of NTB using configurable endpoints see +Documentation/PCI/endpoint/pci-ntb-function.rst + +Endpoint Device +=============== + +Endpoint Controller Devices +--------------------------- + +For implementing NTB functionality at least two endpoint controller devices +are required. + +To find the list of endpoint controller devices in the system:: + + # ls /sys/class/pci_epc/ + 2900000.pcie-ep 2910000.pcie-ep + +If PCI_ENDPOINT_CONFIGFS is enabled:: + + # ls /sys/kernel/config/pci_ep/controllers + 2900000.pcie-ep 2910000.pcie-ep + + +Endpoint Function Drivers +------------------------- + +To find the list of endpoint function drivers in the system:: + + # ls /sys/bus/pci-epf/drivers + pci_epf_ntb pci_epf_ntb + +If PCI_ENDPOINT_CONFIGFS is enabled:: + + # ls /sys/kernel/config/pci_ep/functions + pci_epf_ntb pci_epf_ntb + + +Creating pci-epf-ntb Device +---------------------------- + +PCI endpoint function device can be created using the configfs. To create +pci-epf-ntb device, the following commands can be used:: + + # mount -t configfs none /sys/kernel/config + # cd /sys/kernel/config/pci_ep/ + # mkdir functions/pci_epf_ntb/func1 + +The "mkdir func1" above creates the pci-epf-ntb function device that will +be probed by pci_epf_ntb driver. + +The PCI endpoint framework populates the directory with the following +configurable fields:: + + # ls functions/pci_epf_ntb/func1 + baseclass_code deviceid msi_interrupts pci-epf-ntb.0 + progif_code secondary subsys_id vendorid + cache_line_size interrupt_pin msix_interrupts primary + revid subclass_code subsys_vendor_id + +The PCI endpoint function driver populates these entries with default values +when the device is bound to the driver. The pci-epf-ntb driver populates +vendorid with 0xffff and interrupt_pin with 0x0001:: + + # cat functions/pci_epf_ntb/func1/vendorid + 0xffff + # cat functions/pci_epf_ntb/func1/interrupt_pin + 0x0001 + + +Configuring pci-epf-ntb Device +------------------------------- + +The user can configure the pci-epf-ntb device using its configfs entry. In order +to change the vendorid and the deviceid, the following +commands can be used:: + + # echo 0x104c > functions/pci_epf_ntb/func1/vendorid + # echo 0xb00d > functions/pci_epf_ntb/func1/deviceid + +The PCI endpoint framework also automatically creates a sub-directory in the +function attribute directory. This sub-directory has the same name as the name +of the function device and is populated with the following NTB specific +attributes that can be configured by the user:: + + # ls functions/pci_epf_ntb/func1/pci_epf_ntb.0/ + db_count mw1 mw2 mw3 mw4 num_mws + spad_count + +A sample configuration for NTB function is given below:: + + # echo 4 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/db_count + # echo 128 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/spad_count + # echo 2 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/num_mws + # echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw1 + # echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw2 + +Binding pci-epf-ntb Device to EP Controller +-------------------------------------------- + +NTB function device should be attached to two PCI endpoint controllers +connected to the two hosts. Use the 'primary' and 'secondary' entries +inside NTB function device to attach one PCI endpoint controller to +primary interface and the other PCI endpoint controller to the secondary +interface:: + + # ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary + # ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary + +Once the above step is completed, both the PCI endpoint controllers are ready to +establish a link with the host. + + +Start the Link +-------------- + +In order for the endpoint device to establish a link with the host, the _start_ +field should be populated with '1'. For NTB, both the PCI endpoint controllers +should establish link with the host:: + + # echo 1 > controllers/2900000.pcie-ep/start + # echo 1 > controllers/2910000.pcie-ep/start + + +RootComplex Device +================== + +lspci Output +------------ + +Note that the devices listed here correspond to the values populated in +"Creating pci-epf-ntb Device" section above:: + + # lspci + 0000:00:00.0 PCI bridge: Texas Instruments Device b00d + 0000:01:00.0 RAM memory: Texas Instruments Device b00d + + +Using ntb_hw_epf Device +----------------------- + +The host side software follows the standard NTB software architecture in Linux. +All the existing client side NTB utilities like NTB Transport Client and NTB +Netdev, NTB Ping Pong Test Client and NTB Tool Test Client can be used with NTB +function device. + +For more information on NTB see +:doc:`Non-Transparent Bridge <../../driver-api/ntb>` diff --git a/Documentation/PCI/endpoint/pci-vntb-function.rst b/Documentation/PCI/endpoint/pci-vntb-function.rst new file mode 100644 index 000000000000..0c51f53ab972 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-vntb-function.rst @@ -0,0 +1,129 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +PCI vNTB Function +================= + +:Author: Frank Li <Frank.Li@nxp.com> + +The difference between PCI NTB function and PCI vNTB function is + +PCI NTB function need at two endpoint instances and connect HOST1 +and HOST2. + +PCI vNTB function only use one host and one endpoint(EP), use NTB +connect EP and PCI host + +.. code-block:: text + + + +------------+ +---------------------------------------+ + | | | | + +------------+ | +--------------+ + | NTB | | | NTB | + | NetDev | | | NetDev | + +------------+ | +--------------+ + | NTB | | | NTB | + | Transfer | | | Transfer | + +------------+ | +--------------+ + | | | | | + | PCI NTB | | | | + | EPF | | | | + | Driver | | | PCI Virtual | + | | +---------------+ | NTB Driver | + | | | PCI EP NTB |<------>| | + | | | FN Driver | | | + +------------+ +---------------+ +--------------+ + | | | | | | + | PCI BUS | <-----> | PCI EP BUS | | Virtual PCI | + | | PCI | | | BUS | + +------------+ +---------------+--------+--------------+ + PCI RC PCI EP + +Constructs used for Implementing vNTB +===================================== + + 1) Config Region + 2) Self Scratchpad Registers + 3) Peer Scratchpad Registers + 4) Doorbell (DB) Registers + 5) Memory Window (MW) + + +Config Region: +-------------- + +It is same as PCI NTB Function driver + +Scratchpad Registers: +--------------------- + +It is appended after Config region. + +.. code-block:: text + + + +--------------------------------------------------+ Base + | | + | | + | | + | Common Config Register | + | | + | | + | | + +-----------------------+--------------------------+ Base + span_offset + | | | + | Peer Span Space | Span Space | + | | | + | | | + +-----------------------+--------------------------+ Base + span_offset + | | | + span_count * 4 + | | | + | Span Space | Peer Span Space | + | | | + +-----------------------+--------------------------+ + Virtual PCI Pcie Endpoint + NTB Driver NTB Driver + + +Doorbell Registers: +------------------- + + Doorbell Registers are used by the hosts to interrupt each other. + +Memory Window: +-------------- + + Actual transfer of data between the two hosts will happen using the + memory window. + +Modeling Constructs: +==================== + +32-bit BARs. + +====== =============== +BAR NO CONSTRUCTS USED +====== =============== +BAR0 Config Region +BAR1 Doorbell +BAR2 Memory Window 1 +BAR3 Memory Window 2 +BAR4 Memory Window 3 +BAR5 Memory Window 4 +====== =============== + +64-bit BARs. + +====== =============================== +BAR NO CONSTRUCTS USED +====== =============================== +BAR0 Config Region + Scratchpad +BAR1 +BAR2 Doorbell +BAR3 +BAR4 Memory Window 1 +BAR5 +====== =============================== + + diff --git a/Documentation/PCI/endpoint/pci-vntb-howto.rst b/Documentation/PCI/endpoint/pci-vntb-howto.rst new file mode 100644 index 000000000000..70d3bc90893f --- /dev/null +++ b/Documentation/PCI/endpoint/pci-vntb-howto.rst @@ -0,0 +1,164 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================================================== +PCI Non-Transparent Bridge (NTB) Endpoint Function (EPF) User Guide +=================================================================== + +:Author: Frank Li <Frank.Li@nxp.com> + +This document is a guide to help users use pci-epf-vntb function driver +and ntb_hw_epf host driver for NTB functionality. The list of steps to +be followed in the host side and EP side is given below. For the hardware +configuration and internals of NTB using configurable endpoints see +Documentation/PCI/endpoint/pci-vntb-function.rst + +Endpoint Device +=============== + +Endpoint Controller Devices +--------------------------- + +To find the list of endpoint controller devices in the system:: + + # ls /sys/class/pci_epc/ + 5f010000.pcie_ep + +If PCI_ENDPOINT_CONFIGFS is enabled:: + + # ls /sys/kernel/config/pci_ep/controllers + 5f010000.pcie_ep + +Endpoint Function Drivers +------------------------- + +To find the list of endpoint function drivers in the system:: + + # ls /sys/bus/pci-epf/drivers + pci_epf_ntb pci_epf_test pci_epf_vntb + +If PCI_ENDPOINT_CONFIGFS is enabled:: + + # ls /sys/kernel/config/pci_ep/functions + pci_epf_ntb pci_epf_test pci_epf_vntb + + +Creating pci-epf-vntb Device +---------------------------- + +PCI endpoint function device can be created using the configfs. To create +pci-epf-vntb device, the following commands can be used:: + + # mount -t configfs none /sys/kernel/config + # cd /sys/kernel/config/pci_ep/ + # mkdir functions/pci_epf_vntb/func1 + +The "mkdir func1" above creates the pci-epf-ntb function device that will +be probed by pci_epf_vntb driver. + +The PCI endpoint framework populates the directory with the following +configurable fields:: + + # ls functions/pci_epf_ntb/func1 + baseclass_code deviceid msi_interrupts pci-epf-ntb.0 + progif_code secondary subsys_id vendorid + cache_line_size interrupt_pin msix_interrupts primary + revid subclass_code subsys_vendor_id + +The PCI endpoint function driver populates these entries with default values +when the device is bound to the driver. The pci-epf-vntb driver populates +vendorid with 0xffff and interrupt_pin with 0x0001:: + + # cat functions/pci_epf_vntb/func1/vendorid + 0xffff + # cat functions/pci_epf_vntb/func1/interrupt_pin + 0x0001 + + +Configuring pci-epf-vntb Device +------------------------------- + +The user can configure the pci-epf-vntb device using its configfs entry. In order +to change the vendorid and the deviceid, the following +commands can be used:: + + # echo 0x1957 > functions/pci_epf_vntb/func1/vendorid + # echo 0x0809 > functions/pci_epf_vntb/func1/deviceid + +The PCI endpoint framework also automatically creates a sub-directory in the +function attribute directory. This sub-directory has the same name as the name +of the function device and is populated with the following NTB specific +attributes that can be configured by the user:: + + # ls functions/pci_epf_vntb/func1/pci_epf_vntb.0/ + db_count mw1 mw2 mw3 mw4 num_mws + spad_count + +A sample configuration for NTB function is given below:: + + # echo 4 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/db_count + # echo 128 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/spad_count + # echo 1 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/num_mws + # echo 0x100000 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/mw1 + +A sample configuration for virtual NTB driver for virtual PCI bus:: + + # echo 0x1957 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_vid + # echo 0x080A > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_pid + # echo 0x10 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vbus_number + +Binding pci-epf-ntb Device to EP Controller +-------------------------------------------- + +NTB function device should be attached to PCI endpoint controllers +connected to the host. + + # ln -s controllers/5f010000.pcie_ep functions/pci-epf-ntb/func1/primary + +Once the above step is completed, the PCI endpoint controllers are ready to +establish a link with the host. + + +Start the Link +-------------- + +In order for the endpoint device to establish a link with the host, the _start_ +field should be populated with '1'. For NTB, both the PCI endpoint controllers +should establish link with the host (imx8 don't need this steps):: + + # echo 1 > controllers/5f010000.pcie_ep/start + +RootComplex Device +================== + +lspci Output at Host side +------------------------- + +Note that the devices listed here correspond to the values populated in +"Creating pci-epf-ntb Device" section above:: + + # lspci + 00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0000 (rev 01) + 01:00.0 RAM memory: Freescale Semiconductor Inc Device 0809 + +Endpoint Device / Virtual PCI bus +================================= + +lspci Output at EP Side / Virtual PCI bus +----------------------------------------- + +Note that the devices listed here correspond to the values populated in +"Creating pci-epf-ntb Device" section above:: + + # lspci + 10:00.0 Unassigned class [ffff]: Dawicontrol Computersysteme GmbH Device 1234 (rev ff) + +Using ntb_hw_epf Device +----------------------- + +The host side software follows the standard NTB software architecture in Linux. +All the existing client side NTB utilities like NTB Transport Client and NTB +Netdev, NTB Ping Pong Test Client and NTB Tool Test Client can be used with NTB +function device. + +For more information on NTB see +:doc:`Non-Transparent Bridge <../../driver-api/ntb>` |