Loading...
 

ODP-MUSDK User Guide

 Note

The source code branches xxx-17.04 are based on Marvell engineering drop, minimum tests are performed on MACCHIATObin board with this version of software. Please proceed carefully.

Introduction

The ODP project has been established to produce an open-source, cross-platform set of application programming interfaces (APIs) for the networking data plane.

The purpose of this document is to describe the steps required for enabling OpenDataPlane API over MUSDK framework on Marvell's SoC A7040/8040, using Marvell's User Space SDK and network driver. The original ODP is available at http://www.opendataplane.org/

ODP framework was not developed by Marvell. All related code is provided as is, without any liability on the part of Marvell.

Marvell ODP-MUSDK implementation for the Armada family drivers is based on the standard ODP linux-generic platform with some changes that were done to best match the specific MUSDK architecture. Please refer to MUSDK documentation for more information on Marvell User Space SDK.

Features

Buffer Headroom

Marvell PPv2.2 HW requires every buffer to contain at least 32 bytes plus 2 extra bytes headroom. On the other hand, ODP also required some headroom (66Bytes by default) for every first packet buffer. Therefore, in ODP-MUSDK implementation, is not a real limitation.

PPv2.2 MUSDK implementation reserves 130Bytes headroom for every packet (defined as 'NET_SKB_PAD' + 'MV_ETH_MH_SIZE'). This allows to map MUSDK buffers directly to odp-packet structure and prevent from copy of the buffers.

Pad and CRC Offload

Marvell PPv2.2 HW supports Pad and CRC offload. This feature is enabled by default in in Marvell's ODP-MUSDK and it cannot be changed.

Checksums Offload

Marvell PPv2.2 HW supports L3 and L4 checksum offload. Marvell's ODP-MUSDK driver enables this feature. To use feature this feature, the appropriate 'odp_packet_header' fields should be set correctly by an ODP application (fields are: 'l3_offset', 'l4_offset', 'input_flags.ipv4'/'input_flags.ipv6' and 'input_flags.udp'/'input_flags.tcp').

Scatter-Gather Frame Support

Marvell PPv2.2 HW doesn't support S/G frames on the receive side but it does support it on transmit side. However, the MUSDK driver does not support S/G frames at all. Therefore, ODP-MUSDK implementation does not support this feature neither on receive side nor in transmit side.

MTU Support

ODP-MUSDK driver does not support MTU changes yet; ODP-MUSDK implementation supports only one size buffers of 2KB. MUSDK also does not support S/G frames, so in ODP-MUSDK implementation, it is possible to set MTU only up to 1800 bytes and it requires re-build of ODP-MUSDK.

CoS Support

Current Marvell PPv2.2 Linux driver supports CoS by using 'sysfs' mechanism to configure the CoS mapping. However, current MUSDK driver does not support CoS. Therefore, current ODP-MUSDK implementation does not allow CoS support (i.e. ODP l2-priority-mapping and l3-priority-mapping should not be used).

ODP Queues Mapping

The ODP supports multiple Rx and Tx queue per PacketIO. Usually, most of the odp example applications are allocating one Rx and one Tx queue per ODP thread/worker per port. The ODP-MUSDK implementation is spreading all the Rx and Tx PP2 queues equally among the Rx and Tx ODP queues.

NOTE: there should be a match between the MUSDK PP2 Rx/Tx queues and the ODP Rx/Tx queues. When using 4 ODP threads/workers that should handle single port, 4 Rx/Tx ODP queue will be allocated and 4 Rx/Tx PP2 queues should be initialized by settings the CoS to '1' and enable RSS to spread to 4 queues.

Installing ODP

Pre-requisites

To install the ODP over MUSDK, the following needs to be pre-installed:

  • MUSDK: For details on how to compile and install MUSDK, please refer the Getting Started with MUSDK
  • OpenSSL: OpenSSL must be installed in order to have crypto support in ODP. For further details on how to compile and install OpenSSL on MACCHIATObin, please refer to the OpenSSL Installation Guide.


And make sure you have exported the environment variable correctly on your build machine:

mcbin@buildserver:~$ export CROSS_COMPILE=aarch64-linux-gnu-
mcbin@buildserver:~$ export ARCH=arm64

 

ODP source files

ODP source code repository is available at https://github.com/MarvellEmbeddedProcessors/odp-marvell, on your Linux machine, make a directory to hold the source code and pull the repository to your machine.

mcbin@buildserver:~$ mkdir odp
mcbin@buildserver:~$ cd odp/
mcbin@buildserver:~/odp$ git clone https://github.com/MarvellEmbeddedProcessors/odp-marvell.git .


Then checkout to the branch odp-armada-17.04. This step can be skipped if the desired branch is the default branch to the repository, git clone will automatically checkout the default tree.

mcbin@buildserver:~/odp$ git checkout odp-armada-17.04
Branch odp-armada-17.04 set up to track remote branch odp-armada-17.04 from origin.
Switched to a new branch 'odp-armada-17.04'

 

Build process

To build the ODP package you need to configure the package firstly.

mcbin@buildserver:~/odp$ ./bootstrap
mcbin@buildserver:~/odp$ ./configure CC=${CROSS_COMPILE}gcc CXX=${CROSS_COMPILE}c++ --host=aarch64-linux-gnu --prefix=/home/mcbin/odp/install --with-openssl-path=/home/mcbin/openssl/install --with-platform=linux-musdk --with-sdk-install-path=/home/mcbin/musdk/usr/local --enable-mvpp2-support --enable-static --disable-shared


You'll need to provide the paths for OpenSSL and MUSDK correctly in the ODP configure stage, building ODP package will need them. Please make sure the configure process finishes without any errors. Next use the make command to build ODP binaries.

mcbin@buildserver:~/odp$ make
mcbin@buildserver:~/odp$ make install


Once the make install is done, the ODP binaries and libraries will be available in the ODP install directory. In the example instructions above, the ODP binaries will appear in the /home/mcbin/odp/install.

Install on board

Now copy all the ODP binaries install directories over to the root file system for MACCHIATObin. For the instructions on how to build Buildroot file system for MACCHIATObin, please refer to Build from source - Buildroot.

Running Application Examples

Before running any ODP application examples, please make sure you have updated your root file system for MACCHIATObin with the ODP binaries you built on your Linux machine, including its dependencies such as MUSDK, OpenSSL libraries, etc.

First, bring up the Ethernet ports you are going to use for ODP applications.

# ifconfig eth0 up
# ifconfig eth0 promisc
# ifconfig eth1 up
# ifconfig eth1 promisc


Next, insert the MUSDK kernel modules.

# insmod musdk_uio.ko
# insmod mv_pp_uio.ko


Now we can run the odp_bridge application, this application demonstrates usage and performance of the ODP-MUSDK infrastructure with the relevant network driver for receiving traffic on one network interface and sending it to another network interface with or without changes based on L2 forwarding.

To receive packets from interface eth0 and send them to interface eth1 using single-core, we can issue command below:

# odp_bridge -i eth0,eth1 -c 1


ODP framework supplies a Basic L3 forwarding application and is based on Destination IP forwarding. The application demonstrates usage and performance of the ODP-MUSDK infrastructure with the relevant network driver for receiving traffic on one network interface and sending it to another network interface based on L3 (IPv4) forwarding.

# odp_l3fwd -i eth0,eth1 -r 1.0.0.0/24,eth0 -r 2.0.0.0/24,eth1 -t 1


In the example above, eth0 will send packets to eth1 and vice versa.