M7350/shortcut-fe/README

Shortcut Forwarding Engine
--------------------------

Welcome to "Shortcut" :-)

Here's a quick FAQ:


Q) What is Shortcut?

A) Shortcut is an in-Linux-kernel IP packet forwarding engine.  It's designed
to offer very high speed IP packet forwarding based on IP connection tracking.
It's dramatically faster than the standard netfilter-based NAT forwarding path
but is designed to synchronise state back to netfilter/conntrack so that it
doesn't need to deal with all of the complexities of special cases.


Q) What versions of IP does it support?

A) The current version only supports IPv4 but will be extended to support IPv6 in
the future.


Q) What transport protocols does it support?

A) TCP and UDP.  It also knows enough about ICMP to spot ICMP error messages
related to TCP and UDP and handle things accordingly.


Q) Is there a design spec for this software?

A) Not at the moment.  I'll write one when I get more time.  The code is
intended to be a good tutorial though - it's very heavily commented.  If you
find yourself reading something and not understanding it then I take that to
mean I've probably not done a sufficently good job of explaining what it's
doing in the comments.  Let me know - I will try to fix it :-)


Q) Why was it written?

A) It was written as a demonstration of what can be done to provide high
performance forwarding inside the kernel.  There were two initial motivations:

1) To provide a platform to enable research into how QoS analysis systems can
offload work and avoid huge Linux overheads.

2) To provide a tool to investigate the behaviour of various processors, SoCs
and software sets so that we can characterize and design new network processor
SoCs.


Q) How much faster is it than the Linux kernel forwarding path?

A) At the time of pushing this to github it's been tested on a QCA AP135.
This has a Scorpion (QCA Scopion, not the QMC one :-)) SoC, QCA9550.  The
SoC's processor is a MIPS74K running at 720 MHz and with a DDR2 memory
subsystem that offers a peak of 600 MT/s (16-bit transfers).

Running IPv4 NAT forwarding of UDP between the board's 2 GMAC ports and
using a SmartBits 200 as a traffic generator Linux is able to forward 70k PPS.
Once the SFE code is invoked this will increase to 350k PPS!

There's also a slightly hacky mode which causes SFE to bypass the Linux
bridge layer, but this isn't really ready for use because it doesn't have
sufficient MAC address checks or integration of statistics back to the
Ethernet bridge, but that runs at 436k PPS.


Q) Are there any diagnostics?

A) Yes, this is a research tool after all!  There's a complex way to do this
that's more general purpose and a simple one - here's the simple one:

	mknod /dev/sfe c 253 0

The file /dev/sfe is an XML-ish output and provides details of all the
network connections currently being offloaded.  It also reports the numbers
of packets that took various "exception" paths within the code.  In addition
it provides a summary of the number of connections, attempts to accelerate
connections, cancel accelerations, etc.  It also reports the numbers of
packets that were forwarded and not forwarded by the engine and has some
stats on the effectiveness of the hashing algorithm it uses.


Q) How does the code interact with Linux?

A) There are four minor patches required to make this software run with
Linux.  These are currently against a 3.3.8 or 3.4.0 kernel:

* (net/core/dev.c) adds a hook to allow packets to be extracted out.

* (net/netfilter/nf_conntrack_proto_tcp.c) exposes a state variable inside
  netfilter that's necessary to enable TCP sequence and ACK checking within
  the offload path.  Note that this specific patch is against the QCA QSDK
  patched version of 3.3.8 - there's a slightly braindead "performance"
  patch in that kernel, courtesy of the OpenWrt community that makes the
  Linux forwarding path slightly faster at the expense of losing
  functionality :-(

* (net/Kconfig) adds the shortcut-fe option.

* (net/Makefile) adds the shortcut-fe build support.

Once these are applied and the module is loaded then everything else
is automatic :-)  The patches are in this git repo.


Q) Are any of the pieces reused from other projects?

A) Yes!  Some of the forwarding concepts are reused from the Ubicom Network
Accelerator that morphed into part of the Akronite NSS.  This code has all
been substantially changed though to accomodate Linux's needs.

There are also some pieces that I borrowed from the QCA "FastNAT" software
written by Xiaoping Fan <xfan@qca.qualcomm.com>.  Xiaoping's code was the
first actual demonstration within QCA that this in-kernel concept could yield
signficant performance gains.


Enjoy!
Dave Hudson <dhudson@qti.qualcomm.com>