123 lines
4.6 KiB
Plaintext
123 lines
4.6 KiB
Plaintext
Shortcut Forwarding Engine
|
|
--------------------------
|
|
|
|
Welcome to "Shortcut" :-)
|
|
|
|
Here's a quick FAQ:
|
|
|
|
|
|
Q) What is Shortcut?
|
|
|
|
A) Shortcut is an in-Linux-kernel IP packet forwarding engine. It's designed
|
|
to offer very high speed IP packet forwarding based on IP connection tracking.
|
|
It's dramatically faster than the standard netfilter-based NAT forwarding path
|
|
but is designed to synchronise state back to netfilter/conntrack so that it
|
|
doesn't need to deal with all of the complexities of special cases.
|
|
|
|
|
|
Q) What versions of IP does it support?
|
|
|
|
A) The current version only supports IPv4 but will be extended to support IPv6 in
|
|
the future.
|
|
|
|
|
|
Q) What transport protocols does it support?
|
|
|
|
A) TCP and UDP. It also knows enough about ICMP to spot ICMP error messages
|
|
related to TCP and UDP and handle things accordingly.
|
|
|
|
|
|
Q) Is there a design spec for this software?
|
|
|
|
A) Not at the moment. I'll write one when I get more time. The code is
|
|
intended to be a good tutorial though - it's very heavily commented. If you
|
|
find yourself reading something and not understanding it then I take that to
|
|
mean I've probably not done a sufficently good job of explaining what it's
|
|
doing in the comments. Let me know - I will try to fix it :-)
|
|
|
|
|
|
Q) Why was it written?
|
|
|
|
A) It was written as a demonstration of what can be done to provide high
|
|
performance forwarding inside the kernel. There were two initial motivations:
|
|
|
|
1) To provide a platform to enable research into how QoS analysis systems can
|
|
offload work and avoid huge Linux overheads.
|
|
|
|
2) To provide a tool to investigate the behaviour of various processors, SoCs
|
|
and software sets so that we can characterize and design new network processor
|
|
SoCs.
|
|
|
|
|
|
Q) How much faster is it than the Linux kernel forwarding path?
|
|
|
|
A) At the time of pushing this to github it's been tested on a QCA AP135.
|
|
This has a Scorpion (QCA Scopion, not the QMC one :-)) SoC, QCA9550. The
|
|
SoC's processor is a MIPS74K running at 720 MHz and with a DDR2 memory
|
|
subsystem that offers a peak of 600 MT/s (16-bit transfers).
|
|
|
|
Running IPv4 NAT forwarding of UDP between the board's 2 GMAC ports and
|
|
using a SmartBits 200 as a traffic generator Linux is able to forward 70k PPS.
|
|
Once the SFE code is invoked this will increase to 350k PPS!
|
|
|
|
There's also a slightly hacky mode which causes SFE to bypass the Linux
|
|
bridge layer, but this isn't really ready for use because it doesn't have
|
|
sufficient MAC address checks or integration of statistics back to the
|
|
Ethernet bridge, but that runs at 436k PPS.
|
|
|
|
|
|
Q) Are there any diagnostics?
|
|
|
|
A) Yes, this is a research tool after all! There's a complex way to do this
|
|
that's more general purpose and a simple one - here's the simple one:
|
|
|
|
mknod /dev/sfe c 253 0
|
|
|
|
The file /dev/sfe is an XML-ish output and provides details of all the
|
|
network connections currently being offloaded. It also reports the numbers
|
|
of packets that took various "exception" paths within the code. In addition
|
|
it provides a summary of the number of connections, attempts to accelerate
|
|
connections, cancel accelerations, etc. It also reports the numbers of
|
|
packets that were forwarded and not forwarded by the engine and has some
|
|
stats on the effectiveness of the hashing algorithm it uses.
|
|
|
|
|
|
Q) How does the code interact with Linux?
|
|
|
|
A) There are four minor patches required to make this software run with
|
|
Linux. These are currently against a 3.3.8 or 3.4.0 kernel:
|
|
|
|
* (net/core/dev.c) adds a hook to allow packets to be extracted out.
|
|
|
|
* (net/netfilter/nf_conntrack_proto_tcp.c) exposes a state variable inside
|
|
netfilter that's necessary to enable TCP sequence and ACK checking within
|
|
the offload path. Note that this specific patch is against the QCA QSDK
|
|
patched version of 3.3.8 - there's a slightly braindead "performance"
|
|
patch in that kernel, courtesy of the OpenWrt community that makes the
|
|
Linux forwarding path slightly faster at the expense of losing
|
|
functionality :-(
|
|
|
|
* (net/Kconfig) adds the shortcut-fe option.
|
|
|
|
* (net/Makefile) adds the shortcut-fe build support.
|
|
|
|
Once these are applied and the module is loaded then everything else
|
|
is automatic :-) The patches are in this git repo.
|
|
|
|
|
|
Q) Are any of the pieces reused from other projects?
|
|
|
|
A) Yes! Some of the forwarding concepts are reused from the Ubicom Network
|
|
Accelerator that morphed into part of the Akronite NSS. This code has all
|
|
been substantially changed though to accomodate Linux's needs.
|
|
|
|
There are also some pieces that I borrowed from the QCA "FastNAT" software
|
|
written by Xiaoping Fan <xfan@qca.qualcomm.com>. Xiaoping's code was the
|
|
first actual demonstration within QCA that this in-kernel concept could yield
|
|
signficant performance gains.
|
|
|
|
|
|
Enjoy!
|
|
Dave Hudson <dhudson@qti.qualcomm.com>
|
|
|