Skip to content
Snippets Groups Projects
Commit aaf97ca0 authored by Dave Hudson's avatar Dave Hudson
Browse files

Initial commit

* First commit of the Shortcut Forwarding Engine

* New files, short form FAQ and kernel patches.
parents
No related branches found
No related tags found
No related merge requests found
# Ouptut files
*.o
*.s
README 0 → 100644
Shortcut Forwarding Engine
--------------------------
Welcome to "Shortcut" :-)
Here's a quick FAQ:
Q) What is Shortcut?
A) Shortcut is an in-Linux-kernel IP packet forwarding engine. It's designed
to offer very high speed IP packet forwarding based on IP connection tracking.
It's dramatically faster than the standard netfilter-based NAT forwarding path
but is designed to synchronise state back to netfilter/conntrack so that it
doesn't need to deal with all of the complexities of special cases.
Q) What versions of IP does it support?
A) The current version only supports IPv4 but will be extended to support IPv6 in
the future.
Q) What transport protocols does it support?
A) TCP and UDP. It also knows enough about ICMP to spot ICMP error messages
related to TCP and UDP and handle things accordingly.
Q) Is there a design spec for this software?
A) Not at the moment. I'll write one when I get more time. The code is
intended to be a good tutorial though - it's very heavily commented. If you
find yourself reading something and not understanding it then I take that to
mean I've probably not done a sufficently good job of explaining what it's
doing in the comments. Let me know - I will try to fix it :-)
Q) Why was it written?
A) It was written as a demonstration of what can be done to provide high
performance forwarding inside the kernel. There were two initial motivations:
1) To provide a platform to enable research into how QoS analysis systems can
offload work and avoid huge Linux overheads.
2) To provide a tool to investigate the behaviour of various processors, SoCs
and software sets so that we can characterize and design new network processor
SoCs.
Q) How much faster is it than the Linux kernel forwarding path?
A) At the time of pushing this to github it's been tested on a QCA AP135.
This has a Scorpion (QCA Scopion, not the QMC one :-)) SoC, QCA9550. The
SoC's processor is a MIPS74K running at 720 MHz and with a DDR2 memory
subsystem that offers a peak of 600 MT/s (16-bit transfers).
Running IPv4 NAT forwarding of UDP between the board's 2 GMAC ports and
using a SmartBits 200 as a traffic generator Linux is able to forward 70k PPS.
Once the SFE code is invoked this will increase to 350k PPS!
There's also a slightly hacky mode which causes SFE to bypass the Linux
bridge layer, but this isn't really ready for use because it doesn't have
sufficient MAC address checks or integration of statistics back to the
Ethernet bridge, but that runs at 436k PPS.
Q) Are there any diagnostics?
A) Yes, this is a research tool after all! There's a complex way to do this
that's more general purpose and a simple one - here's the simple one:
mknod /dev/sfe c 253 0
The file /dev/sfe is an XML-ish output and provides details of all the
network connections currently being offloaded. It also reports the numbers
of packets that took various "exception" paths within the code. In addition
it provides a summary of the number of connections, attempts to accelerate
connections, cancel accelerations, etc. It also reports the numbers of
packets that were forwarded and not forwarded by the engine and has some
stats on the effectiveness of the hashing algorithm it uses.
Q) How does the code interact with Linux?
A) There are two minor patches required to make this software run with
Linux. These are currently against a 3.3.8 kernel. The first adds a
hook to allow packets to be extracted out, while the second exposes a
state variable inside netfilter that's necessary to enable TCP sequence
and ACK checking within the offload path. Note that this specific
patch is against the QCA QSDK patched version of 3.3.8 - there's a
slightly braindead "performance" patch in that kernel, courtesy of the
OpenWrt community that makes the Linux forwarding path slightly faster
at the expense of losing functionality :-(
Once these are applied and the module is loaded then everything else
is automatic :-) The patches are in this git repo.
Q) Isn't that patch to dev.c a gross hack?
A) Yes it is and no, it's not thread safe. Fixing this is on my "to do"
list.
Q) Are any of the pieces reused from other projects?
A) Yes! Some of the forwarding concepts are reused from the Ubicom Network
Accelerator that morphed into part of the Akronite NSS. This code has all
been substantially changed though to accomodate Linux's needs.
There are also some pieces that I borrowed from the QCA "FastNAT" software
written by Xiaoping Fan <xfan@qca.qualcomm.com>. Xiaoping's code was the
first actual demonstration within QCA that this in-kernel concept could yield
signficant performance gains.
Enjoy!
Dave Hudson <dhudson@qti.qualcomm.com>
--- /home/dhudson/mips-orig/dev.c 2013-06-13 17:45:45.833036067 +0100
+++ dev.c 2013-04-16 14:56:31.188471976 +0100
@@ -3200,12 +3199,15 @@
ASSERT_RTNL();
RCU_INIT_POINTER(dev->rx_handler, NULL);
RCU_INIT_POINTER(dev->rx_handler_data, NULL);
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_unregister);
+int (*athrs_fast_nat_recv)(struct sk_buff *skb);
+EXPORT_SYMBOL_GPL(athrs_fast_nat_recv);
+
static int __netif_receive_skb(struct sk_buff *skb)
{
struct packet_type *ptype, *pt_prev;
rx_handler_func_t *rx_handler;
struct net_device *orig_dev;
struct net_device *null_or_dev;
@@ -3226,12 +3228,17 @@
orig_dev = skb->dev;
skb_reset_network_header(skb);
skb_reset_transport_header(skb);
skb_reset_mac_len(skb);
+ if (athrs_fast_nat_recv) {
+ if (athrs_fast_nat_recv(skb))
+ return NET_RX_SUCCESS;
+ }
+
pt_prev = NULL;
rcu_read_lock();
another_round:
--- /home/dhudson/mips-orig/nf_conntrack_proto_tcp.c 2013-05-07 21:32:57.153896922 +0100
+++ nf_conntrack_proto_tcp.c 2013-06-13 16:37:40.137102438 +0100
@@ -27,18 +27,20 @@
#include <net/netfilter/nf_conntrack_ecache.h>
#include <net/netfilter/nf_log.h>
#include <net/netfilter/ipv4/nf_conntrack_ipv4.h>
#include <net/netfilter/ipv6/nf_conntrack_ipv6.h>
/* Do not check the TCP window for incoming packets */
-static int nf_ct_tcp_no_window_check __read_mostly = 1;
+int nf_ct_tcp_no_window_check __read_mostly = 0;
+EXPORT_SYMBOL_GPL(nf_ct_tcp_no_window_check);
/* "Be conservative in what you do,
be liberal in what you accept from others."
If it's non-zero, we mark only out of window RST segments as INVALID. */
-static int nf_ct_tcp_be_liberal __read_mostly = 0;
+int nf_ct_tcp_be_liberal __read_mostly = 0;
+EXPORT_SYMBOL_GPL(nf_ct_tcp_be_liberal);
/* If it is set to zero, we disable picking up already established
connections. */
static int nf_ct_tcp_loose __read_mostly = 1;
/* Max number of the retransmitted packets without receiving an (acceptable)
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment