Initial commit

* First commit of the Shortcut Forwarding Engine

* New files, short form FAQ and kernel patches.

.gitignore

+# Ouptut files
+
+*.o
+*.s
+

README

+Shortcut Forwarding Engine
+--------------------------
+
+Welcome to "Shortcut" :-)
+
+Here's a quick FAQ:
+
+
+Q) What is Shortcut?
+
+A) Shortcut is an in-Linux-kernel IP packet forwarding engine.  It's designed
+to offer very high speed IP packet forwarding based on IP connection tracking.
+It's dramatically faster than the standard netfilter-based NAT forwarding path
+but is designed to synchronise state back to netfilter/conntrack so that it
+doesn't need to deal with all of the complexities of special cases.
+
+
+Q) What versions of IP does it support?
+
+A) The current version only supports IPv4 but will be extended to support IPv6 in
+the future.
+
+
+Q) What transport protocols does it support?
+
+A) TCP and UDP.  It also knows enough about ICMP to spot ICMP error messages
+related to TCP and UDP and handle things accordingly.
+
+
+Q) Is there a design spec for this software?
+
+A) Not at the moment.  I'll write one when I get more time.  The code is
+intended to be a good tutorial though - it's very heavily commented.  If you
+find yourself reading something and not understanding it then I take that to
+mean I've probably not done a sufficently good job of explaining what it's
+doing in the comments.  Let me know - I will try to fix it :-)
+
+
+Q) Why was it written?
+
+A) It was written as a demonstration of what can be done to provide high
+performance forwarding inside the kernel.  There were two initial motivations:
+
+1) To provide a platform to enable research into how QoS analysis systems can
+offload work and avoid huge Linux overheads.
+
+2) To provide a tool to investigate the behaviour of various processors, SoCs
+and software sets so that we can characterize and design new network processor
+SoCs.
+
+
+Q) How much faster is it than the Linux kernel forwarding path?
+
+A) At the time of pushing this to github it's been tested on a QCA AP135.
+This has a Scorpion (QCA Scopion, not the QMC one :-)) SoC, QCA9550.  The
+SoC's processor is a MIPS74K running at 720 MHz and with a DDR2 memory
+subsystem that offers a peak of 600 MT/s (16-bit transfers).
+
+Running IPv4 NAT forwarding of UDP between the board's 2 GMAC ports and
+using a SmartBits 200 as a traffic generator Linux is able to forward 70k PPS.
+Once the SFE code is invoked this will increase to 350k PPS!
+
+There's also a slightly hacky mode which causes SFE to bypass the Linux
+bridge layer, but this isn't really ready for use because it doesn't have
+sufficient MAC address checks or integration of statistics back to the
+Ethernet bridge, but that runs at 436k PPS.
+
+
+Q) Are there any diagnostics?
+
+A) Yes, this is a research tool after all!  There's a complex way to do this
+that's more general purpose and a simple one - here's the simple one:
+
+	mknod /dev/sfe c 253 0
+
+The file /dev/sfe is an XML-ish output and provides details of all the
+network connections currently being offloaded.  It also reports the numbers
+of packets that took various "exception" paths within the code.  In addition
+it provides a summary of the number of connections, attempts to accelerate
+connections, cancel accelerations, etc.  It also reports the numbers of
+packets that were forwarded and not forwarded by the engine and has some
+stats on the effectiveness of the hashing algorithm it uses.
+
+
+Q) How does the code interact with Linux?
+
+A) There are two minor patches required to make this software run with
+Linux.  These are currently against a 3.3.8 kernel.  The first adds a
+hook to allow packets to be extracted out, while the second exposes a
+state variable inside netfilter that's necessary to enable TCP sequence
+and ACK checking within the offload path.  Note that this specific
+patch is against the QCA QSDK patched version of 3.3.8 - there's a
+slightly braindead "performance" patch in that kernel, courtesy of the
+OpenWrt community that makes the Linux forwarding path slightly faster
+at the expense of losing functionality :-(
+
+Once these are applied and the module is loaded then everything else
+is automatic :-)  The patches are in this git repo.
+
+
+Q) Isn't that patch to dev.c a gross hack?
+
+A) Yes it is and no, it's not thread safe.  Fixing this is on my "to do"
+list.
+
+
+Q) Are any of the pieces reused from other projects?
+
+A) Yes!  Some of the forwarding concepts are reused from the Ubicom Network
+Accelerator that morphed into part of the Akronite NSS.  This code has all
+been substantially changed though to accomodate Linux's needs.
+
+There are also some pieces that I borrowed from the QCA "FastNAT" software
+written by Xiaoping Fan <xfan@qca.qualcomm.com>.  Xiaoping's code was the
+first actual demonstration within QCA that this in-kernel concept could yield
+signficant performance gains.
+
+
+Enjoy!
+Dave Hudson <dhudson@qti.qualcomm.com>
+

dev.c.patch

+--- /home/dhudson/mips-orig/dev.c	2013-06-13 17:45:45.833036067 +0100
++++ dev.c	2013-04-16 14:56:31.188471976 +0100
+@@ -3200,12 +3199,15 @@
+ 	ASSERT_RTNL();
+ 	RCU_INIT_POINTER(dev->rx_handler, NULL);
+ 	RCU_INIT_POINTER(dev->rx_handler_data, NULL);
+ }
+ EXPORT_SYMBOL_GPL(netdev_rx_handler_unregister);
+ 
++int (*athrs_fast_nat_recv)(struct sk_buff *skb);
++EXPORT_SYMBOL_GPL(athrs_fast_nat_recv);
++
+ static int __netif_receive_skb(struct sk_buff *skb)
+ {
+ 	struct packet_type *ptype, *pt_prev;
+ 	rx_handler_func_t *rx_handler;
+ 	struct net_device *orig_dev;
+ 	struct net_device *null_or_dev;
+@@ -3226,12 +3228,17 @@
+ 	orig_dev = skb->dev;
+ 
+ 	skb_reset_network_header(skb);
+ 	skb_reset_transport_header(skb);
+ 	skb_reset_mac_len(skb);
+ 
++	if (athrs_fast_nat_recv) {
++		if (athrs_fast_nat_recv(skb))
++			return NET_RX_SUCCESS;
++	}
++
+ 	pt_prev = NULL;
+ 
+ 	rcu_read_lock();
+ 
+ another_round:
+ 

nf_conntrack_proto_tcp.c.patch

+--- /home/dhudson/mips-orig/nf_conntrack_proto_tcp.c	2013-05-07 21:32:57.153896922 +0100
++++ nf_conntrack_proto_tcp.c	2013-06-13 16:37:40.137102438 +0100
+@@ -27,18 +27,20 @@
+ #include <net/netfilter/nf_conntrack_ecache.h>
+ #include <net/netfilter/nf_log.h>
+ #include <net/netfilter/ipv4/nf_conntrack_ipv4.h>
+ #include <net/netfilter/ipv6/nf_conntrack_ipv6.h>
+ 
+ /* Do not check the TCP window for incoming packets  */
+-static int nf_ct_tcp_no_window_check __read_mostly = 1;
++int nf_ct_tcp_no_window_check __read_mostly = 0;
++EXPORT_SYMBOL_GPL(nf_ct_tcp_no_window_check);
+ 
+ /* "Be conservative in what you do,
+     be liberal in what you accept from others."
+     If it's non-zero, we mark only out of window RST segments as INVALID. */
+-static int nf_ct_tcp_be_liberal __read_mostly = 0;
++int nf_ct_tcp_be_liberal __read_mostly = 0;
++EXPORT_SYMBOL_GPL(nf_ct_tcp_be_liberal);
+ 
+ /* If it is set to zero, we disable picking up already established
+    connections. */
+ static int nf_ct_tcp_loose __read_mostly = 1;
+ 
+ /* Max number of the retransmitted packets without receiving an (acceptable)