Alignment of IP4 header struct in C++ with ASAN

[bryan-kressler-aq]

I'm building and running a debug version of a project using ef_vi in ASAN, and am running into problems with alignment issues.

Packet buffers should be page (i.e. 64 byte) aligned. I'm working with an ethernet header that includes a vlan tag, so sizeof(ci_ethhdr_vlan_t) == 18 (though the same problem arises with ci_ether_hdr, which is 14 bytes). If I place one of these at the beginning of the packet buffer, then the IPv4 header that should immediately follow (ci_ip4_hdr) ends up 2 byte aligned. The C++ compiler thinks that this struct should be 4 byte aligned: https://godbolt.org/z/vzvje4ezP, which causes failures running under ASAN with errors like:

runtime error: member access within misaligned address 0x7ffe73d04052 for type 'struct ci_ip4_hdr_s', which requires 4 byte alignment
0x7ffe73d04052: note: pointer points here
 08 9a  08 00 45 00 00 00 00 00  00 00 40 11 00 00 de ad  be ef fe ed fa ce f0 01  13 37 00 08 00 00

The alignment requirements of C also appear to be the same: https://godbolt.org/z/6P8oTbd73

Note that adding a [[gnu::packed]] annotation to the struct resolves the issue: https://godbolt.org/z/P4aj5Ej7W.

Should these structs be specified as packed to resolve these alignment issues?

[rhughes-xilinx]

You can specify the structs as packed - that wouldn't be wrong, and x86 CPUs are mostly efficient at unaligned I/O.

What's generally done, however, is to leave the first 2 bytes of the packet buffer unused, i.e. start the Ethernet header misaligned so that the IP header (and everything after it) becomes aligned. The reason for this is all in the "mostly" I used above: a single write which crosses a cache line is always going to create more work for the core and/or data fabric, so if it's easy to avoid it then that's better. ef_vi_transmit() accepts a start address by byte index, so it's happy to accept any alignment (within ef_memreg_alloc()ed memory and not crossing a page boundary).