Things happen rarely if you’re in an application-level development and want to write endian-independant code.

Occasionally, you need to consider about endian in low-level view, but that might confusing and even erroneous if you don’t fully understand it.

Before we begin to talking about why endian matters, a brief introduction of endian is needed.

Endian for short, is the physical organization schema of the sequential stream. Note that it’s not exclusive only to computer science field.

Historically reasons, now there’re three endianness types mainly:

• Little endianness

• Big endianness

• Middle endianness(Mixed endianess, PDP endianness, rarely seen nowadays)

Now, assuming x is a 32-bit integer. Its value, say, 0x0a0b0c0d. Memory layouts of those three endianness can be:

Little endianness:

Big endianness:

Middle endianness:

Clearly, the big endianness follows the writing(reading) habit of right-handed people. Also the little endianness follows the habit of left-handed people.

As for middle endianness, it’s rarely used. It first appeared in PDP-11 machine, you might come across in some embedded devices hardwares like ARM.

So what’s the audience of endianness?

Endianness only makes sense when you’re breaking up a multi-byte quantity and are trying to store the bytes at consecutive memory locations. However, if you have a 32-bit register storing a 32-bit value, it makes no sense to talk about endianness. The register is neither big-endian nor little-endian; it’s just a register holding a 32-bit value. The rightmost bit is the least significant bit, and the leftmost bit is the most significant bit.

Some people classify a register as a big-endian, because it stores its most significant byte at the lowest memory address.

Quoted from Registers and endianness - Writing endian-independent code in C

So there’re serveral cases you need to consider about endianness:

• Reading/Writing integers from file between different platforms.

• Sending/Receiving integers between different networks.

Endianness doesn’t apply to everything, bit operations doesn’t depend on endianness.

The same time, C-style string also doesn’t depend on endianness(It’s byte-wise, so the results are same for different endianness)

First I’ll show you how position-dependent bit operations affects the result:

Now let’s look how reading/writing integers affects the result:

#include <stdio.h>
#include <string.h>
 
int main (void)
{
    FILE* fp;
 
    /* Our example data structure */
    struct {
        char one[4];
        int  two;
        char three[4];
    } data;
 
    /* Fill our structure with data */
    strcpy(data.one, "foo");
    data.two = 0x0a0b0c0d;
    strcpy(data.three, "bar");
 
    /* Write it to a file */
    fp = fopen("output", "wb");
    if (fp) {
        fwrite(&data, sizeof(data), 1, fp);
        fclose(fp);
    }

    return 0;
}

If you compile above code, contents of file output varies(Using hexdump utilities to show the result, with argument -C):

Little endianness:

00000000  66 6f 6f 00 0d 0c 0b 0a  62 61 72 00              |foo.....bar.|
0000000c

Big endianness:

00000000  66 6f 6f 00 0a 0b 0c 0d  62 61 72 00              |foo.....bar.|
0000000c

Middle endianness:

00000000  66 6f 6f 00 0b 0a 0d 0c  62 61 72 00              |foo.....bar.|
0000000c

The byte between offset 4 to 7 varies in different architectures. If you want to read/write the data.two in a correct way, you may need to add an extra field to indicate the endianness when writing to the file output, like:

struct {
    char endian;    /* Indicate what's endianness when writing to this struct on your machine */
    /* (data fields) ... */
} data;

Or, alternatively, convert the numeric value into its corresponding string representation.

You can see how module-init-tools/util.c:native_endianness() implment endianness check:

/*
 * Get CPU endianness. 0 = unknown, 1 = ELFDATA2LSB = little, 2 = ELFDATA2MSB = big
 */
int __attribute__ ((pure)) native_endianness()
{
    /* Encoding the endianness enums in a string and then reading that
     * string as a 32-bit int, returns the correct endianness automagically.
     */
    return (char) *((uint32_t*)("\1\0\0\2"));
}

It’s a concise solution indeed, the idea behind this routine is simple, making "\1\0\0\2" to a uint32_t * pointer and fetch its least significant byte. the result is the type of endianness, revealed in above table.

Also, ffe-0.3.7-1/src/endian.c:check_system_endianess() implement the check in a novel way:

#define F_UNKNOWN_ENDIAN 0
#define F_BIG_ENDIAN 1
#define F_LITTLE_ENDIAN 3

#define LONG_INT 0x0a0b0c0d

uint8_t be[4]={0x0a,0x0b,0x0c,0x0d};
uint8_t le[4]={0x0d,0x0c,0x0b,0x0a};
uint8_t pe[4]={0x0b,0x0a,0x0d,0x0c};

size_t target_size = 16;
uint8_t *target = NULL;

int
check_system_endianess()
{
    uint32_t l = LONG_INT;

    if(target == NULL) target = xmalloc(target_size); // conversion buffer is reserved with malloc in order to ensure proper aligment

    if(memcmp(&l,be,4) == 0) return F_BIG_ENDIAN;
    if(memcmp(&l,le,4) == 0) return F_LITTLE_ENDIAN;
    if(memcmp(&l,pe,4) == 0)
    {
        fprintf(stderr,"Pdp endianess is not supported");
    }
    return F_UNKNOWN_ENDIAN;
}

With above discussion, a more genetic check routine is given, it fixed the problem with native_endianness() and less tedious:

/*
 * A more generic endianness check based on module-init-tools/util.c
 * NOTE: Assuming `unsigned int *' sized 32-bit
 *
 * Return values(endianness types):
 *        0x00: little
 *        0x01: big
 *        0x02: middle
 *
 *        0xff: unknown?
 *        0x??: bug?
 */
int get_endian(void)
{
    unsigned int mask = 0x0102ff00;
    return (char) *((unsigned int *) &mask);
}

/*
 * Implement as a macro
 * Note that it used an unnamed array, which may have portability issues
 *
 * 0x01 - Little endian
 * 0x02 - Big endian
 * 0x?? - Unknown endian
 */
#define _ENDIAN (*((char *) ((int []) {0x02080401})))
#define IS_LENDIAN (_ENDIAN & 0x1)
#define IS_GENDIAN (_ENDIAN & 0x2)

Issues also happen in network layers when sending/receiving data between different machine architectures.

All of the protocol layers in the Transmission Control Protocol and the Internet Protocol (TCP/IP) suite are defined to be big-endian. Any 16-bit or 32-bit value within the various layer headers (such as an IP address, a packet length, or a checksum) must be sent and received with its most significant byte first.

Assuming A establish a TCP socket connection to B whose IP is 10.8.6.1. IPv4 uses a unique 32-bit integer to indicate each network host. The dotted IP can be converted in to decimal in the following formula:

a.b.c.d = a × 2563 + b × 2562 + c × 2561 + d

So, 10.8.6.1 can be converted to decimal number 168297985.

http://168297985 is a valid url, http://0x0a080601 is its hex form.

Suppose an 80x86-based machine made a connection to a SPARC-based machine over the internet. 80x86-based machine will convert 10.8.6.1 into its little-endianness form 0x0106080a and transmit byte-by-byte, e.g. 01, 06, 08, 0a.

The SPARC-based machine received the bytes in order of 01, 06, 08, 0a, construct it into a big-endianness integer 0x0106080a, which misinterpret it as 1.6.8.10.

Situations also applied to those middle-endianness machines.

If the stack runs on a little-endian processor, it has to reorder, at run time, the bytes of every multi-byte data field within the various headers of the layers. If the stack runs on a big-endian processor, there’s nothing to worry about. For the stack to be portable (so it runs on processors of both types), it has to decide whether or not to do this reordering, typically at compile time.

For other endianness types(like middle-endianness), the reorder also needed, we don’t care now, just follow the similiar tacts.

System(e.g. Linux, macOS, BSD, etc.), already provided those conversional functions:

#include <arpa/inet.h>

uint32_t htonl(uint32_t hostlong);
uint16_t htons(uint16_t hostshort);
uint32_t ntohl(uint32_t netlong);
uint16_t ntohs(uint16_t netshort);

For the real example, you can refer to the references at bottom.

Also, if you want to convert from host byte order to other byte orders, you should use the library functions.

The header file endian.h in Linux consist many byte order related functions you might use.

The conclusion is that endianness issues do not affect sequences that combined with single bytes(e.g. C-string), because byte is considered an atomic unit from a storage point of view. On the other hand, sequences based on multi-byte are affected by endianness and you need to take care while coding.

Finally, I’d like to confess that while writing this article, many sources quoted from Writing endian-independent code in C, further revision is needed.

What I am talking now is just tip of the iceberg, if you get interested in those topics, you definitely should read 19-th chapter Protability of Linux Kernel Development (3rd Edition) by Robert Love, it walk you through many architecture-related issues you may come across.

References

Writing endian-independent code in C

Endianness - Wikipedia

Endianness - ChessProgramming Wiki