Real world use cases of bitwise operators

What are some real world use cases of the following bitwise operators?

• AND
• XOR
• NOT
• OR
• Left/Right shift

• Bit fields (flags)
  They're the most efficient way of representing something whose state is defined by several "yes or no" properties. ACLs are a good example; if you have let's say 4 discrete permissions (read, write, execute, change policy), it's better to store this in 1 byte rather than waste 4. These can be mapped to enumeration types in many languages for added convenience.

• Communication over ports/sockets
  Always involves checksums, parity, stop bits, flow control algorithms, and so on, which usually depend on the logic values of individual bytes as opposed to numeric values, since the medium may only be capable of transmitting one bit at a time.

• Compression, Encryption
  Both of these are heavily dependent on bitwise algorithms. Look at the deflate algorithm for an example - everything is in bits, not bytes.

• Finite State Machines
  I'm speaking primarily of the kind embedded in some piece of hardware, although they can be found in software too. These are combinatorial in nature - they might literally be getting "compiled" down to a bunch of logic gates, so they have to be expressed as AND, OR, NOT, etc.

• Graphics There's hardly enough space here to get into every area where these operators are used in graphics programming. XOR (or ^) is particularly interesting here because applying the same input a second time will undo the first. Older GUIs used to rely on this for selection highlighting and other overlays, in order to eliminate the need for costly redraws. They're still useful in slow graphics protocols (i.e. remote desktop).

Those were just the first few examples I came up with - this is hardly an exhaustive list.

Is it odd?

(value & 0x1) > 0

Is it divisible by two (even)?

(value & 0x1) == 0

I've used bitwise operations in implementing a security model for a CMS. It had pages which could be accessed by users if they were in appropriate groups. A user could be in multiple groups, so we needed to check if there was an intersection between the users groups and the pages groups. So we assigned each group a unique power-of-2 identifier, e.g.:

Group A = 1 --> 00000001
Group B = 2 --> 00000010
Group C = 3 --> 00000100

We OR these values together, and store the value (as a single int) with the page. E.g. if a page could be accessed by groups A & B, we store the value 3 (which in binary is 00000011) as the pages access control. In much the same way, we store a value of ORed group identifiers with a user to represent which groups they are in.

So to check if a given user can access a given page, you just need to AND the values together and check if the value is non-zero. This is very fast as this check is implemented in a single instruction, no looping, no database round-trips.

Here's some common idioms dealing with flags stored as individual bits.

enum CDRIndicators {
  Local = 1 << 0,
  External = 1 << 1,
  CallerIDMissing = 1 << 2,
  Chargeable = 1 << 3
};

unsigned int flags = 0;

Set the Chargeable flag:

flags |= Chargeable;

Clear CallerIDMissing flag:

flags &= ~CallerIDMissing;

Test whether CallerIDMissing and Chargeable are set:

if((flags & (CallerIDMissing | Chargeable )) == (CallerIDMissing | Chargeable)) {

}

Low-level programming is a good example. You may, for instance, need to write a specific bit to a memory-mapped register to make some piece of hardware do what you want it to:

volatile uint32_t *register = (volatile uint32_t *)0x87000000;
uint32_t          value;
uint32_t          set_bit   = 0x00010000;
uint32_t          clear_bit = 0x00001000;

value = *register;            // get current value from the register
value = value & ~clear_bit;   // clear a bit
value = value | set_bit;      // set a bit
*register = value;            // write it back to the register

Also, htonl() and htons() are implemented using the & and | operators (on machines whose endianness(Byte order) doesn't match network order):

#define htons(a) ((((a) & 0xff00) >> 8) | \
                  (((a) & 0x00ff) << 8))

#define htonl(a) ((((a) & 0xff000000) >> 24) | \
                  (((a) & 0x00ff0000) >>  8) | \
                  (((a) & 0x0000ff00) <<  8) | \
                  (((a) & 0x000000ff) << 24))

I use them to get RGB(A) values from packed colorvalues, for instance.

When I have a bunch of boolean flags, I like to store them all in an int.

I get them out using bitwise-AND. For example:

int flags;
if (flags & 0x10) {
  // Turn this feature on.
}

if (flags & 0x08) {
  // Turn a second feature on.
}

etc.

& = AND:
Mask out specific bits.
You are defining the specific bits which should be displayed or not displayed. 0x0 & x will clear all bits in a byte while 0xFF will not change x. 0x0F will display the bits in the lower nibble.

Conversion:
To cast shorter variables into longer ones with bit identity it is necessary to adjust the bits because -1 in an int is 0xFFFFFFFF while -1 in a long is 0xFFFFFFFFFFFFFFFF. To preserve the identity you apply a mask after conversion.

|=OR
Set bits. The bits will be set indepently if they are already set. Many datastructures (bitfields) have flags like IS_HSET = 0, IS_VSET = 1 which can be indepently set. To set the flags, you apply IS_HSET | IS_VSET (In C and assembly this is very convenient to read)

^=XOR
Find bits which are the same or different.

~= NOT
Flip bits.

It can be shown that all possible local bit operations can be implemented by these operations. So if you like you can implement an ADD instruction solely by bit operations.

Some wonderful hacks:

http://www.ugcs.caltech.edu/~wnoise/base2.html
http://www.jjj.de/bitwizardry/bitwizardrypage.html

Encryption is all bitwise operations.

You can use them as a quick and dirty way to hash data.

int a = 1230123;
int b = 1234555;
int c = 5865683;
int hash = a ^ b ^ c;

I just used bitwise-XOR (^) about three minutes ago to calculate a checksum for serial communication with a PLC...

This is an example to read colours from a bitmap image in byte format

byte imagePixel = 0xCCDDEE; /* Image in RRGGBB format R=Red, G=Green, B=Blue */

//To only have red
byte redColour = imagePixel & 0xFF0000; /*Bitmasking with AND operator */

//Now, we only want red colour
redColour = (redColour >> 24) & 0xFF;  /* This now returns a red colour between 0x00 and 0xFF.

I hope this tiny examples helps....

Bitwise & is used to mask/extract a certain part of a byte.

1 Byte variable

 01110010
&00001111 Bitmask of 0x0F to find out the lower nibble
 --------
 00000010

Specially the shift operator (<< >>) are often used for calculations.

In the abstracted world of today's modern language, not too many. File IO is an easy one that comes to mind, though that's exercising bitwise operations on something already implemented and is not implementing something that uses bitwise operations. Still, as an easy example, this code demonstrates removing the read-only attribute on a file (so that it can be used with a new FileStream specifying FileMode.Create) in c#:

//Hidden files posses some extra attibutes that make the FileStream throw an exception
//even with FileMode.Create (if exists -> overwrite) so delete it and don't worry about it!
if(File.Exists(targetName))
{
	FileAttributes attributes = File.GetAttributes(targetName);

	if ((attributes & FileAttributes.ReadOnly) == FileAttributes.ReadOnly)
		File.SetAttributes(targetName, attributes & (~FileAttributes.ReadOnly));

	File.Delete(targetName);
}

As far as custom implementations, here's a recent example: I created a "message center" for sending secure messages from one installation of our distributed application to another. Basically, it's analogous to email, complete with Inbox, Outbox, Sent, etc, but it also has guaranteed delivery with read receipts, so there are additional subfolders beyond "inbox" and "sent." What this amounted to was a requirement for me to define generically what's "in the inbox" or what's "in the sent folder". Of the sent folder, I need to know what's read and what's unread. Of what's unread, I need to know what's received and what's not received. I use this information to build a dynamic where clause which filters a local datasource and displays the appropriate information.

Here's how the enum is put together:

	public enum MemoView :int
	{
		InboundMemos = 1,					//     0000 0001
		InboundMemosForMyOrders = 3,		//     0000 0011
		SentMemosAll = 16,					//     0001 0000
		SentMemosNotReceived = 48,			//     0011
		SentMemosReceivedNotRead = 80,		//     0101
		SentMemosRead = 144,				//     1001
		Outbox = 272,						//0001 0001 0000
		OutBoxErrors = 784					//0011 0001 0000
	}

Do you see what this does? By anding (&) with the "Inbox" enum value, InboundMemos, I know that InboundMemosForMyOrders is in the inbox.

Here's a boiled down version of the method that builds and returns the filter that defines a view for the currently selected folder:

	private string GetFilterForView(MemoView view, DefaultableBoolean readOnly)
	{
		string filter = string.Empty;
		if((view & MemoView.InboundMemos) == MemoView.InboundMemos)
		{
			filter = "<inbox filter conditions>";

			if((view & MemoView.InboundMemosForMyOrders) == MemoView.InboundMemosForMyOrders)
			{
				filter += "<my memo filter conditions>";
			}
		}
		else if((view & MemoView.SentMemosAll) == MemoView.SentMemosAll)
		{
			//all sent items have originating system = to local
			filter = "<memos leaving current system>";

			if((view & MemoView.Outbox) == MemoView.Outbox)
			{
				...
			}
			else
			{
				//sent sub folders
				filter += "<all sent items>";

				if((view & MemoView.SentMemosNotReceived) == MemoView.SentMemosNotReceived)
				{
					if((view & MemoView.SentMemosReceivedNotRead) == MemoView.SentMemosReceivedNotRead)
					{
						filter += "<not received and not read conditions>";
					}
					else
						filter += "<received and not read conditions>";
				}
			}
		}

		return filter;
	}

Extremely simple, but a neat implementation at a level of abstraction that doesn't typically require bitwise operations.

Usually bitwise operations are faster than doing multiply/divide. So if you need to multiply a variable x by say 9, you will do x<<3 + x which would be a few cycles faster than x*9. If this code is inside an ISR, you will save on response time.

Similarly if you want to use an array as a circular queue, it'd be faster (and more elegant) to handle wrap around checks with bit wise operations. (your array size should be a power of 2). Eg: , you can use tail = ((tail & MASK) + 1) instead of tail = ((tail +1) < size) ? tail+1 : 0, if you want to insert/delete.

Also if you want a error flag to hold multiple error codes together, each bit can hold a separate value. You can AND it with each individual error code as a check. This is used in Unix error codes.

Also a n-bit bitmap can be a really cool and compact data structure. If you want to allocate a resource pool of size n, we can use a n-bits to represent the current status.

Base64 encoding is an example. Base64 encoding is used to represent binary data as a printable characters for sending over email systems (and other purposes). Base64 encoding converts a series of 8 bit bytes into 6 bit character lookup indexes. Bit operations, shifting, and'ing, or'ing, not'ing are very useful for implementing the bit operations necessary for Base64 encoding and decoding.

This of course is only 1 of countless examples.

I'm suprised no one picked the obvious answer for the Internet age. Calculating valid network addresses for a subnet.

http://www.topwebhosts.org/tools/netmask.php

Nobody seems to have mentioned fixed point maths.

(Yeah, I'm old, ok?)

Bitwise operators are useful for looping arrays which length is power of 2. As many people mentioned, bitwise operators are extremely useful and are used in Flags, Graphics, Networking, Encryption. Not only that, but they are extremely fast. My personal favorite use is to loop an array without conditionals. Suppose you have a zero-index based array(e.g. first element's index is 0) and you need to loop it indefinitely. By indefinitely I mean going from first element to last and returning to first. One way to implement this is:

int[] arr = new int[8];
int i = 0;
while (true) {
    print(arr[i]);
    i = i + 1;
    if (i >= arr.length) 
        i = 0;
}

This is the simplest approach, if you'd like to avoid if statement, you can use modulus approach like so:

int[] arr = new int[8];
int i = 0;
while (true) {
    print(arr[i]);
    i = i + 1;
    i = i % arr.length;
}

The down side of these two methods is that modulus operator is expensive, since it looks for a remainder after integer division. And the first method runs an if statement on each iteration. With bitwise operator however if length of your array is a power of 2, you can easily generate a sequence like 0 .. length - 1 by using & (bitwise and) operator like so i & length. So knowing this, the code from above becomes

int[] arr = new int[8];
int i = 0;
while (true){
    print(arr[i]);
    i = i + 1;
    i = i & (arr.length - 1);
}

Here is how it works. In binary format every number that is power of 2 subtracted by 1 is expressed only with ones. For example 3 in binary is 11, 7 is 111, 15 is 1111 and so on, you get the idea. Now, what happens if you & any number against a number consisting only of ones in binary? Let's say we do this:

num & 7;

If num is smaller or equal to 7 then the result will be num because each bit &-ed with 1 is itself. If num is bigger than 7, during the & operation computer will consider 7's leading zeros which of course will stay as zeros after & operation only the trailing part will remain. Like in case of 9 & 7 in binary it will look like

1001 & 0111

the result will be 0001 which is 1 in decimal and addresses second element in array.

Is a number x a power of 2? (Useful for example in algorithms where a counter is incremented, and an action is to be taken only logarithmic number of times)

(x & (x - 1)) == 0

Which is the highest bit of an integer x? (This for example can be used to find the minimum power of 2 that is larger than x)

x |= (x >>  1);
x |= (x >>  2);
x |= (x >>  4);
x |= (x >>  8);
x |= (x >> 16);
return x - (x >>> 1); // ">>>" is unsigned right shift

Which is the lowest 1 bit of an integer x? (Helps find number of times divisible by 2.)

x & -x

If you ever want to calculate your number mod(%) a certain power of 2, you can use yourNumber & 2^N-1, which in this case is the same as yourNumber % 2^N.

number % 16 = number & 15;
number % 128 = number & 127;

This is probably only useful being an alternative to modulus operation with a very big dividend that is 2^N... But even then its speed boost over the modulus operation is negligible in my test on .NET 2.0. I suspect modern compilers already perform optimizations like this. Anyone know more about this?

I use them for multi select options, this way I only store one value instead of 10 or more

When you only want to change some bits of a microcontroller's Outputs, but the register to write to is a byte, you do something like this (pseudocode):

char newOut = OutRegister & 0b00011111 //clear 3 msb's
newOut = newOut | 0b10100000 //write '101' to the 3 msb's
OutRegister = newOut //Update Outputs

Of course, many microcontrollers allow you to change each bit individually...

it can also be handy in a sql relational model, let's say you have the following tables: BlogEntry, BlogCategory

traditonally you could create a n-n relationship between them using a BlogEntryCategory table or when there are not that much BlogCategory records you could use one value in BlogEntry to link to multiple BlogCategory records just like you would do with flagged enums, in most RDBMS there are also a very fast operators to select on that 'flagged' column...

I've seen them used in role based access control systems.

Whenever I first started C programming, I understood truth tables and all that, but it didn't all click with how to actually use it until I read this article http://www.gamedev.net/reference/articles/article1563.asp (which gives real life examples)

I often use bitwise operations for storing combinations of options in a single integer.

int options = 0;

where OPTION1 might be defined as 1, OPTION2 as 2, OPTION3 as 4, OPTION4 as 8, OPTION5 as 16, ...

void addOption(int option) would use the | operator to add an option to options.

boolean hasOption(int option) would use the & operator to test for the option within options.

There is a real world use in my question here -
https://stackoverflow.com/questions/1362799/respond-to-only-the-first-wmkeydown-notification">https://stackoverflow.com/questions/1362799/respond-to-only-the-first-wmkeydown-notification</a>

When consuming a WM_KEYDOWN message in the windows C api bit 30 specifies the previous key state. The value is 1 if the key is down before the message is sent, or it is zero if the key is up

They are mostly used for bitwise operations (surprise). Here are a few real-world examples found in PHP codebase.

Character encoding:

if (s <= 0 && (c & ~MBFL_WCSPLANE_MASK) == MBFL_WCSPLANE_KOI8R) {

Data structures:

ar_flags = other->ar_flags & ~SPL_ARRAY_INT_MASK;

Database drivers:

dbh->transaction_flags &= ~(PDO_TRANS_ACCESS_MODE^PDO_TRANS_READONLY);

Compiler implementation:

opline->extended_value = (opline->extended_value & ~ZEND_FETCH_CLASS_MASK) | ZEND_FETCH_CLASS_INTERFACE;

I've seen it in a few game development books as a more efficient way to multiply and divide.

2 << 3 == 2 * 8 
32 >> 4 == 32 / 16

I don't think this counts as bitwise, but ruby's Array defines set operations through the normal integer bitwise operators. So [1,2,4] & [1,2,3] # => [1,2]. Similarly for a ^ b #=> set difference and a | b #=> union.

Tower Of Hanoi linear solution uses bitwise operations to solve the problem.

public static void linear(char start, char temp, char end, int discs)
{
	int from,to;
	for (int i = 1; i < (1 << discs); i++) {
		from = (i & i-1) % 3;
		to = ((i | i-1) + 1) % 3;
		System.out.println(from+" => "+to);
	}
}

The explanation for this solution can be found here

I use them as an option handlers, e.g. in Acces Control Lists to describe specific resources.

Take a look at this article <http://planetozh.com/blog/2006/05/php-bitwise-operators-example-of-use/>

Edit:

One more link: <http://blog.code-head.com/how-to-write-a-permission-system-using-bits-and-bitwise-operations-in-php>

I wrote a small wiki article a while back showing a binary writer/reader. It works on the bit level and shows how bitwise operators can be used to pack data. That would probably be a "real world" example since it has application in games.

I was always under the assumption that bitwise operations are fairly simple operations to be performed, so when running time is crucial a solution which is implemented via bitsets can improve running time by a constant amount, algorithm depending.

Another real world application in the database world is MySQL which has a datatype called SET.

Bitwise operators are by the DBMS to store the SET datatype. Set can save space.

Element	   SET Value	Decimal Value
Travel	    00000001	1
Sports	    00000010	2
Dancing	   00000100	   4
Fine Dining   00001000	8

I use them to implement fast BCD calculations (accountants and auditors get upset with fp rounding).

We use Bitwise Flags to make the session smaller for login permissions on our internal website.

A very specific example, but I used them to make my sudoku solver run faster (I was having a competition with a friend)

Each column, row and 3x3 was represented as an unsigned integer and as I set numbers I'd flag the appriate bit for the number set in the relevent column,row and 3x3 square.

This then made it very easy to see what possible numbers I could place in a given square since I would OR together the right column,row and 3x3 square and then NOT this to leave me with a mask that represented the possible legal values for a given position.

Hope that makes sense.

Nobody's mentioned collections yet. Sometimes you have a smallish collection of possible values, say only 10 or 20 possible values, and you want to keep some of them in a set. Sure you can use a regular Set implementation which will most likely use a backing hashtable. But since the set of possible values is so small this is really just a waste of time and space. Instead, you can store the set in a single int or long value which is exactly what the java EnumSet does if I remember correctly.

A common use is for alignment e.g I need my data aligned on 4-byte or 16-byte boundaries.This is very common with RISC processors where an unaligned load/store is either expensive ( because it triggers an exception handler that then needs to fix up the non-aligned load ) or not allowed at all.

For any alignment that is a power of 2, the next aligned pos can be calculated as follows :

aligned_offset = alignment + ((current_offset - 1) & ~(alignment - 1))

So in the case of 4 byte alignment and a current offset of 9 then :

aligned_offset = 4 + ((9-1) & ~(4-1)) = 4 + (8 & 0xFFFFFFFC) = 4+ 8  = 12  

so the next 4 byte aligned offset would be 12