Grab a segment of an array in Java without creating a new array on heap

JavaArraysSlice

Java Problem Overview


I'm looking for a method in Java that will return a segment of an array. An example would be to get the byte array containing the 4th and 5th bytes of a byte array. I don't want to have to create a new byte array in the heap memory just to do that. Right now I have the following code:

doSomethingWithTwoBytes(byte[] twoByteArray);

void someMethod(byte[] bigArray)
{
      byte[] x = {bigArray[4], bigArray[5]};
      doSomethingWithTwoBytes(x);
}

I'd like to know if there was a way to just do doSomething(bigArray.getSubArray(4, 2)) where 4 is the offset and 2 is the length, for example.

Java Solutions


Solution 1 - Java

Disclaimer: This answer does not conform to the constraints of the question:

> I don't want to have to create a new byte array in the heap memory just to do that.

(Honestly, I feel my answer is worthy of deletion. The answer by @unique72 is correct. Imma let this edit sit for a bit and then I shall delete this answer.)


I don't know of a way to do this directly with arrays without additional heap allocation, but the other answers using a sub-list wrapper have additional allocation for the wrapper only – but not the array – which would be useful in the case of a large array.

That said, if one is looking for brevity, the utility method Arrays.copyOfRange() was introduced in Java 6 (late 2006?):

byte [] a = new byte [] {0, 1, 2, 3, 4, 5, 6, 7};

// get a[4], a[5]

byte [] subArray = Arrays.copyOfRange(a, 4, 6);

Solution 2 - Java

Arrays.asList(myArray) delegates to new ArrayList(myArray), which doesn't copy the array but just stores the reference. Using List.subList(start, end) after that makes a SubList which just references the original list (which still just references the array). No copying of the array or its contents, just wrapper creation, and all lists involved are backed by the original array. (I thought it'd be heavier.)

Solution 3 - Java

If you're seeking a pointer style aliasing approach, so that you don't even need to allocate space and copy the data then I believe you're out of luck.

System.arraycopy() will copy from your source to destination, and efficiency is claimed for this utility. You do need to allocate the destination array.

Solution 4 - Java

One way is to wrap the array in java.nio.ByteBuffer, use the absolute put/get functions, and slice the buffer to work on a subarray.

For instance:

doSomething(ByteBuffer twoBytes) {
    byte b1 = twoBytes.get(0);
    byte b2 = twoBytes.get(1);
    ...
}

void someMethod(byte[] bigArray) {
      int offset = 4;
      int length = 2;
      doSomething(ByteBuffer.wrap(bigArray, offset, length).slice());
}

Note that you have to call both wrap() and slice(), since wrap() by itself only affects the relative put/get functions, not the absolute ones.

ByteBuffer can be a bit tricky to understand, but is most likely efficiently implemented, and well worth learning.

Solution 5 - Java

Use java.nio.Buffer's. It's a lightweight wrapper for buffers of various primitive types and helps manage slicing, position, conversion, byte ordering, etc.

If your bytes originate from a Stream, the NIO Buffers can use "direct mode" which creates a buffer backed by native resources. This can improve performance in a lot of cases.

Solution 6 - Java

You could use the ArrayUtils.subarray in apache commons. Not perfect but a bit more intuitive than System.arraycopy. The downside is that it does introduce another dependency into your code.

Solution 7 - Java

I see the subList answer is already here, but here's code that demonstrates that it's a true sublist, not a copy:

public class SubListTest extends TestCase {
	public void testSubarray() throws Exception {
		Integer[] array = {1, 2, 3, 4, 5};
		List<Integer> list = Arrays.asList(array);
		List<Integer> subList = list.subList(2, 4);
		assertEquals(2, subList.size());
		assertEquals((Integer) 3, subList.get(0));
		list.set(2, 7);
		assertEquals((Integer) 7, subList.get(0));
	}
}

I don't believe there's a good way to do this directly with arrays, however.

Solution 8 - Java

List.subList(int startIndex, int endIndex)

Solution 9 - Java

The Lists allow you to use and work with subList of something transparently. Primitive arrays would require you to keep track of some kind of offset - limit. ByteBuffers have similar options as I heard.

Edit: If you are in charge of the useful method, you could just define it with bounds (as done in many array related methods in java itself:

doUseful(byte[] arr, int start, int len) {
    // implementation here
}
doUseful(byte[] arr) {
    doUseful(arr, 0, arr.length);
}

It's not clear, however, if you work on the array elements themselves, e.g. you compute something and write back the result?

Solution 10 - Java

One option would be to pass the whole array and the start and end indices, and iterate between those instead of iterating over the whole array passed.

void method1(byte[] array) {
    method2(array,4,5);
}
void method2(byte[] smallarray,int start,int end) {
    for ( int i = start; i <= end; i++ ) {
        ....
    }
}

Solution 11 - Java

Java references always point to an object. The object has a header that amongst other things identifies the concrete type (so casts can fail with ClassCastException). For arrays, the start of the object also includes the length, the data then follows immediately after in memory (technically an implementation is free to do what it pleases, but it would be daft to do anything else). So, you can;t have a reference that points somewhere into an array.

In C pointers point anywhere and to anything, and you can point to the middle of an array. But you can't safely cast or find out how long the array is. In D the pointer contains an offset into the memory block and length (or equivalently a pointer to the end, I can't remember what the implementation actually does). This allows D to slice arrays. In C++ you would have two iterators pointing to the start and end, but C++ is a bit odd like that.

So getting back to Java, no you can't. As mentioned, NIO ByteBuffer allows you to wrap an array and then slice it, but gives an awkward interface. You can of course copy, which is probably very much faster than you would think. You could introduce your own String-like abstraction that allows you to slice an array (the current Sun implementation of String has a char[] reference plus a start offset and length, higher performance implementation just have the char[]). byte[] is low level, but any class-based abstraction you put on that is going to make an awful mess of the syntax, until JDK7 (perhaps).

Solution 12 - Java

@unique72 answer as a simple function or line, you may need to replace Object, with the respective class type you wish to 'slice'. Two variants are given to suit various needs.

/// Extract out array from starting position onwards
public static Object[] sliceArray( Object[] inArr, int startPos ) {
	return Arrays.asList(inArr).subList(startPos, inArr.length).toArray();
}

/// Extract out array from starting position to ending position
public static Object[] sliceArray( Object[] inArr, int startPos, int endPos ) {
	return Arrays.asList(inArr).subList(startPos, endPos).toArray();
}

Solution 13 - Java

How about a thin List wrapper?

List<Byte> getSubArrayList(byte[] array, int offset, int size) {
   return new AbstractList<Byte>() {
      Byte get(int index) {
         if (index < 0 || index >= size) 
           throw new IndexOutOfBoundsException();
         return array[offset+index];
      }
      int size() {
         return size;
      }
   };
}

(Untested)

Solution 14 - Java

I needed to iterate through the end of an array and didn't want to copy the array. My approach was to make an Iterable over the array.

public static Iterable<String> sliceArray(final String[] array, 
                                          final int start) {
  return new Iterable<String>() {
    String[] values = array;
    int posn = start;
  
    @Override
    public Iterator<String> iterator() {
      return new Iterator<String>() {
        @Override
        public boolean hasNext() {
          return posn < values.length;
        }

        @Override
        public String next() {
          return values[posn++];
        }

        @Override
        public void remove() {
          throw new UnsupportedOperationException("No remove");
        }
      };
    }
  };
}

Solution 15 - Java

This is a little more lightweight than Arrays.copyOfRange - no range or negative

public static final byte[] copy(byte[] data, int pos, int length )
{
	byte[] transplant = new byte[length];
	
	System.arraycopy(data, pos, transplant, 0, length);
	
	return transplant;
}

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionjbuView Question on Stackoverflow
Solution 1 - JavaDavid J. LiszewskiView Answer on Stackoverflow
Solution 2 - Javaunique72View Answer on Stackoverflow
Solution 3 - JavadjnaView Answer on Stackoverflow
Solution 4 - JavaSoulmanView Answer on Stackoverflow
Solution 5 - JavaJames SchekView Answer on Stackoverflow
Solution 6 - JavasethView Answer on Stackoverflow
Solution 7 - JavaCarl ManasterView Answer on Stackoverflow
Solution 8 - JavaManuel SelvaView Answer on Stackoverflow
Solution 9 - JavaakarnokdView Answer on Stackoverflow
Solution 10 - JavaSam DeFabbia-KaneView Answer on Stackoverflow
Solution 11 - JavaTom Hawtin - tacklineView Answer on Stackoverflow
Solution 12 - JavaPicoCreatorView Answer on Stackoverflow
Solution 13 - JavaRoToRaView Answer on Stackoverflow
Solution 14 - JavaOwen O'MalleyView Answer on Stackoverflow
Solution 15 - JavabondView Answer on Stackoverflow