In C#, what is the best technique to join two or more byte arrays?

Problem

In C#, I have three byte arrays that I need to merge into one. What is the most efficient way of completing this task?

Asked by RoboDev

Solution #1

Use System for primitive types (including bytes). Buffer. Instead of System, use BlockCopy. Array. Copy. It’s more efficient.

Using three 10-byte arrays, I timed each of the suggested ways in a loop that ran a million times. The following are the outcomes:

I re-ran the test after increasing the size of each array to 100 elements:

I re-ran the test with the array sizes increasing to 1000 elements each:

Finally, I increased the size of each array to 1 million elements and performed the test again, this time only repeating each loop 4000 times:

As a result, if you require a new byte array, use

byte[] rv = new byte[a1.Length + a2.Length + a3.Length];
System.Buffer.BlockCopy(a1, 0, rv, 0, a1.Length);
System.Buffer.BlockCopy(a2, 0, rv, a1.Length, a2.Length);
System.Buffer.BlockCopy(a3, 0, rv, a1.Length + a2.Length, a3.Length);

However, if you have access to an IEnumerablebyte>, you should definitely utilize LINQ’s Concat> method. It’s only a fraction of a second slower than the yield operator in C#, but it’s more concise and elegant.

IEnumerable<byte> rv = a1.Concat(a2).Concat(a3);

You can construct the System.Buffer if you have an arbitrary number of arrays and are using.NET 3.5. This is a more generic BlockCopy solution:

private byte[] Combine(params byte[][] arrays)
{
    byte[] rv = new byte[arrays.Sum(a => a.Length)];
    int offset = 0;
    foreach (byte[] array in arrays) {
        System.Buffer.BlockCopy(array, 0, rv, offset, array.Length);
        offset += array.Length;
    }
    return rv;
}

*Note: In order for the above block to work, you must first add the following namespace to the top.

using System.Linq;

To Jon Skeet’s point about iteration of subsequent data structures (byte array vs. IEnumerablebyte>), I re-ran the previous timing test (1 million elements, 4000 iterations), this time with a loop that iterates over the entire array with each pass:

The point is that understanding the efficiency of both the construction and use of the final data structure is critical. Putting all of your attention on the efficiency of the producing process may cause you to miss the inefficiency of the usage process. Congratulations, Jon.

Answered by Matt Davis

Solution #2

Many of the responses appear to disregard the specified requirements:

These two combined rule out a LINQ sequence of bytes: anything with yield will make getting the final size impossible without iterating through the entire sequence.

Of again, if those aren’t the genuine needs, LINQ (or the IListT> implementation) could be a suitable fit. I’ll assume, however, that Superdumbell knows exactly what he wants.

(EDIT: I’ve just had a new idea.) Making a copy of the arrays versus reading them slowly has a significant conceptual difference. Consider what happens if you update the data in one of the “source” arrays after executing the Combine (or whatever) method but before using the result: the change will be evident due to lazy evaluation. It won’t work with an instant copy. Different situations will necessitate different responses – just something to keep in mind.)

Here are my recommended methods, which are undoubtedly similar to those in some of the other responses:)

public static byte[] Combine(byte[] first, byte[] second)
{
    byte[] ret = new byte[first.Length + second.Length];
    Buffer.BlockCopy(first, 0, ret, 0, first.Length);
    Buffer.BlockCopy(second, 0, ret, first.Length, second.Length);
    return ret;
}

public static byte[] Combine(byte[] first, byte[] second, byte[] third)
{
    byte[] ret = new byte[first.Length + second.Length + third.Length];
    Buffer.BlockCopy(first, 0, ret, 0, first.Length);
    Buffer.BlockCopy(second, 0, ret, first.Length, second.Length);
    Buffer.BlockCopy(third, 0, ret, first.Length + second.Length,
                     third.Length);
    return ret;
}

public static byte[] Combine(params byte[][] arrays)
{
    byte[] ret = new byte[arrays.Sum(x => x.Length)];
    int offset = 0;
    foreach (byte[] data in arrays)
    {
        Buffer.BlockCopy(data, 0, ret, offset, data.Length);
        offset += data.Length;
    }
    return ret;
}

Of course, the “params” variant necessitates first generating an array of byte arrays, which adds to the inefficiencies.

Answered by Jon Skeet

Solution #3

For code purity, I took Matt’s LINQ example a step further:

byte[] rv = a1.Concat(a2).Concat(a3).ToArray();

In my instance, the arrays are small, therefore performance isn’t an issue.

Answered by Nate Barbettini

Solution #4

Use the following if you just need a fresh byte array:

byte[] Combine(byte[] a1, byte[] a2, byte[] a3)
{
    byte[] ret = new byte[a1.Length + a2.Length + a3.Length];
    Array.Copy(a1, 0, ret, 0, a1.Length);
    Array.Copy(a2, 0, ret, a1.Length, a2.Length);
    Array.Copy(a3, 0, ret, a1.Length + a2.Length, a3.Length);
    return ret;
}

If you only require a single IEnumerable, try utilizing the yield operator in C# 2.0:

IEnumerable<byte> Combine(byte[] a1, byte[] a2, byte[] a3)
{
    foreach (byte b in a1)
        yield return b;
    foreach (byte b in a2)
        yield return b;
    foreach (byte b in a3)
        yield return b;
}

Answered by FryGuy

Solution #5

I really came into some problems when I tried to use Concat.. (with arrays in the 10-million, it actually crashed).

I found the following to be straightforward, easy, and reliable, and it works for any number of arrays (not just three) (using LINQ):

public static byte[] ConcatByteArrays(params byte[][]  arrays)
{
    return arrays.SelectMany(x => x).ToArray();
}

Answered by 00jt

Post is based on https://stackoverflow.com/questions/415291/best-way-to-combine-two-or-more-byte-arrays-in-c-sharp