I have a PHP script that can encode a PNG image to a Base64 string.

I&#39;d like to do the same thing using JavaScript. I know how to open files, but I&#39;m not sure how to do the encoding. I&#39;m not used to working with binary data.


How can you encode a string to Base64 in JavaScript?

I am looking for opinions of how to handle large binary files on which my source code (web application) is dependent. We are currently discussing several alternatives:

1. Copy the binary files by hand.
 - Pro: Not sure.
 - Contra: I am strongly against this, as it increases the likelihood of errors when setting up a new site/migrating the old one. Builds up another hurdle to take.
2. Manage them all with _Git_.
 - Pro: Removes the possibility to &#39;forget&#39; to copy a important file
 - Contra: Bloats the repository and decreases flexibility to manage the code-base and checkouts, clones, etc. will take quite a while.
3. Separate repositories.
 - Pro: Checking out/cloning the source code is fast as ever, and the images are properly archived in their own repository.
 - Contra: Removes the simpleness of having _the one and only_ Git repository on the project. It surely introduces some other things I haven&#39;t thought about.

What are your experiences/thoughts regarding this?

Also: Does anybody have experience with multiple Git repositories and managing them in one project?

The files are images for a program which generates PDFs with those files in it. The files will not change very often (as in years), but they are very relevant to a program. The program will not work without the files.


Managing large binary files with Git

My Express app is receiving a base64-encoded PNG from the browser (generated from canvas with toDataURL() ) and writing it to a file. But the file isn&#39;t a valid image file, and the &quot;file&quot; utility simply identifies it as &quot;data&quot;.

    var body = req.rawBody,
      base64Data = body.replace(/^data:image\/png;base64,/,&quot;&quot;),
      binaryData = new Buffer(base64Data, &#39;base64&#39;).toString(&#39;binary&#39;);
    
    require(&quot;fs&quot;).writeFile(&quot;out.png&quot;, binaryData, &quot;binary&quot;, function(err) {
      console.log(err); // writes out file without error, but it&#39;s not a valid image
    });

How can I save a base64-encoded image to disk?

I want to edit a binary file, but I don&#39;t want to use another tool other than Visual Studio because it&#39;s a pain to switch back and forth.  

Is there perhaps an add-in or some built in functionality that can do that in Visual Studio?


Can I hex edit a file in Visual Studio?

I&#39;m writing an application that allows users to upload images onto the server. I expect about 20 images per day all jpeg and probably not edited/resized. (This is another question, how to resize the images on the server side before storing. Maybe someone can please drop a .NET resource for that in the comment or so). 
I wonder now what the best place for storing uploaded images is.

- Store the images as a file in the file system and create a record in a table with the exact path to that image.

- Or, store the image itself in a table using an &quot;image&quot; or &quot;binary data&quot; data type of the database server.

I see advantages and disadvantages in both. 
I like a) because I can easily relocate the files and just have to change the table entry. On the other hand I don&#39;t like storing business data on the web server and I don&#39;t really want to connect the web server to any other datasource that holds business data (for security reasons) 
I like b) because all the information is in one place and easily accessible by a query. On the other hand the database will get very big very soon. Outsourcing that data could be more difficult.

What is the best place for storing uploaded images, SQL database or disk file system?

I was trying to `GET` a binary data using [`request`][1], and had something like:

    var requestSettings = {
        method: &#39;GET&#39;,
        url: url,
    };
    request(requestSettings, function(error, response, body) {
        // Use body as a binary Buffer
    }

But `body` was always a few bytes different from expected. After further investigation I found out that `request` assumed `body` is string and replaced all non-unicode bytes.

I tried to add

    encoding: &#39;binary&#39;
to `requestSettings` but it didn&#39;t help.

How can I get the binary data?


  [1]: https://github.com/request/request

Getting binary content in Node.js using request

I have this binary file on my Linux system...

     udit@udit-Dabba ~ $ cat file.enc
     Salted__s�bO��&lt;0�F���Jw!���]�:`C�LKȆ�l

Using the hexdump command, I see its information like this:

     udit@udit-Dabba ~ $ hexdump -C file.enc
     00000000  53 61 6c 74 65 64 5f 5f  1b 73 a1 62 4f 15 be f6  |Salted__.s.bO...|
     00000010  3c 30 cc 46 ee 10 13 11  84 bf 4a 77 21 a4 84 99  |&lt;0.F......Jw!...|
     00000020  0e 5d ef 11 18 3a 60 43  a0 4c 4b 1e c8 86 e6 6c  |.]...:`C.LK....l|
     00000030

Now I am given a file on some other system whose contents are like this:

     53 61 6c 74 65 64 5f 5f  1b 73 a1 62 4f 15 be f6
     3c 30 cc 46 ee 10 13 11  84 bf 4a 77 21 a4 84 99
     0e 5d ef 11 18 3a 60 43  a0 4c 4b 1e c8 86 e6 6c

And I need to find out that same exact binary information from this hexdump.

How can I proceed for that?

If there isn&#39;t any switch for that then C code will also work fine.

(But a Linux command with some switch is preferable)

**Limitation:**

The binary information in the file is output of an encryption algorithm, so contents should exactly match...






Transform hexadecimal information to binary using a Linux command

Lately I&#39;ve been asked to write a function that reads the binary file into the `std::vector&lt;BYTE&gt;` where `BYTE` is an `unsigned char`. Quite quickly I came with something like this:

    #include &lt;fstream&gt;
    #include &lt;vector&gt;
    typedef unsigned char BYTE;

    std::vector&lt;BYTE&gt; readFile(const char* filename)
    {
        // open the file:
        std::streampos fileSize;
        std::ifstream file(filename, std::ios::binary);

        // get its size:
        file.seekg(0, std::ios::end);
        fileSize = file.tellg();
        file.seekg(0, std::ios::beg);

        // read the data:
        std::vector&lt;BYTE&gt; fileData(fileSize);
        file.read((char*) &amp;fileData[0], fileSize);
        return fileData;
    }

which seems to be unnecessarily complicated and the explicit cast to `char*` that I was forced to use while calling `file.read` doesn&#39;t make me feel any better about it.  

---
Another option is to use [`std::istreambuf_iterator`](http://www.cplusplus.com/reference/iterator/istreambuf_iterator/):

    std::vector&lt;BYTE&gt; readFile(const char* filename)
    {
        // open the file:
        std::ifstream file(filename, std::ios::binary);

        // read the data:
        return std::vector&lt;BYTE&gt;((std::istreambuf_iterator&lt;char&gt;(file)),
                                  std::istreambuf_iterator&lt;char&gt;());
    }

which is pretty simple and short, but still I have to use the `std::istreambuf_iterator&lt;char&gt;` even when I&#39;m reading into `std::vector&lt;unsigned char&gt;`.

---
The last option that seems to be perfectly straightforward is to use [`std::basic_ifstream&lt;BYTE&gt;`](http://en.cppreference.com/w/cpp/io/basic_ifstream), which kinda expresses it explicitly that *&quot;I want an input file stream and I want to use it to read `BYTE`s&quot;*:

    std::vector&lt;BYTE&gt; readFile(const char* filename)
    {
        // open the file:
        std::basic_ifstream&lt;BYTE&gt; file(filename, std::ios::binary);

        // read the data:
        return std::vector&lt;BYTE&gt;((std::istreambuf_iterator&lt;BYTE&gt;(file)),
                                  std::istreambuf_iterator&lt;BYTE&gt;());
    }

but I&#39;m not sure whether [`basic_ifstream`](http://en.cppreference.com/w/cpp/io/basic_ifstream) is an appropriate choice in this case.

*What is the best way of reading a binary file into the `vector`?* I&#39;d also like to know what&#39;s happening *&quot;behind the scene&quot;* and what are the possible problems I might encounter (apart from stream not being opened properly which might be avoided by simple [`is_open`](http://www.cplusplus.com/reference/fstream/ifstream/is_open/) check).

Is there any good reason why one would prefer to use [`std::istreambuf_iterator`](http://www.cplusplus.com/reference/iterator/istreambuf_iterator/) here?  
(the only advantage that I can see is simplicity)

How to read a binary file into a vector of unsigned chars

Working on a Fedora Constantine box. I am looking to `diff` two directories recursively to check for source changes. Due to the setup of the project (prior to my own engagement with said project! *sigh*), the directories contain both source and binaries, as well as large binary datasets. While diffing eventually works on these directories, it would take perhaps twenty seconds if I could ignore the binary files.

As far as I understand, diff does not have an &#39;ignore binary file&#39; mode, but does have an ignore argument which will ignore regular expression **within** a file. I don&#39;t know what to write there to ignore binary files, regardless of extension.

I&#39;m using the following command, but it does not ignore binary files. Does anyone know how to modify this command to do this?

    diff -rq dir1 dir2

diff a directory recursively, ignoring all binary files

For one and a half years, I have been keeping my eyes on the git community in hopes of making the switch away from SVN.  One particular issue holding me back is the inability to lock binary files. Throughout the past year I have yet to see developments on this issue.  I understand that locking files goes against the fundamental principles of distributed source control, but I don&#39;t see how a web development company can take advantage of git to track source code and image file changes when there is the potential for binary file conflicts.

To achieve the effects of locking, a &quot;central&quot; repository must be identified.  Regardless of the distributed nature of git, most companies will have a &quot;central&quot; repository for a software project.  We should be able to mark a file as requiring a lock from the governing git repository at a specified address.  Perhaps this is made difficult because git tracks file contents not files?

Do any of you have experience in dealing with git and binary files that should be locked before modification?

NOTE: It looks like Source Gear&#39;s new open source distributed version control project, Veracity, has locking as one of its goals.

Locking binary files using git version control system

Informally, most of us understand that there are &#39;binary&#39; files (object files, images, movies, executables, proprietary document formats, etc) and &#39;text&#39; files (source code, XML files, HTML files, email, etc).

In general, you need to know the contents of a file to be able to do anything useful with it, and form that point of view if the encoding is &#39;binary&#39; or &#39;text&#39;, it doesn&#39;t really matter. And of course files just store bytes of data so they are all &#39;binary&#39; and &#39;text&#39; doesn&#39;t mean anything without knowing the encoding. And yet, it is still useful to talk about &#39;binary&#39; and &#39;text&#39; files, but to avoid offending anyone with this imprecise definition, I will continue to use &#39;scare&#39; quotes.

However, there are various tools that work on a wide range of files, and in practical terms, you want to do something different based on whether the file is &#39;text&#39; or &#39;binary&#39;. An example of this is any tool that outputs data on the console. Plain &#39;text&#39; will look fine, and is useful. &#39;binary&#39; data messes up your terminal, and is generally not useful to look at. GNU grep at least uses this distinction when determining if it should output matches to the console.

So, the question is, how do you tell if a file is &#39;text&#39; or &#39;binary&#39;? And to restrict is further, how do you tell on a Linux like file-system? I am not aware of any filesystem meta-data that indicates the &#39;type&#39; of a file, so the question further becomes, by inspecting the content of a file, how do I tell if it is &#39;text&#39; or &#39;binary&#39;? And for simplicity, lets restrict &#39;text&#39; to mean characters which are printable on the user&#39;s console. And in particular how would you *implement* this? (I thought this was implied on this site, but I guess it is helpful, in general, to be pointed at existing code that does this, I should have specified), I&#39;m not really after what existing programs can I use to do this.

How do I distinguish between &#39;binary&#39; and &#39;text&#39; files?

 * Do I have to do something to tell Git whether some files are binary (like in Subversion)? Or, can Git handle binary data automatically?
 * If I change the binary file, so that I have 100 binary revisions, will git just store all 100 versions individually in the repository?
 * What are submodules for with git? 


How does Git deal with binary files?

On Windows machines there are lots of third party editors available to edit a binary file.

Any idea how can I edit a binary file on Unix system?

How to edit binary file on Unix systems

How can I know if a file is a binary file?

For example, compiled c file.

I want to read all files from some directory, but I want ignore binary files.

How to check if the file is a binary file and read all the files which are not?

I&#39;m trying to read binary data using C#. I have all the information about the layout of the data in the files I want to read. I&#39;m able to read the data &quot;chunk by chunk&quot;, i.e. getting the first 40 bytes of data converting it to a string, get the next 40 bytes.

Since there are at least three slightly different version of the data, I would like to read the data directly into a struct. It just feels so much more right than by reading it &quot;line by line&quot;.

**I have tried the following approach but to no avail:**

    StructType aStruct;
    int count = Marshal.SizeOf(typeof(StructType));
    byte[] readBuffer = new byte[count];
    BinaryReader reader = new BinaryReader(stream);
    readBuffer = reader.ReadBytes(count);
    GCHandle handle = GCHandle.Alloc(readBuffer, GCHandleType.Pinned);
    aStruct = (StructType) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(StructType));
    handle.Free();

The stream is an opened FileStream from which I have began to read from. I get an `AccessViolationExceptio`n when using `Marshal.PtrToStructure`.

The stream contains more information than I&#39;m trying to read since I&#39;m not interested in data at the end of the file.

**The struct is defined like:**

    [StructLayout(LayoutKind.Explicit)]
    struct StructType
    {
        [FieldOffset(0)]
        public string FileDate;
        [FieldOffset(8)]
        public string FileTime;
        [FieldOffset(16)]
        public int Id1;
        [FieldOffset(20)]
        public string Id2;
    }

The examples code is changed from original to make this question shorter.

How would I read binary data from a file into a struct?

Read binary file into a struct

I was searching for a way to check whether I&#39;ve reached the end of a file for my binary reader and one suggestion was to use PeekChar as such

    while (inFile.PeekChar() &gt; 0)
    {
        ...
    }


However, it looks like I&#39;ve run into an issue

&lt;pre&gt;
Unhandled Exception: System.ArgumentException: The output char buffer is too sma
ll to contain the decoded characters, encoding &#39;Unicode (UTF-8)&#39; fallback &#39;Syste
m.Text.DecoderReplacementFallback&#39;.
Parameter name: chars
   at System.Text.Encoding.ThrowCharsOverflow()
   at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothin
gDecoded)
   at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* char
s, Int32 charCount, DecoderNLS baseDecoder)
   at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars,
 Int32 charCount, Boolean flush)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteC
ount, Char[] chars, Int32 charIndex, Boolean flush)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteC
ount, Char[] chars, Int32 charIndex)
   at System.IO.BinaryReader.InternalReadOneChar()
   at System.IO.BinaryReader.PeekChar()
&lt;/pre&gt;

So maybe PeekChar isn&#39;t the best way to do it, and I don&#39;t think it should even be used that way because I&#39;m checking the current position of my reader and not really what the next character is supposed to be.

C# checking for binary reader end of file

I have to read a binary file in a legacy format with Java. 

In a nutshell the file has a header consisting of several integers, bytes and fixed-length char arrays, followed by a list of records which also consist of integers and chars.

In any other language I would create `struct`s (C/C++) or `record`s (Pascal/Delphi) which  are byte-by-byte representations of the header and the record. Then I&#39;d read `sizeof(header)` bytes into a header variable and do the same for the records.

Something like this: (Delphi)

    type
      THeader = record
        Version: Integer;
        Type: Byte;
        BeginOfData: Integer;
        ID: array[0..15] of Char;
      end;

    ...

    procedure ReadData(S: TStream);
    var
      Header: THeader;
    begin
      S.ReadBuffer(Header, SizeOf(THeader));
      ...
    end;

What is the best way to do something similar with Java? Do I have to read every single value on its own or is there any other way to do this kind of &quot;block-read&quot;?

Best way to read structured binary files with Java

Why should we distinguish between text file and binary files when transmitting them? Why there are some channels designed only for textual data? At the bottom level, they are all bits.

All Binaryfiles Solutions on Gang of Coders

Total of 18 Binaryfiles Solutions