Strange \n in base64 encoded string in Ruby

RubyBase64Encode

Ruby Problem Overview


The inbuilt Base64 library in Ruby is adding some '\n's. I'm unable to find out the reason. For this special example:

irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'base64'
=> true
irb(main):003:0> str =  "1110--ad6ca0b06e1fbeb7e6518a0418a73a6e04a67054"
=> "1110--ad6ca0b06e1fbeb7e6518a0418a73a6e04a67054"
irb(main):004:0> Base64.encode64(str)
=> "MTExMC0tYWQ2Y2EwYjA2ZTFmYmViN2U2NTE4YTA0MThhNzNhNmUwNGE2NzA1\nNA==\n"

The \n's are at the last and 6th position from end. The decoder (Base64.decode64) returns back the old string perfectly. Strange thing is, these \n's don't add any value to the encoded string. When I remove the newlines from the output string, the decoder decodes it again perfectly.

irb(main):005:0> Base64.decode64(Base64.encode64(str).gsub("\n", '')) == str
=> true

More of this, I used an another JS library to produce the base64 encoded output of the same input string, the output comes without the \n's.

Is this a bug or anything else? Has anybody faced this issue before?

FYI,

$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Ruby Solutions


Solution 1 - Ruby

Edit: Since I wrote this answer Base64.strict_encode64() was added, which does not add newlines.


The docs are somewhat confusing, the b64encode method is supposed to add a newline for every 60th character, and the example for the encode64 method is actually using the b64encode method.

It seems the pack("m") method for the Array class used by encode64 also adds the newlines. I would consider it a design bug that this is not optional.

You could either remove the newlines yourself, or if you're using rails, there's ActiveSupport::CoreExtensions::Base64::Encoding with the encode64s method.

Solution 2 - Ruby

In ruby-1.9.2 you have Base64.strict_encode64 which doesn't add that \n (newline) at the end.

Solution 3 - Ruby

Yeah, this is quite normal. The doc gives an example demonstrating the line-splitting. base64 does the same thing in other languages too (eg. Python).

The reason content-free newlines are added at the encode stage is because base64 was originally devised as an encoding mechanism for sending binary content in e-mail, where the line length is limited. Feel free to replace them away if you don't need them.

Solution 4 - Ruby

Use strict_encode64 method. encode64 adds \n every 60 symbols

Solution 5 - Ruby

Seems they've got to be stripped/ignored, like:

Base64.encode64(str).gsub(/\n/, '')

Solution 6 - Ruby

The \n added when using Base64#encode64 is correct, check this post out: https://glaucocustodio.github.io/2014/09/27/a-reminder-about-base64encode64-in-ruby/

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionintellidiotView Question on Stackoverflow
Solution 1 - RubyChristoffer HammarströmView Answer on Stackoverflow
Solution 2 - RubyghtnView Answer on Stackoverflow
Solution 3 - RubybobinceView Answer on Stackoverflow
Solution 4 - RubyАлександр ТихоновичView Answer on Stackoverflow
Solution 5 - RubymeesernView Answer on Stackoverflow
Solution 6 - Rubyuser1519240View Answer on Stackoverflow