json_encode(): Invalid UTF-8 sequence in argument

Php Problem Overview

I'm calling json_encode() on data that comes from a MySQL database with utf8_general_ci collation. The problem is that some rows have weird data which I can't clean. For example symbol �, so once it reaches json_encode(), it fails with json_encode(): Invalid UTF-8 sequence in argument.

I've tried utf8_encode() and utf8_decode(), even with mb_check_encoding() but it keeps getting through and causing havoc.

Running PHP 5.3.10 on Mac. So the question is - how can I clean up invalid utf8 symbols, keeping the rest of data, so that json_encoding() would work?

Update. Here is a way to reproduce it:

echo json_encode(pack("H*" ,'c32e'));

Php Solutions

Solution 1 - Php

I had a similar error which caused json_encode to return a null field whenever there was a hi-ascii character such as a curly apostrophe in a string, due to the wrong character set being returned in the query.

The solution was to make sure it comes as utf8 by adding:

mysql_set_charset('utf8');

after the mysql connect statement.

Solution 2 - Php

Seems like the symbol was Å, but since data consists of surnames that shouldn't be public, only first letter was shown and it was done by just $lastname[0], which is wrong for multibyte strings and caused the whole hassle. Changed it to mb_substr($lastname, 0, 1) - works like a charm.

Solution 3 - Php

The problem is that this character is UTF8, but json_encode does not handle it correctly. To say more, there is a list of other characters (see [Unicode characters list][1]), that will trigger the same error, so stripping off this one (Å) will not correct an issue to the end.

What we have used is to convert these chars to html entities like this:

htmlentities( (string) $value, ENT_QUOTES, 'utf-8', FALSE);

[1]: http://www.utf8-chartable.de/unicode-utf8-table.pl?start=128&number=128&utf8=string-literal&unicodeinhtml=hex "Unicode characters list"

Solution 4 - Php

Make sure that your connection charset to MySQL is UTF-8. It often defaults to ISO-8859-1 which means that the MySQL driver will convert the text to ISO-8859-1.

You can set the connection charset with mysql_set_charset, mysqli_set_charset or with the query SET NAMES 'utf-8'

Solution 5 - Php

Using this code might help. It solved my problem!

mb_convert_encoding($post["post"],'UTF-8','UTF-8');

or like that

mb_convert_encoding($string,'UTF-8','UTF-8');

Solution 6 - Php

The symbol you posted is the placeholder symbol for a broken byte sequence. Basically, it's not a real symbol but an error in your string.

What is the exact byte value of the symbol? Blindly applying utf8_encode is not a good idea, it's better to find out first where the byte(s) came from and what they mean.

Solution 7 - Php

Another thing that throws this error, when you use php's json_encode function, is when unicode characters are upper case \U and not lower case \u

Solution 8 - Php

json_encode works only with UTF-8 data. You'll have to ensure that your data is in UTF-8. alternatively, you can use iconv() to convert your results to UTF-8 before feeding them to json_encode()

Solution 9 - Php

Updated.. I solved this issue by stating the charset on PDO connection as below:

"mysql:host=$host;dbname=$db;charset=utf8"

All data received was then in the correct charset for the rest of the code to use

Solution 10 - Php

I am very late but if some one working on SLIM to make rest api and getting same error can solve this problem by adding below line as:

<?php

// DbConnect.php file
class DbConnect
{
    //Variable to store database link
    private $con;

    //Class constructor
    function __construct()
    {

    }

    //This method will connect to the database
    function connect()
    {
        //Including the constants.php file to get the database constants
        include_once dirname(__FILE__) . '/Constants.php';

        //connecting to mysql database
        $this->con = new mysqli(DB_HOST, DB_USERNAME, DB_PASSWORD, DB_NAME);

		mysqli_set_charset($this->con, "utf8"); // add this line 
        //Checking if any error occured while connecting
        if (mysqli_connect_errno()) {
            echo "Failed to connect to MySQL: " . mysqli_connect_error();
        }

        //finally returning the connection link
        return $this->con;
    }
}

Solution 11 - Php

Using setLocale('fr_FR.UTF8') before json_encode solved the problem.

Content Type	Original Author	Original Content on Stackoverflow
Question	Artjom Kurapov	View Question on Stackoverflow
Solution 1 - Php	Robert Imhoff	View Answer on Stackoverflow
Solution 2 - Php	Artjom Kurapov	View Answer on Stackoverflow
Solution 3 - Php	serge.k	View Answer on Stackoverflow
Solution 4 - Php	Emil Vikström	View Answer on Stackoverflow
Solution 5 - Php	Can Uludağ	View Answer on Stackoverflow
Solution 6 - Php	Evert	View Answer on Stackoverflow
Solution 7 - Php	rharvey	View Answer on Stackoverflow
Solution 8 - Php	Deepika Patel	View Answer on Stackoverflow
Solution 9 - Php	Jamie Deakin	View Answer on Stackoverflow
Solution 10 - Php	inrsaurabh	View Answer on Stackoverflow
Solution 11 - Php	KeizerBridge	View Answer on Stackoverflow