PHP json encode - Malformed UTF-8 characters, possibly incorrectly encoded

I'm using json_encode($data) to an data array and there's a field contains Russian characters.

I used this mb_detect_encoding() to display what encoding it is for that field and it displays UTF-8.

I think the json encode failed due to some bad characters in it like "ра▒". I tried alot of things utf8_encode on the data and it will by pass that error but then the data doesn't look correct anymore.

What can be done with this issue?

The issue happens if there are some non-utf8 characters inside even though most of them are utf8 chars. This will remove any non-utf8 characters and now it works.

$data['name'] = mb_convert_encoding($data['name'], 'UTF-8', 'UTF-8');

If you have a multidimensional array to encode in JSON format then you can use below function:

If JSON_ERROR_UTF8 occurred :

$encoded = json_encode( utf8ize( $responseForJS ) );

Below function is used to encode Array data recursively

/* Use it for json_encode some corrupt UTF-8 chars
 * useful for = malformed utf-8 characters possibly incorrectly encoded by json_encode
 */
function utf8ize( $mixed ) {
    if (is_array($mixed)) {
        foreach ($mixed as $key => $value) {
            $mixed[$key] = utf8ize($value);
        }
    } elseif (is_string($mixed)) {
        return mb_convert_encoding($mixed, "UTF-8", "UTF-8");
    }
    return $mixed;
}

Please, make sure to initiate your Pdo object with the charset iso as utf8. This should fix this problem avoiding any re-utf8izing dance.

$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');

With php 7.2, two options allow to manage invalid UTF-8 direcly in json_encode :

https://www.php.net/manual/en/function.json-encode

json_encode($text, JSON_INVALID_UTF8_IGNORE);

Or

json_encode($text, JSON_INVALID_UTF8_SUBSTITUTE);

you just add in your pdo connection charset=utf8 like below line of pdo connection:

$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');

hope this will help you

Remove HTML entities before JSON encoding. I used html_entity_decode() in PHP and the problem was solved

$json = html_entity_decode($source);
$data = json_decode($json,true);

Do you by any chance have UUIDs in your result set? In that case the following database flag will help:

PDO::DBLIB_ATTR_STRINGIFY_UNIQUEIDENTIFIER => true

If your data is well encoded in the database for example, make sure to use the mb_ * functions for string handling, before json_encode. Functions like substr or strlen do not work well with utf8mb4 and can cut your text and leave a malformed UTF8

I know this is kind of an old topic, but for me it was what I needed. I just needed to modify the answer 'jayashan perera'.

//...code
        $stmt->execute();
        $result = $stmt->fetchAll(PDO::FETCH_ASSOC);


        for ($i=0; $i < sizeof($result) ; $i++) { 
            $tempCnpj = $result[$i]['CNPJ'];
            $tempFornecedor = json_encode(html_entity_decode($result[$i]['Nome_fornecedor']),true) ;
            $tempData = $result[$i]['efetivado_data'];
            $tempNota = $result[$i]['valor_nota'];
            $arrResposta[$i] = ["Status"=>"true", "Cnpj"=>"$tempCnpj", "Fornecedor"=>$tempFornecedor, "Data"=>"$tempData", "Nota"=>"$tempNota" ];
        }

        echo json_encode($arrResposta);

And no .js i have use

obj = JSON.parse(msg);