MySQL truncates concatenated result of a GROUP_CONCAT function

MysqlViewTypesTruncate

Mysql Problem Overview


I've created a view which uses GROUP_CONCAT to concatenate results from a query on products column with data type of 'varchar(7) utf8_general_ci' in a column named concat_products.

The problem is that MySQL truncates value of "concat_products" column. phpMyAdmin says the data type of "concat_products" column is varchar(341) utf8_bin

Table products:

CREATE TABLE `products`(
    `productId` tinyint(2) unsigned NOT NULL AUTO_INCREMENT, 
    `product` varchar(7) COLLATE utf8_general_ci NOT NULL, 
    `price` mediumint(5) unsigned NOT NULL, 
    PRIMARY KEY (`productId`)
) ENGINE=InnoDB AUTO_INCREMENT=28 DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci

The "concat_products_vw" view:

CREATE VIEW concat_products_vw AS
SELECT
  `userId`,
  	GROUP_CONCAT(CONCAT_WS('_', `product`, `productId`, `price`) 
  		ORDER BY `productId` ASC SEPARATOR '*') AS concat_products
FROM
  `users`
LEFT JOIN `products` 
ON `users`.`accountBalance` >= `product`.`price`
GROUP BY `productId` 

According to MySQL manual:

> Values in VARCHAR columns are variable-length strings
Length can be specified as a value from 1 to 255 before MySQL 4.0.2 and 0 to 255 as of MySQL 4.0.2.


EDIT

Values in VARCHAR columns are variable-length strings. The length can be specified as a value from 0 to 65,535.

  1. Why MySQL specifies more than 255 characters for varchar "concat_products" column? (solved!)

  2. Why uf8_bin instead of utf8_general_ci?

  3. Is it possible to change the data type of a column in a view for example in my case to text for "concat_products" column?

  4. If not what can I do to prevent MySQL from truncating "concat_products" column?

Mysql Solutions


Solution 1 - Mysql

As I already wrote in an earlier comment, the MySQL manual says:

> Values in VARCHAR columns are variable-length strings. The length can > be specified as a value from 0 to 65,535.

So the problem is not with the data type of the field.

The MySQL manual also says:

> The result is truncated to the maximum length that is given by the > group_concat_max_len system variable, which has a default value of > 1024. The value can be set higher, although the effective maximum length of the return value is constrained by the value of > max_allowed_packet. The syntax to change the value of > group_concat_max_len at runtime is as follows, where val is an > unsigned integer: > SET [GLOBAL | SESSION] group_concat_max_len = val;

Your options for changing the value of group_concat_max_len are:

  1. changing the value at MySQL startup by appending this to the command:
    --group_concat_max_len=your_value_here
  2. adding this line in your MySQL configuration file (mysql.ini): group_concat_max_len=your_value_here
  3. running this command after MySQL startup:
    SET GLOBAL group_concat_max_len=your_value_here;
  4. running this command after opening a MySQL connection:
    SET SESSION group_concat_max_len=your_value_here;

Documentation: SET, Server System Variables: group_concat_max_len

Solution 2 - Mysql

As Jocelyn mentioned, the size of a GROUP_CONCAT() result is bounded by group_concat_max_len, however there is an additional interaction with ORDER BY that results in a further truncation to 1/3 of group_concat_max_len. For an example, see this related answer.

The default value for group_concat_max_len is 1024, and 1024 / 3 = 341 probably explains why the type of concat_products shows up as varchar(341) in the original example. If you were to remove the GROUP BY productId clause, concat_products should show up as varchar(1024).

I have not found this interaction between GROUP_CONCAT() and ORDER BY mentioned in the MySQL Manual, but it affects at least MySQL Server 5.1.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionpouyaView Question on Stackoverflow
Solution 1 - MysqlJocelynView Answer on Stackoverflow
Solution 2 - MysqlJoshSView Answer on Stackoverflow