Optimize mySql for faster alter table add column

MysqlPerformanceAlter

Mysql Problem Overview


I have a table that has 170,002,225 rows with about 35 columns and two indexes. I want to add a column. The alter table command took about 10 hours. Neither the processor seemed busy during that time nor were there excessive IO waits. This is on a 4 way high performance box with tons of memory.

Is this the best I can do? Is there something I can look at to optimize the add column in tuning of the db?

Mysql Solutions


Solution 1 - Mysql

I faced a very similar situation in the past and i improve the performance of the operation in this way :

  1. Create a new table (using the structure of the current table) with the new column(s) included.
  2. execute a INSERT INTO new_table (column1,..columnN) SELECT (column1,..columnN) FROM current_table;
  3. rename the current table
  4. rename the new table using the name of the current table.

Solution 2 - Mysql

ALTER TABLE in MySQL is actually going to create a new table with new schema, then re-INSERT all the data and delete the old table. You might save some time by creating the new table, loading the data and then renaming the table.

From "High Performance MySQL book" (the percona guys):

> The usual trick for loading MyISAM table efficiently is to disable keys, load the data and renalbe the keys:

mysql> ALTER TABLE test.load_data DISABLE KEYS;
-- load data
mysql> ALTER TABLE test.load_data ENABLE KEYS;

Solution 3 - Mysql

Well, I would recommend using latest Percona MySQL builds plus since there is the following note in MySQL manual

> In other cases, MySQL creates a > temporary table, even if the data > wouldn't strictly need to be copied. > For MyISAM tables, you can speed up > the index re-creation operation (which > is the slowest part of the alteration > process) by setting the > myisam_sort_buffer_size system > variable to a high value.

You can do ALTER TABLE DISABLE KEYS first, then add column and then ALTER TABLE ENABLE KEYS. I don't see anything can be done here.

BTW, can't you go MongoDB? It doesn't rebuild anything when you add column.

Solution 4 - Mysql

Maybe you can remove the index before alter the table because what is take most of the time to build is the index?

Solution 5 - Mysql

Combining some of the comments on the other answers, this was the solution that worked for me (MySQL 5.6):

  1. create table mytablenew like mytable;
  2. alter table mytablenew add column col4a varchar(12) not null after col4;
  3. alter table mytablenew drop index index1, drop index index2,...drop index indexN;
  4. insert into mytablenew (col1,col2,...colN) select col1,col2,...colN from mytable;
  5. alter table mytablenew add index index1 (col1), add index index2 (col2),...add index indexN (colN);
  6. rename table mytable to mytableold, mytablenew to mytable

On a 75M row table, dropping the indexes before the insert caused the query to complete in 24 minutes rather than 43 minutes.

Other answers/comments have insert into mytablenew (col1) select (col1) from mytable, but this results in ERROR 1241 (21000): Operand should contain 1 column(s) if you have the parenthesis in the select query.

Other answers/comments have insert into mytablenew select * from mytable;, but this results in ERROR 1136 (21S01): Column count doesn't match value count at row 1 if you've already added a column.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAndrewView Question on Stackoverflow
Solution 1 - MysqlRRUZView Answer on Stackoverflow
Solution 2 - MysqlFoneyOpView Answer on Stackoverflow
Solution 3 - MysqlVladislav RastrusnyView Answer on Stackoverflow
Solution 4 - MysqlMicromegaView Answer on Stackoverflow
Solution 5 - MysqlenharmonicView Answer on Stackoverflow