django.db.utils.ProgrammingError: relation already exists
PythonDjangoPostgresqlUbuntuPython Problem Overview
I'm trying to set up the tables for a new django project (that is, the tables do NOT already exist in the database); the django version is 1.7 and the db back end is PostgreSQL. The name of the project is crud. Results of migration attempt follow:
python manage.py makemigrations crud
Migrations for 'crud':
0001_initial.py:
- Create model AddressPoint
- Create model CrudPermission
- Create model CrudUser
- Create model LDAPGroup
- Create model LogEntry
- Add field ldap_groups to cruduser
- Alter unique_together for crudpermission (1 constraint(s))
python manage.py migrate crud
Operations to perform:
Apply all migrations: crud
Running migrations:
Applying crud.0001_initial...Traceback (most recent call last):
File "manage.py", line 18, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 161, in handle
executor.migrate(targets, plan, fake=options.get("fake", False))
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 68, in migrate
self.apply_migration(migration, fake=fake)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 102, in apply_migration
migration.apply(project_state, schema_editor)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 108, in apply
operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/models.py", line 36, in database_forwards
schema_editor.create_model(model)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 262, in create_model
self.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 103, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 82, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 66, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 66, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "crud_crudpermission" already exists
Some highlights from the migration file:
dependencies = [
('auth', '0001_initial'),
('contenttypes', '0001_initial'),
]
migrations.CreateModel(
name='CrudPermission',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('_created_by', models.CharField(default=b'', max_length=64, null=True, editable=False, blank=True)),
('_last_updated_by', models.CharField(default=b'', max_length=64, null=True, editable=False, blank=True)),
('_created', models.DateTimeField(null=True, editable=False, blank=True)),
('_last_updated', models.DateTimeField(null=True, editable=False, blank=True)),
('domain', models.CharField(max_length=32, choices=[(b'town', b'Town'), (b'boe', b'BOE'), (b'police', b'Police')])),
('ldap_group', models.CharField(max_length=128, verbose_name=b'LDAP group')),
('can_add', models.BooleanField(default=False, verbose_name=b'add')),
('can_change', models.BooleanField(default=False, verbose_name=b'change')),
('restrict_change_to_own', models.BooleanField(default=False)),
('can_delete', models.BooleanField(default=False, verbose_name=b'delete')),
('restrict_delete_to_own', models.BooleanField(default=False)),
('models', models.ManyToManyField(to='contenttypes.ContentType', null=True, blank=True)),
],
options={
'verbose_name': 'CRUD permission',
},
bases=(models.Model,),
),
migrations.AlterUniqueTogether(
name='crudpermission',
unique_together=set([('ldap_group', 'can_add', 'can_change', 'can_delete', 'domain')]),
)
,
The crud app is not meant to actually do anything, but I use it another app, so when I try migrate from that app, I trigger the above problem.
I've found other examples on the web of people with similar issues, but none of their cases seem to apply because
- The problem affects an entire relation, not just one column
- I am not using multiple inheritance.
Where should I look next to find the underlying problem?
Python Solutions
Solution 1 - Python
This works pretty fine
./manage.py migrate --fake default
https://docs.djangoproject.com/en/2.2/ref/django-admin/#cmdoption-migrate-fake
Solution 2 - Python
Do python manage.py migrate --fake
initally.
https://docs.djangoproject.com/en/3.2/ref/django-admin/#django-admin-migrate
Solution 3 - Python
I've faced similar issue when added couple new fields to existing model. I'm using Django 1.9, which introduced --run-syncdb
option. Running manage.py migrate --run-syncdb
fixed my tables.
Solution 4 - Python
Now (I'm using Django 1.9) you can make:
./manage.py migrate [--database DATABASE] --fake [app_label] [migration_name]
This way you're targeting the problem with more accuracy, and you can fake only the problematic migration on the specific database.
So, looking at the question, you could:
./manage.py migrate --database default --fake crud crud.0001_initial
Solution 5 - Python
Been facing a similar issue, eventually deleted all .py files in migration folder (django 1.7 creates one automatically), worked perfectly after that.
Solution 6 - Python
I was facing the similar issues, where i had changed column name. I was getting same error as mentioned in the stack-trace provided with he question.
Here's what I did.
I ran fake migrations first. Then i removed it's(migrations i wanted to run) entry from django_migrations table and ran migrations again(no fake this time).
Changes appeared as expected for me.
hope this is helpful.
Solution 7 - Python
Django provides a --fake-initial
option which I found effective for my use. From the Django Migration Documentation:
> --fake-initial > > Allows Django to skip an app’s initial migration if all database > tables with the names of all models created by all CreateModel > operations in that migration already exist. This option is intended > for use when first running migrations against a database that > preexisted the use of migrations. This option does not, however, check > for matching database schema beyond matching table names and so is > only safe to use if you are confident that your existing schema > matches what is recorded in your initial migration.
For my use, I had just pulled a project from version control and was preparing to add some new model fields. I added the fields, ran ./manage.py makemigrations
and then attempted to run ./manage.py migrate
which threw the error since, as one would expect, many of the fields already existed in the existing database.
What I should have done was to run makemigrations
immediately after pulling the project from versions control to create a snapshot of existing models' state. Then, running the ./manage.py migrate --fake-initial
would be the next step.
After that you can add away and makemigrations
> migrate
as normal.
NOTE: I do not know if a --fake-initial
would skip existing fields and add new ones. I opted to comment out the new fields I'd created up to that point, run the --fake-initial
as if it were the first thing I did after pulling from version control, then added in updated fields in the next migration.
Other related documentation: https://docs.djangoproject.com/en/dev/topics/migrations/#initial-migrations
Solution 8 - Python
I've been dealing with this for several years. There could be different scenarios:
Scenario 1: as in the original post, you had no tables to start with. In this case, I'd
- comment out the relationship in models.py
- run python manage.py
- migrate assuming that it now succeeds
- uncomment what you
- commented out in step 1 run python manage.py migrate --fake
Scenario 2: multiple apps: One possibility is that you might have different apps and the data model of one app is using some tables from the other app. In this case, if the data model is designed properly, you should be able to create the tables for one app only (by specifying only that one in setting.py), then add the other app and migrate. If it is not design with care, and there are recursive dependencies, I suggest changing the design rather than making a temporary fix.
Scenario 3: you had some tables and something went wrong with your migration, then I'd
- revert models.py to what it was and only introduce the new relationship that appears to already exist in models.py.
- delete migration folder
- run python manage.py makemigrations
- introduce new changes to models.py if any and continue with makemigrations and migrate commands as usual.
Solution 9 - Python
I found and solved a particular example of this error in a Django 1.10 project while I was changing a foreign key field named member
to point to a different table. I was changing this in three different models and I hit this error on all of them. In my first attempt I renamed member
to member_user
and tried to create a new field member
as a foreign key pointing at the new table, but this didn't work.
What I found is that when I renamed the member
column it did not modify the index name in the form <app>_<model>_<hash>
and when I tried to create a new member
column it tried to create the same index name because the hash portion of the name was the same.
I resolved the problem by creating a new member_user
relation temporarily and copying the data. This created a new index with a different hash. I then deleted member
and recreated it pointing at the new table and with it the would-be conflicting index name. Once that was done I ran the RunPython
step to populate the new member
column with references to the applicable table. I finished by adding RemoveField
migrations to clean up the temporary member_user
columns.
I did have to split my migrations into two files because I received this error:
> psycopg2.OperationalError: cannot ALTER TABLE "<table_name>" because it has pending trigger events
After the creation and copy of data into member_user
I was not able to remove member
in the same migration transaction. This may be a postgres specific limitation, but it was easily resolved by creating another transaction and moving everything after creating and copying member_user
into the second migration.
Solution 10 - Python
In my case, a migration file was deleted and a new one was auto-generated, which had a different name. Because of the name difference, Django tried to apply the new migration file, which was exactly same as the previously applied one, which was now removed. In both of them, a new model had to be created which resulted in django.db.utils.ProgrammingError: relation "app_space" already exists
. I tried to reverse the migration, but the missing migration file prevented django from actually reversing it. Lesson learnt, migration files should be checked into git.
Here are some steps that helped me to get to the bottom of this. --fake
solved it temporarily but the same issue happened in the next migration. I wouldn't recommend --fake
unless you know for sure it is the right use case for it.
This answer was really key for me.
-
Check previous migrations
python manage.py showmigrations
-
Check applied migrations in Django DB
select * from django_migrations;
(usepsql
to access postgres db console:psql -h 0.0.0.0 -U <your-db-user>
then use target db\c <your-db-name>
). -
I saw an applied migration that was no longer in my migrations folder.
20 | app | 0001_initial | 2021-03-05 07:40:20.14602+00
21 | app | 0002_some_auto_name | 07:40:20.269775+00
22 | app | 0003_auto_20210318_2350 <---here | 2021-03-18 23:50:09.064971+00
But in migrations folder I had 0003_auto_20210318_2355
, the same file generated 5 minutes later. I renamed the migration file to the name above so that I could reverse it.
- Reverse the migration by passing the migration to which you want to return.
python manage.py migrate <app-name> <latest-migration-to-which-to-return>
python manage.py migrate app 0002_some_auto_name
- Do the right thing from here and check in the migrations to git. Then you can do
makemigrations
andmigrate
and have a more peaceful life.
Solution 11 - Python
For me, When I faced this exception, I solve it using the Django dbshell utility or any kind of MY_DATABASE Viewer / interactive command line.
DBShell:
python manage.py dbshell
ALTER TABLE [name_of_field_that_already_exists] DROP column [field_table];
Solution 12 - Python
I found this problem in web2pyframework
in models/config.py
.
Change
settings.base.migrate = True
on config file to
settings.base.migrate = False
Problem solved.
Solution 13 - Python
I recently had the same issue and tried some of the solutions here. manage.py migrate --fake
led to a "django_content_type" does not exist
error. Equally deleting old migrations might cause problems for other users if the migrations are shared.
The manage.py squashmigrations
command (docs) seems to be the ideal way to deal with this. Condenses old migrations into a single migration (which prevents applying them out of sequence etc), and preserves the old migrations for any other users. It worked in my case at least.
Solution 14 - Python
I am not sure about using the solution with fake. Most likely that error will occur again at the next migration.
Find out which columns are creating that problem
python manage.py dbshell
select * from <tablename> where false;
(Now you see which columns are saved by postgresql and can delete them in the database)alter table <tablename> drop column <columnname>;
(Start the migrations process)python manage.py makemigrations
python manage.py migrate
Solution 15 - Python
If you're getting this error when you run python manage.py test --k
, you can fix it by deleting the test database: python manage.py test
Solution 16 - Python
Do not try to use --fake
, because with this you risk corrupting your database.
Instead, you can backup your current database, recreate it, and then apply backup.
-
Create backup:
pg_dump -F c -v -h localhost <database_name> -f tmp/<pick_a_file_name>.psql
-
Recreate it:
rails db:drop db:create
-
Restore backup:
pg_restore --exit-on-error --verbose --dbname=<database_name> tmp/<pick_a_file_name>.psql