Bulk create model objects in django

DjangoDjango Models

Django Problem Overview


I have a lot of objects to save in database, and so I want to create Model instances with that.

With django, I can create all the models instances, with MyModel(data), and then I want to save them all.

Currently, I have something like that:

for item in items:
    object = MyModel(name=item.name)
    object.save()

I'm wondering if I can save a list of objects directly, eg:

objects = []
for item in items:
    objects.append(MyModel(name=item.name))
objects.save_all()

How to save all the objects in one transaction?

Django Solutions


Solution 1 - Django

as of the django development, there exists bulk_create as an object manager method which takes as input an array of objects created using the class constructor. check out django docs

Solution 2 - Django

Use bulk_create() method. It's standard in Django now.

Example:

Entry.objects.bulk_create([
    Entry(headline="Django 1.0 Released"),
    Entry(headline="Django 1.1 Announced"),
    Entry(headline="Breaking: Django is awesome")
])

Solution 3 - Django

worked for me to use manual transaction handling for the loop(postgres 9.1):

from django.db import transaction
with transaction.atomic():
    for item in items:
        MyModel.objects.create(name=item.name)

in fact it's not the same, as 'native' database bulk insert, but it allows you to avoid/descrease transport/orms operations/sql query analyse costs

Solution 4 - Django

Here is how to bulk-create entities from column-separated file, leaving aside all unquoting and un-escaping routines:

SomeModel(Model):
    @classmethod
    def from_file(model, file_obj, headers, delimiter):
        model.objects.bulk_create([
            model(**dict(zip(headers, line.split(delimiter))))
            for line in file_obj],
            batch_size=None)

Solution 5 - Django

name = request.data.get('name')
period = request.data.get('period')
email = request.data.get('email')
prefix = request.data.get('prefix')
bulk_number = int(request.data.get('bulk_number'))

bulk_list = list()
for _ in range(bulk_number):
    code = code_prefix + uuid.uuid4().hex.upper()
    bulk_list.append(
        DjangoModel(name=name, code=code, period=period, user=email))

bulk_msj = DjangoModel.objects.bulk_create(bulk_list)

Solution 6 - Django

Using create will cause one query per new item. If you want to reduce the number of INSERT queries, you'll need to use something else.

I've had some success using the Bulk Insert snippet, even though the snippet is quite old. Perhaps there are some changes required to get it working again.

http://djangosnippets.org/snippets/446/

Solution 7 - Django

Check out this blog post on the bulkops module.

On my django 1.3 app, I have experienced significant speedup.

Solution 8 - Django

for a single line implementation, you can use a lambda expression in a map

map(lambda x:MyModel.objects.get_or_create(name=x), items)

Here, lambda matches each item in items list to x and create a Database record if necessary.

Lambda Documentation

Solution 9 - Django

The easiest way is to use the create Manager method, which creates and saves the object in a single step.

for item in items:
    MyModel.objects.create(name=item.name)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAlexis MétaireauView Question on Stackoverflow
Solution 1 - DjangoecbtlnView Answer on Stackoverflow
Solution 2 - DjangoDanilView Answer on Stackoverflow
Solution 3 - DjangoThorin SchifferView Answer on Stackoverflow
Solution 4 - DjangoIvan KlassView Answer on Stackoverflow
Solution 5 - DjangoYagmur SAHINView Answer on Stackoverflow
Solution 6 - DjangoOmerGertelView Answer on Stackoverflow
Solution 7 - DjangoMrJView Answer on Stackoverflow
Solution 8 - DjangoFallenAngelView Answer on Stackoverflow
Solution 9 - DjangoDaniel RosemanView Answer on Stackoverflow