They aren&#39;t used in parallel. NGINX is a *[reverse proxy][1]*. It&#39;s first in line. It accepts incoming connections and decides where they should go next. It also (usually) serves static media such as CSS, JS and images. It can also do other things such as encryption via SSL, caching etc.


Gunicorn is the next layer and is an *[application server][2]*. NGINX sees that the incoming connection is for `www.domain.com` and knows (via configuration files) that it should pass that connection onto Gunicorn. Gunicorn is a [WSGI][3] server which is basically a:

&gt; simple and universal interface between web servers and web applications or frameworks

Gunicorn&#39;s job is to manage and run the Django instance(s) (similar to using `django-admin runserver` during development)

The contrast to this setup is to use Apache with the `mod_wsgi` module. In this situation, the application server is actually a part of Apache, running as a module.


  [1]: http://en.wikipedia.org/wiki/Reverse_proxy
  [2]: http://en.wikipedia.org/wiki/Application_server
  [3]: http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface

*Nginx and Gunicorn are not used in parrallel.*

 - Gunicorn, is a Web Server Gateway Interface (WSGI) server
   implementation that is commonly used to run Python web applications.
 - NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server.
 - Nginx responsible for serving static content, gzip compression, ssl,
   proxy_buffers and other HTTP stuff.While gunicorn is a Python HTTP server that interfaces with both nginx and your actual python web-app code to serve dynamic content.

*The following diagrams shows how nginx and Gunicorn interact.*


[![Nginx and Gunicorn][1]][1][![Overall idea of nginx and Gunicorn][2]][2]

  


  [1]: https://i.stack.imgur.com/9GAQc.png
  [2]: https://i.stack.imgur.com/Bxq7E.png

I&#39;m trying to convert a chunk of HTML text with BeautifulSoup. Here is an example:

    &lt;div&gt;
        &lt;p&gt;
            Some text
            &lt;span&gt;more text&lt;/span&gt;
            even more text
        &lt;/p&gt;
        &lt;ul&gt;
            &lt;li&gt;list item&lt;/li&gt;
            &lt;li&gt;yet another list item&lt;/li&gt;
        &lt;/ul&gt;
    &lt;/div&gt;
    &lt;p&gt;Some other text&lt;/p&gt;
    &lt;ul&gt;
        &lt;li&gt;list item&lt;/li&gt;
        &lt;li&gt;yet another list item&lt;/li&gt;
    &lt;/ul&gt;


I tried doing something like:

    def parse_text(contents_string)
        Newlines = re.compile(r&#39;[\r\n]\s+&#39;)
        bs = BeautifulSoup.BeautifulSoup(contents_string, convertEntities=BeautifulSoup.BeautifulSoup.HTML_ENTITIES)
        txt = bs.getText(&#39;\n&#39;)
        return Newlines.sub(&#39;\n&#39;, txt)

...but that way my span element is always on a new line. This is of course a simple example. Is there a way to get the text in the HTML page as the way it will be rendered in the browser (no css rules required, just the regular way div, span, li, etc. elements are rendered) in Python?

Rendered HTML to plain text using Python

I&#39;ve got a fairly straightforward function which returns a jQuery .ajax() promise as such:

    CLAW.controls.validateLocation = function(val, $inputEl) {
        return $.ajax({
            url: locationServiceUrl + &#39;ValidateLocation/&#39;,
            data: {
                &#39;locationName&#39;: val
            },
            beforeSend: function() {
                $inputEl.addClass(&#39;busy&#39;);
            }
        }).done(function(result) {
            // some success clauses
        }).fail(function(result) {
            // some failure clauses
        }).always(function() {
            // some always clauses
        });
    }

For the most part, this new promises interface works like a dream, and eliminating callback pyramids when using jQuery&#39;s .ajax() is great. However, I cannot for the life of me figure out how to properly test these promises using Jasmine and/or Sinon:

 1. All of Sinon&#39;s documentation assumes you&#39;re using old-school
    callbacks; I don&#39;t see a single example of how to use it with
    promises/deferreds

 2. When attempting to use a Jasmine or Sinon spy to spy on $.ajax, the
    spy is effectively overwriting the promise, so its `done`, `fail`,
    and `always` clauses no longer exist on the ajax function, so the promise never resolves and tosses an error instead

I&#39;d really just love an example or two of how to test these new jQuery .ajax() promises with the aforementioned testing libs. I&#39;ve scoured the &#39;net fairly intensely and haven&#39;t really dredged up anything on doing so. The one resource I did find mentioned using Jasmine.ajax, but I&#39;d like to avoid that if possible, seeing as Sinon provides most of the same capabilities out-of-the-box.

How to properly unit test jQuery&#39;s .ajax() promises using Jasmine and/or Sinon?

A lot of Django app deployments over Amazon&#39;s EC2 use HTTP servers NGINX and Gunicorn. 

I was wondering what they actually do and why both are used in parallel. What is the purpose of running them both in parallel?

What is the purpose of NGINX and Gunicorn running in parallel?

<p>A lot of Django app deployments over Amazon's EC2 use HTTP servers NGINX and Gunicorn.</p>
<p>I was wondering what they actually do and why both are used in parallel. What is the purpose of running them both in parallel?</p>


Hi I have an infuriating problem.

I have a url pattern like this:

    # mproject/myapp.urls.py
    
    url(r&#39;^project/(?P&lt;project_id&gt;\d+)/$&#39;,&#39;user_profile.views.EditProject&#39;,name=&#39;edit_project&#39;),

it works fine in the browser but for testing, when I do this in the shell:

    from django.test import Client
    from django.core.urlresolvers import reverse

    client= Client()
    response = client.get(reverse(&#39;edit_project&#39;), project_id=4)

I get the dreaded:

    NoReverseMatch: Reverse for &#39;edit_project&#39; with arguments &#39;()&#39; and keyword arguments &#39;{}&#39; not found.


What am I missing here? 

Django Reverse with arguments &#39;()&#39; and keyword arguments &#39;{}&#39; not found

Which is it better to use generally?

 - https://docs.djangoproject.com/en/dev/topics/http/shortcuts/#redirect

 - https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpResponseRedirect

Is there any advantage to writing:

    return redirect(my_url)

over:

    return HttpResponseRedirect(my_url)

Or is it a direct alias? Is there any difference? Which is more pythonic/django-nic?

What the difference between using Django redirect and HttpResponseRedirect?

Let&#39;s pretend I have the following QueryDict: 

    &lt;QueryDict: {u&#39;num&#39;: [0], u&#39;var1&#39;: [u&#39;value1&#39;, u&#39;value2&#39;], u&#39;var2&#39;: [u&#39;8&#39;]}&gt;

I&#39;d like to have a dictionary out of this, eg:

    {&#39;num&#39;: [0], &#39;var1&#39;:[&#39;value1&#39;, &#39;value2&#39;], &#39;var2&#39;:[&#39;8&#39;]}

(I don&#39;t care if the unicode symbol `u` stays or goes.)

If I do `queryDict.dict()`, as suggested by the [django site][1], I lose the extra values belonging to `var1`, eg:

    {&#39;num&#39;: [0], &#39;var1&#39;:[&#39;value2&#39;], &#39;var2&#39;:[&#39;8&#39;]}

I was thinking of doing this:

    myDict = {}
    for key in queryDict.iterkeys():
        myDict[key] = queryDict.getlist(key)

Is there a better way? 


  [1]: https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.QueryDict.dict

How to change a django QueryDict to Python Dict?

I&#39;m trying to get Celery logging working with `Django`. I have logging set-up in `settings.py` to go to console (that works fine as I&#39;m hosting on `Heroku`). At the top of each module, I have:

    import logging
    logger = logging.getLogger(__name__)

And in my tasks.py, I have:

    from celery.utils.log import get_task_logger
    logger = get_task_logger(__name__)

That works fine for logging calls from a task and I get output like this:

    2012-11-13T18:05:38+00:00 app[worker.1]: [2012-11-13 18:05:38,527: INFO/PoolWorker-2] Syc feed is starting

But if that task then calls a method in another module, e.g. a `queryset` method, I get duplicate log entries, e.g.

    2012-11-13T18:00:51+00:00 app[worker.1]: [INFO] utils.generic_importers.ftp_processor process(): File xxx.csv already imported. Not downloaded
    2012-11-13T18:00:51+00:00 app[worker.1]: [2012-11-13 18:00:51,736: INFO/PoolWorker-6] File xxx.csv already imported. Not downloaded

I think I could use

    CELERY_HIJACK_ROOT_LOGGER = False
to just use the `Django` logging but this didn&#39;t work when I tried it and even if I did get it to work, I would lose the `&quot;PoolWorker-6&quot;` bit which I do want. (Incidentally, I can&#39;t figure out how to get the task name to display in the log entry from Celery, as [the docs][docs] seems to indicate that it should).

I suspect I&#39;m missing something simple here.

[docs]: http://docs.celeryproject.org/en/master/configuration.html?highlight=celeryd_task_log_format#std:setting-CELERYD_TASK_LOG_FORMAT

Django Celery Logging Best Practice

I have model that looks like this:

    class Category(models.Model):
        parentCategory = models.ForeignKey(&#39;self&#39;, blank=True, null=True, related_name=&#39;subcategories&#39;)
        name = models.CharField(max_length=200)
        description = models.CharField(max_length=500)

I managed to get flat json representation of all categories with serializer:

    class CategorySerializer(serializers.HyperlinkedModelSerializer):
        parentCategory = serializers.PrimaryKeyRelatedField()
        subcategories = serializers.ManyRelatedField()

        class Meta:
            model = Category
            fields = (&#39;parentCategory&#39;, &#39;name&#39;, &#39;description&#39;, &#39;subcategories&#39;)

Now what I want to do is for subcategories list to have inline json representation of subcategories instead of their ids. How would I do that with django-rest-framework? I tried to find it in documentation, but it seems incomplete.



Django rest framework nested self-referential objects

When is it necessary to restart nginx and reload will not suffice?  
Does it make a difference if an extension like [passenger][1] is used?

Should the service be restarted if it consumes too much memory. Any other reasons for restarting Nginx, particularly after a configuration change either in an extension or a Nginx core config?

After making a configuration change, one can either restart or reload nginx, via the [binary][2] itself or the init.d script &quot;/etc/init.d/nginx -h&quot; on Ubuntu.  Which method should be preferred? 

  [1]: http://www.modrails.com/documentation/Users%20guide%20Nginx.html
  [2]: http://wiki.nginx.org/CommandLine#Starting.2C_Stopping.2C_and_Restarting_Nginx

When to restart and not reload Nginx?

In my nginx conf file, I have :

      listen       80;
        server_name  $hostname;

however if I do netstat I see that it is listening on 0.0.0.0:80 

what I want to happen, is the nginx to listen to $hostname:80 , is there a way to configure it to do that? 

I tried different settings with no success so far. Appreciate your help. 

How to make nginx to listen to server_name:port

I want to make a Flask+Nginx+Gunicorn deployment. I have Nginx setup and running and I run gunicorn as described in the docs:

    gunicorn app:app

But when I logout of the server the gunicorn process exits? What is the correct way to make sure it stay running for Nginx to connect to, and restarts if it crashes?

What is the correct way to leave gunicorn running?

On server-side using Sinatra with a `stream` block.

    get &#39;/stream&#39;, :provides =&gt; &#39;text/event-stream&#39; do
      stream :keep_open do |out|
        connections &lt;&lt; out
        out.callback { connections.delete(out) }
      end
    end

On client side:

    var es = new EventSource(&#39;/stream&#39;);
    es.onmessage = function(e) { $(&#39;#chat&#39;).append(e.data + &quot;\n&quot;) };

When i using app directly, via `http://localhost:9292/`, everything works perfect. The connection is persistent and all messages are passed to all clients.

However when it goes through Nginx, `http://chat.dev`, the connection are dropped and a reconnection fires every second or so.

Nginx setup looks ok to me:

    upstream chat_dev_upstream {
      server 127.0.0.1:9292;
    }

    server {
      listen       80;
      server_name  chat.dev;

      location / {
        proxy_pass http://chat_dev_upstream;
        proxy_buffering off;
        proxy_cache off;
        proxy_set_header Host $host;
      }
    }

Tried `keepalive 1024` in `upstream` section as well as `proxy_set_header Connection keep-alive;`in `location`.

Nothing helps :(

No persistent connections and messages not passed to any clients.





EventSource / Server-Sent Events through Nginx

I have a Rails app up and running on my server and now I&#39;d like to add another one.

I want Nginx to check what the request is for and split traffic based on domain name

Both sites have their own nginx.conf symlinked into sites-enabled, but I get an error starting nginx `Starting nginx: nginx: [emerg] duplicate listen options for 0.0.0.0:80 in /etc/nginx/sites-enabled/bubbles:6`

They are both listening on 80 but for different things.

Site #1

    upstream blog_unicorn {
      server unix:/tmp/unicorn.blog.sock fail_timeout=0;
    }
    
    server {
      listen 80 default deferred;
      server_name walrus.com www.walrus.com;
      root /home/deployer/apps/blog/current/public;
    
      location ^~ /assets/ {
        gzip_static on;
        expires max;
        add_header Cache-Control public;
      }
    
      try_files $uri/index.html $uri @blog_unicorn;
      location @blog_unicorn {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://blog_unicorn;
      }
    
      error_page 500 502 503 504 /500.html;
      client_max_body_size 4G;
      keepalive_timeout 10;
    }

Site two:

    upstream bubbles_unicorn {
      server unix:/tmp/unicorn.bubbles.sock fail_timeout=0;
    }
    
    server {
      listen 80 default deferred;
      server_name bubbles.com www.bubbles.com;
      root /home/deployer/apps/bubbles/current/public;
    
      location ^~ /assets/ {
        gzip_static on;
        expires max;
        add_header Cache-Control public;
      }
    
      try_files $uri/index.html $uri @bubbles_unicorn;
      location @bubbles_unicorn {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://bubbles_unicorn;
      }
    
      error_page 500 502 503 504 /500.html;
      client_max_body_size 4G;
      keepalive_timeout 10;
    }

Serving two sites from one server with Nginx

I&#39;m starting gunicorn with the Django command `python manage.py run_gunicorn`. How can I stop gunicorn properly?

Note: I have a semi-automated server deployment with fabric. Thus using something like `ps aux | grep gunicorn` to kill the process manually by pid is not an option. 

How to stop gunicorn properly

I am running a **Flask/Gunicorn** Python app on a Heroku Cedar dyno. The app returns `JSON responses` to its clients (it&#39;s an `API server`, really).

Once in a while clients get 0-byte responses. It&#39;s not me returning them, however. Here is a snippet of my app&#39;s log:

&gt; Mar 14 13:13:31 d.0b1adf0a-0597-4f5c-8901-dfe7cda9bce0 app[web.1]
&gt; [2013-03-14 13:13:31 UTC] 10.104.41.136 apisrv -
&gt; api_get_credits_balance(): session_token=[MASKED]

The first line above is me starting to handle the request. 

&gt; Mar 14 13:13:31 d.0b1adf0a-0597-4f5c-8901-dfe7cda9bce0 app[web.1]
&gt; [2013-03-14 13:13:31 UTC] 10.104.41.136 apisrv 1252148511
&gt; api_get_credits_balance(): returning [{&#39;credits_balance&#39;: 0}]

The second line is me returning a value (to Flask -- it&#39;s a Flask &quot;Response&quot; object).

&gt; Mar 14 13:13:31 d.0b1adf0a-0597-4f5c-8901-dfe7cda9bce0 app[web.1]
&gt; &quot;10.104.41.136 - - [14/Mar/2013:13:13:31] &quot;POST
&gt; /get_credits_balance?session_token=MASKED HTTP/1.1&quot; 200 22 &quot;-&quot;
&gt; &quot;Appcelerator Titanium/3.0.0.GA (iPhone/6.1.2; iPhone OS; en_US;)&quot;

The third line is Gnicorn&#39;s, where you can see the Gunicorn got the 200 status and 22 bytes HTTP body (&quot;`200 22`&quot;).

However, the client got 0 bytes. Here is the Heroku router log:

&gt; Mar 14 13:13:30 d.0b1adf0a-0597-4f5c-8901-dfe7cda9bce0 heroku[router]
&gt; at=info method=POST path=/get_credits_balance?session_token=MASKED
&gt; host=matchspot-apisrv.herokuapp.com fwd=&quot;66.87.116.128&quot; dyno=web.1
&gt; queue=0 wait=0ms connect=1ms service=19ms status=200 bytes=0

Why does Gunicorn return 22 bytes, but Heroku sees 0, and indeed passes back 0 bytes to the client? Is this a Heroku bug?




Heroku truncates HTTP responses?

I&#39;m new to heroku and gunicorn so I&#39;m not sure how this works.  But I&#39;ve done some searching and I think I&#39;m close to deploying my Django app (1.5.1).  So I know I need a Procfile which has


    web: gunicorn app.wsgi

Because my directories are a bit different. I can&#39;t run gunicorn in the root directory

    app_project
        requirements/
        contributors/
        app/
            app/
                settings/
                wsgi.py
            # Normally Procfile goes here
        Procfile

Normally app/ would be the root directory, but I decided to structure my folders this way to separate my django app from some other things.  Since I have to put the Procfile in the root directory for heroku to recognize it, what should I put in the Procfile and/or what parameters should I place in the gunicorn command?


Note:

    web: gunicorn app.wsgi # won&#39;t work because Procfile is in a directory above
                           # I also want to keep the directories as is
                           # I also don&#39;t want to create a secondary git inside the app folder just for heroku
    web: gunicorn app.app.wsgi # won&#39;t work because I don&#39;t want to convert the folder into a python module




How can I modify Procfile to run Gunicorn process in a non-standard folder on Heroku?

I dont know much about python. I want to start working on the project and the setup instruction says:

    pip install -r requirements-dev.txt

Simple enougth. The problem is that I get this: 

        Downloading/unpacking gunicorn==0.13.4 (from -r requirements.txt (line 7))
      Running setup.py egg_info for package gunicorn
        Traceback (most recent call last):
          File &quot;&lt;string&gt;&quot;, line 16, in &lt;module&gt;
          File &quot;/tmp/pip-build-root/gunicorn/setup.py&quot;, line 18, in &lt;module&gt;
            long_description = file(
        NameError: name &#39;file&#39; is not defined
        Complete output from command python setup.py egg_info:
        Traceback (most recent call last):
    
      File &quot;&lt;string&gt;&quot;, line 16, in &lt;module&gt;
    
      File &quot;/tmp/pip-build-root/gunicorn/setup.py&quot;, line 18, in &lt;module&gt;
    
        long_description = file(
    
    NameError: name &#39;file&#39; is not defined

I dont understand the problem. Maybe somebody can help out?

I run this on Arch Linux, python defaults to python 3 and the project is not python 3 but Im not sure if thats it.

Thanks.


Content Type	Original Author	Original Content on Stackoverflow
Question	deadlock	View Question on Stackoverflow
Solution 1 - Django	Timmy O'Mahony	View Answer on Stackoverflow
Solution 2 - Django	Thusitha Deepal	View Answer on Stackoverflow

What is the purpose of NGINX and Gunicorn running in parallel?

Django Problem Overview

Django Solutions

Solution 1 - Django

Solution 2 - Django

How to properly unit test jQuery's .ajax() promises using Jasmine and/or Sinon?

Rendered HTML to plain text using Python

Attributions