Try setting the system default encoding as `utf-8` at the start of the script, so that all strings are encoded using that.

Example -

    import sys
    reload(sys)
    sys.setdefaultencoding(&#39;utf-8&#39;)

The above should set the default encoding as `utf-8` .

You really want to do this

    flog.write(&quot;\nCompany Name: &quot;+ pCompanyName.encode(&#39;utf-8&#39;))

This is the &quot;encode late&quot; strategy described in this [unicode presentation][1] (slides 32 through 35).

  [1]: http://farmdev.com/talks/unicode/

I have trouble understanding the difference (if there is one) between ```roc_auc_score()``` and ```auc()``` in scikit-learn.

Im tying to predict a binary output with imbalanced classes (around 1.5% for Y=1).

###Classifier

    model_logit = LogisticRegression(class_weight=&#39;auto&#39;)
    model_logit.fit(X_train_ridge, Y_train)

###Roc curve

    false_positive_rate, true_positive_rate, thresholds = roc_curve(Y_test, clf.predict_proba(xtest)[:,1])

###AUC&#39;s

    auc(false_positive_rate, true_positive_rate)
    Out[490]: 0.82338034042531527
and

    roc_auc_score(Y_test, clf.predict(xtest))
    Out[493]: 0.75944737191205602

Somebody can explain this difference ? I thought both were just calculating the area under the ROC curve. Might be because of the imbalanced dataset but I could not figure out why.

Thanks!

Different result with roc_auc_score() and auc()

Some libraries follow different conventions for their filenames, such as the PAM libs -- `pam_unix.so`, not `libpam_unix.so`.

How do you override the target library filename in CMake to get something like `new_thing.so` instead of the default `libnew_thing.so`?

How do you rename a library filename in CMake?

My python (ver 2.7) script is running well to get some company name from local html files but when it comes to some specific country name, it gives this error &quot;UnicodeEncodeError: &#39;ascii&#39; codec can&#39;t encode character&quot;

Specially getting error when this company name comes

Company Name: **K&#252;hlfix K&#228;lteanlagen Ing.Gerhard Doczekal &amp; Co. KG**

The link cannot be processed

    Traceback (most recent call last): 
      File &quot;C:\Python27\Process2.py&quot;, line 261, in &lt;module&gt;
        flog.write(&quot;\nCompany Name: &quot;+str(pCompanyName))
    UnicodeEncodeError: &#39;ascii&#39; codec can&#39;t encode character u&#39;\xfc&#39; in position 9: ordinal not in range(128)

**Error gives in this line of code:** 

    if companyAlreadyKnown == 0:
       for hit in soup2.findAll(&quot;h1&quot;):
           print &quot;Company Name: &quot;+hit.text
           pCompanyName = hit.text
           flog.write(&quot;\nCompany Name: &quot;+str(pCompanyName))
           companyObj.setCompanyName(pCompanyName)


UnicodeEncodeError: &#39;ascii&#39; codec can&#39;t encode character at special name

My python (ver 2.7) script is running well to get some company name from local html files but when it comes to some specific country name, it gives this error "UnicodeEncodeError: 'ascii' codec can't encode character"
Specially getting error when this company name comes
Company Name: Kühlfix Kälteanlagen Ing.Gerhard Doczekal &#x26; Co. KG
The link cannot be processed
<pre><code class="hljs language-arduino">Traceback (most recent call last): 
 File "C:\Python27\Process2.py", line 261, in &#x3C;module>
 flog.write("\nCompany Name: "+str(pCompanyName))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 9: ordinal not in range(128)
</code></pre>
Error gives in this line of code:
<pre><code class="hljs language-css">if companyAlreadyKnown == 0:
 for hit in soup2.findAll("h1"):
 print "Company Name: "+hit.text
 pCompanyName = hit.text
 flog.write("\nCompany Name: "+str(pCompanyName))
 companyObj.setCompanyName(pCompanyName)
</code></pre>

I&#39;m trying to write a script to check a website. It&#39;s the first time I&#39;m using selenium. I&#39;m trying to run the script on a OSX system. Although I checked in /Library/Python/2.7/site-packages and selenium-2.46.0-py2.7.egg is present, when I run the script it keeps telling me that there is no selenium module to import.


This is the log that I get when I run my code:

&gt;     Traceback (most recent call last):
&gt;       File &quot;/Users/GiulioColleluori/Desktop/Class_Checker.py&quot;, line 10, in &lt;module&gt;
&gt;         from selenium import webdriver
&gt;     ImportError: No module named &#39;selenium&#39;

 

ImportError: No module named &#39;selenium&#39;

I&#39;m trying to send an array(list) of requests to the WheniWork API using requests.post, and I keep getting one of two errors. When I send the list as a list, I get an unpacking error, and when I send it as a string, I get an error asking me to submit an array. I think it has something to do with how requests handles lists. Here are the examples:

    url=&#39;https://api.wheniwork.com/2/batch&#39;
	headers={&quot;W-Token&quot;: &quot;Ilovemyboss&quot;}
    data=[{&#39;url&#39;: &#39;/rest/shifts&#39;, &#39;params&#39;: {&#39;user_id&#39;: 0,&#39;other_stuff&#39;:&#39;value&#39;}, &#39;method&#39;:&#39;post&#39;,{&#39;url&#39;: &#39;/rest/shifts&#39;, &#39;params&#39;: {&#39;user_id&#39;: 1,&#39;other_stuff&#39;:&#39;value&#39;}, &#39;method&#39;:&#39;post&#39;}]
    r = requests.post(url, headers=headers,data=data)
    print r.text

    # ValueError: too many values to unpack

Simply wrapping the value for data in quotes:

    url=&#39;https://api.wheniwork.com/2/batch&#39;
	headers={&quot;W-Token&quot;: &quot;Ilovemyboss&quot;}
    data=&quot;[]&quot; #removed the data here to emphasize that the only change is the quotes
    r = requests.post(url, headers=headers,data=data)
    print r.text

    #{&quot;error&quot;:&quot;Please include an array of requests to make.&quot;,&quot;code&quot;:5000}



How to send an array using requests.post (Python)? &quot;Value Error: Too many values to unpack&quot;

&lt;!-- language-all: python --&gt;

Let&#39;s say I have a `Product` model with products in a storefront, and a `ProductImages` table with images of the product, which can have zero or more images. Here&#39;s a simplified example:

    class Product(models.Model):
      product_name = models.CharField(max_length=255)
      # ...
    
    class ProductImage(models.Model):
      product = models.ForeignKey(Product, related_name=&#39;images&#39;)
      image_file = models.CharField(max_length=255)
      # ...
  
When displaying search results for products, I want to prioritize products which have images associated with them. I can easily get the number of images:

    from django.db.models import Count
    Product.objects.annotate(image_count=Count(&#39;images&#39;))

But that&#39;s not actually what I want. I&#39;d like to annotate it with a boolean field, `have_images`, indicating whether the product has one or more images, so that I can sort by that:

    Product.objects.annotate(have_images=(?????)).order_by(&#39;-have_images&#39;, &#39;product_name&#39;)

How can I do that? Thanks!

Django query annotation with boolean field

I am trying to run `pip install mitmproxy` on Windows, but I keep getting access denied, even with `cmd` and `PowerShell` using the `Run as Administrator` option.

    WindowsError: [Error 5] Access is denied: &#39;c:\\users\\bruno\\appdata\\local\\temp\\easy_install-0fme6u\\cryptography-0.9.1\\.eggs\\cffi-1.1.2-py2.7-win-amd64.egg\\_cffi_backend.pyd&#39;

How can I make this work?

pip install access denied on Windows

If you are not interested in the details of Mongolian but just want a quick answer about using and converting Unicode values in Swift, then skip down to the first part of the [accepted answer][1].

---

#Background

I want to render Unicode text for [traditional Mongolian to be used in iOS apps][2]. The better and long term solution is to use an [AAT smart font][3] that would render this complex script. ([Such fonts do exist][4] but their license does not allow modification and non-personal use.) However, since I have never made a font, let alone all of the rendering logic for an AAT font, I just plan to do the rendering myself in Swift for now. Maybe at some later date I can learn to make a smart font. 

Externally I will use Unicode text, but internally (for display in a `UITextView`) I will convert the Unicode to individual glyphs that are stored in a dumb font (coded with Unicode [PUA][5] values). So my rendering engine needs to convert Mongolian Unicode values (range: U+1820 to U+1842) to glyph values stored in the PUA (range: U+E360 to U+E5CF). Anyway, this is my plan since it is [what I did in Java in the past][6], but maybe I need to change my whole way of thinking.

#Example

The following image shows *su* written twice in Mongolian using two different forms for the letter *u* (in red). (Mongolian is written vertically with letters being connected like cursive letters in English.) 

![enter image description here][7]

In Unicode these two strings would be expressed as 

    var suForm1: String = &quot;\u{1830}\u{1826}&quot;
    var suForm2: String = &quot;\u{1830}\u{1826}\u{180B}&quot;

The Free Variation Selector (U+180B) in `suForm2` is recognized (correctly) by Swift `String` to be a unit with the *u* (U+1826) that precedes it. It is considered by Swift to be a single character, an extended grapheme cluster. However, for the purposes of doing the rendering myself, I need to differentiate *u* (U+1826) and FVS1 (U+180B) as two distinct UTF-16 code points.

For internal display purposes, I would convert the above Unicode strings to the following rendered glyph strings:

    suForm1 = &quot;\u{E46F}\u{E3BA}&quot; 
    suForm2 = &quot;\u{E46F}\u{E3BB}&quot;

#Question

I have been playing around with Swift `String` and `Character`. There are a lot of convenient things about them, but since in my particular case I deal exclusively with UTF-16 code units, I wonder if I should be using the old `NSString` rather than Swift&#39;s `String`. I realize that I can use `String.utf16` to get UTF-16 code points, but [the conversion back to `String` isn&#39;t very nice][8].

Would it be better to stick with [`String`][9] and [`Character`][10] or should I use [`NSString`][11] and [`unichar`][12]?

#What I have read

- [Strings and Characters documentation][13]
- [Strings in Swift][14]
- [NSString and Unicode][15]

*Updates to this question have been hidden in order to clean the page up. See the edit history.*

  [1]: https://stackoverflow.com/a/31483262/3681880
  [2]: https://stackoverflow.com/questions/28544714/how-do-you-make-a-vertical-text-uilabel-and-uitextview-for-ios-in-swift
  [3]: https://en.wikipedia.org/wiki/Apple_Advanced_Typography
  [4]: http://www.mongolfont.com/en/font/mnglwhiteaat.html
  [5]: https://en.wikipedia.org/wiki/Private_Use_Areas
  [6]: https://github.com/suragch/Chimee/blob/master/app/src/main/java/net/studymongolian/chimee/MongolUnicodeRenderer.java
  [7]: http://i.stack.imgur.com/xtEJJ.png
  [8]: https://stackoverflow.com/questions/24542170/is-there-a-way-to-create-a-string-from-utf16-array-in-swift
  [9]: https://developer.apple.com/library/prerelease/ios/documentation/Swift/Reference/Swift_String_Structure/index.html
  [10]: https://developer.apple.com/library/prerelease/ios/documentation/Swift/Reference/Swift_Character_Structure/index.html
  [11]: https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html
  [12]: https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html#//apple_ref/c/tdef/unichar
  [13]: https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html
  [14]: http://oleb.net/blog/2014/07/swift-strings/
  [15]: http://www.objc.io/issues/9-strings/unicode/
  [16]: https://stackoverflow.com/a/31327086/3681880
  [17]: https://stackoverflow.com/a/31026358/3681880
  [18]: https://stackoverflow.com/questions/31421054/convert-uint32-utf-32-to-string-in-swift
  [19]: https://stackoverflow.com/questions/31438210/how-to-implement-the-hashable-protocol-in-swift-for-an-int-array-a-custom-strin
  [20]: https://stackoverflow.com/questions/31421054/convert-uint32-utf-32-to-string-in-swift#comment50842032_31423768

Working with Unicode code points in Swift

I was baffled when a colleague showed me this line of JavaScript alerting 42.

    

&lt;!-- begin snippet: js hide: false --&gt;

&lt;!-- language: lang-js --&gt;

    alert(2+ 40);

&lt;!-- end snippet --&gt;

It quickly turns out that what looks like a minus sign is actually an arcane Unicode character with clearly different semantics.

This left me wondering why that character doesn&#39;t produce a syntax error when the expression is parsed. I&#39;d also like to know if there are more characters behaving like this.


Why does 2+ 40 equal 42?

It&#39;s possible to include Emoji characters in modern browsers, but how can one make it a single color and choose that color?

For example, here is some Emoji and some regular (plane 0) Unicode symbols. All should be red, but only the symbols are rendered in red.

[![Emoji color attempt][1]][1]

Associated [HTML + CSS](http://jsbin.com/moxusosila/edit?html,css,js,output):

    &lt;p&gt;
      &#128024;&#128039;&#128060;
    &lt;/p&gt;
    &lt;p&gt;
      ♥★ℹ
    &lt;/div&gt;

    p {
      font-size: 3em;
      color: red
    }

  [1]: http://i.stack.imgur.com/eloTn.png

Color for Unicode Emoji

I&#39;m finding Unicode for special characters from [FileFormat.Info&#39;s search](http://www.fileformat.info/info/unicode/char/search.htm). 

Some characters are rendering as the classic black-and-white glyphs, such as ⚠ (warning sign, `\u26A0` or `&amp;#x26a0;`). These are preferable, since I can apply CSS styles (such as color) to them. 

[![image of warning glyph][1]][1]

Others are rendering as newer cartoony emoji, such as ⌛ (hourglass, `\u231B` or `&amp;#x231b;`). These are not preferable, since I cannot fully style them. 

[![image of hourglass emoji][2]][2]

It appears that the browser is making this change, since I&#39;m able to see the hourglass glyph on Mac Firefox, just not Mac Chrome nor Mac Safari. 

Is there a way to force browsers to display the older (flat monotone) versions to display? 

**Update**: It seems (from comments below) there is a [text presentation selector](http://www.unicode.org/reports/tr51/#def_text_presentation_selector), `FE0E`, available to enforce text-vs-emoji. The selector is concatenated as a suffix without space onto the character&#39;s code, such as `&amp;#x231b;&amp;#xFE0E;` for HTML hex or `\u231B\uFE0E` for JS. However, *it is simply not honored by all browsers* (eg Chrome and Edge).

  [1]: http://i.stack.imgur.com/mRrUw.png
  [2]: http://i.stack.imgur.com/niqCz.png

How to prevent Unicode characters from rendering as emoji in HTML from JavaScript?

Does there exist a Unicode symbol that represents a person? I would expect something like this:

[![pictogram of a stick figure][1]][1]

I need a short way to represent a *price per person*, such as “25€/person”, but with the word “person” replaced with a Unicode character for a person. The benefit will be that the picture isn’t tied to the English language.

  [1]: http://i.stack.imgur.com/tDjcv.png

What Unicode symbol represents a person?

I had a script in Python2 that was working great. 

    def _generate_signature(data):
       return hmac.new(&#39;key&#39;, data, hashlib.sha256).hexdigest()

Where data was the output of `json.dumps`. 

Now, if I try to run the same kind of code in Python 3, I get the following:

    Traceback (most recent call last):
      File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
      File &quot;/usr/lib/python3.4/hmac.py&quot;, line 144, in new
        return HMAC(key, msg, digestmod)
      File &quot;/usr/lib/python3.4/hmac.py&quot;, line 42, in __init__
        raise TypeError(&quot;key: expected bytes or bytearray, but got %r&quot; %type(key).__name__)
    TypeError: key: expected bytes or bytearray, but got &#39;str&#39;

If I try something like transforming the key to bytes like so:

    bytes(&#39;key&#39;)

I get

    Traceback (most recent call last):
      File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
    TypeError: string argument without an encoding

I&#39;m still struggling to understand the encodings in Python 3.

Python3 and hmac . How to handle string not being binary

In iOS8 and prior I can use:

    NSString *str = ...; // some URL
    NSString *result = [str stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding];

in iOS9 `stringByAddingPercentEscapesUsingEncoding` has been replaced with `stringByAddingPercentEncodingWithAllowedCharacters`:

    NSString *str = ...; // some URL
    NSCharacterSet *set = ???; // where to find set for NSUTF8StringEncoding?
    NSString *result = [str stringByAddingPercentEncodingWithAllowedCharacters:set];

and my question is: where to find needed `NSCharacterSet` (`NSUTF8StringEncoding`) for proper replacement of `stringByAddingPercentEscapesUsingEncoding`?

Replacement for stringByAddingPercentEscapesUsingEncoding in ios9?

I keep getting UnicodeEncodeError when trying to print a &#39;&#193;&#39; that I get from a website requested using selenium in python 3.4.

I already defined at the top of my .py file

`#  -*- coding: utf-8 -*-`

the def is something like this:

    from selenium import webdriver

    b = webdriver.Firefox()
    b.get(&#39;http://fisica.uniandes.edu.co/personal/profesores-de-planta&#39;)
    dataProf = b.find_elements_by_css_selector(&#39;td[width=&quot;508&quot;]&#39;)
    for dato in dataProf:
            print(datos.text)

and the exception:

    Traceback (most recent call last):
      File &quot;C:/Users/Andres/Desktop/scrap/scrap.py&quot;, line 444, in &lt;module&gt;
        dar_p_fisica()
      File &quot;C:/Users/Andres/Desktop/scrap/scrap.py&quot;, line 390, in dar_p_fisica
        print(datos.text) #.encode().decode(&#39;ascii&#39;, &#39;ignore&#39;)
      File &quot;C:\Python34\lib\encodings\cp1252.py&quot;, line 19, in encode
        return codecs.charmap_encode(input,self.errors,encoding_table)[0]
    UnicodeEncodeError: &#39;charmap&#39; codec can&#39;t encode character &#39;\u2010&#39; in position 173: character maps to &lt;undefined&gt;

thanks in advance

UnicodeEncodeError: &#39;charmap&#39; codec can&#39;t encode character &#39;\u2010&#39;: character maps to &lt;undefined&gt;

I have installed python 2(64 bit), on windows 8.1 (64 bit) and wanted to know pip version and for that I fired `pip --version` but it is giving error.

        C:\Users\ADMIN&gt;pip --version
    Traceback (most recent call last):
      File &quot;c:\dev\python27\lib\runpy.py&quot;, line 162, in _run_module_as_main
        &quot;__main__&quot;, fname, loader, pkg_name)
      File &quot;c:\dev\python27\lib\runpy.py&quot;, line 72, in _run_code
        exec code in run_globals
      File &quot;C:\dev\Python27\Scripts\pip.exe\__main__.py&quot;, line 5, in &lt;module&gt;
      File &quot;c:\dev\python27\lib\site-packages\pip\__init__.py&quot;, line 15, in &lt;module&gt;
        from pip.vcs import git, mercurial, subversion, bazaar  # noqa
      File &quot;c:\dev\python27\lib\site-packages\pip\vcs\mercurial.py&quot;, line 10, in &lt;module&gt;
        from pip.download import path_to_url
      File &quot;c:\dev\python27\lib\site-packages\pip\download.py&quot;, line 35, in &lt;module&gt;
        from pip.utils.ui import DownloadProgressBar, DownloadProgressSpinner
      File &quot;c:\dev\python27\lib\site-packages\pip\utils\ui.py&quot;, line 51, in &lt;module&gt;
        _BaseBar = _select_progress_class(IncrementalBar, Bar)
      File &quot;c:\dev\python27\lib\site-packages\pip\utils\ui.py&quot;, line 44, in _select_progress_class
        six.text_type().join(characters).encode(encoding)
    LookupError: unknown encoding: cp65001


Note : The same command works fine for python 3. I have uninstalled both and installed again but still no success.

Python 2.7 : LookupError: unknown encoding: cp65001

I have a CSV file and I wish to understand its encoding. Is there a menu option in Microsoft Excel that can help me detect it

OR do I need to make use of programming languages like C# or PHP to deduce it.

How to check encoding of a CSV file

Observe the following problem:

    import re
    from bs4 import BeautifulSoup as BS
    
    soup = BS(&quot;&quot;&quot;
    &lt;a href=&quot;/customer-menu/1/accounts/1/update&quot;&gt;
        Edit
    &lt;/a&gt;
    &quot;&quot;&quot;)
    
    # This returns the &lt;a&gt; element
    soup.find(
        &#39;a&#39;,
        href=&quot;/customer-menu/1/accounts/1/update&quot;,
        text=re.compile(&quot;.*Edit.*&quot;)
    )
    
    soup = BS(&quot;&quot;&quot;
    &lt;a href=&quot;/customer-menu/1/accounts/1/update&quot;&gt;
        &lt;i class=&quot;fa fa-edit&quot;&gt;&lt;/i&gt; Edit
    &lt;/a&gt;
    &quot;&quot;&quot;)
    
    # This returns None
    soup.find(
        &#39;a&#39;,
        href=&quot;/customer-menu/1/accounts/1/update&quot;,
        text=re.compile(&quot;.*Edit.*&quot;)
    )

For some reason, BeautifulSoup will not match the text, when the `&lt;i&gt;` tag is there as well. Finding the tag and showing its text produces

    &gt;&gt;&gt; a2 = soup.find(
            &#39;a&#39;,
            href=&quot;/customer-menu/1/accounts/1/update&quot;
        )
    &gt;&gt;&gt; print(repr(a2.text))
    &#39;\n Edit\n&#39;

Right. According to the [Docs](http://www.crummy.com/software/BeautifulSoup/bs4/doc/#a-regular-expression), soup uses the match function of the regular expression, not the search function. So I need to provide the DOTALL flag:

    pattern = re.compile(&#39;.*Edit.*&#39;)
    pattern.match(&#39;\n Edit\n&#39;)  # Returns None
    
    pattern = re.compile(&#39;.*Edit.*&#39;, flags=re.DOTALL)
    pattern.match(&#39;\n Edit\n&#39;)  # Returns MatchObject

Alright. Looks good. Let&#39;s try it with soup

    soup = BS(&quot;&quot;&quot;
    &lt;a href=&quot;/customer-menu/1/accounts/1/update&quot;&gt;
        &lt;i class=&quot;fa fa-edit&quot;&gt;&lt;/i&gt; Edit
    &lt;/a&gt;
    &quot;&quot;&quot;)
    
    soup.find(
        &#39;a&#39;,
        href=&quot;/customer-menu/1/accounts/1/update&quot;,
        text=re.compile(&quot;.*Edit.*&quot;, flags=re.DOTALL)
    )  # Still return None... Why?!


### **Edit**

My solution based on geckons answer: I implemented these helpers:

    import re
    
    MATCH_ALL = r&#39;.*&#39;
    
    
    def like(string):
        &quot;&quot;&quot;
        Return a compiled regular expression that matches the given
        string with any prefix and postfix, e.g. if string = &quot;hello&quot;,
        the returned regex matches r&quot;.*hello.*&quot;
        &quot;&quot;&quot;
        string_ = string
        if not isinstance(string_, str):
            string_ = str(string_)
        regex = MATCH_ALL + re.escape(string_) + MATCH_ALL
        return re.compile(regex, flags=re.DOTALL)
    
    
    def find_by_text(soup, text, tag, **kwargs):
        &quot;&quot;&quot;
        Find the tag in soup that matches all provided kwargs, and contains the
        text.
        
        If no match is found, return None.
        If more than one match is found, raise ValueError.
        &quot;&quot;&quot;
        elements = soup.find_all(tag, **kwargs)
        matches = []
        for element in elements:
            if element.find(text=like(text)):
                matches.append(element)
        if len(matches) &gt; 1:
            raise ValueError(&quot;Too many matches:\n&quot; + &quot;\n&quot;.join(matches))
        elif len(matches) == 0:
            return None
        else:
            return matches[0]

Now, when I want to find the element above, I just run `find_by_text(soup, &#39;Edit&#39;, &#39;a&#39;, href=&#39;/customer-menu/1/accounts/1/update&#39;)`

BeautifulSoup - search by text inside a tag

After I installed BeautifulSoup, Whenever I run my Python in cmd, this warning comes out.

    D:\Application\python\lib\site-packages\beautifulsoup4-4.4.1-py3.4.egg\bs4\__init__.py:166:
    UserWarning: No parser was explicitly specified, so I&#39;m using the best
    available HTML parser for this system (&quot;html.parser&quot;). This usually isn&#39;t a
    problem, but if you run this code on another system, or in a different
    virtual environment, it may use a different parser and behave differently.

    To get rid of this warning, change this:

     BeautifulSoup([your markup])

    to this:

     BeautifulSoup([your markup], &quot;html.parser&quot;)

I have no ideal why it comes out and how to solve it.


How to get rid of BeautifulSoup user warning?

&gt;TypeError: a bytes-like object is required, not &#39;str&#39; 

getting above error while Executing below python code to save the HTML table data in Csv file. don&#39;t know how to get rideup.pls help me.

    import csv
    import requests
    from bs4 import BeautifulSoup
    
    url=&#39;http://www.mapsofindia.com/districts-india/&#39;
    response=requests.get(url)
    html=response.content
    
    soup=BeautifulSoup(html,&#39;html.parser&#39;)
    table=soup.find(&#39;table&#39;, attrs={&#39;class&#39;:&#39;tableizer-table&#39;})
    list_of_rows=[]
    for row in table.findAll(&#39;tr&#39;)[1:]:
        list_of_cells=[]
        for cell in row.findAll(&#39;td&#39;):
            list_of_cells.append(cell.text)
        list_of_rows.append(list_of_cells)
    outfile=open(&#39;./immates.csv&#39;,&#39;wb&#39;)
    writer=csv.writer(outfile)
    writer.writerow([&quot;SNo&quot;, &quot;States&quot;, &quot;Dist&quot;, &quot;Population&quot;])
    writer.writerows(list_of_rows)

on above the last line.

TypeError: a bytes-like object is required, not &#39;str&#39; in python and CSV

I wanted to write a piece of code like the following:

    from bs4 import BeautifulSoup
    import urllib2
    
    url = &#39;http://www.thefamouspeople.com/singers.php&#39;
    html = urllib2.urlopen(url)
    soup = BeautifulSoup(html)

But I found that I have to install `urllib3` package now.

Moreover, I couldn&#39;t find any tutorial or example to understand how to rewrite the above code, for example, `urllib3` does not have `urlopen`.

Any explanation or example, please?!

P/S: I&#39;m using python 3.4.

What should I use to open a url instead of urlopen in urllib3

I am trying to use python and beautiful soup to extract the content part of the tags below:

    &lt;meta property=&quot;og:title&quot; content=&quot;Super Fun Event 1&quot; /&gt;
    &lt;meta property=&quot;og:url&quot; content=&quot;http://superfunevents.com/events/super-fun-event-1/&quot; /&gt;

I&#39;m getting BeautifulSoup to load the page just fine and find other stuff (this also grabs the article id from the id tag hidden in the source), but I don&#39;t know the correct way to search the html and find these bits, I&#39;ve tried variations of find and findAll to no avail. The code iterates over a list of urls at present...


    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    
    #importing the libraries
    from urllib import urlopen
    from bs4 import BeautifulSoup
    
    def get_data(page_no):
        webpage = urlopen(&#39;http://superfunevents.com/?p=&#39; + str(i)).read()
        soup = BeautifulSoup(webpage, &quot;lxml&quot;)
        for tag in soup.find_all(&quot;article&quot;) :
            id = tag.get(&#39;id&#39;)
            print id
    # the hard part that doesn&#39;t work - I know this example is well off the mark!        
        title = soup.find(&quot;og:title&quot;, &quot;content&quot;)
        print (title.get_text())
        url = soup.find(&quot;og:url&quot;, &quot;content&quot;)
        print (url.get_text())
    # end of problem

    for i in range (1,100):
        get_data(i)

If anyone can help me sort the bit to find the og:title and og:content that&#39;d be fantastic!

Get meta tag content property with BeautifulSoup and Python

Typically we all using HTML numbers or names in web pages. For example, `&amp;` is `&amp;#38;` or `&amp;amp;`, and `$`, `@`, `&#169;`, `&#174;`, etc.

Is there an HTML number or name for `&lt;br&gt;`?.


Is there any ASCII character for &lt;br&gt;?

So this web page is rendering with these symbols and they are found throughout this website/application but on no other sites. Can anyone tell me

 1. What this symbol is?
 2. Why it is showing up only in one browser?

[![Excerpt][1]][1]

  [1]: http://i.stack.imgur.com/oLDdA.jpg

Why is this LSEP symbol showing up on Chrome and not Firefox or Edge?

How can I detect either numbers or letters in a string? I am aware you use the ASCII codes, but what functions take advantage of them?

Detect whether a Python string is a number or a letter

I noticed on my website, http://www.cscc.org.sg/, there&#39;s this odd symbol that shows up.

[![enter image description here][1]][1]

It says L SEP. In the HTML Code, it display the same thing.

[![enter image description here][2]][2]

Can someone shows me how to remove them?


  [1]: https://i.stack.imgur.com/NR8n9.png
  [2]: https://i.stack.imgur.com/NiO8L.png

Strange symbol shows up on website (L SEP)?

I was solving some problem on codeforces. Normally I first check if the character is upper or lower English letter then subtract or add `32` to convert it to the corresponding letter. But I found someone do `^= 32` to do the same thing. Here it is: 

    char foo = &#39;a&#39;;
    foo ^= 32;
    char bar = &#39;A&#39;;
    bar ^= 32;
    cout &lt;&lt; foo &lt;&lt; &#39; &#39; &lt;&lt; bar &lt;&lt; &#39;\n&#39;; // foo is A, and bar is a

I have searched for an explanation for this and didn&#39;t find out. So why this works? 



Content Type	Original Author	Original Content on Stackoverflow
Question	rhb1	View Question on Stackoverflow
Solution 1 - Python	Anand S Kumar	View Answer on Stackoverflow
Solution 2 - Python	David Cullen	View Answer on Stackoverflow

UnicodeEncodeError: 'ascii' codec can't encode character at special name

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

How do you rename a library filename in CMake?

Different result with roc_auc_score() and auc()

Attributions