Because the default *is* UTF-8, you only need to use that declaration when you deviate from the default, or if you rely on other tools (like your IDE or text editor) to make use of that information.

In other words, *as far as Python is concerned*, only when you want to use an encoding that differs do you have to use that declaration.

Other tools, such as your editor, can support similar syntax, which is why the [PEP 263 specification](https://www.python.org/dev/peps/pep-0263/) allows for considerable flexibility in the syntax (it must be a comment, the text `coding` must be there, followed by either a `:` or `=` character and optional whitespace, followed by a recognised codec).

Note that it only applies to how Python *reads the source code*. It doesn&#39;t apply to executing that code, so not to how printing, opening files, or any other I/O operations translate between bytes and Unicode. For more details on Python, Unicode, and encodings, I strongly urge you to read the [Python Unicode HOWTO](http://docs.python.org/3/howto/unicode.html), or the  very thorough [*Pragmatic Unicode* talk](https://nedbatchelder.com/text/unipain.html) by Ned Batchelder.

No, if:
---
- entire project use only the `UTF-8`, which is a default.
- and you&#39;re sure your IDE tool doesn&#39;t need that encoding declaration in each file.

Yes, if
---

- your project relies on different encoding
- or relies on many encodings.

For multi-encodings projects:

&gt; If some files are encoded in the `non-utf-8`, then even for these
&gt; encoded in `UTF-8` you should add encoding declaration too, because
&gt; the golden rule is `Explicit is better than implicit.`

Reference:
--
- PyCharm doesn&#39;t need that declaration:

[configuring encoding for specific file in pycharm][1]

- vim doesn&#39;t need that declaration, but:

&gt;     # vim: set fileencoding=&lt;encoding name&gt; :

  [1]: https://www.jetbrains.com/help/pycharm/configuring-individual-file-encoding.html

I&#39;m trying to convert image from `PIL` to `OpenCV` format. I&#39;m using `OpenCV 2.4.3`.
here is what I&#39;ve attempted till now.

    &gt;&gt;&gt; from PIL import Image
    &gt;&gt;&gt; import cv2 as cv
    &gt;&gt;&gt; pimg = Image.open(&#39;D:\\traffic.jpg&#39;)                           #PIL Image
    &gt;&gt;&gt; cimg = cv.cv.CreateImageHeader(pimg.size,cv.IPL_DEPTH_8U,3)    #CV Image
    &gt;&gt;&gt; cv.cv.SetData(cimg,pimg.tostring())
    &gt;&gt;&gt; cv.cv.NamedWindow(&#39;cimg&#39;)
    &gt;&gt;&gt; cv.cv.ShowImage(&#39;cimg&#39;,cimg)
    &gt;&gt;&gt; cv.cv.WaitKey()

But I think the image is not getting converted to CV format. The Window shows me a large brown image.
Where am I going wrong in Converting image from `PIL` to `CV` format?

 Also , why do i need to type `cv.cv` to access functions?



Convert image from PIL to openCV format

&gt; **Similar Question:**  
&gt; [Find() vs. Where().FirstOrDefault()](https://stackoverflow.com/questions/9335015/find-vs-where-firstordefault)  

&lt;!-- End of automatically inserted text --&gt;

Got an interesting outcome searching for Diana within a large sequence of a simple reference type having a single string property.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    
    public class Customer{
        public string Name {get;set;}
    }
    
    Stopwatch watch = new Stopwatch();        
        const string diana = &quot;Diana&quot;;
    
        while (Console.ReadKey().Key != ConsoleKey.Escape)
        {
            //Armour with 1000k++ customers. Wow, should be a product with a great success! :)
            var customers = (from i in Enumerable.Range(0, 1000000)
                             select new Customer
                             {
                                Name = Guid.NewGuid().ToString()
                             }).ToList();
    
            customers.Insert(999000, new Customer { Name = diana }); // Putting Diana at the end :)
                    
            //1. System.Linq.Enumerable.DefaultOrFirst()
            watch.Restart();
            customers.FirstOrDefault(c =&gt; c.Name == diana);
            watch.Stop();
            Console.WriteLine(&quot;Diana was found in {0} ms with System.Linq.Enumerable.FirstOrDefault().&quot;, watch.ElapsedMilliseconds);
    
            //2. System.Collections.Generic.List&lt;T&gt;.Find()
            watch.Restart();
            customers.Find(c =&gt; c.Name == diana);
            watch.Stop();
            Console.WriteLine(&quot;Diana was found in {0} ms with System.Collections.Generic.List&lt;T&gt;.Find().&quot;, watch.ElapsedMilliseconds);
        }

![enter image description here][1]

Is this because of no Enumerator overhead in List&lt;T&gt;.Find() or this plus maybe something else?

`Find()` runs almost as twice as faster, hoping *.Net* team will not mark it Obsolete in the future.


  [1]: http://i.stack.imgur.com/mtv05.png

Performance of Find() vs. FirstOrDefault()

Python 3 uses UTF-8 encoding for source-code files by default. Should I still use the encoding declaration at the beginning of every source file? Like `# -*- coding: utf-8 -*-`

Should I use encoding declaration in Python 3?

<p>Python 3 uses UTF-8 encoding for source-code files by default. Should I still use the encoding declaration at the beginning of every source file? Like <code># -*- coding: utf-8 -*-</code></p>


I&#39;m punching way above my weight here, but please bear with this Python amateur. I&#39;m a PHP developer by trade and I&#39;ve hardly touched this language before.

What I&#39;m trying to do is call a method in a class...sounds simple enough? I&#39;m utterly baffled about what &#39;self&#39; refers to, and what is the correct procedure to call such a method inside a class and outside a class. 

Could someone *explain* to me, how to call the `move` method with the variable `RIGHT`. I&#39;ve tried researching this on several &#39;learn python&#39; sites and searches on StackOverflow, but to no avail. Any help will be appreciated.

The following class works in Scott&#39;s Python script which is accessed by a terminal GUI (urwid). 

The function I&#39;m working with is a Scott Weston&#39;s missile launcher Python script, which I&#39;m trying to hook into a PHP web-server.

    class MissileDevice:
      INITA     = (85, 83, 66, 67,  0,  0,  4,  0)
      INITB     = (85, 83, 66, 67,  0, 64,  2,  0)
      CMDFILL   = ( 8,  8,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0,
                    0,  0,  0,  0,  0,  0,  0,  0)
      STOP      = ( 0,  0,  0,  0,  0,  0)
      LEFT      = ( 0,  1,  0,  0,  0,  0)
      RIGHT     = ( 0,  0,  1,  0,  0,  0)
      UP        = ( 0,  0,  0,  1,  0,  0)
      DOWN      = ( 0,  0,  0,  0,  1,  0)
      LEFTUP    = ( 0,  1,  0,  1,  0,  0)
      RIGHTUP   = ( 0,  0,  1,  1,  0,  0)
      LEFTDOWN  = ( 0,  1,  0,  0,  1,  0)
      RIGHTDOWN = ( 0,  0,  1,  0,  1,  0)
      FIRE      = ( 0,  0,  0,  0,  0,  1)
    
      def __init__(self, battery):
        try:
          self.dev=UsbDevice(0x1130, 0x0202, battery)
          self.dev.open()
          self.dev.handle.reset()
        except NoMissilesError, e:
          raise NoMissilesError()
    
      def move(self, direction):
        self.dev.handle.controlMsg(0x21, 0x09, self.INITA, 0x02, 0x01)
        self.dev.handle.controlMsg(0x21, 0x09, self.INITB, 0x02, 0x01)
        self.dev.handle.controlMsg(0x21, 0x09, direction+self.CMDFILL, 0x02, 0x01)

Python calling method in class

I am trying to `pip install` the `MySQL-python` package, but I get an `ImportError`.

    

    Jans-MacBook-Pro:~ jan$ /Library/Frameworks/Python.framework/Versions/3.3/bin/pip-3.3 install MySQL-python
    Downloading/unpacking MySQL-python
      Running setup.py egg_info for package MySQL-python
        Traceback (most recent call last):
          File &quot;&lt;string&gt;&quot;, line 16, in &lt;module&gt;
          File &quot;/var/folders/lf/myf7bjr57_jg7_5c4014bh640000gn/T/pip-build/MySQL-python/setup.py&quot;, line 14, in &lt;module&gt;
            from setup_posix import get_config
          File &quot;./setup_posix.py&quot;, line 2, in &lt;module&gt;
            from ConfigParser import SafeConfigParser
        ImportError: No module named &#39;ConfigParser&#39;
        Complete output from command python setup.py egg_info:
        Traceback (most recent call last):
    
      File &quot;&lt;string&gt;&quot;, line 16, in &lt;module&gt;
    
      File &quot;/var/folders/lf/myf7bjr57_jg7_5c4014bh640000gn/T/pip-build/MySQL-python/setup.py&quot;, line 14, in &lt;module&gt;
    
        from setup_posix import get_config
    
      File &quot;./setup_posix.py&quot;, line 2, in &lt;module&gt;
    
        from ConfigParser import SafeConfigParser
    
    ImportError: No module named &#39;ConfigParser&#39;
    
    ----------------------------------------
    Command python setup.py egg_info failed with error code 1 in /var/folders/lf/myf7bjr57_jg7_5c4014bh640000gn/T/pip-build/MySQL-python
    Storing complete log in /Users/jan/.pip/pip.log
    Jans-MacBook-Pro:~ jan$ 

Any ideas?

Python 3 ImportError: No module named &#39;ConfigParser&#39;

I&#39;m trying to create multithreaded web server in python, but it only responds to one request at a time and I can&#39;t figure out why. Can you help me, please?

    #!/usr/bin/env python2
    # -*- coding: utf-8 -*-
    
    from SocketServer import ThreadingMixIn
    from  BaseHTTPServer import HTTPServer
    from SimpleHTTPServer import SimpleHTTPRequestHandler
    from time import sleep
    
    class ThreadingServer(ThreadingMixIn, HTTPServer):
    	pass
    
    class RequestHandler(SimpleHTTPRequestHandler):
    	def do_GET(self):
    		self.send_response(200)
    		self.send_header(&#39;Content-type&#39;, &#39;text/plain&#39;)
    		sleep(5)
    		response = &#39;Slept for 5 seconds..&#39;
    		self.send_header(&#39;Content-length&#39;, len(response))
    		self.end_headers()
    		self.wfile.write(response)
    
    ThreadingServer((&#39;&#39;, 8000), RequestHandler).serve_forever()

Multithreaded web server in python

I am making a scatter plot in matplotlib and need to change the background of the actual plot to black.  I know how to change the face color of the plot using:

    fig = plt.figure()
    fig.patch.set_facecolor(&#39;xkcd:mint green&#39;)

[![enter image description here][1]][1]

My issue is that this changes the color of the space around the plot. How to I change the actual background color of the plot?


  [1]: https://i.stack.imgur.com/UN6YB.png

How to change plot background color?

I want to add a debug print statement test, if I enable `--verbose` from the command line and if I have the following in the script.

    logger.info(&quot;test&quot;)

I went through the following questions, but couldn&#39;t get the answer...

- https://stackoverflow.com/questions/5980042/how-to-implement-the-verbose-or-v-option-into-a-python-script

- https://stackoverflow.com/questions/9042919/python-logging-is-there-something-below-debug


Easier way to enable verbose logging

Does range function allows concatenation ? Like i want to make a `range(30)` &amp; concatenate it with `range(2000, 5002)`.  So my concatenated range will be `0, 1, 2, ... 29, 2000, 2001,  ... 5001` 

Code like this does not work on my latest python (ver: 3.3.0)

    range(30) + range(2000, 5002)


Concatenating two range function results

I have a very simple Python 3 script:

    f1 = open(&#39;a.txt&#39;, &#39;r&#39;)
    print(f1.readlines())
    f2 = open(&#39;b.txt&#39;, &#39;r&#39;)
    print(f2.readlines())
    f3 = open(&#39;c.txt&#39;, &#39;r&#39;)
    print(f3.readlines())
    f4 = open(&#39;d.txt&#39;, &#39;r&#39;)
    print(f4.readlines())
    f1.close()
    f2.close()
    f3.close()
    f4.close()

But it always says:
    
    IOError: [Errno 32] Broken pipe

I saw on the internet all the complicated ways to fix this, but I copied this code directly, so I think that there is something wrong with the code and not Python&#39;s SIGPIPE.

I am redirecting the output, so if the above script was named &quot;open.py&quot;, then my command to run would be:

    open.py | othercommand

IOError: [Errno 32] Broken pipe when piping: `prog.py | othercmd`

When iterating over a `bytes` object in Python 3, one gets the individual `bytes` as `ints`:

    &gt;&gt;&gt; [b for b in b&#39;123&#39;]
    [49, 50, 51]

How to get 1-length `bytes` objects instead?

The following is possible, but not very obvious for the reader and most likely performs bad:

    &gt;&gt;&gt; [bytes([b]) for b in b&#39;123&#39;]
    [b&#39;1&#39;, b&#39;2&#39;, b&#39;3&#39;]

Iterate over individual bytes in Python 3

What does a bare asterisk in the arguments of a function do?

When I looked at the [pickle module][1], I see this:

    pickle.dump(obj, file, protocol=None, *, fix_imports=True)

I know about a single and double asterisks preceding arguments (for variable number of arguments), but this precedes nothing. And I&#39;m pretty sure this has nothing to do with pickle. That&#39;s probably just an example of this happening. I only learned its name when I sent this to the interpreter:

    &gt;&gt;&gt; def func(*):
    ...     pass
    ...
      File &quot;&lt;stdin&gt;&quot;, line 1
    SyntaxError: named arguments must follow bare *

If it matters, I&#39;m on python 3.3.0.

  [1]: http://docs.python.org/3.3/library/pickle.html#pickle.dump

Bare asterisk in function arguments?

I am using [Font Awesome][1] on my webpage and I want to display an icon inside the `:before` pseudo element.

According to the [documentation/cheatsheet][2], I have to type `&amp;#xf066;` to get this font, but it isn&#39;t working. I believe that is normal because HTML entities aren&#39;t supported in `:before`.  
So I googled a bit and found out, that if you want to display HTML entities in `:before`, you have to use the escaped hex reference.  
So I was searching for the hex reference to `&amp;#xf066;` but I found nothing. I believe that is because these are &quot;private use&quot; values, whatever that means.

Is there any way to get it working in `:before`?


  [1]: http://fortawesome.github.com/Font-Awesome/
  [2]: http://fortawesome.github.io/Font-Awesome/cheatsheet/

Unicode via CSS :before

I was implementing password hashing with salt, so I generated salt as binary, hashed the password, base64 encoded the password and salt then stored them into database.

Now when I am checking password, I am supposed to decode the salt back into binary data, use it to hash the supplied password, base64 encode the result and check if the result matches the one in database.

The problem is, I cannot find a method to decode the salt back into binary data. I encoded them using the Buffer.toString method but there doesn&#39;t seem to be reverse function.


NodeJS: How to decode base64 encoded string back to binary?

I am writing a Python (Python 3.3) program to send some data to a webpage using POST method.  Mostly for debugging process I am getting the page result and displaying it on the screen using `print()` function.

The code is like this:

    conn.request(&quot;POST&quot;, resource, params, headers)
    response = conn.getresponse()
    print(response.status, response.reason)
    data = response.read()
    print(data.decode(&#39;utf-8&#39;));

the `HTTPResponse` `.read()` method returns a `bytes` element encoding the page (which is a well formated UTF-8 document)  It seemed okay until I stopped using IDLE GUI for Windows and used the Windows console instead.  The returned page has a U+2014 character (em-dash) which the print function translates well in the Windows GUI (I presume Code Page 1252) but does not in the Windows Console (Code Page 850).  Given the `strict` default behavior I get the following error:

    UnicodeEncodeError: &#39;charmap&#39; codec can&#39;t encode character &#39;\u2014&#39; in position 10248: character maps to &lt;undefined&gt;

I could fix it using this quite ugly code:

    print(data.decode(&#39;utf-8&#39;).encode(&#39;cp850&#39;,&#39;replace&#39;).decode(&#39;cp850&#39;))

Now it replace the offending character &quot;—&quot; with a `?`.  Not the ideal case (a hyphen should be a better replacement) but good enough for my purpose.

There are several things I do not like from my solution.

 1. The code is ugly with all that decoding, encoding, and decoding.
 2. It solves the problem for just this case.  If I port the program for a system using some other encoding (latin-1, cp437, back to cp1252, etc.) it should recognize the target encoding.  It does not.  (for instance, when using again the IDLE GUI, the emdash is also lost, which didn&#39;t happen before)
 3. It would be nicer if the emdash translated to a hyphen instead of a interrogation bang.

The problem is not the emdash (I can think of several ways to solve that particularly problem) but I need to write robust code.  I am feeding the page with data from a database and that data can come back.  I can anticipate many other conflicting cases: an &#39;&#193;&#39; U+00c1 (which is possible in my database) could translate into CP-850 (DOS/Windows Console encodign for Western European Languages) but not into CP-437 (encoding for US English, which is default in many Windows instalations).

So, the question:

Is there a nicer solution that makes my code agnostic from the output interface encoding?

UnicodeEncodeError: &#39;charmap&#39; codec can&#39;t encode - character maps to &lt;undefined&gt;, print function

I am doing some scripts in python. I create a string that I save in a file. This string got lot of data, coming from the arborescence and filenames of a directory.
According to convmv, all my arborescence is in UTF-8.

I want to keep everything in UTF-8 because I will save it in MySQL after.
For now, in MySQL, which is in UTF-8, I got some problem with some characters (like &#233; or &#232; - I&#39;am French).

I want that python always use string as UTF-8. I read some informations on the internet and i did like this.

My script begin with this :

     #!/usr/bin/python
     # -*- coding: utf-8 -*-
     def createIndex():
         import codecs
         toUtf8=codecs.getencoder(&#39;UTF8&#39;)
         #lot of operations &amp; building indexSTR the string who matter
         findex=open(&#39;config/index/music_vibration_&#39;+date+&#39;.index&#39;,&#39;a&#39;)
         findex.write(codecs.BOM_UTF8)
         findex.write(toUtf8(indexSTR)) #this bugs!

And when I execute, here is the answer : `UnicodeDecodeError: &#39;ascii&#39; codec can&#39;t decode byte 0xc3 in position 2171: ordinal not in range(128)`


Edit:
I see, in my file, the accent are nicely written. After creating this file, I read it and I write it into MySQL.
But I dont understand why, but I got problem with encoding.
My MySQL database is in utf8, or seems to be SQL query `SHOW variables LIKE &#39;char%&#39;` returns me only utf8 or binary.

My function looks like this : 

    #!/usr/bin/python
    # -*- coding: utf-8 -*-
	
    def saveIndex(index,date):
        import MySQLdb as mdb
        import codecs
	
	    sql = mdb.connect(&#39;localhost&#39;,&#39;admin&#39;,&#39;*******&#39;,&#39;music_vibration&#39;)
	    sql.charset=&quot;utf8&quot;
	    findex=open(&#39;config/index/&#39;+index,&#39;r&#39;)
	    lines=findex.readlines()
	    for line in lines:
		    if line.find(&#39;#artiste&#39;) != -1:
			    artiste=line.split(&#39;[:::]&#39;)
			    artiste=artiste[1].replace(&#39;\n&#39;,&#39;&#39;)
			
			    c=sql.cursor()
			    c.execute(&#39;SELECT COUNT(id) AS nbr FROM artistes WHERE nom=&quot;&#39;+artiste+&#39;&quot;&#39;)
			    nbr=c.fetchone()
			    if nbr[0]==0:
				    c=sql.cursor()
				    iArt+=1
				    c.execute(&#39;INSERT INTO artistes(nom,status,path) VALUES(&quot;&#39;+artiste+&#39;&quot;,99,&quot;&#39;+artiste+&#39;/&quot;)&#39;.encode(&#39;utf8&#39;)

And artiste who are nicely displayed in the file writes bad into the BDD.
What is the problem ?

python encoding utf-8

I&#39;m looking to create a comma-separated list of values from a SQL Server 2005 table, just like in [JanetOhara&#39;s question][1].  I&#39;m using a query similar to the one presented in [techdo&#39;s answer][2] to the question.

Everything is working, except the list of values is getting XML encoded.  What should be:

    Sports &amp; Recreation,x &gt;= y

Is instead returning as:

    Sports &amp;amp; Recreation,x &amp;lt;= y

Is there a way to disable the XML character encoding when using &quot;FOR XML&quot; in SQL Server?

  [1]: https://stackoverflow.com/q/12435284/1383298
  [2]: https://stackoverflow.com/a/12435331/1383298

How do I avoid character encoding when using &quot;FOR XML PATH&quot;?

[From here][1]

&gt; Essentially, string uses the UTF-16 character encoding form

But when saving vs [StreamWriter][2] : 

&gt; This constructor creates a StreamWriter with UTF-8 encoding without a
&gt; Byte-Order Mark (BOM),


I&#39;ve seen this sample (broken link removed):

![enter image description here][4]

And it looks like `utf8` is smaller for some strings while `utf-16` is smaller in some other strings.

* So why does .net use `utf16` as default encoding for string and `utf8` for saving files? 

Thank you.

*p.s. Ive already read [the famous article][5]*


  [1]: http://csharpindepth.com/Articles/General/strings.aspx
  [2]: http://msdn.microsoft.com/en-us/library/wtbhzte9.aspx
  [4]: http://i.stack.imgur.com/QYDgv.jpg
  [5]: http://www.joelonsoftware.com/articles/Unicode.html

Why does .net use the UTF16 encoding for string, but uses UTF-8 as default for saving files?

I have used [Java Mail API][1], for sending emails. I am using a contact formular to send the input, which has to be send to a specific email.

The email is send without problems, though I am a danish guy, and I am therefore in need of three danish characters which is &#39;&#230;&#39;, &#39;&#248;&#39; and &#39;&#229;&#39;, in the subject and the email text.

I have therefore seen that I can use UTF-8 character encoding, to provide these characters, but when my mail is send I only see some strange letters - &#39;&#227;&#166;&#39;, &#39;&#227;&#184;&#39; and &#39;&#227;&#165;&#39; - instead of the danish letters - &#39;&#230;&#39;, &#39;&#248;&#39; and &#39;&#229;&#39;.

My method to send the email is looking like this:

    public void sendEmail(String name, String fromEmail, String subject, String message) throws AddressException, MessagingException, UnsupportedEncodingException, SendFailedException
    {
        //Set Mail properties
        Properties props = System.getProperties();
        props.setProperty(&quot;mail.smtp.starttls.enable&quot;, &quot;true&quot;);
        props.setProperty(&quot;mail.smtp.host&quot;, &quot;smtp.gmail.com&quot;);
        props.setProperty(&quot;mail.smtp.socketFactory.port&quot;, &quot;465&quot;);
        props.setProperty(&quot;mail.smtp.socketFactory.class&quot;, &quot;javax.net.ssl.SSLSocketFactory&quot;);
        props.setProperty(&quot;mail.smtp.auth&quot;, &quot;true&quot;);
        props.setProperty(&quot;mail.smtp.port&quot;, &quot;465&quot;);
        Session session = Session.getDefaultInstance(props, new javax.mail.Authenticator() {
            @Override
            protected PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication(&quot;my_username&quot;, &quot;my_password&quot;);
            }
        });

        //Create the email with variable input
        MimeMessage mimeMessage = new MimeMessage(session);
        mimeMessage.setHeader(&quot;Content-Type&quot;, &quot;text/plain; charset=UTF-8&quot;);
        mimeMessage.setFrom(new InternetAddress(fromEmail, name));
        mimeMessage.setRecipient(Message.RecipientType.TO, new InternetAddress(&quot;my_email&quot;));
        mimeMessage.setSubject(subject, &quot;utf-8&quot;);
        mimeMessage.setContent(message, &quot;text/plain&quot;);
        
        //Send the email
        Transport.send(mimeMessage);
    }

Please help me find out how I can correct this &#39;error&#39;.


  [1]: https://javaee.github.io/javamail/

UTF-8 charset doesn&#39;t work with javax.mail

If I run the following Perl program:

    perl -e &#39;use utf8; print &quot;鸡\n&quot;;&#39;

I get this warning:

    Wide character in print at -e line 1.

If I run this Perl program:

    perl -e &#39;print &quot;鸡\n&quot;;&#39;

I do not get a warning.

I thought `use utf8` was required to use UTF-8 characters in a Perl script.  Why does this not work and how can I fix it? I&#39;m using Perl 5.16.2. I have the same issue if this is in a file instead of being a one liner on the command line.



Use of &#39;use utf8;&#39; gives me &#39;Wide character in print&#39;

I am trying to create a &#39;normalized&#39; copy of a string, to help reduce duplicate names in a database. The names contain many international characters (ie. accented letters), and I want to create a copy with the accents removed. 

I did come across the method below, but cannot get it to work. I can&#39;t seem to find what the Unicode Hacks plugin is.

      # Utility method that retursn an ASCIIfied, downcased, and sanitized string.
      # It relies on the Unicode Hacks plugin by means of String#chars. We assume
      # $KCODE is &#39;u&#39; in environment.rb. By now we support a wide range of latin
      # accented letters, based on the Unicode Character Palette bundled inMacs.
      def self.normalize(str)
         n = str.chars.downcase.strip.to_s
         n.gsub!(/[&#195; &#195;&#161;&#195;&#162;&#195;&#163;&#195;&#164;&#195;&#165;&#196;&#196;?]/u,    &#39;a&#39;)
         n.gsub!(/&#195;&#166;/u,                  &#39;ae&#39;)
         n.gsub!(/[&#196;&#196;?]/u,                &#39;d&#39;)
         n.gsub!(/[&#195;&#167;&#196;?&#196;&#196;?&#196;?]/u,          &#39;c&#39;)
         n.gsub!(/[&#195;&#168;&#195;&#169;&#195;&#170;&#195;&#171;&#196;?&#196;?&#196;?&#196;?&#196;?]/u, &#39;e&#39;)
         n.gsub!(/&#198;?/u,                   &#39;f&#39;)
         n.gsub!(/[&#196;&#196;?&#196;&#161;&#196;&#163;]/u,            &#39;g&#39;)
         n.gsub!(/[&#196;&#165;&#196;&#167;]/,                &#39;h&#39;)
         n.gsub!(/[&#195;&#172;&#195;&#172;&#195;&#173;&#195;&#174;&#195;&#175;&#196;&#171;&#196;&#169;&#196;&#173;]/u,     &#39;i&#39;)
         n.gsub!(/[&#196;&#175;&#196;&#177;&#196;&#179;&#196;&#181;]/u,           &#39;j&#39;)
         n.gsub!(/[&#196;&#183;&#196;&#184;]/u,               &#39;k&#39;)
         n.gsub!(/[&#197;?&#196;&#190;&#196;&#186;&#196;&#188;&#197;?]/u,         &#39;l&#39;)
         n.gsub!(/[&#195;&#177;&#197;?&#197;?&#197;?&#197;?&#197;?]/u,       &#39;n&#39;)
         n.gsub!(/[&#195;&#178;&#195;&#179;&#195;&#180;&#195;&#181;&#195;&#182;&#195;&#184;&#197;&#197;?&#197;&#197;]/u,  &#39;o&#39;)
         n.gsub!(/&#197;?/u,                  &#39;oe&#39;)
         n.gsub!(/&#196;?/u,                   &#39;q&#39;)
         n.gsub!(/[&#197;?&#197;?&#197;?]/u,             &#39;r&#39;)
         n.gsub!(/[&#197;?&#197;&#161;&#197;?&#197;&#200;?]/u,          &#39;s&#39;)
         n.gsub!(/[&#197;&#165;&#197;&#163;&#197;&#167;&#200;?]/u,           &#39;t&#39;)
         n.gsub!(/[&#195;&#185;&#195;&#186;&#195;&#187;&#195;&#188;&#197;&#171;&#197;&#175;&#197;&#177;&#197;&#173;&#197;&#169;&#197;&#179;]/u,&#39;u&#39;)
         n.gsub!(/&#197;&#181;/u,                   &#39;w&#39;)
         n.gsub!(/[&#195;&#189;&#195;&#191;&#197;&#183;]/u,             &#39;y&#39;)
         n.gsub!(/[&#197;&#190;&#197;&#188;&#197;&#186;]/u,             &#39;z&#39;)
         n.gsub!(/\s+/,                   &#39; &#39;)
         n.gsub!(/[^\sa-z0-9_-]/,          &#39;&#39;)
         n
      end

Do I need to &#39;require&#39; a particular library/gem? Or maybe someone could recommend another way to go about this.

I am not using Rails, nor do I plan on doing so.

Content Type	Original Author	Original Content on Stackoverflow
Question	Mateusz Jagiełło	View Question on Stackoverflow
Solution 1 - Python	Martijn Pieters	View Answer on Stackoverflow
Solution 2 - Python	Sławomir Lenart	View Answer on Stackoverflow

Should I use encoding declaration in Python 3?

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

No, if:

Yes, if

Reference:

Performance of Find() vs. FirstOrDefault()

Convert image from PIL to openCV format

Attributions