I have an xml doc that I am trying to parse using Etree.lxml

    &lt;Envelope xmlns=&quot;http://www.example.com/zzz/yyy&quot;&gt;
      &lt;Header&gt;
        &lt;Version&gt;1&lt;/Version&gt;
      &lt;/Header&gt;
      &lt;Body&gt;
        some stuff
      &lt;Body&gt;
    &lt;Envelope&gt;

My code is:

    path = &quot;path to xml file&quot;
    from lxml import etree as ET
    parser = ET.XMLParser(ns_clean=True)
    dom = ET.parse(path, parser)
    dom.getroot()

When I try to get dom.getroot() I get:

    &lt;Element {http://www.example.com/zzz/yyy}Envelope at 28adacac&gt;
However I only want:

    &lt;Element Envelope at 28adacac&gt;

When i do 

    dom.getroot().find(&quot;Body&quot;)

I get nothing returned. However, when I
     
    dom.getroot().find(&quot;{http://www.example.com/zzz/yyy}Body&quot;) 

I get a result.

I thought passing ns_clean=True to the parser would prevent this.

Any ideas?



lxml etree xmlparser remove unwanted namespace

When it comes to [generating XML data in Python][1], there are two libraries I often see recommended: ***[lxml][2]*** and ***[ElementTree][3]***

From what I can tell, the two libraries are very similar to each other. They both seem to have similar module names, usage guidelines, and functionality. Even the import statements are fairly similar.

     # Importing lxml and ElementTree
    import lxml.etree
    import xml.etree.ElementTree

What are the differences between the `lxml` and `ElementTree` libraries for Python?


  [1]: https://stackoverflow.com/q/3844360/3357935
  [2]: https://lxml.de/index.html
  [3]: https://docs.python.org/3/library/xml.etree.elementtree.html

What are the differences between lxml and ElementTree?

This is my error:

    (mysite)zjm1126@zjm1126-G41MT-S2:~/zjm_test/mysite$ pip install lxml
    Downloading/unpacking lxml
      Running setup.py egg_info for package lxml
        Building lxml version 2.3.
        Building without Cython.
        ERROR: /bin/sh: xslt-config: not found
        
        ** make sure the development packages of libxml2 and libxslt are installed **
        
        Using build configuration of libxslt
    Installing collected packages: lxml
      Running setup.py install for lxml
        Building lxml version 2.3.
        Building without Cython.
        ERROR: /bin/sh: xslt-config: not found
        
        ** make sure the development packages of libxml2 and libxslt are installed **
        
        Using build configuration of libxslt
        building &#39;lxml.etree&#39; extension
        gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.6/src/lxml/lxml.etree.o -w
        src/lxml/lxml.etree.c:4: fatal error: Python.h: 没有那个文件或目录
        compilation terminated.
        error: command &#39;gcc&#39; failed with exit status 1
        Complete output from command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-jOhgvD-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6:
        Building lxml version 2.3.
    
    Building without Cython.
    
    ERROR: /bin/sh: xslt-config: not found
    
    
    
    ** make sure the development packages of libxml2 and libxslt are installed **
    
    
    
    Using build configuration of libxslt
    
    running install
    
    running build
    
    running build_py
    
    running build_ext
    
    building &#39;lxml.etree&#39; extension
    
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.6/src/lxml/lxml.etree.o -w
    
    src/lxml/lxml.etree.c:4: fatal error: Python.h: 没有那个文件或目录
    
    compilation terminated.
    
    error: command &#39;gcc&#39; failed with exit status 1
    
    ----------------------------------------
    Command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-jOhgvD-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6 failed with error code 1
    Storing complete log in /home/zjm1126/.pip/pip.log

What can I do?

**updated:**

    (mysite)zjm1126@zjm1126-G41MT-S2:~/zjm_test/mysite$ pip install lxml
    Downloading/unpacking lxml
      Running setup.py egg_info for package lxml
        Building lxml version 2.3.
        Building without Cython.
        Using build configuration of libxslt 1.1.26
        Building against libxml2/libxslt in the following directory: /usr/lib
    Installing collected packages: lxml
      Running setup.py install for lxml
        Building lxml version 2.3.
        Building without Cython.
        Using build configuration of libxslt 1.1.26
        Building against libxml2/libxslt in the following directory: /usr/lib
        building &#39;lxml.etree&#39; extension
        gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.6/src/lxml/lxml.etree.o -w
        src/lxml/lxml.etree.c:4: fatal error: Python.h: 没有那个文件或目录
        compilation terminated.
        error: command &#39;gcc&#39; failed with exit status 1
        Complete output from command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-NJw2ws-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6:
        Building lxml version 2.3.
    
    Building without Cython.
    
    Using build configuration of libxslt 1.1.26
    
    Building against libxml2/libxslt in the following directory: /usr/lib
    
    running install
    
    running build
    
    running build_py
    
    running build_ext
    
    building &#39;lxml.etree&#39; extension
    
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.6/src/lxml/lxml.etree.o -w
    
    src/lxml/lxml.etree.c:4: fatal error: Python.h: 没有那个文件或目录
    
    compilation terminated.
    
    error: command &#39;gcc&#39; failed with exit status 1
    
    ----------------------------------------
    Command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-NJw2ws-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6 failed with error code 1
    Storing complete log in /home/zjm1126/.pip/pip.log

the log:

    ------------------------------------------------------------
    /home/zjm1126/zjm_test/mysite/bin/pip run on Thu Mar  3 17:07:27 2011
    Downloading/unpacking mysql-python
      Running setup.py egg_info for package mysql-python
        running egg_info
        creating pip-egg-info/MySQL_python.egg-info
        writing pip-egg-info/MySQL_python.egg-info/PKG-INFO
        writing top-level names to pip-egg-info/MySQL_python.egg-info/top_level.txt
        writing dependency_links to pip-egg-info/MySQL_python.egg-info/dependency_links.txt
        writing pip-egg-info/MySQL_python.egg-info/PKG-INFO
        writing top-level names to pip-egg-info/MySQL_python.egg-info/top_level.txt
        writing dependency_links to pip-egg-info/MySQL_python.egg-info/dependency_links.txt
        writing manifest file &#39;pip-egg-info/MySQL_python.egg-info/SOURCES.txt&#39;
        warning: manifest_maker: standard file &#39;-c&#39; not found
        reading manifest file &#39;pip-egg-info/MySQL_python.egg-info/SOURCES.txt&#39;
        reading manifest template &#39;MANIFEST.in&#39;
        warning: no files found matching &#39;MANIFEST&#39;
        warning: no files found matching &#39;ChangeLog&#39;
        warning: no files found matching &#39;GPL&#39;
        writing manifest file &#39;pip-egg-info/MySQL_python.egg-info/SOURCES.txt&#39;
    Installing collected packages: mysql-python
      Running setup.py install for mysql-python
        Running command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-XuVIux-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6
        running install
        running build
        running build_py
        creating build
        creating build/lib.linux-i686-2.6
        copying _mysql_exceptions.py -&gt; build/lib.linux-i686-2.6
        creating build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/__init__.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/converters.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/connections.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/cursors.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/release.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        copying MySQLdb/times.py -&gt; build/lib.linux-i686-2.6/MySQLdb
        creating build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/__init__.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/CR.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/FIELD_TYPE.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/ER.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/FLAG.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/REFRESH.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        copying MySQLdb/constants/CLIENT.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
        running build_ext
        building &#39;_mysql&#39; extension
        creating build/temp.linux-i686-2.6
        gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,3,&#39;final&#39;,0) -D__version__=1.2.3 -I/usr/include/mysql -I/usr/include/python2.6 -c _mysql.c -o build/temp.linux-i686-2.6/_mysql.o -DBIG_JOINS=1 -fno-strict-aliasing -DUNIV_LINUX -DUNIV_LINUX
        In file included from _mysql.c:29:
        pymemcompat.h:10: fatal error: Python.h: 没有那个文件或目录
        compilation terminated.
        error: command &#39;gcc&#39; failed with exit status 1
        Complete output from command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-XuVIux-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6:
        running install
    
    running build
    
    running build_py
    
    creating build
    
    creating build/lib.linux-i686-2.6
    
    copying _mysql_exceptions.py -&gt; build/lib.linux-i686-2.6
    
    creating build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/__init__.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/converters.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/connections.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/cursors.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/release.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    copying MySQLdb/times.py -&gt; build/lib.linux-i686-2.6/MySQLdb
    
    creating build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/__init__.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/CR.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/FIELD_TYPE.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/ER.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/FLAG.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/REFRESH.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    copying MySQLdb/constants/CLIENT.py -&gt; build/lib.linux-i686-2.6/MySQLdb/constants
    
    running build_ext
    
    building &#39;_mysql&#39; extension
    
    creating build/temp.linux-i686-2.6
    
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,3,&#39;final&#39;,0) -D__version__=1.2.3 -I/usr/include/mysql -I/usr/include/python2.6 -c _mysql.c -o build/temp.linux-i686-2.6/_mysql.o -DBIG_JOINS=1 -fno-strict-aliasing -DUNIV_LINUX -DUNIV_LINUX
    
    In file included from _mysql.c:29:
    
    pymemcompat.h:10: fatal error: Python.h: 没有那个文件或目录
    
    compilation terminated.
    
    error: command &#39;gcc&#39; failed with exit status 1
    
    ----------------------------------------
    Command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-XuVIux-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6 failed with error code 1
    Exception information:
    Traceback (most recent call last):
      File &quot;/home/zjm1126/zjm_test/mysite/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/basecommand.py&quot;, line 130, in main
        self.run(options, args)
      File &quot;/home/zjm1126/zjm_test/mysite/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/commands/install.py&quot;, line 228, in run
        requirement_set.install(install_options, global_options)
      File &quot;/home/zjm1126/zjm_test/mysite/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/req.py&quot;, line 1043, in install
        requirement.install(install_options, global_options)
      File &quot;/home/zjm1126/zjm_test/mysite/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/req.py&quot;, line 559, in install
        cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
      File &quot;/home/zjm1126/zjm_test/mysite/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/__init__.py&quot;, line 249, in call_subprocess
        % (command_desc, proc.returncode))
    InstallationError: Command /home/zjm1126/zjm_test/mysite/bin/python -c &quot;import setuptools;__file__=&#39;/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-XuVIux-record/install-record.txt --install-headers /home/zjm1126/zjm_test/mysite/include/site/python2.6 failed with error code 1



libxml install error using pip

I&#39;m trying to get the links from a page with xpath. The problem is that I only want the links inside a table, but if I apply the xpath expression on the whole page I&#39;ll capture links which I don&#39;t want.

For example:

    tree = lxml.html.parse(some_response)
    links = tree.xpath(&quot;//a[contains(@href, &#39;http://www.example.com/filter/&#39;)]&quot;)

The problem is that applies the expression to the whole document. I located the element I want, for example:

    tree = lxml.html.parse(some_response)
    root = tree.getroot()
    table = root[1][5] #for example
    links = table.xpath(&quot;//a[contains(@href, &#39;http://www.example.com/filter/&#39;)]&quot;)

But that seems to be performing the query in the whole document as well, as I still am capturing the links outside of the table. &lt;a href=&quot;http://codespeak.net/lxml/xpathxslt.html&quot;&gt;This page&lt;/a&gt; says that &quot;When xpath() is used on an Element, the XPath expression is evaluated against the element (if relative) or against the root tree (if absolute):&quot;. So, what I using is an absolute expression and I need to make it relative? Is that it?

Basically, how can I go about filtering only elements that exist inside of this table?

Python: Using xpath locally / on a specific element

I&#39;ve converted my scripts from Python 2.7 to 3.2, and I have a bug. 

    # -*- coding: utf-8 -*-
    import time
    from datetime import date
    from lxml import etree
    from collections import OrderedDict
    
    # Create the root element
    page = etree.Element(&#39;results&#39;)
    
    # Make a new document tree
    doc = etree.ElementTree(page)
    
    # Add the subelements
    pageElement = etree.SubElement(page, &#39;Country&#39;,Tim = &#39;Now&#39;, 
                                          name=&#39;Germany&#39;, AnotherParameter = &#39;Bye&#39;,
                                          Code=&#39;DE&#39;,
                                          Storage=&#39;Basic&#39;)
    pageElement = etree.SubElement(page, &#39;City&#39;, 
                                          name=&#39;Germany&#39;,
                                          Code=&#39;PZ&#39;,
                                          Storage=&#39;Basic&#39;,AnotherParameter = &#39;Hello&#39;)
    # For multiple multiple attributes, use as shown above
    
    # Save to XML file
    outFile = open(&#39;output.xml&#39;, &#39;w&#39;)
    doc.write(outFile) 

On the last line, I got this error:

    builtins.TypeError: must be str, not bytes
    File &quot;C:\PythonExamples\XmlReportGeneratorExample.py&quot;, line 29, in &lt;module&gt;
      doc.write(outFile)
    File &quot;c:\Python32\Lib\site-packages\lxml\etree.pyd&quot;, line 1853, in lxml.etree._ElementTree.write (src/lxml/lxml.etree.c:44355)
    File &quot;c:\Python32\Lib\site-packages\lxml\etree.pyd&quot;, line 478, in lxml.etree._tofilelike (src/lxml/lxml.etree.c:90649)
    File &quot;c:\Python32\Lib\site-packages\lxml\etree.pyd&quot;, line 282, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:7972)
    File &quot;c:\Python32\Lib\site-packages\lxml\etree.pyd&quot;, line 378, in lxml.etree._FilelikeWriter.write (src/lxml/lxml.etree.c:89527)

I&#39;ve installed Python 3.2, and I&#39;ve installed lxml-2.3.win32-py3.2.exe.

On Python 2.7 it works.

builtins.TypeError: must be str, not bytes

I want to install Lxml so I can then install Scrapy.

When I updated my Mac today it wouldn&#39;t let me reinstall lxml, I get the following error:

    

    In file included from src/lxml/lxml.etree.c:314:
    /private/tmp/pip_build_root/lxml/src/lxml/includes/etree_defs.h:9:10: fatal error: &#39;libxml/xmlversion.h&#39; file not found
    #include &quot;libxml/xmlversion.h&quot;
             ^
    1 error generated.
    error: command &#39;cc&#39; failed with exit status 1

I have tried using brew to install libxml2 and libxslt, both installed fine but I still cannot install lxml.

Last time I was installing I needed to enable the developer tools on Xcode but since it&#39;s updated to Xcode 5 it doesn&#39;t give me that option anymore.

Does anyone know what I need to do?

Cannot install Lxml on Mac OS X 10.9

I have an HTML file (from Newegg) and their HTML is organized like below.  All of the data in their specifications table is &#39;*desc*&#39; while the titles of each section are in &#39;*name.*&#39; Below are two examples of data from Newegg pages.

    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Brand&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;Intel&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Series&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;Core i5&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Cores&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Socket&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;LGA 1156&lt;/td&gt;
    &lt;/tr&gt;
---------------------------------
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Brand&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;AMD&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Series&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;Phenom II X4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Cores&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;4&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
        &lt;td class=&quot;name&quot;&gt;Socket&lt;/td&gt;
        &lt;td class=&quot;desc&quot;&gt;Socket AM3&lt;/td&gt;
    &lt;/tr&gt;

In the end I would like to have a class for a CPU (which is already set up) that consists of a Brand, Series, Cores, and Socket type to store each of the data.  This is the only way I can think of to go about doing this:

    if(parsedDocument.xpath(tr/td[@class=&quot;name&quot;])==&#39;Brand&#39;):
        CPU.brand = parsedDocument.xpath(tr/td[@class=&quot;name&quot;]/nextsibling?).text

And doing this for the rest of the values. How would I accomplish the nextsibling and is there an easier way of doing this?

How to select following sibling/XML tag using XPath

I am trying to parse xml which contains the some non ASCII cheracter, 

the code looks like below

    from lxml import etree
    from lxml import objectify
    content = u&#39;&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;&lt;div&gt;Order date &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160; &#160;: 05/08/2013 12:24:28&lt;/div&gt;&#39;
    mail.replace(&#39;\xa0&#39;,&#39; &#39;)
    xml = etree.fromstring(mail)

but it shows me error on the line &#39;content = ...&#39;
like 

    syntaxError: Non-ASCII character &#39;\xc2&#39; in file /home/projects/ztest/responce.py on line 3, 
    but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

in the terminal it&#39;s working but while running on the eclipse IDE it&#39;s giving me a error.

Don&#39;t know how to overcome..

SyntaxError of Non-ASCII character

1. I am running the following comand for installing the packages in that file &quot;
`pip install -r requirements.txt --download-cache=~/tmp/pip-cache`&quot;.

2. requirement.txt contains pacakages like 
 
&lt;!-- language-all: none --&gt;

    # Data formats
    # ------------
    PIL==1.1.7 # 

    html5lib==0.90
    httplib2==0.7.4
    lxml==2.3.1
    
    # Documentation
    # -------------
    Sphinx==1.1
    docutils==0.8.1
    
    # Testing
    # -------
    behave==1.1.0
    dingus==0.3.2
    django-testscenarios==0.7.2
    mechanize==0.2.5
    mock==0.7.2
    testscenarios==0.2
    testtools==0.9.14
    wsgi_intercept==0.5.1

while comming to install &quot;lxml&quot; packages i am getting the following eror


    Requirement already satisfied (use --upgrade to upgrade): django-testproject&gt;=0.1.1 in /usr/lib/python2.7/site-packages/django_testproject-0.1.1-py2.7.egg (from django-testscenarios==0.7.2-&gt;-r requirements.txt (line 33))
    Installing collected packages: lxml, Sphinx, docutils, behave, dingus, mock, testscenarios, testtools, South
      Running setup.py install for lxml
        Building lxml version 2.3.1.
        Building without Cython.
        ERROR: /bin/sh: xslt-config: command not found
        
        ** make sure the development packages of libxml2 and libxslt are installed **
        
        Using build configuration of libxslt
        building &#39;lxml.etree&#39; extension
        gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.7/src/lxml/lxml.etree.o -w
        In file included from src/lxml/lxml.etree.c:239:0:
        src/lxml/etree_defs.h:9:31: fatal error: libxml/xmlversion.h: No such file or directory
        compilation terminated.
        error: command &#39;gcc&#39; failed with exit status 1
        Complete output from command /usr/bin/python -c &quot;import setuptools;__file__=&#39;/root/Projects/ir/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-SwjFm3-record/install-record.txt:
        Building lxml version 2.3.1.
    
    Building without Cython.
    
    ERROR: /bin/sh: xslt-config: command not found
    
    
    
    ** make sure the development packages of libxml2 and libxslt are installed **
    
    
    
    Using build configuration of libxslt
    
    running install
    
    running build
    
    running build_py
    
    copying src/lxml/cssselect.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/__init__.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/sax.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/pyclasslookup.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/usedoctest.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/doctestcompare.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/_elementpath.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/ElementInclude.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/builder.py -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/html/clean.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/__init__.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/_dictmixin.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/ElementSoup.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/usedoctest.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/defs.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/builder.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/_html5builder.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/diff.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/html5parser.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/_diffcommand.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/_setmixin.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/soupparser.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/html/formfill.py -&gt; build/lib.linux-i686-2.7/lxml/html
    
    copying src/lxml/isoschematron/__init__.py -&gt; build/lib.linux-i686-2.7/lxml/isoschematron
    
    copying src/lxml/etreepublic.pxd -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/tree.pxd -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/etree_defs.h -&gt; build/lib.linux-i686-2.7/lxml
    
    copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/rng
    
    copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl
    
    copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -&gt; build/lib.linux-i686-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    running build_ext
    
    building &#39;lxml.etree&#39; extension
    
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.7/src/lxml/lxml.etree.o -w
    
    In file included from src/lxml/lxml.etree.c:239:0:
    
    src/lxml/etree_defs.h:9:31: fatal error: libxml/xmlversion.h: No such file or directory
    
    compilation terminated.
    
    error: command &#39;gcc&#39; failed with exit status 1
    
    ----------------------------------------
    Command /usr/bin/python -c &quot;import setuptools;__file__=&#39;/root/Projects/ir/build/lxml/setup.py&#39;;execfile(__file__)&quot; install --single-version-externally-managed --record /tmp/pip-SwjFm3-record/install-record.txt failed with error code 1
    Storing complete log in /root/.pip/pip.log

  Can anyone check and guide me what would be the problem and why?Any package missing to install.


src/lxml/etree_defs.h:9:31: fatal error: libxml/xmlversion.h: No such file or directory

pip is not able to install packages correctly: Permission denied error

while running a python script, I got this error

      from lxml import etree
    ImportError: No module named lxml

now I tried to install lxml

    sudo easy_install lmxl

but it gives me the following error

    Building lxml version 2.3.beta1.
    NOTE: Trying to build without Cython, pre-generated &#39;src/lxml/lxml.etree.c&#39; needs to be available.
    ERROR: /bin/sh: xslt-config: not found

    ** make sure the development packages of libxml2 and libxslt are installed **

Using build configuration of libxslt 

    src/lxml/lxml.etree.c:4: fatal error: Python.h: No such file or directory
    compilation terminated.
    error: Setup script exited with error: command &#39;gcc&#39; failed with exit status 1

Installing lxml module in python

I am using [lxml.html][1] to generate some HTML. I want to pretty print (with indentation) my final result into an html file. How do I do that?

This is what I have tried and got till now
    

    import lxml.html as lh
    from lxml.html import builder as E
    sliderRoot=lh.Element(&quot;div&quot;, E.CLASS(&quot;scroll&quot;), style=&quot;overflow-x: hidden; overflow-y: hidden;&quot;)
    scrollContainer=lh.Element(&quot;div&quot;, E.CLASS(&quot;scrollContainer&quot;), style=&quot;width: 4340px;&quot;)
    sliderRoot.append(scrollContainer)
    print lh.tostring(sliderRoot, pretty_print = True, method=&quot;html&quot;)

As you can see I am using the `pretty_print=True` attribute. I thought that would give indented code, but it doesn&#39;t really help. This is the output :

`&lt;div style=&quot;overflow-x: hidden; overflow-y: hidden;&quot; class=&quot;scroll&quot;&gt;&lt;div style=&quot;width: 4340px;&quot; class=&quot;scrollContainer&quot;&gt;&lt;/div&gt;&lt;/div&gt;`


  [1]: http://lxml.de/lxmlhtml.html

How to Pretty Print HTML to a file, with indentation

I need to completely remove elements, based on the contents of an attribute, using python&#39;s lxml.  Example:

    import lxml.etree as et

    xml=&quot;&quot;&quot;
    &lt;groceries&gt;
      &lt;fruit state=&quot;rotten&quot;&gt;apple&lt;/fruit&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;pear&lt;/fruit&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;starfruit&lt;/fruit&gt;
      &lt;fruit state=&quot;rotten&quot;&gt;mango&lt;/fruit&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;peach&lt;/fruit&gt;
    &lt;/groceries&gt;
    &quot;&quot;&quot;

    tree=et.fromstring(xml)

    for bad in tree.xpath(&quot;//fruit[@state=\&#39;rotten\&#39;]&quot;):
      #remove this element from the tree

    print et.tostring(tree, pretty_print=True)

I would like this to print:

    &lt;groceries&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;pear&lt;/fruit&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;starfruit&lt;/fruit&gt;
      &lt;fruit state=&quot;fresh&quot;&gt;peach&lt;/fruit&gt;
    &lt;/groceries&gt;

Is there a way to do this without storing a temporary variable and printing to it manually, as:

    newxml=&quot;&lt;groceries&gt;\n&quot;
    for elt in tree.xpath(&#39;//fruit[@state=\&#39;fresh\&#39;]&#39;):
      newxml+=et.tostring(elt)

    newxml+=&quot;&lt;/groceries&gt;&quot;

how to remove an element in lxml

I&#39;m having difficulty installing lxml with easy_install on Ubuntu 11.

When I type `$ easy_install lxml` I get:

    Searching for lxml
    Reading http://pypi.python.org/simple/lxml/
    Reading http://codespeak.net/lxml
    Best match: lxml 2.3
    Downloading http://lxml.de/files/lxml-2.3.tgz
    Processing lxml-2.3.tgz
    Running lxml-2.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-7UdQOZ/lxml-2.3/egg-dist-tmp-GacQGy
    Building lxml version 2.3.
    Building without Cython.
    ERROR: /bin/sh: xslt-config: not found

    ** make sure the development packages of libxml2 and libxslt are installed **

    Using build configuration of libxslt 
    In file included from src/lxml/lxml.etree.c:227:0:
    src/lxml/etree_defs.h:9:31: fatal error: libxml/xmlversion.h: No such file or directory
    compilation terminated.

It seems that `libxslt` or `libxml2` is not installed. I&#39;ve tried following the instructions at http://www.techsww.com/tutorials/libraries/libxslt/installation/installing_libxslt_on_ubuntu_linux.php and http://www.techsww.com/tutorials/libraries/libxml/installation/installing_libxml_on_ubuntu_linux.php with no success.

If I try `wget ftp://xmlsoft.org/libxml2/libxml2-sources-2.6.27.tar.gz` I get

    &lt;successful connection info&gt;
    ==&gt; SYST ... done.    ==&gt; PWD ... done.
    ==&gt; TYPE I ... done.  ==&gt; CWD (1) /libxml2 ... done.
    ==&gt; SIZE libxml2-sources-2.6.27.tar.gz ... done.
    ==&gt; PASV ... done.    ==&gt; RETR libxml2-sources-2.6.27.tar.gz ... 
    No such file `libxml2-sources-2.6.27.tar.gz&#39;.

If I try the other first, I&#39;ll get to `./configure --prefix=/usr/local/libxslt --with-libxml-prefix=/usr/local/libxml2` and that will fail eventually with:

    checking for libxml libraries &gt;= 2.6.27... configure: error: Could not find libxml2 anywhere, check ftp://xmlsoft.org/.

I&#39;ve tried both versions `2.6.27` and `2.6.29` of `libxml2` with no difference.

Leaving no stone unturned, I have successfully done `sudo apt-get install libxml2-dev`, but this changes nothing.

How to install lxml on Ubuntu

I&#39;d like to write a code snippet that would grab all of the text inside the `&lt;content&gt;` tag, in lxml, in all three instances below, including the code tags. I&#39;ve tried `tostring(getchildren())` but that would miss the text in between the tags. I didn&#39;t have very much luck searching the API for a relevant function. Could you help me out?

    &lt;!--1--&gt;
    &lt;content&gt;
    &lt;div&gt;Text inside tag&lt;/div&gt;
    &lt;/content&gt;
    #should return &quot;&lt;div&gt;Text inside tag&lt;/div&gt;

    &lt;!--2--&gt;
    &lt;content&gt;
    Text with no tag
    &lt;/content&gt;
    #should return &quot;Text with no tag&quot;


    &lt;!--3--&gt;
    &lt;content&gt;
    Text outside tag &lt;div&gt;Text inside tag&lt;/div&gt;
    &lt;/content&gt;
    #should return &quot;Text outside tag &lt;div&gt;Text inside tag&lt;/div&gt;&quot;



Get all text inside a tag in lxml

Suppose the following route which accesses an xml file to replace the text of a specific tag with a given xpath (?key=):

    @app.route(&#39;/resource&#39;, methods = [&#39;POST&#39;])
    def update_text():
        # CODE
Then, I would use cURL like this:

    curl -X POST http://ip:5000/resource?key=listOfUsers/user1 -d &quot;John&quot;

The xpath expreesion `listOfUsers/user1` should access the tag `&lt;user1&gt;` to change its current text to &quot;John&quot;.

I have no idea on how to achieve this because I&#39;m just starting to learn Flask and REST and I can&#39;t find any good example for this specific case. Also, I&#39;d like to use lxml to manipulate the xml file since I already know it. 

Could somebody help and provide an example to guide me?

Flask example with POST

I am having problems with installing `lxml`. I have tried the solutions of the relative questions in this site and other sites but could not fix the problem. Need some suggestions/solution on this.

I am providing the full log after executing `pip install lxml`, 

    Downloading/unpacking lxml
      Downloading lxml-3.3.5.tar.gz (3.5MB): 3.5MB downloaded
      Running setup.py (path:/tmp/pip_build_root/lxml/setup.py) egg_info for package lxml
        /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: &#39;bugtrack_url&#39;
          warnings.warn(msg)
        Building lxml version 3.3.5.
        Building without Cython.
        Using build configuration of libxslt 1.1.28
        
        warning: no previously-included files found matching &#39;*.py&#39;
    Installing collected packages: lxml
      Running setup.py install for lxml
        /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: &#39;bugtrack_url&#39;
          warnings.warn(msg)
        Building lxml version 3.3.5.
        Building without Cython.
        Using build configuration of libxslt 1.1.28
        building &#39;lxml.etree&#39; extension
        x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
        x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
        Please submit a full bug report,
        with preprocessed source if appropriate.
        See &lt;file:///usr/share/doc/gcc-4.8/README.Bugs&gt; for instructions.
        error: command &#39;x86_64-linux-gnu-gcc&#39; failed with exit status 4
        Complete output from command /usr/bin/python -c &quot;import setuptools, tokenize;__file__=&#39;/tmp/pip_build_root/lxml/setup.py&#39;;exec(compile(getattr(tokenize, &#39;open&#39;, open)(__file__).read().replace(&#39;\r\n&#39;, &#39;\n&#39;), __file__, &#39;exec&#39;))&quot; install --record /tmp/pip-KUq9VD-record/install-record.txt --single-version-externally-managed --compile:
        /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: &#39;bugtrack_url&#39;
    
      warnings.warn(msg)
    
    Building lxml version 3.3.5.
    
    Building without Cython.
    
    Using build configuration of libxslt 1.1.28
    
    running install
    
    running build
    
    running build_py
    
    creating build
    
    creating build/lib.linux-x86_64-2.7
    
    creating build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/pyclasslookup.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/doctestcompare.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/builder.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/cssselect.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/sax.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/_elementpath.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/ElementInclude.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/__init__.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/usedoctest.py -&gt; build/lib.linux-x86_64-2.7/lxml
    
    creating build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/__init__.py -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    creating build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/html5parser.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/ElementSoup.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/builder.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/clean.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/_diffcommand.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/diff.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/_html5builder.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/_setmixin.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/defs.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/formfill.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/soupparser.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/__init__.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    copying src/lxml/html/usedoctest.py -&gt; build/lib.linux-x86_64-2.7/lxml/html
    
    creating build/lib.linux-x86_64-2.7/lxml/isoschematron
    
    copying src/lxml/isoschematron/__init__.py -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron
    
    copying src/lxml/lxml.etree.h -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/lxml.etree_api.h -&gt; build/lib.linux-x86_64-2.7/lxml
    
    copying src/lxml/includes/config.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xpath.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/tree.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/dtdvalid.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xmlparser.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/etreepublic.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xslt.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/schematron.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/uri.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/c14n.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xinclude.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xmlschema.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/xmlerror.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/relaxng.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/htmlparser.pxd -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/etree_defs.h -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    copying src/lxml/includes/lxml-version.h -&gt; build/lib.linux-x86_64-2.7/lxml/includes
    
    creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources
    
    creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/rng
    
    copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/rng
    
    creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
    
    copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
    
    copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
    
    creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -&gt; build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
    
    running build_ext
    
    building &#39;lxml.etree&#39; extension
    
    creating build/temp.linux-x86_64-2.7
    
    creating build/temp.linux-x86_64-2.7/src
    
    creating build/temp.linux-x86_64-2.7/src/lxml
    
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
    
    x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
    
    Please submit a full bug report,
    
    with preprocessed source if appropriate.
    
    See &lt;file:///usr/share/doc/gcc-4.8/README.Bugs&gt; for instructions.
    
    error: command &#39;x86_64-linux-gnu-gcc&#39; failed with exit status 4
    
    ----------------------------------------
    Cleaning up...
    Command /usr/bin/python -c &quot;import setuptools, tokenize;__file__=&#39;/tmp/pip_build_root/lxml/setup.py&#39;;exec(compile(getattr(tokenize, &#39;open&#39;, open)(__file__).read().replace(&#39;\r\n&#39;, &#39;\n&#39;), __file__, &#39;exec&#39;))&quot; install --record /tmp/pip-KUq9VD-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
    Storing debug log for failure in /root/.pip/pip.log


Also, the `pip.log` file looks like this,

    status = self.run(options, args)
      File &quot;/usr/lib/python2.7/dist-packages/pip/commands/install.py&quot;, line 283, in run
        requirement_set.install(install_options, global_options, root=options.root_path)
      File &quot;/usr/lib/python2.7/dist-packages/pip/req.py&quot;, line 1435, in install
        requirement.install(install_options, global_options, *args, **kwargs)
      File &quot;/usr/lib/python2.7/dist-packages/pip/req.py&quot;, line 706, in install
        cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
      File &quot;/usr/lib/python2.7/dist-packages/pip/util.py&quot;, line 697, in call_subprocess
        % (command_desc, proc.returncode, cwd))
    InstallationError: Command /usr/bin/python -c &quot;import setuptools, tokenize;__file__=&#39;/tmp/pip_build_root/lxml/setup.py&#39;;exec(compile(getattr(tokenize, &#39;open&#39;, open)(__file__).read().replace(&#39;\r\n&#39;, &#39;\n&#39;), __file__, &#39;exec&#39;))&quot; install --record /tmp/pip-KUq9VD-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml


`dmesg | tail` command shows output like this,

    [1744367.676147] Out of memory: Kill process 25518 (cc1) score 388 or sacrifice child
    [1744367.676665] Killed process 25518 (cc1) total-vm:242352kB, anon-rss:200608kB, file-rss:0kB

It&#39;s seems like a memory issue. I am taking reference form [this question][1]


  [1]: https://stackoverflow.com/questions/16149613/installing-lxml-with-pip-in-virtualenv-ubuntu-12-10-error-command-gcc-failed

lxml installation error ubuntu 14.04 (internal compiler error)

    ...
    soup = BeautifulSoup(html, &quot;lxml&quot;)
    File &quot;/Library/Python/2.7/site-packages/bs4/__init__.py&quot;, line 152, in __init__
    % &quot;,&quot;.join(features))
    bs4.FeatureNotFound: Couldn&#39;t find a tree builder with the features you requested: lxml. Do you need to install a parser library?
The above outputs on my Terminal. I am on Mac OS 10.7.x. I have Python 2.7.1, and followed [this tutorial][1] to get Beautiful Soup and lxml, which both installed successfully and work with a separate test file [located here][2]. In the Python script that causes this error, I have included this line:
    `from pageCrawler import comparePages`
And in the pageCrawler file I have included the following two lines:
    `from bs4 import BeautifulSoup`
    `from urllib2 import urlopen`

Any help in figuring out what the problem is and how it can be solved would much be appreciated.

  [1]: http://www.gregreda.com/2013/03/03/web-scraping-101-with-python/
  [2]: http://web.engr.illinois.edu/~shauk2/testWebScrape.py

bs4.FeatureNotFound: Couldn&#39;t find a tree builder with the features you requested: lxml. Do you need to install a parser library?

I&#39;m using [lxml][1] to create an XML file from scratch;
having a code like this:

    from lxml import etree
    
    root = etree.Element(&quot;root&quot;)
    root.set(&quot;interesting&quot;, &quot;somewhat&quot;)
    child1 = etree.SubElement(root, &quot;test&quot;)

How do I write root `Element` object to an xml file using `write()` method of `ElementTree` class?


  [1]: http://codespeak.net/lxml/tutorial.html

Write xml file using lxml library in Python

I need to parse a xml file to extract some data.
I only need some elements with certain attributes, here&#39;s an example of document:
    
    &lt;root&gt;
        &lt;articles&gt;
            &lt;article type=&quot;news&quot;&gt;
                 &lt;content&gt;some text&lt;/content&gt;
            &lt;/article&gt;
            &lt;article type=&quot;info&quot;&gt;
                 &lt;content&gt;some text&lt;/content&gt;
            &lt;/article&gt;
            &lt;article type=&quot;news&quot;&gt;
                 &lt;content&gt;some text&lt;/content&gt;
            &lt;/article&gt;
        &lt;/articles&gt;
    &lt;/root&gt;

Here I would like to get only the article with the type &quot;news&quot;.
What&#39;s the most efficient and elegant way to do it with lxml?

I tried with the find method but it&#39;s not very nice:

    from lxml import etree
    f = etree.parse(&quot;myfile&quot;)
    root = f.getroot()
    articles = root.getchildren()[0]
    article_list = articles.findall(&#39;article&#39;)
    for article in article_list:
        if &quot;type&quot; in article.keys():
            if article.attrib[&#39;type&#39;] == &#39;news&#39;:
                content = article.find(&#39;content&#39;)
                content = content.text


        








finding elements by attribute with lxml

I want to use an xpath expression to get the value of an attribute.

I expected the following to work

    from lxml import etree

    for customer in etree.parse(&#39;file.xml&#39;).getroot().findall(&#39;BOB&#39;):
        print customer.find(&#39;./@NAME&#39;)

but this gives an error :

    Traceback (most recent call last):
      File &quot;bob.py&quot;, line 22, in &lt;module&gt;
        print customer.find(&#39;./@ID&#39;)
      File &quot;lxml.etree.pyx&quot;, line 1409, in lxml.etree._Element.find (src/lxml/lxml.etree.c:39972)
      File &quot;/usr/local/lib/python2.7/dist-packages/lxml/_elementpath.py&quot;, line 272, in find
        it = iterfind(elem, path, namespaces)
      File &quot;/usr/local/lib/python2.7/dist-packages/lxml/_elementpath.py&quot;, line 262, in iterfind
        selector = _build_path_iterator(path, namespaces)
      File &quot;/usr/local/lib/python2.7/dist-packages/lxml/_elementpath.py&quot;, line 246, in _build_path_iterator
        selector.append(ops[token[0]](_next, token))
    KeyError: &#39;@&#39;

Am I wrong to expect this to work?

selecting attribute values from lxml

From what I can make out, the two main HTML parsing libraries in Python are lxml and BeautifulSoup. I&#39;ve chosen BeautifulSoup for a project I&#39;m working on, but I chose it for no particular reason other than finding the syntax a bit easier to learn and understand. But I see a lot of people seem to favour lxml and I&#39;ve heard that lxml is faster.

So I&#39;m wondering what are the advantages of one over the other? When would I want to use lxml and when would I be better off using BeautifulSoup? Are there any other libraries worth considering?

Parsing HTML in python - lxml or BeautifulSoup? Which of these is better for what kinds of purposes?

    &lt;?xml version=&quot;1.0&quot; ?&gt;
    &lt;data&gt;
        &lt;test &gt;
            &lt;f1 /&gt;
        &lt;/test &gt;
        &lt;test2 &gt;
            &lt;test3&gt;
             &lt;f1 /&gt;
            &lt;/test3&gt;
        &lt;/test2&gt;
        &lt;f1 /&gt;
    &lt;/data&gt;

Using lxml is it possible to find recursively for tag &quot; f1 &quot;? I tried findall method but it works only for immediate children.

I think I should go for BeautifulSoup for this !!!


All Lxml Solutions on Gang of Coders

Total of 23 Lxml Solutions