    &gt;&gt;&gt; a = np.array([[1,2,3], [4,5,np.nan], [7,8,9]])
    array([[  1.,   2.,   3.],
           [  4.,   5.,  nan],
           [  7.,   8.,   9.]])
    
    &gt;&gt;&gt; a[~np.isnan(a).any(axis=1)]
    array([[ 1.,  2.,  3.],
           [ 7.,  8.,  9.]])

and reassign this to `a`.

Explanation: `np.isnan(a)` returns a similar array with `True` where `NaN`, `False` elsewhere. `.any(axis=1)` reduces an `m*n` array to `n` with an logical `or` operation on the whole rows, `~` inverts `True/False` and `a[  ]` chooses just the rows from the original array, which have `True` within the brackets.

    

I have the following Java regex, which I didn&#39;t write and I am trying to modify:

    ^class-map(?:(\\s+match-all)|(\\s+match-any))?(\\s+[\\x21-\\x7e]{1,40})$
               ^                                 ^

It&#39;s similar to [this one][1].

Note the first question mark. Does it mean that the group is optional? There is already a question mark after the corresponding `)`. Does the colon have a special meaning in regex?

The regex compiles fine, and there are already JUnit tests that show how it works. It&#39;s just that I&#39;m a bit confused about why the first question mark and colon are there.


  [1]: https://stackoverflow.com/questions/11529071/please-explain-this-java-regex

Regex: ?: notation (Question mark and colon notation)

I am running into issues when trying to use the DOMParser in my js code.  In my code, I retrieve an xml file via xmlhttp.responseText soap response.  I want to be able to access its elements in JSON format, so my code looks like:
      
    var xml = new DOMParser();
    xml = xml.parseFromString(xmlhttp.responseText, &#39;text/xml&#39;);
    var result = xmlToJson(xml);

I get this error message:
    ReferenceError: DOMParser is not defined

Edit:
This link hasn&#39;t worked for me because my javascript isn&#39;t in the HTML page, as it is a node.js file.  &lt;https://stackoverflow.com/questions/9250545/javascript-domparser-access-innerhtml-and-other-properties&gt;  

Trying to use the DOMParser with node js

Basically, I&#39;m doing some data analysis.  I read in a dataset as a numpy.ndarray and some of the values are missing (either by just not being there, being `NaN`, or by being a string written &quot;`NA`&quot;).

I want to clean out all rows containing any entry like this.  How do I do that with a numpy ndarray?

How to remove all rows in a numpy.ndarray that contain non-numeric values

<p>Basically, I'm doing some data analysis.  I read in a dataset as a numpy.ndarray and some of the values are missing (either by just not being there, being <code>NaN</code>, or by being a string written "<code>NA</code>").</p>
<p>I want to clean out all rows containing any entry like this.  How do I do that with a numpy ndarray?</p>


I have a tuple called `values` which contains the following:

    (&#39;275&#39;, &#39;54000&#39;, &#39;0.0&#39;, &#39;5000.0&#39;, &#39;0.0&#39;)

I want to change the first value (i.e., `275`) in this tuple but I understand that tuples are immutable so `values[0] = 200` will not work. How can I achieve this?

How to change values in a tuple?

How do I list the dependencies for a package using pip?

Listing the dependencies of a package using pip

I am using BeautifulSoup to scrape an URL and I had the following code, to find the `td` tag whose class is `&#39;empformbody&#39;`:

    import urllib
    import urllib2
    from BeautifulSoup import BeautifulSoup
    
    url =  &quot;http://www.example.com/servlet/av/ResultTemplate=AVResult.html&quot;
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    the_page = response.read()
    soup = BeautifulSoup(the_page)

    soup.findAll(&#39;td&#39;,attrs={&#39;class&#39;:&#39;empformbody&#39;})

Now in the above code we can use `findAll` to get tags and information related to them, but I want to use XPath. Is it possible to use XPath with BeautifulSoup? If possible, please provide me example code.

can we use XPath with BeautifulSoup?

To implement a subscriptable object is easy, just implement `__getitem__` in this object&#39;s class definition.&lt;br/&gt;
But now I want to implement a subscriptable class. For example, I want to implement this code:

    class Fruit(object):
        Apple = 0
        Pear = 1
        Banana = 2
        #________________________________ 
        #/ Some other definitions,         \
        #\ make class &#39;Fruit&#39; subscriptable. /
        # -------------------------------- 
        #        \   ^__^
        #         \  (oo)\_______
        #            (__)\       )\/\
        #                ||----w |
        #                ||     ||

    print Fruit[&#39;Apple&#39;], Fruit[&#39;Banana&#39;]
    #Output: 0 2
I know `getattr` can do the same thing, but I feel subscript accessing is more elegant.

How to implement a subscriptable class in Python (subscriptable class, not subscriptable object)?

I have the following code:

    import re
    #open the xml file for reading:
    file = open(&#39;path/test.xml&#39;,&#39;r+&#39;)
    #convert to string:
    data = file.read()
    file.write(re.sub(r&quot;&lt;string&gt;ABC&lt;/string&gt;(\s+)&lt;string&gt;(.*)&lt;/string&gt;&quot;,r&quot;&lt;xyz&gt;ABC&lt;/xyz&gt;\1&lt;xyz&gt;\2&lt;/xyz&gt;&quot;,data))
    file.close()

where I&#39;d like to replace the old content that&#39;s in the file with the new content. However, when I execute my code, the file &quot;test.xml&quot; is appended, i.e. I have the old content follwed by the new &quot;replaced&quot; content. What can I do in order to delete the old stuff and only keep the new?

Replace and overwrite instead of appending

I can&#39;t seem to find any python libraries that do multiple regression. The only things I find only do simple regression. I need to regress my dependent variable (y) against several independent variables (x1, x2, x3, etc.).

For example, with this data:

    print &#39;y        x1      x2       x3       x4      x5     x6       x7&#39;
    for t in texts:
        print &quot;{:&gt;7.1f}{:&gt;10.2f}{:&gt;9.2f}{:&gt;9.2f}{:&gt;10.2f}{:&gt;7.2f}{:&gt;7.2f}{:&gt;9.2f}&quot; /
       .format(t.y,t.x1,t.x2,t.x3,t.x4,t.x5,t.x6,t.x7)

(output for above:)

    

          y        x1       x2       x3        x4     x5     x6       x7
       -6.0     -4.95    -5.87    -0.76     14.73   4.02   0.20     0.45
       -5.0     -4.55    -4.52    -0.71     13.74   4.47   0.16     0.50
      -10.0    -10.96   -11.64    -0.98     15.49   4.18   0.19     0.53
       -5.0     -1.08    -3.36     0.75     24.72   4.96   0.16     0.60
       -8.0     -6.52    -7.45    -0.86     16.59   4.29   0.10     0.48
       -3.0     -0.81    -2.36    -0.50     22.44   4.81   0.15     0.53
       -6.0     -7.01    -7.33    -0.33     13.93   4.32   0.21     0.50
       -8.0     -4.46    -7.65    -0.94     11.40   4.43   0.16     0.49
       -8.0    -11.54   -10.03    -1.03     18.18   4.28   0.21     0.55

How would I regress these in python, to get the linear regression formula:

&gt; Y = a1x1 + a2x2 + a3x3 + a4x4 + a5x5 + a6x6 + +a7x7 + c



Multiple linear regression in Python

For a minimal working example, let&#39;s digitize a 2D array. `numpy.digitize` requires a 1D array:

    import numpy as np
    N = 200
    A = np.random.random((N, N))
    X = np.linspace(0, 1, 20)
    print np.digitize(A.ravel(), X).reshape((N, N))

Now the [documentation says][1]:

&gt; ... A copy is made only if needed.

How do I know if the `ravel` copy it is &quot;needed&quot; in this case? In general - is there a way I can determine if a particular operation creates a copy or a view?

  [1]: http://docs.scipy.org/doc/numpy/reference/generated/numpy.ravel.html


How can I tell if NumPy creates a view or a copy?

Suppose I have an array

    a = np.array([1, 2, 1, 3, 3, 3, 0])

How can I (efficiently, Pythonically) find which elements of `a` are duplicates (i.e., non-unique values)?  In this case the result would be `array([1, 3, 3])` or possibly `array([1, 3])` if efficient.

I&#39;ve come up with a few methods that appear to work:

### Masking ###

    m = np.zeros_like(a, dtype=bool)
    m[np.unique(a, return_index=True)[1]] = True
    a[~m]

### Set operations ###

    a[~np.in1d(np.arange(len(a)), np.unique(a, return_index=True)[1], assume_unique=True)]

This one is cute but probably illegal (as `a` isn&#39;t actually unique):

    np.setxor1d(a, np.unique(a), assume_unique=True)

### Histograms ###

    u, i = np.unique(a, return_inverse=True)
    u[np.bincount(i) &gt; 1]

### Sorting ###

    s = np.sort(a, axis=None)
    s[:-1][s[1:] == s[:-1]]

### Pandas ###

    s = pd.Series(a)
    s[s.duplicated()]

Is there anything I&#39;ve missed?  I&#39;m not necessarily looking for a numpy-only solution, but it has to work with numpy data types and be efficient on medium-sized data sets (up to 10 million in size).

&lt;hr/&gt;

## Conclusions ##

Testing with a 10 million size data set (on a 2.8GHz Xeon):

    a = np.random.randint(10**7, size=10**7)

The fastest is sorting, at 1.1s.  The dubious `xor1d` is second at 2.6s, followed by masking and Pandas `Series.duplicated` at 3.1s, `bincount` at 5.6s, and `in1d` and senderle&#39;s `setdiff1d` both at 7.3s.  Steven&#39;s `Counter` is only a little slower, at 10.5s; trailing behind are Burhan&#39;s `Counter.most_common` at 110s and DSM&#39;s `Counter` subtraction at 360s.

I&#39;m going to use sorting for performance, but I&#39;m accepting Steven&#39;s answer because the performance is acceptable and it *feels* clearer and more Pythonic.

Edit: discovered the Pandas solution.  If Pandas is available it&#39;s clear and performs well.

Determining duplicate values in an array

Is there a preferred way to keep the data type of a `numpy` array fixed as `int` (or `int64` or whatever), while still having an element inside listed as `numpy.NaN`?

In particular, I am converting an in-house data structure to a Pandas DataFrame. In our structure, we have integer-type columns that still have NaN&#39;s (but the dtype of the column is int). It seems to recast everything as a float if we make this a DataFrame, but we&#39;d really like to be `int`.

Thoughts?

**Things tried:**

I tried using the `from_records()` function under pandas.DataFrame, with `coerce_float=False` and this did not help. I also tried using NumPy masked arrays, with NaN fill_value, which also did not work. All of these caused the column data type to become a float.

NumPy or Pandas: Keeping array type as integer while having a NaN value

I came across the fact that `numpy` arrays are passed by reference at multiple places, but then when I execute the following code, why is there a difference between the behavior of `foo` and `bar`

    import numpy as np
    
    def foo(arr):
       arr = arr - 3
    
    def bar(arr):
       arr -= 3
    
    a = np.array([3, 4, 5])
    foo(a)
    print a # prints [3, 4, 5]

    bar(a)
    print a # prints [0, 1, 2]

I&#39;m using python 2.7 and numpy version 1.6.1



Content Type	Original Author	Original Content on Stackoverflow
Question	zebra	View Question on Stackoverflow
Solution 1 - Python	eumiro	View Answer on Stackoverflow

How to remove all rows in a numpy.ndarray that contain non-numeric values

Python Problem Overview

Python Solutions

Solution 1 - Python

Trying to use the DOMParser with node js

Regex: ?: notation (Question mark and colon notation)

Attributions