I have a pandas data frame with few columns.

Now I know that certain rows are outliers based on a certain column value.

For instance

&gt; column &#39;Vol&#39; has all values around `12xx` and one value is `4000` (outlier).

Now I would like to exclude those rows that have `Vol` column like this.

So, essentially I need to put a filter on the data frame such that we select all rows where the values of a certain column are within, say, 3 standard deviations from mean.

What is an elegant way to achieve this?

Detect and exclude outliers in a pandas DataFrame

ECMAScript 5 has the `filter()` prototype for `Array` types, but not `Object` types, if I understand correctly.

How would I implement a `filter()` for `Object`s in JavaScript? 

Let&#39;s say I have this object:

    var foo = {
        bar: &quot;Yes&quot;
    };

And I want to write a `filter()` that works on `Object`s:

    Object.prototype.filter = function(predicate) {
        var result = {};
        
        for (key in this) {
            if (this.hasOwnProperty(key) &amp;&amp; !predicate(this[key])) {
                result[key] = this[key];
            }
        }
        
        return result;
    };

This works when I use it in the following demo, but when I add it to my site that uses jQuery 1.5 and jQuery UI 1.8.9, I get JavaScript errors in FireBug.

&lt;!-- begin snippet: js hide: true --&gt;

&lt;!-- language: lang-js --&gt;

    Object.prototype.filter = function(predicate) {
      var result = {};
      for (key in this) {
        if (this.hasOwnProperty(key) &amp;&amp; !predicate(this[key])) {
          console.log(&quot;copying&quot;);
          result[key] = this[key];
        }
      }
      return result;
    };

    var foo = {
      bar: &quot;Yes&quot;,
      moo: undefined
    };

    foo = foo.filter(function(property) {
      return typeof property === &quot;undefined&quot;;
    });

    document.getElementById(&#39;disp&#39;).innerHTML = JSON.stringify(foo, undefined, &#39;  &#39;);
    console.log(foo);

&lt;!-- language: lang-css --&gt;

    #disp {
      white-space: pre;
      font-family: monospace
    }

&lt;!-- language: lang-html --&gt;

    &lt;div id=&quot;disp&quot;&gt;&lt;/div&gt;

&lt;!-- end snippet --&gt;


  [1]: http://jsfiddle.net/MPUnL/4/

JavaScript: filter() for Objects

Rails has built in log filtering so you don&#39;t log passwords and credit cards.  Works great for that but when you want to trigger a custom log (like to email) and send your own params or other data along with it, the parameters are obviously not auto-filtered. I have been digging and trying to find this in the rails source but have had no luck so far.

I have configured rails to filter parameters as follows and it works properly for keeping the data out of rails logs:

    config.filter_parameters += [:password, :password_confirmation, :credit_card]

How would you filter sensitive data from the params hash before dumping it into an email, api call or custom (non-rails) log?


How to filter parameters in rails?

I sometimes have the need to make sure some instances are excluded from a queryset.  
This is the way I do it usually:  

    unwanted_instance = MyModel.objects.get(pk=bad_luck_number)
    uninteresting_stuff_happens()
    my_results = MyModel.objects.exclude(id=unwanted_instance.id)

or, if I have more of them:

    my_results = MyModel.objects.exclude(id_in=[uw_in1.id, uw_in2.id, uw_in3.id])

This &#39;feels&#39; a bit clunky, so I tried:

    my_ideally_obtained_results = MyModel.objects.exclude(unwanted_instance)

Which doesn&#39;t work. But I read [here on SO][1] that a subquery can be used as parameter for exclude.  
Am I out of luck? Am I missing some functionality (checked the docs, but didn&#39;t find any useful pointer)


  [1]: https://stackoverflow.com/questions/1637897/django-queryset-excluding-entries-in-second-model/1638163#1638163

Django excluding specific instances from queryset without using field lookup

I was told that in ZSH you could do something like `command` and then when you hit up it would filter the history based on the given command. But when I try this it just cycles the history like bash does. Is this disabled by default?

Filtering ZSH history by command

I have a Maven2 project, and I need to add, in a properties file, the current version and the current date.

For the current version, I&#39;ve used `${project.version}`, which works correctly.

My question is how can I set the current date (i.e. the date when the build is done by Maven2) in my properties file:

    client.version=Version ${project.version}
    client.build=???

(in addition, if I can specify the format for the date, it will be really great)

Adding the current date with Maven2 filtering

I have a rank-1 `numpy.array` of which I want to make a boxplot. However, I want to exclude all values equal to zero in the array. Currently, I solved this by looping the array and copy the value to a new array if not equal to zero. However, as the array consists of 86 000 000 values and I have to do this multiple times, this takes a lot of patience.

Is there a more intelligent way to do this?

How do I remove all zero elements from a NumPy array?

I have a Pandas DataFrame with a &#39;date&#39; column. Now I need to filter out all rows in the DataFrame that have dates outside of the next two months. Essentially, I only need to retain the rows that are within the next two months. 

What is the best way to achieve this?

Filtering Pandas DataFrames on dates

How can I query/filter in Django and ignore the cases of my query-string?

I&#39;ve got something like and like to ignore the case of `my_parameter`:

    MyClass.objects.filter(name=my_parameter)

Django Model - Case-insensitive Query / Filtering

I&#39;m working with a Boolean index in Pandas.

The question is why the statement:

    a[(a[&#39;some_column&#39;]==some_number) &amp; (a[&#39;some_other_column&#39;]==some_other_number)]

works fine whereas

    a[(a[&#39;some_column&#39;]==some_number) and (a[&#39;some_other_column&#39;]==some_other_number)]

exits with error?

Example:

    a = pd.DataFrame({&#39;x&#39;:[1,1],&#39;y&#39;:[10,20]})

    In: a[(a[&#39;x&#39;]==1)&amp;(a[&#39;y&#39;]==10)]
    Out:    x   y
         0  1  10

    In: a[(a[&#39;x&#39;]==1) and (a[&#39;y&#39;]==10)]
    Out: ValueError: The truth value of an array with more than one element is ambiguous.     Use a.any() or a.all()




Logical operators for Boolean indexing in Pandas

I am working with Javascript(ES6) /FaceBook react and trying to get the first 3 elements of an array that varies in size. I would like do the equivalent of Linq take(n).

In my Jsx file I have  the following:

    var items = list.map(i =&gt; {
      return (
        &lt;myview item={i} key={i.id} /&gt;
      );
    });

Then to get the first 3 items I tried

      var map = new Map(list);
        map.size = 3;
        var items = map(i =&gt; {
          return (&lt;SpotlightLandingGlobalInboxItem item={i} key={i.id} /&gt;);
        });

This didn&#39;t work  as map doesn&#39;t have a set function.

Can you please help?

How to get first N number of elements from an array

I am looking for a very fast way to filter down a collection in C#.  I am currently using generic `List&lt;object&gt;` collections, but am open to using other structures if they perform better.

Currently, I am just creating a new `List&lt;object&gt;` and looping thru the original list.  If the filtering criteria matches, I put a copy into the new list.

Is there a better way to do this?  Is there a way to filter in place so there is no temporary list required?

Filtering collections in C#

I want to filter my dataframe with an `or` condition to keep rows with a particular column&#39;s values that are outside the range `[-0.25, 0.25]`. I tried:

    df = df[(df[&#39;col&#39;] &lt; -0.25) or (df[&#39;col&#39;] &gt; 0.25)]

But I get the error:

&gt; Truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

Truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

Is it possible to filter out some requests using Chrome developer tools, say, filter out all image requests?

Is there a way to filter network requests using Google Chrome developer tools?

I&#39;m using DataTables ([datatables.net][1]) and I would like my search box to be outside of the table (for example in my header div).

Is this possible ?


  [1]: http://www.datatables.net

Datatables - Search Box outside datatable

I&#39;m a C coder developing something in python. I know how to do the following in C (and hence in C-like logic applied to python), but I&#39;m wondering what the &#39;Python&#39; way of doing it is.

I have a dictionary d, and I&#39;d like to operate on a subset of the items, only those who&#39;s key (string) contains a specific substring.

i.e. the C logic would be:

    for key in d:
        if filter_string in key:
            # do something
        else
            # do nothing, continue

I&#39;m imagining the python version would be something like

    filtered_dict = crazy_python_syntax(d, substring)
    for key,value in filtered_dict.iteritems():
        # do something


I&#39;ve found a lot of posts on here regarding filtering dictionaries, but couldn&#39;t find one which involved exactly this.

My dictionary is not nested and i&#39;m using python 2.7



filter items in a python dictionary where keys contain a specific string

I&#39;m trying to create a sub-table from another table of all the last name fields sorted A-Z which have a phone number field that isn&#39;t null. I could do this pretty easy with SQL, but I have no clue how to go about running a SQL query within Excel. I&#39;m tempted to import the data into postgresql and just query it there, but that seems a little excessive. 

For what I&#39;m trying to do, the SQL query `SELECT lastname, firstname, phonenumber WHERE phonenumber IS NOT NULL ORDER BY lastname` would do the trick. It seems too simple for it to be something that Excel can&#39;t do natively. How can I run a SQL query like this from within Excel?

How to run a SQL query on an Excel table?

I have an array:

    var month: List&lt;String&gt; = arrayListOf(&quot;January&quot;, &quot;February&quot;, &quot;March&quot;)

I have to filter the list so I am left with only `&quot;January&quot;`.


How can I filter an ArrayList in Kotlin so I only have elements which match my condition?

I have an array that looks like this:

    array(
      &#39;abc&#39; =&gt; 0,
      &#39;foo-bcd&#39; =&gt; 1,
      &#39;foo-def&#39; =&gt; 1,
      &#39;foo-xyz&#39; =&gt; 0,
      // ...
    )

How can I retain only the elements that start with `foo-`?

Remove all elements from array that do not start with a certain string

I am trying to filter a noisy heart rate signal with python. Because heart rates should never be above about 220 beats per minute, I want to filter out all noise above 220 bpm. I converted 220/minute into 3.66666666 Hertz and then converted that Hertz to rad/s to get 23.0383461 rad/sec.

The sampling frequency of the chip that takes data is 30Hz so I converted that to rad/s to get 188.495559 rad/s.

After looking up some stuff online I found some functions for a bandpass filter that I wanted to make into a lowpass. [Here is the link the bandpass code][1], so I converted it to be this:

    from scipy.signal import butter, lfilter
    from scipy.signal import freqs
    
    def butter_lowpass(cutOff, fs, order=5):
        nyq = 0.5 * fs
        normalCutoff = cutOff / nyq
        b, a = butter(order, normalCutoff, btype=&#39;low&#39;, analog = True)
        return b, a
    
    def butter_lowpass_filter(data, cutOff, fs, order=4):
        b, a = butter_lowpass(cutOff, fs, order=order)
        y = lfilter(b, a, data)
        return y
    
    cutOff = 23.1 #cutoff frequency in rad/s
    fs = 188.495559 #sampling frequency in rad/s
    order = 20 #order of filter
    
    #print sticker_data.ps1_dxdt2
    
    y = butter_lowpass_filter(data, cutOff, fs, order)
    plt.plot(y)


I am very confused by this though because I am pretty sure the butter function takes in the cutoff and sampling frequency in rad/s but I seem to be getting a weird output. Is it actually in Hz?

Secondly, what is the purpose of these two lines:

        nyq = 0.5 * fs
        normalCutoff = cutOff / nyq

I know it&#39;s something about normalization but I thought the nyquist was 2 times the sampling requency, not one half. And why are you using the nyquist as a normalizer?

Can someone explain more about how to create filters with these functions?

I plotted the filter using:

    w, h = signal.freqs(b, a)
    plt.plot(w, 20 * np.log10(abs(h)))
    plt.xscale(&#39;log&#39;)
    plt.title(&#39;Butterworth filter frequency response&#39;)
    plt.xlabel(&#39;Frequency [radians / second]&#39;)
    plt.ylabel(&#39;Amplitude [dB]&#39;)
    plt.margins(0, 0.1)
    plt.grid(which=&#39;both&#39;, axis=&#39;both&#39;)
    plt.axvline(100, color=&#39;green&#39;) # cutoff frequency
    plt.show()

and got this which clearly does not cut-off at 23 rad/s:

![result][2]

  [1]: http://wiki.scipy.org/Cookbook/ButterworthBandpass
  [2]: http://i.imgur.com/weqB1sw.png

Creating lowpass filter in SciPy - understanding methods and units

I have a C# dictionary, `Dictionary&lt;Guid, MyObject&gt;` that I need to be filtered based on a property of `MyObject`.

For example, I want to remove all records from the dictionary where `MyObject.BooleanProperty = false`. What is the best way of acheiving this?

Filtering out values from a C# Generic Dictionary

Sorry just getting into Pandas, this seems like it should be a very straight forward question. How can I use the `isin(&#39;X&#39;)` to remove rows that **are in** the list `X`? In R I would write `!which(a %in% b)`.


Remove rows not .isin(&#39;X&#39;)

The callback function in [`array_filter()`][1] only passes in the array&#39;s values, not the keys.

If I have:
    
    $my_array = array(&quot;foo&quot; =&gt; 1, &quot;hello&quot; =&gt; &quot;world&quot;);
    
    $allowed = array(&quot;foo&quot;, &quot;bar&quot;);

What&#39;s the best way to delete all keys in `$my_array` that are not in the `$allowed` array?

Desired output:

    $my_array = array(&quot;foo&quot; =&gt; 1);

  [1]: http://php.net/manual/en/function.array-filter.php

Filter array by its keys using an array of allowed keys

In the Music app of the new iOS, we can see an album cover behind a view that blurs it. 

How can something like that be accomplished? I&#39;ve read the documentation, but did not find anything there.

![][1]


  [1]: http://i.stack.imgur.com/QXEg4m.jpg

Creating a blurring overlay view

We have an array of objects as such

    var myArr = [ {name: &quot;john&quot;, age: 23},
                  {name: &quot;john&quot;, age: 43},
                  {name: &quot;jim&quot;, age: 101},
                  {name: &quot;bob&quot;, age: 67} ];

how do I get the list of objects from myArr where name is john with lodash?


Filtering array of objects with lodash based on property value

Like just about everyone, I&#39;m still learning the intricacies (and loving them) of the new Java 8 Streams API. I have a question concerning usage of streams. I&#39;ll provide a simplified example.

Java Streams allows us to take a `Collection`, and use the `stream()` method on it to receive a stream of all of its elements. Within it, there are a number of useful methods, such as `filter()`, `map()`, and `forEach()`, which allow us to use lambda operations on the contents.

I have code that looks something like this (simplified):

    set.stream().filter(item -&gt; item.qualify())
        .map(item -&gt; (Qualifier)item).forEach(item -&gt; item.operate());
    set.removeIf(item -&gt; item.qualify());

The idea is to get a mapping of all items in the set, which match a certain qualifier, and then operate through them. After the operation, they serve no further purpose, and should be removed from the original set. The code works well, but I can&#39;t shake the feeling that there&#39;s an operation in `Stream` that could do this for me, in a single line.

If it&#39;s in the Javadocs, I may be overlooking it.

Does anyone more familiar with the API see something like that?

Can Java 8 Streams operate on an item in a collection, and then remove it?

I have two array lists e.g. 

    List&lt;Date&gt; a;
    contains : 10/10/2014, 10/11/2016

    List&lt;Date&gt; b;
    contains : 10/10/2016

How can i do a check between list `a` and `b` so the value that is missing in `b` is returned?e.g. `10/10/2014`

 

How can I return the difference between two lists?

I&#39;m working in Python with a pandas DataFrame of video games, each with a genre. I&#39;m trying to remove any video game with a genre that appears less than some number of times in the DataFrame, but I have no clue how to go about this. I did find [a StackOverflow question](https://stackoverflow.com/questions/6796569/how-to-filter-a-dataframe-based-on-category-counts) that seems to be related, but I can&#39;t decipher the solution at all (possibly because I&#39;ve never heard of R and my memory of functional programming is rusty at best).

Help?

How do I filter a pandas DataFrame based on value counts?

I have been trying to get the container id of docker instance using docker process command, but when i&#39;m trying with filter by **name** it works **fine** for me.


    sudo -S docker ps -q --filter=&quot;name=romantic_rosalind&quot;

&gt; Results container id : 
&gt; 
&gt; 3c7e865f1dfb


But when i filter using **image** i&#39;m getting all the instance container ids :

    sudo -S docker ps -q  --filter=&quot;image=docker-mariadb:1.0.1&quot;

&gt; Results Container ids :
&gt; 
&gt; 5570dc09b581
&gt; 
&gt; 3c7e865f1dfb


But i wish to get only container id of mariadb. 

**How to get container id of docker process using filter as image ?**


How to filter docker process based on image

I have a pandas `DataFrame` called `data` with a column called `ms`.  I want to eliminate all the rows where `data.ms` is above the 95% percentile.  For now, I&#39;m doing this:

    limit = data.ms.describe(90)[&#39;95%&#39;]
    valid_data = data[data[&#39;ms&#39;] &lt; limit]

which works, but I want to generalize that to any percentile.  What&#39;s the best way to do that?


Eliminating all data over a given percentile

I have an array of _ids and I want to get all docs accordingly, what&#39;s the best way to do it ?

Something like ...

       
    // doesn&#39;t work ... of course ...

    model.find({
	    &#39;_id&#39; : [
		    &#39;4ed3ede8844f0f351100000c&#39;,
		    &#39;4ed3f117a844e0471100000d&#39;, 
		    &#39;4ed3f18132f50c491100000e&#39;
	    ]
    }, function(err, docs){
	    console.log(docs);
    });

The array might contain hundreds of _ids.

All Filtering Solutions on Gang of Coders

Total of 31 Filtering Solutions