How to delete columns in numpy.array
PythonNumpyScipyPython Problem Overview
I would like to delete selected columns in a numpy.array . This is what I do:
n [397]: a = array([[ NaN, 2., 3., NaN],
.....: [ 1., 2., 3., 9]])
In [398]: print a
[[ NaN 2. 3. NaN]
[ 1. 2. 3. 9.]]
In [399]: z = any(isnan(a), axis=0)
In [400]: print z
[ True False False True]
In [401]: delete(a, z, axis = 1)
Out[401]:
array([[ 3., NaN],
[ 3., 9.]])
In this example my goal is to delete all the columns that contain NaN's. I expect the last command to result in:
array([[2., 3.],
[2., 3.]])
How can I do that?
Python Solutions
Solution 1 - Python
Given its name, I think the standard way should be delete:
import numpy as np
A = np.delete(A, 1, 0) # delete second row of A
B = np.delete(B, 2, 0) # delete third row of B
C = np.delete(C, 1, 1) # delete second column of C
According to numpy's documentation page, the parameters for numpy.delete are as follow:
numpy.delete(arr, obj, axis=None)
arrrefers to the input array,objrefers to which sub-arrays (e.g. column/row no. or slice of the array) andaxisrefers to either column wise (axis = 1) or row-wise (axis = 0) delete operation.
Solution 2 - Python
Example from the numpy documentation:
>>> a = numpy.array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> numpy.delete(a, numpy.s_[1:3], axis=0) # remove rows 1 and 2
array([[ 0, 1, 2, 3],
[12, 13, 14, 15]])
>>> numpy.delete(a, numpy.s_[1:3], axis=1) # remove columns 1 and 2
array([[ 0, 3],
[ 4, 7],
[ 8, 11],
[12, 15]])
Solution 3 - Python
Another way is to use masked arrays:
import numpy as np
a = np.array([[ np.nan, 2., 3., np.nan], [ 1., 2., 3., 9]])
print(a)
# [[ NaN 2. 3. NaN]
# [ 1. 2. 3. 9.]]
The np.ma.masked_invalid method returns a masked array with nans and infs masked out:
print(np.ma.masked_invalid(a))
[[-- 2.0 3.0 --]
[1.0 2.0 3.0 9.0]]
The np.ma.compress_cols method returns a 2-D array with any column containing a masked value suppressed:
a=np.ma.compress_cols(np.ma.masked_invalid(a))
print(a)
# [[ 2. 3.]
# [ 2. 3.]]
Solution 4 - Python
This creates another array without those columns:
b = a.compress(logical_not(z), axis=1)
Solution 5 - Python
From Numpy Documentation
np.delete(arr, obj, axis=None) Return a new array with sub-arrays along an axis deleted.
>>> arr
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> np.delete(arr, 1, 0)
array([[ 1, 2, 3, 4],
[ 9, 10, 11, 12]])
>>> np.delete(arr, np.s_[::2], 1)
array([[ 2, 4],
[ 6, 8],
[10, 12]])
>>> np.delete(arr, [1,3,5], None)
array([ 1, 3, 5, 7, 8, 9, 10, 11, 12])
Solution 6 - Python
In your situation, you can extract the desired data with:
a[:, -z]
"-z" is the logical negation of the boolean array "z". This is the same as:
a[:, logical_not(z)]
Solution 7 - Python
>>> A = array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> A = A.transpose()
>>> A = A[1:].transpose()
Solution 8 - Python
Removing Matrix columns that contain NaN. This is a lengthy answer, but hopefully easy to follow.
def column_to_vector(matrix, i):
return [row[i] for row in matrix]
import numpy
def remove_NaN_columns(matrix):
import scipy
import math
from numpy import column_stack, vstack
columns = A.shape[1]
#print("columns", columns)
result = []
skip_column = True
for column in range(0, columns):
vector = column_to_vector(A, column)
skip_column = False
for value in vector:
# print(column, vector, value, math.isnan(value) )
if math.isnan(value):
skip_column = True
if skip_column == False:
result.append(vector)
return column_stack(result)
### test it
A = vstack(([ float('NaN'), 2., 3., float('NaN')], [ 1., 2., 3., 9]))
print("A shape", A.shape, "\n", A)
B = remove_NaN_columns(A)
print("B shape", B.shape, "\n", B)
A shape (2, 4)
[[ nan 2. 3. nan]
[ 1. 2. 3. 9.]]
B shape (2, 2)
[[ 2. 3.]
[ 2. 3.]]