Find the index position of the first non-NA value in an R vector?
RR Problem Overview
I have a problem where a vector has a bunch of NAs at the beginning, and data thereafter. However the peculiarity of my data is that the first n values that are non NA, are probably unreliable, so I would like to remove them and replace them with NA.
For example, if I have a vector of length 20, and non-NAs start at index position 4:
> z
[1] NA NA NA -1.64801942 -0.57209233 0.65137286 0.13324344 -2.28339326
[9] 1.29968050 0.10420776 0.54140323 0.64418164 -1.00949072 -1.16504423 1.33588892 1.63253646
[17] 2.41181291 0.38499825 -0.04869589 0.04798073
I would like to remove the first 3 non-NA values, which I believe to be unreliable, to give this:
> z
[1] NA NA NA NA NA NA 0.13324344 -2.28339326
[9] 1.29968050 0.10420776 0.54140323 0.64418164 -1.00949072 -1.16504423 1.33588892 1.63253646
[17] 2.41181291 0.38499825 -0.04869589 0.04798073
Of course I need a general solution and I never know when the first non-NA value starts. How would I go about doing this? IE how do I find out the index position of the first non-NA value?
For completeness, my data is actually arranged in a data frame with lots of these vectors in columns, and each vector can have a different non-NA starting position. Also once the data starts, there may be sporadic NAs further down, which prevents me from simply counting their number, as a solution.
R Solutions
Solution 1 - R
Use a combination of is.na
and which
to find the non-NA index locations.
NonNAindex <- which(!is.na(z))
firstNonNA <- min(NonNAindex)
# set the next 3 observations to NA
is.na(z) <- seq(firstNonNA, length.out=3)
Solution 2 - R
Similar idea to that of @Joshua, but using which.min()
## dummy data
set.seed(1)
dat <- runif(10)
dat[seq_len(sample(10, 1))] <- NA
## start of data
start <- which.min(is.na(dat))
which gives:
> (start <- which.min(is.na(dat)))
[1] 4
Use this to set start:(start+2)
to NA
is.na(dat) <- seq(start, length.out = 3)
resulting in:
> dat
[1] NA NA NA NA NA
[6] NA 0.94467527 0.66079779 0.62911404 0.06178627
Solution 3 - R
If dealing with large data, Position
is considerably faster than which
, because it only evaluates until a match is found, rather than evaluating the whole vector.
x=c(rep(NA,3),1:1e8)
Position(function(x) !is.na(x), x)
# 4
We can assign NA to the following N values (or the end of the vector, whichever comes first) by
pos = Position(function(x)!is.na(x), x)
x[pos:min(pos+N-1, length(x))] <- NA
Solution 4 - R
I would do it something along the lines of
# generate some data
tb <- runif(10)
tb[1:3] <- NA
# I convert vector to TRUE/FALSE based on whether it's NA or not
# rle function will tell you when something "changes" in the vector
# (in our case from TRUE to FALSE)
tb.rle <- rle(is.na(tb))
# this is where vector goes from all TRUE to (at least one) FALSE
# your first true number is one position ahead, so +1
tb.rle$lengths[1]
# you can now subset your vector with the first non-NA value
# and do with it whatever you want. I assign it a fantastic
# non-believable number
tb[tb.rle$lengths[1] + 1] <- 42
Solution 5 - R
na.trim() in the zoo package can help.
library(zoo)
dummy.data <- c(rep(NA, 5), seq(1:7), NA)
x <- length(dummy.data) - length(na.trim(dummy.data, sides = "left"))
dummy.data[(x+1):(x+3)] <- NA
dummy.data
[1] NA NA NA NA NA NA NA NA 4 5 6 7 NA
Solution 6 - R
You can directly use replace() function also, I know answer is already there but like replace() is too good with these kind of things
For Example-:
A <- c(1,2,3,4,5,NA,58,NA,98,NA,NA,NA)
which(is.na(A))
A <- replace(A,1:3,NA)