Find the index position of the first non-NA value in an R vector?

R

R Problem Overview


I have a problem where a vector has a bunch of NAs at the beginning, and data thereafter. However the peculiarity of my data is that the first n values that are non NA, are probably unreliable, so I would like to remove them and replace them with NA.

For example, if I have a vector of length 20, and non-NAs start at index position 4:

> z
 [1]          NA          NA          NA -1.64801942 -0.57209233  0.65137286  0.13324344 -2.28339326
 [9]  1.29968050  0.10420776  0.54140323  0.64418164 -1.00949072 -1.16504423  1.33588892  1.63253646
[17]  2.41181291  0.38499825 -0.04869589  0.04798073

I would like to remove the first 3 non-NA values, which I believe to be unreliable, to give this:

> z
 [1]          NA          NA          NA          NA          NA          NA  0.13324344 -2.28339326
 [9]  1.29968050  0.10420776  0.54140323  0.64418164 -1.00949072 -1.16504423  1.33588892  1.63253646
[17]  2.41181291  0.38499825 -0.04869589  0.04798073

Of course I need a general solution and I never know when the first non-NA value starts. How would I go about doing this? IE how do I find out the index position of the first non-NA value?

For completeness, my data is actually arranged in a data frame with lots of these vectors in columns, and each vector can have a different non-NA starting position. Also once the data starts, there may be sporadic NAs further down, which prevents me from simply counting their number, as a solution.

R Solutions


Solution 1 - R

Use a combination of is.na and which to find the non-NA index locations.

NonNAindex <- which(!is.na(z))
firstNonNA <- min(NonNAindex)

# set the next 3 observations to NA
is.na(z) <- seq(firstNonNA, length.out=3)

Solution 2 - R

Similar idea to that of @Joshua, but using which.min()

## dummy data
set.seed(1)
dat <- runif(10)
dat[seq_len(sample(10, 1))] <- NA

## start of data
start <- which.min(is.na(dat))

which gives:

> (start <- which.min(is.na(dat)))
[1] 4

Use this to set start:(start+2) to NA

is.na(dat) <- seq(start, length.out = 3)

resulting in:

> dat
 [1]         NA         NA         NA         NA         NA
 [6]         NA 0.94467527 0.66079779 0.62911404 0.06178627

Solution 3 - R

If dealing with large data, Position is considerably faster than which, because it only evaluates until a match is found, rather than evaluating the whole vector.

x=c(rep(NA,3),1:1e8)
Position(function(x) !is.na(x), x)
# 4

We can assign NA to the following N values (or the end of the vector, whichever comes first) by

pos = Position(function(x)!is.na(x), x)
x[pos:min(pos+N-1, length(x))] <- NA

Solution 4 - R

I would do it something along the lines of

# generate some data
tb <- runif(10)
tb[1:3] <- NA

# I convert vector to TRUE/FALSE based on whether it's NA or not
# rle function will tell you when something "changes" in the vector
# (in our case from TRUE to FALSE)
tb.rle <- rle(is.na(tb))

# this is where vector goes from all TRUE to (at least one) FALSE
# your first true number is one position ahead, so +1
tb.rle$lengths[1] 

# you can now subset your vector with the first non-NA value
# and do with it whatever you want. I assign it a fantastic 
# non-believable number
tb[tb.rle$lengths[1] + 1] <- 42

Solution 5 - R

na.trim() in the zoo package can help.

library(zoo)
dummy.data <- c(rep(NA, 5), seq(1:7), NA)
x <- length(dummy.data) - length(na.trim(dummy.data, sides = "left"))
dummy.data[(x+1):(x+3)] <- NA
dummy.data
[1] NA NA NA NA NA NA NA NA  4  5  6  7 NA

Solution 6 - R

You can directly use replace() function also, I know answer is already there but like replace() is too good with these kind of things

For Example-:

A <- c(1,2,3,4,5,NA,58,NA,98,NA,NA,NA)
which(is.na(A))
A <- replace(A,1:3,NA)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionThomas BrowneView Question on Stackoverflow
Solution 1 - RJoshua UlrichView Answer on Stackoverflow
Solution 2 - RGavin SimpsonView Answer on Stackoverflow
Solution 3 - RdwwView Answer on Stackoverflow
Solution 4 - RRoman LuštrikView Answer on Stackoverflow
Solution 5 - RInColoradoView Answer on Stackoverflow
Solution 6 - RBharat KaushikView Answer on Stackoverflow