EF: Validation failing on update when using lazy-loaded, required properties

Given this extremely simple model:

public class MyContext : BaseContext
{
    public DbSet<Foo> Foos { get; set; }
    public DbSet<Bar> Bars { get; set; }
}

public class Foo
{
    public int Id { get; set; }
    public int Data { get; set; }
    [Required]
    public virtual Bar Bar { get; set; }
}

public class Bar
{
    public int Id { get; set; }
}

The following program fails:

object id;
using (var context = new MyContext())
{
    var foo = new Foo { Bar = new Bar() };
    context.Foos.Add(foo);
    context.SaveChanges();
    id = foo.Id;
}
using (var context = new MyContext())
{
    var foo = context.Foos.Find(id);
    foo.Data = 2;
    context.SaveChanges(); //Crash here
}

With a DbEntityValidationException. The message found in EntityValidationErrors is The Bar field is required..

However, if I force loading of the Bar property by adding the following line before SaveChanges:

var bar = foo.Bar;

Everything works fine. This also works if I remove the [Required] attribute.

Is this really the expected behavior? Are there any workarounds (besides loading every single required reference every time I want to update an entity)

I found the following post that had an answer for the same problem:

> The cause of this problem is that in > RC and RTM validation no longer lazy > loads any properties. The reason this > change was made is because when saving > a lot of entities at once that have > lazy loaded properties validation > would get them one by one potentially > causing a lot of unexpected > transactions and crippling > performance. > > The workaround is to explicitly load > all validated properties before saving > or validating by using .Include(), you > can read more on how to do this here: > http://blogs.msdn.com/b/adonet/archive/2011/01/31/using-dbcontext-in-ef-feature-ctp5-part-6-loading-related-entities.aspx

My take on this is that is a pretty crappy proxy implementation. While unnecesarily walking the object graph and retriveing lazy-loaded properties is naturally something to be avoided (but apparently overlooked in Microsoft's first incarnation of EF), you shouldn't have to need to go un-proxying a wrapper to validate that it exists. On second thoughts, I'm not sure why you need to go walking the object graph anyway, surely the change tracker of the ORM knows what objects require validation.

I'm not sure why the problem exists, but I'm sure I wouldn't be having this problem if I was using say, NHibernate.

My 'workaround' - What I've done is define the Required nature of the relationship in a EntityTypeConfiguration class, and removed the Required attribute. This should make it work fine. It means that you will not validate the relationship, but it will fail the update. Not an ideal result.

Ok, here is the real answer =)

First a little explanation

if you have a property (like your Bar) noting a FK (ForeignKey), you can also have the corresponding FK field in your model so if we only need the FK and not the actual Bar we don't need it to go to the database:

[ForeignKey("BarId")]
public virtual Bar Bar { get; set; }
public int BarId { get; set; }

Now, to answer your question, what you can do to make the Bar as Required is to flag the BarId property as required, but not the Bar itself:

[ForeignKey("BarId")]
public virtual Bar Bar { get; set; }
[Required] //this makes the trick
public int BarId { get; set; }

this works like a charm =)

Transparent workaround to ignore error on unloaded references

In your DbContext, override ValidateEntity method to remove validation error on references that are not loaded.

    private static bool IsReferenceAndNotLoaded(DbEntityEntry entry, string memberName)
    {
        var reference = entry.Member(memberName) as DbReferenceEntry;
        return reference != null && !reference.IsLoaded;
    }

    protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry,
                                                 IDictionary<object, object> items)
    {
        var result = base.ValidateEntity(entityEntry, items);
        if (result.IsValid || entityEntry.State != EntityState.Modified)
        {
            return result;
        }
        return new DbEntityValidationResult(entityEntry,
            result.ValidationErrors
                  .Where(e => !IsReferenceAndNotLoaded(entityEntry, e.PropertyName)));
    }

Pros :

• Transparent and will not crash when you use inheritance, complex types, doesn't require modification on your model...
• Only when validation fails
• No reflection
• Iterates only on invalid unloaded references
• No useless data loading

Here's a semi-acceptable work-around:

var errors = this.context.GetValidationErrors();
foreach (DbEntityValidationResult result in errors) {
	Type baseType = result.Entry.Entity.GetType().BaseType;
	foreach (PropertyInfo property in result.Entry.Entity.GetType().GetProperties()) {
		if (baseType.GetProperty(property.Name).GetCustomAttributes(typeof(RequiredAttribute), true).Any()) {
			property.GetValue(result.Entry.Entity, null);
		}
	}
}

If anyone wants a general approach to solve this problem, here you have a custom DbContext which finds out properties based on these constraints:

• Lazy Load is ON.
• Properties with virtual
• Properties having any ValidationAttribute attribute.

After retrieving this list, on any SaveChanges in which have something to modify it will load all references and collections automatically avoiding any unexpected exception.

public abstract class ExtendedDbContext : DbContext
{
    public ExtendedDbContext(string nameOrConnectionString)
        : base(nameOrConnectionString)
    {
    }

    public ExtendedDbContext(DbConnection existingConnection, bool contextOwnsConnection)
        : base(existingConnection, contextOwnsConnection)
    {
    }

    public ExtendedDbContext(ObjectContext objectContext, bool dbContextOwnsObjectContext)
        : base(objectContext, dbContextOwnsObjectContext)
    {
    }

    public ExtendedDbContext(string nameOrConnectionString, DbCompiledModel model)
        : base(nameOrConnectionString, model)
    {
    }

    public ExtendedDbContext(DbConnection existingConnection, DbCompiledModel model, bool contextOwnsConnection)
        : base(existingConnection, model, contextOwnsConnection)
    {
    }

    #region Validation + Lazy Loading Hack

    /// <summary>
    /// Enumerator which identifies lazy loading types.
    /// </summary>
    private enum LazyEnum
    {
        COLLECTION,
        REFERENCE,
        PROPERTY,
        COMPLEX_PROPERTY
    }

    /// <summary>
    /// Defines a lazy load property
    /// </summary>
    private class LazyProperty
    {
        public string Name { get; private set; }
        public LazyEnum Type { get; private set; }

        public LazyProperty(string name, LazyEnum type)
        {
            this.Name = name;
            this.Type = type;
        }
    }

    /// <summary>
    /// Concurrenct dictinary which acts as a Cache.
    /// </summary>
    private ConcurrentDictionary<Type, IList<LazyProperty>> lazyPropertiesByType =
        new ConcurrentDictionary<Type, IList<LazyProperty>>();

    /// <summary>
    /// Obtiene por la caché y si no lo tuviese lo calcula, cachea y obtiene.
    /// </summary>
    private IList<LazyProperty> GetLazyProperties(Type entityType)
    {
        return
            lazyPropertiesByType.GetOrAdd(
                entityType,
                innerEntityType =>
                {
                    if (this.Configuration.LazyLoadingEnabled == false)
                        return new List<LazyProperty>();

                    return
                        innerEntityType
                            .GetProperties(BindingFlags.Public | BindingFlags.Instance)
                            .Where(pi => pi.CanRead)
                            .Where(pi => !(pi.GetIndexParameters().Length > 0))
                            .Where(pi => pi.GetGetMethod().IsVirtual)
                            .Where(pi => pi.GetCustomAttributes().Exists(attr => typeof(ValidationAttribute).IsAssignableFrom(attr.GetType())))
                            .Select(
                                pi =>
                                {
                                    Type propertyType = pi.PropertyType;
                                    if (propertyType.HasGenericInterface(typeof(ICollection<>)))
                                        return new LazyProperty(pi.Name, LazyEnum.COLLECTION);
                                    else if (propertyType.HasGenericInterface(typeof(IEntity<>)))
                                        return new LazyProperty(pi.Name, LazyEnum.REFERENCE);
                                    else
                                        return new LazyProperty(pi.Name, LazyEnum.PROPERTY);
                                }
                            )
                            .ToList();
                }
            );
    }

    #endregion

    #region DbContext

    public override int SaveChanges()
    {
        // Get all Modified entities
        var changedEntries =
            this
                .ChangeTracker
                .Entries()
                .Where(p => p.State == EntityState.Modified);

        foreach (var entry in changedEntries)
        {
            foreach (LazyProperty lazyProperty in GetLazyProperties(ObjectContext.GetObjectType(entry.Entity.GetType())))
            {
                switch (lazyProperty.Type)
                {
                    case LazyEnum.REFERENCE:
                        entry.Reference(lazyProperty.Name).Load();
                        break;
                    case LazyEnum.COLLECTION:
                        entry.Collection(lazyProperty.Name).Load();
                        break;
                }
            }
        }

        return base.SaveChanges();
    }

    #endregion
}

Where IEntity<T> is:

public interface IEntity<T>
{
    T Id { get; set; }
}

These extensions were used in this code:

public static bool HasGenericInterface(this Type input, Type genericType)
{
    return
        input
            .GetInterfaces()
            .Any(x => x.IsGenericType && x.GetGenericTypeDefinition() == genericType);
}

public static bool Exists<T>(this IEnumerable<T> source, Predicate<T> predicate)
{
    foreach (T item in source)
    {
        if (predicate(item))
            return true;
    }

    return false;
} 

Hope it helps,

I know it's a bit late... However, ill post this here. Since i too got horribly annoyed with this. Just tell EF to Include the required field.

Notice the SMALL change

using (var context = new MyContext())
{
    var foo = context.Foos.Include("Bar").Find(id);
    foo.Data = 2;
    context.SaveChanges(); //Crash here
}

Since this is still a problem in EF 6.1.1 I thought I would provide another answer that may suit some people, depending on their exact model requirements. To summarize the issue:

1. You need to use a proxy for lazy loading.

2. The property you are lazy loading is marked Required.

3. You want to modify and save the proxy without having to force-load the lazy references.

3 is not possible with the current EF proxies (either of them), which is a serious shortcoming in my opinion.

In my case the lazy property behaves like a value type so its value is provided when we add the entity and never changed. I can enforce this by making its setter protected and not providing a method to update it, that is, it must be created through a constructor, eg:

var myEntity = new MyEntity(myOtherEntity);

MyEntity has this property:

public virtual MyOtherEntity Other { get; protected set; }

So EF will not perform validation on this property but I can ensure it is not null in the constructor. That is one scenario.

Assuming you do not want to use the constructor in that way, you can still ensure validation using a custom attribute, such as:

[RequiredForAdd]
public virtual MyOtherEntity Other { get; set; }

The RequiredForAdd attribute is a custom attribute that inherits from Attribute not RequiredAttribute. It has no properties or methods apart from its base ones.

In my DB Context class I have a static constructor which finds all the properties with those attributes:

private static readonly List<Tuple<Type, string>> validateOnAddList = new List<Tuple<Type, string>>();

static MyContext()
{
    FindValidateOnAdd();
}

private static void FindValidateOnAdd()
{
    validateOnAddList.Clear();

    var modelType = typeof (MyEntity);
    var typeList = modelType.Assembly.GetExportedTypes()
        .Where(t => t.Namespace.NotNull().StartsWith(modelType.Namespace.NotNull()))
        .Where(t => t.IsClass && !t.IsAbstract);

    foreach (var type in typeList)
    {
        validateOnAddList.AddRange(type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
            .Where(pi => pi.CanRead)
            .Where(pi => !(pi.GetIndexParameters().Length > 0))
            .Where(pi => pi.GetGetMethod().IsVirtual)
            .Where(pi => pi.GetCustomAttributes().Any(attr => attr is RequiredForAddAttribute))
            .Where(pi => pi.PropertyType.IsClass && pi.PropertyType != typeof (string))
            .Select(pi => new Tuple<Type, string>(type, pi.Name)));
    }
}

Now that we have a list of properties we need to check manually, we can override validation and manually validate them, adding any errors to the collection returned from the base validator:

protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry, IDictionary<object, object> items)
{
    return CustomValidateEntity(entityEntry, items);
}

private DbEntityValidationResult CustomValidateEntity(DbEntityEntry entry, IDictionary<object, object> items)
{
    var type = ObjectContext.GetObjectType(entry.Entity.GetType());

    // Always use the default validator.    
    var result = base.ValidateEntity(entry, items);

    // In our case, we only wanted to validate on Add and our known properties.
    if (entry.State != EntityState.Added || !validateOnAddList.Any(t => t.Item1 == type))
        return result;

    var propertiesToCheck = validateOnAddList.Where(t => t.Item1 == type).Select(t => t.Item2);

    foreach (var name in propertiesToCheck)
    {
        var realProperty = type.GetProperty(name);
        var value = realProperty.GetValue(entry.Entity, null);
        if (value == null)
        {
            logger.ErrorFormat("Custom validation for RequiredForAdd attribute validation exception. {0}.{1} is null", type.Name, name);
            result.ValidationErrors.Add(new DbValidationError(name, string.Format("RequiredForAdd validation exception. {0}.{1} is required.", type.Name, name)));
        }
    }

    return result;
}

Note that I am only interested in validating for an Add; if you wanted to check during Modify as well, you would need to either do the force-load for the property or use a Sql command to check the foreign key value (shouldn't that already be somewhere in the context)?

Because the Required attribute has been removed, EF will create a nullable FK; to ensure you DB integrity you could alter the FKs manually in a Sql script that you run against your database after it has been created. This will at least catch the Modify with null issues.

Just had the same problem in EF 6.1.2. To solve this your class should be like the following:

public class Foo {
    public int Id { get; set; }
    public int Data { get; set; }

    public int BarId { get; set; }

    public virtual Bar Bar { get; set; }

}

As you can see, the "Required" attribute is not needed, because the Bar property is already required since the BarId property is not nullable.

So if you wanted the Bar property to be nullable, you would have to write:

public class Foo {
    public int Id { get; set; }
    public int Data { get; set; }

    public int? BarId { get; set; }

    public virtual Bar Bar { get; set; }
}