Puzzling Enumerable.Cast InvalidCastException

Question

The following throws a InvalidCastException .

 IEnumerable<int> list = new List<int>() { 1 }; IEnumerable<long> castedList = list.Cast<long>(); Console.WriteLine(castedList.First()); 

Why?

I am using Visual Studio 2008 SP1.

Answer 1

It is very strange! There is a blog post here that describes how Cast<T>() behavior was changed between .NET 3.5 and .NET 3.5 SP1, but it still doesn't work, t explains InvalidCastException, which you even get if you rewrite your code this way:

 var list = new[] { 1 }; var castedList = from long l in list select l; Console.WriteLine(castedList.First()); 

Obviously, you can get around this by throwing yourself

 var castedList = list.Select(i => (long)i); 

This works, but it does not explain the error in the first place. I tried to make the list short and swim, and they threw the same exception.

Edit

This blog post explains why it doesn't work!

Cast<T>() is an extension method on IEnumerable , not IEnumerable<T> . This means that by the time when each value reaches the moment when it will be launched, it is already placed in the System.Object block. Essentially this is trying to do this:

 int i = 1; object o = i; long l = (long)o; 

This code throws an InvalidCastException that you get. If you try to impose an int directly on a long one, you are fine, but discarding the boxed int back in long does not work.

Of course, weird!

Answer 2

The Enumerable.Cast method is defined as follows:

 public static IEnumerable<TResult> Cast<TResult>( this IEnumerable source ) 

And there is no information about the original type of IEnumerable elements, so I think that each of your ints was originally converted to System.Object through boxing, and then it tried to unpack into a long variable, and this is not true.

Similar code to play:

 int i = 1; object o = i; // boxing long l = (long)o; // unboxing, incorrect // long l = (int)o; // this will work 

So the solution for your problem would be:

 ints.Select(i => (long)i) 

Answer 3

Hmm ... interesting puzzle. All the more interesting is that I just ran it in Visual Studio 2008, and it didn't quit at all.

I am not using Service Pack 1, and maybe this could be a problem. I know that in SP1 there were some performance improvements in .Cast () that could cause a problem. Some reading:

Blog Post 1

Blog Post 2

Answer 4

I'm on it again! Here you will find a solution to all problems with the conversion of List<T> and Enumerable<T> . ~ 150 lines of code
Just remember to define at least one explicit or implicit conversion operator for the input / output types involved (if they do not exist), as it should be anyway!

 using System; using System.Collections; using System.Collections.Generic; using System.Linq; using System.Text; using System.Reflection; namespace System.Collections.Generic //purposely in same namespace as List<T>,IEnumerable<T>, so extension methods are available with them { public static class Enumerable { public static List<TOutput> ConvertAll<TInput,TOutput>( this IEnumerable<TInput> input ) { return BuildConvertedList<TInput,TOutput>( input, GetConverterDelegate<TInput,TOutput>() ); } public static IEnumerable<TOutput> ConvertAll<TInput,TOutput>( this IEnumerable<TInput> input, bool lazy ) { if (lazy) return new LazyConverter<TInput,TOutput>( input, GetConverterDelegate<TInput,TOutput>() ); return BuildConvertedList<TInput,TOutput>( input, GetConverterDelegate<TInput,TOutput>() ); } public static List<TOutput> ConvertAll<TInput,TOutput>( this IEnumerable<TInput> input, Converter<TInput, TOutput> converter ) { return BuildConvertedList<TInput,TOutput>( input, converter ); } public static List<TOutput> ConvertAll<TInput, TOutput>( this List<TInput> input ) { Converter<TInput, TOutput> converter = GetConverterDelegate<TInput,TOutput>(); return input.ConvertAll<TOutput>( converter ); } public static IEnumerable<TOutput> ConvertAll<TInput, TOutput>( this List<TInput> input, Converter<TInput, TOutput> converter, bool lazy ) { if (lazy) return new LazyConverter<TInput, TOutput>( input, converter ); return input.ConvertAll<TOutput>( converter ); } public static List<TOutput> ConvertAll<TInput, TOutput>( this List<TInput> input, Converter<TInput, TOutput> converter ) { return input.ConvertAll<TOutput>( converter ); } //Used to manually build converted list when input is IEnumerable, since it doesn't have the ConvertAll method like the List does private static List<TOutput> BuildConvertedList<TInput,TOutput>( IEnumerable<TInput> input, Converter<TInput, TOutput> converter ){ List<TOutput> output = new List<TOutput>(); foreach (TInput input_item in input) output.Add( converter( input_item ) ); return output; } private sealed class LazyConverter<TInput, TOutput>: IEnumerable<TOutput>, IEnumerator<TOutput> { private readonly IEnumerable<TInput> input; private readonly Converter<TInput, TOutput> converter; private readonly IEnumerator<TInput> input_enumerator; public LazyConverter( IEnumerable<TInput> input, Converter<TInput, TOutput> converter ) { this.input = input; this.converter = converter; this.input_enumerator = input.GetEnumerator(); } public IEnumerator<TOutput> GetEnumerator() {return this;} //IEnumerable<TOutput> Member IEnumerator IEnumerable.GetEnumerator() {return this;} //IEnumerable Member public void Dispose() {input_enumerator.Dispose();} //IDisposable Member public TOutput Current {get {return converter.Invoke( input_enumerator.Current );}} //IEnumerator<TOutput> Member object IEnumerator.Current {get {return Current;}} //IEnumerator Member public bool MoveNext() {return input_enumerator.MoveNext();} //IEnumerator Member public void Reset() {input_enumerator.Reset();} //IEnumerator Member } private sealed class TypeConversionPair: IEquatable<TypeConversionPair> { public readonly Type source_type; public readonly Type target_type; private readonly int hashcode; public TypeConversionPair( Type source_type, Type target_type ) { this.source_type = source_type; this.target_type = target_type; //precalc/store hash, since object is immutable; add one to source hash so reversing the source and target still produces unique hash hashcode = (source_type.GetHashCode() + 1) ^ target_type.GetHashCode(); } public static bool operator ==( TypeConversionPair x, TypeConversionPair y ) { if ((object)x != null) return x.Equals( y ); if ((object)y != null) return y.Equals( x ); return true; //x and y are both null, cast to object above ensures reference equality comparison } public static bool operator !=( TypeConversionPair x, TypeConversionPair y ) { if ((object)x != null) return !x.Equals( y ); if ((object)y != null) return !y.Equals( x ); return false; //x and y are both null, cast to object above ensures reference equality comparison } //TypeConversionPairs are equal when their source and target types are equal public bool Equals( TypeConversionPair other ) { if ((object)other == null) return false; //cast to object ensures reference equality comparison return source_type == other.source_type && target_type == other.target_type; } public override bool Equals( object obj ) { TypeConversionPair other = obj as TypeConversionPair; if ((object)other != null) return Equals( other ); //call IEqualityComparer<TypeConversionPair> implementation if obj type is TypeConversionPair return false; //obj is null or is not of type TypeConversionPair; Equals shall not throw errors! } public override int GetHashCode() {return hashcode;} //assigned in constructor; object is immutable } private static readonly Dictionary<TypeConversionPair,Delegate> conversion_op_cache = new Dictionary<TypeConversionPair,Delegate>(); //Uses reflection to find and create a Converter<TInput, TOutput> delegate for the given types. //Once a delegate is obtained, it is cached, so further requests for the delegate do not use reflection* //(*the typeof operator is used twice to look up the type pairs in the cache) public static Converter<TInput, TOutput> GetConverterDelegate<TInput, TOutput>() { Delegate converter; TypeConversionPair type_pair = new TypeConversionPair( typeof(TInput), typeof(TOutput) ); //Attempt to quickly find a cached conversion delegate. lock (conversion_op_cache) //synchronize with concurrent calls to Add if (conversion_op_cache.TryGetValue( type_pair, out converter )) return (Converter<TInput, TOutput>)converter; //Get potential conversion operators (target-type methods are ordered first) MethodInfo[][] conversion_op_sets = new MethodInfo[2][] { type_pair.target_type.GetMethods( BindingFlags.Static | BindingFlags.Public | BindingFlags.FlattenHierarchy ), type_pair.source_type.GetMethods( BindingFlags.Static | BindingFlags.Public | BindingFlags.FlattenHierarchy ) }; //Find appropriate conversion operator, //favoring operators on target type in case functionally equivalent operators exist, //since the target type conversion operator may have access to an appropriate constructor //or a common instance cache (ie immutable objects may be cached and reused). for (int s = 0; s < conversion_op_sets.Length; s++) { MethodInfo[] conversion_ops = conversion_op_sets[s]; for (int m = 0; m < conversion_ops.Length; m++) { MethodInfo mi = conversion_ops[m]; if ((mi.Name == "op_Explicit" || mi.Name == "op_Implicit") && mi.ReturnType == type_pair.target_type && mi.GetParameters()[0].ParameterType.IsAssignableFrom( type_pair.source_type )) //Assuming op_Explicit and op_Implicit always have exactly one parameter. { converter = Delegate.CreateDelegate( typeof(Converter<TInput, TOutput>), mi ); lock (conversion_op_cache) //synchronize with concurrent calls to TryGetValue conversion_op_cache.Add( type_pair, converter ); //Cache the conversion operator reference for future use. return (Converter<TInput, TOutput>)converter; } } } return (TInput x) => ((TOutput)Convert.ChangeType( x, typeof(TOutput) )); //this works well in the absence of conversion operators for types that implement IConvertible //throw new InvalidCastException( "Could not find conversion operator to convert " + type_pair.source_type.FullName + " to " + type_pair.target_type.FullName + "." ); } } } 

Using an example:

 using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Program { static void Main(string[] args) { List<string> list = new List<string>(new string[] { "abcde", "abcd", "abc"/*will break length constraint*/, "ab", "a" }); //Uncomment line below to see non-lazy behavior. All items converted before method returns, and will fail on third item, which breaks the length constraint. //List<ConstrainedString> constrained_list = list.ConvertAll<string,ConstrainedString>(); IEnumerable<ConstrainedString> constrained_list = list.ConvertAll<string,ConstrainedString>( true ); //lazy conversion; conversion is not attempted until that item is read foreach (ConstrainedString constrained_string in constrained_list) //will not fail until the third list item is read/converted System.Console.WriteLine( constrained_string.ToString() ); } public class ConstrainedString { private readonly string value; public ConstrainedString( string value ){this.value = Constrain(value);} public string Constrain( string value ) { if (value.Length > 3) return value; throw new ArgumentException("String length must be > 3!"); } public static explicit operator ConstrainedString( string value ){return new ConstrainedString( value );} public override string ToString() {return value;} } } } 

Answer 5

I wish they did something clever, for example, using any implicit or explicit cast operators defined for the type. Current behavior and inconsistency are unacceptable. Absolutely useless in the current state.

Realizing that Cast<Type> throwing an exception instead of using the translation operators that I defined for the type, I began to annoy and found this thread. If it is defined to be IEnumerable , why don't they just implement it to use reflection to get the type of the object, get the target type, detect any available static conversion operators, and find the right one to execute. It can use heterogeneous IEnumerable in IEnumerable<T> .

The next implementation is a working idea ...

 public static class EnumerableMinusWTF { public static IEnumerable<TResult> Cast<TResult,TSource>(this IEnumerable<TSource> source) { Type source_type = typeof(TSource); Type target_type = typeof(TResult); List<MethodInfo> methods = new List<MethodInfo>(); methods.AddRange( target_type.GetMethods( BindingFlags.Static | BindingFlags.Public ) ); //target methods will be favored in the search methods.AddRange( source_type.GetMethods( BindingFlags.Static | BindingFlags.Public ) ); MethodInfo op_Explicit = FindExplicitConverstion(source_type, target_type, methods ); List<TResult> results = new List<TResult>(); foreach (TSource source_item in source) results.Add((TResult)op_Explicit.Invoke(null, new object[] { source_item })); return results; } public static MethodInfo FindExplicitConverstion(Type source_type, Type target_type, List<MethodInfo> methods) { foreach (MethodInfo mi in methods) { if (mi.Name == "op_Explicit") //will return target and take one parameter if (mi.ReturnType == target_type) if (mi.GetParameters()[0].ParameterType == source_type) return mi; } throw new InvalidCastException( "Could not find conversion operator to convert " + source_type.FullName + " to " + target_type.FullName + "." ); } } 

Then I can successfully execute this code:

  //LessonID inherits RegexConstrainedString, and has explicit conversion operator defined to convert string to LessonID List<string> lessons = new List<String>(new string[] {"l001,l002"}); IEnumerable<LessonID> constrained_lessons = lessons.Cast<LessonID, string>(); 

Answer 6

Here's what to think about ...

• Do you want to do or convert?
  
• You want to get the result as List<T> or IEnumerable<T> .
  
• If the result is IEnumerable<T> , do you want the cast / transform to be applied lazily (i.e. casting / transform does not actually happen until the iterator reaches each element)?

A useful distinction between cast / convert, as the casting operator often involves creating a new object and can be considered a conversion:
Cast implementations should automatically apply the conversion operators defined for the types involved; A new facility may or may not be built.
Implementations of "Convert" should be allowed to specify the delegate System.Converter<TInput,TOutput> .

Potential method headers:

 List<TOutput> Cast<TInput,TOutput>(IEnumerable<TInput> input); List<TOutput> Convert<TInput,TOutput>(IEnumerable<TInput> input, Converter<TInput,TOutput> converter); IEnumerable<TOutput> Cast<TInput,TOutput>(IEnumerable<TInput> input); IEnumerable<TOutput> Convert<TInput,TOutput>(IEnumerable<TInput> input, Converter<TInput,TOutput> converter); 

Problematic implementation of "Cast" using the existing structure; Suppose you pass as the input the List<string> that you want to convert with any of the previous methods.

 //Select can return only a lazy read-only iterator; also fails to use existing explicit cast operator, because such a cast isn't possible in c# for a generic type parameter (so says VS2008) list.Select<TInput,TOutput>( (TInput x) => (TOutput)x ); //Cast fails, unless TOutput has an explicit conversion operator defined for 'object' to 'TOutput'; this confusion is what lead to this topic in the first place list.Cast<TOuput>(); 

Problem Convert Implementations

 //Again, the cast to a generic type parameter not possible in c#; also, this requires a List<T> as input instead of just an IEnumerable<T>. list.ConvertAll<TOutput>( new Converter<TInput,TOuput>( (TInput x) => (TOutput)x ) ); //This would be nice, except reflection is used, and must be used since c# hides the method name for explicit operators "op_Explicit", making it difficult to obtain a delegate any other way. list.ConvertAll<TOutput>( (Converter<TInput,TOutput>)Delegate.CreateDelegate( typeof(Converter<TInput,TOutput>), typeof(TOutput).GetMethod( "op_Explicit", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.Public ) ) ); 

Summary:
Cast / Convert methods must include certain explicit conversion operators or allow the conversion delegate to be specified. The C # language specification for conversion operators — in particular, the lack of a method name — makes it difficult to get a delegate, except for reflection. An alternative is to encapsulate or copy the conversion code, which once again increases (maintains) the complexity of your code, since the really possible / allowed conversions are implicit in the presence or absence of conversion operators and must be processed by the compiler. We do not need to manually search for definitions with capitalized names (for example, "op_Explicit") of the corresponding conversion operators with reflection in WORK TIME for the corresponding types. In addition, the Cast / Convert methods for converting bulk / lists using explicit conversion operators must really be a structure function, and using List.ConvertAll<T> they ... besides the language specification, it makes it difficult to get a delegate for conversion operators effectively !!!

Answer 7

Of course, the smart thing is to use Select(i => (long)i) and what I would recommend for conversions between built-in value types and for custom conversion.

But as a curious remark, since .NET 4 you can create your own extension method, which also works with these types of conversions. But this requires that you use the dynamic keyword. This just happens:

 public static IEnumerable<TResult> CastSuper<TResult>(this IEnumerable source) { foreach (var s in source) yield return (TResult)(dynamic)s; } 

As I said, it works with integral transforms (narrowing or extension of transformations), numerical conversions to / from / between floating-point types and methods of "conversion" of implicit operator and explicit operator types.

And of course, it still works with good old link conversions and conversion unpacking, such as the original System.Enumerable.Cast<TResult> .