将通用的DbDataReader映射到List<T>

5

我在使用属性绑定数据访问类时遇到了一些小问题(更像是烦恼)。问题在于当读取器中不存在与类中对应的属性列时,映射会失败。

代码

这是映射器类:

// Map our datareader object to a strongly typed list
private static IList<T> Map<T>(DbDataReader dr) where T : new()
{
    try
    {
        // initialize our returnable list
        List<T> list = new List<T>();
        // fire up the lamda mapping
        var converter = new Converter<T>();
        while (dr.Read())
        {
            // read in each row, and properly map it to our T object
            var obj = converter.CreateItemFromRow(dr);
            // add it to our list
            list.Add(obj);
        }
        // reutrn it
        return list;
    }
    catch (Exception ex)
    {    
        return default(List<T>);
    }
}

转换器类:

/// <summary>
/// Converter class to convert returned Sql Records to strongly typed classes
/// </summary>
/// <typeparam name="T">Type of the object we'll convert too</typeparam>
internal class Converter<T> where T : new()
{
    // Concurrent Dictionay objects
    private static ConcurrentDictionary<Type, object> _convertActionMap = new ConcurrentDictionary<Type, object>();
    // Delegate action declaration
    private Action<IDataReader, T> _convertAction;

    // Build our mapping based on the properties in the class/type we've passed in to the class
    private static Action<IDataReader, T> GetMapFunc()
    {
        var exps = new List<Expression>();
        var paramExp = Expression.Parameter(typeof(IDataReader), "o7thDR");
        var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
        var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
        var _props = typeof(T).GetProperties();
        foreach (var property in _props)
        {

            var getPropExp = Expression.MakeIndex(paramExp, getPropInfo, new[] { Expression.Constant(property.Name, typeof(string)) });
            var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
            var bindExp = Expression.Assign(Expression.Property(targetExp, property), castExp);
            exps.Add(bindExp);

        }
        // return our compiled mapping, this will ensure it is cached to use through our record looping
        return Expression.Lambda<Action<IDataReader, T>>(Expression.Block(exps), new[] { paramExp, targetExp }).Compile();
    }

    internal Converter()
    {
        // Fire off our mapping functionality
        _convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(typeof(T), (t) => GetMapFunc());
    }

    internal T CreateItemFromRow(IDataReader dataReader)
    {
        T result = new T();
        _convertAction(dataReader, result);
        return result;
    }
}

异常

System.IndexOutOfRangeException {"Mileage"}

Stacktrace

at System.Data.ProviderBase.FieldNameLookup.GetOrdinal(String fieldName)
at System.Data.SqlClient.SqlDataReader.GetOrdinal(String name)
at System.Data.SqlClient.SqlDataReader.get_Item(String name)
at lambda_method(Closure , IDataReader , Typing )
at o7th.Class.Library.Data.Converter`1.CreateItemFromRow(IDataReader dataReader) in d:\Backup Folder\Development\o7th Web Design\o7th.Class.Library.C-Sharp\o7th.Class.Library\Data Access Object\Converter.cs:line 50
at o7th.Class.Library.Data.Wrapper.Map[T](DbDataReader dr) in d:\Backup Folder\Development\o7th Web Design\o7th.Class.Library.C-Sharp\o7th.Class.Library\Data Access Object\Wrapper.cs:line 33

问题

如何处理当我有一个额外的属性而读取器可能没有作为列,以及反过来时不会失败的情况? 当然,快速的解决方法是在示例中的查询中简单地添加NULL As Mileage,但这并不是问题的解决方案 :)


这里是使用反射的Map<T>:

// Map our datareader object to a strongly typed list
private static IList<T> Map<T>(DbDataReader dr) where T : new()
{
    try
    {
        // initialize our returnable list
        List<T> list = new List<T>();
        T item = new T();
        PropertyInfo[] properties = (item.GetType()).GetProperties();
        while (dr.Read()) {
            int fc = dr.FieldCount;
            for (int j = 0; j < fc; ++j) {
                var pn = properties[j].Name;
                var gn = dr.GetName(j);
                if (gn == pn) {
                    properties[j].SetValue(item, dr[j], null);
                }
            }
            list.Add(item);
        }
        // return it
        return list;
    }
    catch (Exception ex)
    {
        // Catch an exception if any, an write it out to our logging mechanism, in addition to adding it our returnable message property
        _Msg += "Wrapper.Map Exception: " + ex.Message;
        ErrorReporting.WriteEm.WriteItem(ex, "o7th.Class.Library.Data.Wrapper.Map", _Msg);
        // make sure this method returns a default List
        return default(List<T>);
    }
}
注意: 与使用表达式树相比,此方法速度慢63%...

你至少可以发布Map方法。 - D Stanley
1
@o7thWebDesign,希望你意识到了问题所在。问题在于读取器中不存在指定属性的列。另一种方法是先按数据库列名循环,然后检查是否存在匹配的属性。我会发布一个粗略的解决方案。 - nawfal
1
@o7thWebDesign 我认为表达式不会使它高效。我看到的唯一受益的领域是当你正在new一个新对象时,但是在那里你根本没有使用表达式树。你为什么认为表达式会加速事情?表达式应该只用于避免样板文件创建代码。 - nawfal
1
@o7thWebDesign 我的意思是,表达式方法不会比(From row In _Results.Cast(Of DbDataRecord)() Select New ObjectToTypeTo() With {.TheProperty = row(0)}).ToList().方法更快。当然它比反射调用要快。我添加了一个解决方案,但不完整。对此我很抱歉。我稍后会完成这个练习! :) 就当做一个入门吧。gn - nawfal
1个回答

3
正如评论中所述,问题在于读取器中不存在指定属性的列。想法是首先通过读取器的列名进行循环,并检查是否存在匹配的属性。但是,如何事先获取列名列表呢?
  1. One idea is to use expression trees itself to build the list of column names from the reader and check it against properties of the class. Something like this

    var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
    
    var loopIncrementVariableExp = Expression.Parameter(typeof(int), "i");
    var columnNamesExp = Expression.Parameter(typeof(List<string>), "columnNames");
    
    var columnCountExp = Expression.Property(paramExp, "FieldCount");
    var getColumnNameExp = Expression.Call(paramExp, "GetName", Type.EmptyTypes, 
        Expression.PostIncrementAssign(loopIncrementVariableExp));
    var addToListExp = Expression.Call(columnNamesExp, "Add", Type.EmptyTypes, 
        getColumnNameExp);
    var labelExp = Expression.Label(columnNamesExp.Type);
    
    var getColumnNamesExp = Expression.Block(
        new[] { loopIncrementVariableExp, columnNamesExp },
        Expression.Assign(columnNamesExp, Expression.New(columnNamesExp.Type)),
        Expression.Loop(
            Expression.IfThenElse(
                Expression.LessThan(loopIncrementVariableExp, columnCountExp),
                addToListExp,
                Expression.Break(labelExp, columnNamesExp)),
            labelExp));
    

    would be the equivalent of

    List<string> columnNames = new List<string>();
    for (int i = 0; i < reader.FieldCount; i++)
    {
        columnNames.Add(reader.GetName(i));
    }
    

    One may continue with the final expression, but there is a catch here making any further effort along this line futile. The above expression tree will be fetching the column names every time the final delegate is called which in your case is for every object creation, which is against the spirit of your requirement.

  2. Another approach is to let the converter class have a pre-defined awareness of the column names for a given type, by means of attributes (see for an example) or by maintaining a static dictionary like (Dictionary<Type, IEnumerable<string>>). Though it gives more flexibility, the flip side is that your query need not always include all the column names of a table, and any reader[notInTheQueryButOnlyInTheTableColumn] would result in exception.

  3. The best approach as I see is to fetch the column names from the reader object, but only once. I would re-write the thing like:

    private static List<string> columnNames;
    
    private static Action<IDataReader, T> GetMapFunc()
    {
        var exps = new List<Expression>();
    
        var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
        var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
    
        var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
    
        foreach (var columnName in columnNames)
        {
            var property = typeof(T).GetProperty(columnName);
            if (property == null)
                continue;
    
            // use 'columnName' instead of 'property.Name' to speed up reader lookups
            //in case of certain readers.
            var columnNameExp = Expression.Constant(columnName);
            var getPropExp = Expression.MakeIndex(
                paramExp, getPropInfo, new[] { columnNameExp });
            var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
            var bindExp = Expression.Assign(
                Expression.Property(targetExp, property), castExp);
            exps.Add(bindExp);
        }
    
        return Expression.Lambda<Action<IDataReader, T>>(
            Expression.Block(exps), paramExp, targetExp).Compile();
    }
    
    internal T CreateItemFromRow(IDataReader dataReader)
    {
        if (columnNames == null)
        {
            columnNames = Enumerable.Range(0, dataReader.FieldCount)
                                    .Select(x => dataReader.GetName(x))
                                    .ToList();
            _convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(
                typeof(T), (t) => GetMapFunc());
        }
    
        T result = new T();
        _convertAction(dataReader, result);
        return result;
    }
    

    Now that begs the question why not pass the data reader directly to constructor? That would be better.

    private IDataReader dataReader;
    
    private Action<IDataReader, T> GetMapFunc()
    {
        var exps = new List<Expression>();
    
        var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
        var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
    
        var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
    
        var columnNames = Enumerable.Range(0, dataReader.FieldCount)
                                    .Select(x => dataReader.GetName(x));
        foreach (var columnName in columnNames)
        {
            var property = typeof(T).GetProperty(columnName);
            if (property == null)
                continue;
    
            // use 'columnName' instead of 'property.Name' to speed up reader lookups
            //in case of certain readers.
            var columnNameExp = Expression.Constant(columnName);
            var getPropExp = Expression.MakeIndex(
                paramExp, getPropInfo, new[] { columnNameExp });
            var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
            var bindExp = Expression.Assign(
                Expression.Property(targetExp, property), castExp);
            exps.Add(bindExp);
        }
    
        return Expression.Lambda<Action<IDataReader, T>>(
            Expression.Block(exps), paramExp, targetExp).Compile();
    }
    
    internal Converter(IDataReader dataReader)
    {
        this.dataReader = dataReader;
        _convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(
            typeof(T), (t) => GetMapFunc());
    }
    
    internal T CreateItemFromRow()
    {
        T result = new T();
        _convertAction(dataReader, result);
        return result;
    }
    

    Call it like

    List<T> list = new List<T>();
    var converter = new Converter<T>(dr);
    while (dr.Read())
    {
        var obj = converter.CreateItemFromRow();
        list.Add(obj);
    }
    

虽然如此,我可以提出一些改进意见。

  1. CreateItemFromRow中调用的通用new T()较慢,因为它在幕后使用反射。你可以将该部分委托给表达式树,这样应该更快

  2. 现在的GetProperty调用不区分大小写,这意味着您的列名必须完全匹配属性名。我会使用其中一个Bindings.Flag使其不区分大小写。

  3. 我不确定为什么在此处使用ConcurrentDictionary作为缓存机制。泛型类<T>中的静态字段将对每个T都是唯一的。泛型字段本身可以充当缓存。另外,为什么ConcurrentDictionaryValue部分是object类型?

  4. 正如我之前所说,强烈将类型和列名绑定在一起并不是最好的做法(你正在通过缓存特定类型的Action委托来实现这一点)。即使对于同一类型,您的查询也可能选择不同的列集。最好让数据读取器决定。

  5. 使用Expression.Convert而不是Expression.TypeAs进行从object的值类型转换。

  6. 还要注意,reader.GetOrdinal是执行数据读取器查找的更快的方法。

我会重新像这样编写整个内容:
readonly Func<IDataReader, T> _converter;
readonly IDataReader dataReader;

private Func<IDataReader, T> GetMapFunc()
{
    var exps = new List<Expression>();

    var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");

    var targetExp = Expression.Variable(typeof(T));
    exps.Add(Expression.Assign(targetExp, Expression.New(targetExp.Type)));

    //does int based lookup
    var indexerInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(int) });

    var columnNames = Enumerable.Range(0, dataReader.FieldCount)
                                .Select(i => new { i, name = dataReader.GetName(i) });
    foreach (var column in columnNames)
    {
        var property = targetExp.Type.GetProperty(
            column.name,
            BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
        if (property == null)
            continue;

        var columnNameExp = Expression.Constant(column.i);
        var propertyExp = Expression.MakeIndex(
            paramExp, indexerInfo, new[] { columnNameExp });
        var convertExp = Expression.Convert(propertyExp, property.PropertyType);
        var bindExp = Expression.Assign(
            Expression.Property(targetExp, property), convertExp);
        exps.Add(bindExp);
    }

    exps.Add(targetExp);
    return Expression.Lambda<Func<IDataReader, T>>(
        Expression.Block(new[] { targetExp }, exps), paramExp).Compile();
}

internal Converter(IDataReader dataReader)
{
    this.dataReader = dataReader;
    _converter = GetMapFunc();
}

internal T CreateItemFromRow()
{
    return _converter(dataReader);
}

1
@o7thWebDesign :) 没有人看美式足球,真正的足球在其他地方 :)。我看到代码中有许多改进的可能性。我很快会发布更新! - nawfal
1
我看了LOL ;) 我同意,真正的足球在别处,橄榄球也是如此 ;) - Kevin
@o7thWebDesign 只有10到11%?我将其与您发布的原始版本进行了比较,在我的测试中显示了100到150%的改进! exps.Add(targetExp)是必需的,因为targetExp是我们的返回参数,并且它是块表达式中的最后一个参数。块表达式中的最后一个表达式将被返回。请参见此线程以获取示例问题:how-do-i-specify-the-object-to-return-from-an-expression-tree-method。这很奇怪,如果没有它,它居然能为您工作 :) - nawfal
我能请你看一下这个链接吗:https://dev59.com/rnnZa4cB1Zd3GeqPwvVF - Kevin

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接