These releases come with the complete support for the DYNAMIC operations. It also comes with a lot of bug fixes and enhancements.
The major release feature is being called DYNAMICS as it is enabling the dynamic capabilities if we are to use it in the context of an ORM. Therefore, do not get confused comparing the terminology against the dynamics within the .NET ecosystem.
This release is now supporting the capability of pushing an instance of ExpandoObject or IDictionary<string, object> object into the push operations mentioned below.
And an instance of IEnumerable<ExpandoObject> or IEnumerable<IDictionary<string, object>> object into the push operations mentioned below.
Though the use-case ratio percentage is low, in some cases (edge-cases), it is quite important to get enabled.
The main target is just to fully enable the support for dynamic capabilities in the context of an ORM.
Let us say, you would like to copy the data from one table into another table that is residing in a different database, then the dynamic code snippet below would do the job for you.
using (var sourceConnection = new SqlConnection(sourceConnection))
{
var people = sourceConnection.QueryAll("[dbo].[Person]");
using (var destinationConnection = new SqlConnection(destinationConnection))
{
destinationConnection.InsertAll("[sales].[Customer]", people);
}
}
When you query all the people from the [dbo].[Person] table, it returns an instance of IEnumerable<ExpandoObject>. The InsertAll is now capable of accepting such argument value.
If you think the data is quite big enough, you can instead use the BulkInsert operation like below.
destinationConnection.BulkInsert("[sales].[Customer]", people);
Also, notice that you have not even created a single model to accomplish the job, for as long as the structure of the source table [dbo].[Person] is identical to the destination table [sales].[Customer], then you can do insert and/or bulk-process it.
You can as well filter the results based on the target fields from the destination table like below.
using (var sourceConnection = new SqlConnection(sourceConnection))
{
using (var destinationConnection = new SqlConnection(destinationConnection))
{
var fields = DbFieldCache
.Get("[sales].[Customer]", destinationConnection, transaction: null)
.Select(e => e.AsField());
var people = sourceConnection.QueryAll("[dbo].[Person]", fields: fields);
destinationConnection.InsertAll("[sales].[Customer]", people);
}
}
Or, just add your own literal string like below.
using (var sourceConnection = new SqlConnection(sourceConnection))
{
using (var destinationConnection = new SqlConnection(destinationConnection))
{
var fields = Field.From("Name", "Address", "SSN");
var people = sourceConnection.QueryAll("[dbo].[Person]", fields: fields);
destinationConnection.InsertAll("[sales].[Customer]", people);
}
}
As the ExpandoObject is also a dictionary object underneath, if the identity column is present in the destination table but is absent from the source list (i.e.:
IEnumerable<ExpandoObject>,IEnumerable<IDictionary<string, object>>) during the operations (i.e.: Insert, InsertAll and BulkInsert), then, the new key-value-pair (KVP) entry will be created in the underlying dictionary object. The same logic applies to the Merge, MergeAll and BulkMerge operations.
Below are the code snippets for each of the affected operations.
Below is a sample code snippet for Insert operation.
using (var connection = new SqlConnection(connection))
{
var person = new Dictionary<string, object>
{
{ "Name", "John Doe" },
{ "Address", "New York" }
};
var id = connection.Insert("[dbo].[Person]", person);
}
If the identity column is present on the
[dbo].[Person]table, then a new entry of key-value-pair will be added to thepersonvariable.
Let us say you have this method.
public IEnumerable<IDictionary<string, object>> GetPeople(int count = 10)
{
for (var i = 0; i < count; i++)
{
yield return new Dictionary<string, object>
{
{ "Name", $"Name {i}" },
{ "Address", $"Address{i}" }
};
}
}
Then, below is a sample code snippet for InsertAll operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(100);
var insertedRows = connection.InsertAll("[dbo].[Person]", people);
}
And below is for BulkInsert operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(10000);
var insertedRows = connection.BulkInsert("[dbo].[Person]", people);
}
If the identity column is present on the
[dbo].[Person]table, then the new entries of the key-value-pairs will be added to each item of thepeoplevariable.
Below is a sample code snippet for Merge operation.
using (var connection = new SqlConnection(connection))
{
var person = new Dictionary<string, object>
{
{ "Id", 10045 }
{ "Name", "James Smith" },
{ "Address", "New York" }
};
var id = connection.Merge("[dbo].[Person]", person);
}
If the identity column is present on the
[dbo].[Person]table, then a new entry of key-value-pair will be added to thepersonvariable.
By default, the primary and/or identity column will be used as qualifier, but you can specify your custom qualifiers like below.
var id = connection.Merge("[dbo].[Person]", people, qualifiers: Field.From("Name"));
Let us say you have this method.
public IEnumerable<IDictionary<string, object>> GetPeople(int count = 10)
{
for (var i = 0; i < count; i++)
{
yield return new Dictionary<string, object>
{
{ "Id", i }
{ "Name", $"Name {i} - Merged" },
{ "Address", $"Address{i} - Merged" }
};
}
}
Then, below is a sample code snippet for MergeAll operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(100);
var mergedRows = connection.MergeAll("[dbo].[Person]", people);
}
And below is for BulkMerge operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(10000);
var mergedRows = connection.BulkMerge("[dbo].[Person]", people);
}
If the identity column is present on the
[dbo].[Person]table, then the new entries of the key-value-pairs will be added to each item of thepeoplevariable.
By default, the primary and/or identity column will be used as qualifier, but you can specify your custom qualifiers like below.
// MergeAll
var mergedRows = connection.MergeAll("[dbo].[Person]",
people,
qualifiers: Field.From("Name"));
// BulkMerge
var mergedRows = connection.BulkMerge("[dbo].[Person]",
people,
qualifiers: Field.From("Name"));
Below is a sample code snippet for Update operation.
using (var connection = new SqlConnection(connection))
{
var person = new Dictionary<string, object>
{
{ "Id", 10045 },
{ "Name", "John Doe" },
{ "Address", "California" },
{ "UpdatedUtc", DateTime.UtcNow }
};
var updatedRows = connection.Update("[dbo].[Person]", person);
}
By default, the primary and/or identity column will be used as qualifier, but you can specify your custom qualifiers like below.
var updatedRows = connection.Update("[dbo].[Person]", people, qualifiers: Field.From("Name"));
Let us say you have this method.
public IEnumerable<IDictionary<string, object>> GetPeople(int count = 10)
{
for (var i = 0; i < count; i++)
{
yield return new Dictionary<string, object>
{
{ "Id", i }
{ "Name", $"Name {i} - Updated" },
{ "Address", $"Address{i} - Updated" }
};
}
}
Then, below is a sample code snippet for Updated operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(100);
var mergedRows = connection.UpdateAll("[dbo].[Person]", people);
}
And below is for BulkUpdate operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(10000);
var updatedRows = connection.BulkUpdate("[dbo].[Person]", people);
}
By default, the primary and/or identity column will be used as qualifier, but you can specify your custom qualifiers like below.
// UpdateAll
var updatedRows = connection.UpdateAll("[dbo].[Person]",
people,
qualifiers: Field.From("Name"));
// BulkUpdate
var updatedRows = connection.BulkUpdate("[dbo].[Person]",
people,
qualifiers: Field.From("Name"));
Let us say you have this method.
public IEnumerable<IDictionary<string, object>> GetKeys(int count = 10)
{
for (var i = 0; i < count; i++)
{
yield return new Dictionary<string, object>
{
{ "Id", i },
{ "Name", $"Name {i}" }
};
}
}
And below is for BulkDelete operation.
using (var connection = new SqlConnection(connection))
{
var people = GetPeople(10000);
var deletedRows = connection.BulkDelete("[dbo].[Person]", people);
}
By default, the primary and/or identity column will be used as qualifier, but you can specify your custom qualifiers like below.
var deletedRows = connection.BulkDelete("[dbo].[Person]", people, qualifiers: Field.From("Name"));
Or, you can also just bulk-delete through the list of primary keys.
var keys = new [] { 10045, ..., 11211 };
using (var connection = new SqlConnection(connection))
{
var deletedRows = connection.BulkDelete("[dbo].[Person]", primaryKeys: keys);
}
Dictionary<string, string> #647v2.1.0.System.ComponentModel.Annotations to v5.0.0.TPropertyHandler has implemented the IPropertyHandler interface when calling the Add() method of the PropertyHandlerCache object.TableName. #234TableName. #238TableName. #236TableName. #235TableName. #239TableName. #237WITH BulkMergeAsync #640Thank you to our contributors and to all the collaborators.
For our active collaborators on this release:
For the other collaborators of this release (filing the Issues/Bugs, Questions, Recommendations, Verification/Validations, etc)
Please continue to do so and
]]>Starting at version v1.12.4 and its corresponding extended libraries for different DB Providers at version 1.1.1, all the Async methods within the library will have an additional argument cancellationToken (defaulted to CancellationToken.None and/or default).
When you upgrade, you can leverage the possibility of cancellable async operations. All the existing calls you made to any operations will not be affected (both compilation/signatures and the behaviors).
See the sample calls below for ExecuteNonQuery.
var tokenSource = new CancellationTokenSource();
using (var connection = new SqlConnection(connectionString))
{
var result = await connection.ExecuteQueryAsync<DateTime>("SELECT GETUTCDATE();",
cancellationToken: tokenSource.Token);
// You can cancel the token anytime
}
And for the Query operation.
var tokenSource = new CancellationTokenSource();
using (var connection = new SqlConnection(connectionString))
{
var people = await connection.QueryAllAsync<Person>(cancellationToken: tokenSource.Token);
// You can cancel the token anytime
}
And for the Insert operation.
var tokenSource = new CancellationTokenSource();
using (var connection = new SqlConnection(connectionString))
{
var id = await connection.InsertAsync<Person, long>(new
{
Name = "John Doe",
Address = "New York"
},
cancellationToken: tokenSource.Token);
// You can cancel the token anytime
}
Such calls will be identical to all other async methods/operations within the library.
There is one breaking change that will be covered by this release. But the use case for this is very minimal and/or almost 0.
If you have created your own customized IDbHelper object, then, you need to change your signature by simply adding the cancellationToken argument in both methods defined in this interface.
public async Task<IEnumerable<DbField>> GetFieldsAsync(IDbConnection connection,
string tableName,
IDbTransaction transaction = null,
CancellationToken cancellationToken = default)
public async Task<object> GetScopeIdentityAsync(IDbConnection connection,
IDbTransaction transaction = null,
CancellationToken cancellationToken = default)
You can visit the actual release from our Github page.
]]>The version RepoDB v1.12.3 is a version that contains the fixes for those issues/bugs.
Allow us to explain and enumerate those issues.
Only in the new compiler. In order for us to ensure that the collision will not happen during the caching of the AOT Compiled Expression, we need to consider the caching of the following.
Without the ordinal caching, the collision may happen in various queries per entity model, resulting in multiple invalid casting exceptions (and/or data assignment issues). See below.
var people = connection.ExecuteQuery<Person>("SELECT Id, Name FROM Person;");
people = connection.ExecuteQuery<Person>("SELECT Name, Id FROM Person;");
This is critical; therefore, we issued the fix immediately.
Affected by the latest release and core compiler updates, the current fetch operations are using the DbDataReader.ToEnumerable() operations. Therefore, all the Async fetch operations (i.e.: BatchQueryAsync, QueryAsync and ExecuteQueryAsync) are not truly async methods.
As a fix, we again reverted the code to DbDataReader.ToEnumerableAsync() and made updates to the mentioned operations to have the proper chain of calls.
As reported by the user, a NullReferenceException is thrown if the Query Expression is being equated to a NULL value.
connection.Query<Person>(e => e.Name == null);
The issue happened because of the changes made in the latest compiler. In addition to that, we added the proper Integration Tests to cover this case.
In the previous version, the code below was working for the table that has no Primary and Identity key.
var model = new { Id = 1, ColumnToUpdate = "Value" };
connection.Update("NonKeyedTable", model);
It was affected by the changes during the updates to the core compiler; therefore, we needed to issue an immediate hotfix for it.
A shared POCO between different data providers is not working if the DB Specific provider attribute is defined.
Let us say, you would like to use the model below for both PostgreSQL and SQLite.
public class Person
{
public long Id { get; set; }
public string Name { get; set; }
[NpgsqlTypeMap(NpgsqlDbType.Json)]
public string ExtraInformation { get; set; }
}
As stated with the attribute NpgsqlTypeMap, such a model is catered for PostgreSQL; however, a user also would like to use such a model for SQLite, and there the exception is being thrown.
As an opportunity to further enhance the library, we have included the fixes for this. The fixes are not limited to the mentioned DB providers.
You can visit the actual release from our Github page
]]>The version v1.12.0 together with its extensions (at v1.1.0) release is one of the biggest suite releases of RepoDB. It has the major updates to its Core Features and Capabilities (Enhancements, Bug Fixes and Requests from the .NET Community). It also includes the initial support for the F# programming language; bundled with the new Enhanced Compiler, Class Handler and Advanced Dynamic Operation Invocations.
Below is the list of the updates/changes covered by this release.
It is important to not skip the Breaking Changes section.
This release package comes with the initial support for the F# programming language together with the various requests from the F# community. Thank you to Isaac Abraham and Angel Munoz for being active collaborators here.
To start with, first, install the package. In this sample, we will use the SQL Server.
> Install-Package RepoDb.SqlServer
Then, initialize the bootstrapper.
> RepoDb.SqlServerBootstrap.Initialize();
Then, create a model.
module Types =
//[<CLIMutable>]
type Person =
{ Id: int64
Name: string
Age: int
Address: string
IsActive: bool }
And also the table.
CREATE TABLE [dbo].[Person]
(
[Id] [bigint] PRIMARY KEY IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](128) NOT NULL,
[Age] [int] NULL,
[Address] [nvarchar](max) NULL,
[IsActive] [bit] NOT NULL
)
GO
ALTER TABLE [dbo].[Person] ADD CONSTRAINT [DF_Person_IsActive] DEFAULT ((1)) FOR [IsActive]
GO
Lastly, call the operations.
// Query
let connection = (new SqlConnection(url))
let queryResult = connection.QueryAll("Person").AsList()
// Insert
let person = { Id = 0L; Name = "John Doe"; Address = "New York"; Age = 32; IsActive = true }
let id = connection.Insert<Person, int64>(person)
// Insert (TableName)
let person = {| Name = "James Smith"; Age = 32; Address = "Washington"; IsActive = true|}
let id = connection.Insert<int64>(ClassMappedNameCache.Get<Person>(), person)
// Merge
let person = { Id = 0L; Name = "John Doe"; }
let id = connection.Merge<Person>(person)
// Update
let person = { Id = 0L; Name = "James Doe"; }
let affectedRows = connection.Update<Person>(person)
// Delete
let affectedRows = connection.Delete<Person>(10045)
By default, all the functionalities and features written for the C# programming language are also inherited by the F# programming language. It is also important to take note that the CLIMutable attribute can be eliminated from all your models starting from this release.
Disclaimer: The option types in F# are not supported if it is being placed at the ClassHandler and PropertyHandler objects.
The core compiler of RepoDB has been rewritten from being a monolithic codebase to multiple smaller codebases. All the compiler classes needed for the AOT compilations have been split into several partial classes. With these updates, the code is much easier, neater and cleaner.
The reasons and the motivations behind this are the following.
In addition to this, with the new compiler, the Anonymous Type Param Argument is now being cached and compiled ahead-of-time (AOT), thus making this library more performant and much more memory-efficient when calling the underlying Execute methods (i.e.: ExecuteQuery, ExecuteNonQuery, ExecuteScalar, ExecuteReader and ExecuteQueryMultiple). See below.
connection.ExecuteQuery<Person>("SELECT * FROM [Person] WHERE Id = @Id;",
new { Id = 10045});
Not to mention, with the new enhanced compiler, the library even becomes faster and more efficient than it was before.
The feature ClassHandler has been introduced as part of the package. It allows you to handle the event during the serialization/deserialization process of the class model and the DbDataReader object (inbound/outbound).
First, implement a class handler by simply implementing the IClassHandler interface.
public class PersonClassHandler : IClassHandler<Person>
{
public Person Get(Person entity,
DbDataReader reader)
{
return entity;
}
public Person Set(Person entity)
{
return entity;
}
}
Then, map the class handler into your model.
[ClassHandler(typeof(PersonClassHandler))]
public class Person
{
...
}
You can also use the FluentMapper class if you wish to have attribute-free models.
FluentMapper
.Entity<Person>()
.ClassHandler<PersonClassHandler>();
Or, the ClassHandlerMapper.
ClassHandlerMapper
.Add<Person, PersonClassHandler>(true);
After the mappings, when you call any of the push operations (i.e.: Insert, Merge and Update), the Set() method of the property handler will be invoked, thus allowing you to intercept the information prior to the actual execution against the database.
On the other hand, when you call any of the push operations (i.e.: Query, QueryAll, BatchQuery and even the raw-SQL based ExecuteQuery), the Get() method of the property handler will be invoked for you to verify the extracted information prior to returning it to the caller.
In the previous versions, unhandled exception messages are being thrown if the immutable classes are used as the models. However, in order to strengthen the support for the F# programming language, support for immutable classes has been introduced.
public class Person
{
public Person(int id,
string name)
{
Id = id;
Name = name;
}
public int Id { get; }
public string Name { get; }
}
The name of the constructor arguments must be identical to the name of the properties. Only the matching arguments and properties will be processed during the hydration process. Therefore, to be safe, ensure that the names of the target arguments are equal to the DbDataReader fields, no matter what.
It is also important to take note that you can even combine the constructor arguments together with the public properties within the model. See below.
public class Person
{
public Person(int id,
string name)
{
Id = id;
Name = name;
}
public int Id { get; }
public string Name { get; }
public string Addres { get; set; } // Writable
public bool IsActive { get; set; } // Writable
}
Historically, the 2nd Layer cache capability was not introduced in the ExecuteQuery operation for the purpose of avoiding the collisions. However, since this release has supported the table-based calls for the Entity Model, it has also validated this feature; therefore, starting from this release, the 2nd Layer cache will be supported in this method.
Let us say you have a cache factory named MemoryCacheFactory that creates a single instance of ICache object. Then, the code below is now valid.
var memoryCache = MemoryCacheFactory.Create();
using (var connection = new SqlConnection(connectionString))
{
var products = connection.ExecuteQuery<Product>("SELECT * FROM Product;",
cacheKey: "AllProducts", cache: memoryCache);
}
The first call will query the database; the 2nd call will query the data from the supplied cache object. By default, the cache item expires within 180 minutes, but such settings can be changed by simply passing the value in the cacheItemExpiration argument during the calls.
var products = connection.ExecuteQuery<Product>("SELECT * FROM Product;",
cacheKey: "AllProducts",
cache: memoryCache,
cacheItemExpiration: 300 /* 5 hours */);
If you are working with a repository (i.e.: BaseRepository and DbRepository), you do not need to pass the instance of the ICache object. It is handled by the repository internally through the MemoryCache object.
using (var repository = new DbRepository(connectionString))
{
var products = repository.ExecuteQuery<Product>("SELECT * FROM Product;",
cacheKey: "AllProducts");
}
Fluent Calls Collisions: For as long as you are pointing to the same ICache object, you can collide the information with the fluent calls.
The call below is valid and collides with the cache item specified in the execute method above.
var memoryCache = MemoryCacheFactory.Create();
using (var connection = new SqlConnection(connectionString))
{
var products = connection.QueryAll<Product>(cacheKey: "AllProducts",
cache: memoryCache);
}
Of course, if the type of the cached items is not equal to the target result type, then an exception will be thrown. See below.
var memoryCache = MemoryCacheFactory.Create();
using (var connection = new SqlConnection(connectionString))
{
var suppliers = connection.QueryAll<Supplier>(cacheKey: "AllProducts",
cache: memoryCache);
}
It is not common for the ORM to support the Anonymous Type Resultset; however, in RepoDB we have supported this. This is also a case for this library to completely support the F# programming language.
connection.QueryAll<{| Id : int64; Name : string |}> "Person"
The code snippet above is for F# programming language, and below is the equivalent C# programming language.
// Create a type def
private void Main(string[] args)
{
var typeDef = new
{
Id = default(long),
Name = default(string)
};
var people = Call(typeDef, "Person");
// Process the 'people' as anonymous
}
private IEnumerable<T> Call<T>(T typeDef,
string table)
{
using (var connection = new SqlConnection(connectionString))
{
return connection.QueryAll<T>(table);)
}
}
Historically, both the ExpandoObject and the IDictionary<string, object> objects are already supported as the query resultset when fetching the data from the database, but only if you are using the table-based calls like below.
var people = connection.QueryAll("Person");
On this release, the calls below will also return the ExpandoObject.
var people = connection.QueryAll<dynamic>("Person");
// Or
var people = connection.QueryAll<ExpandoObject>("Person");
Or, you can even infer as IDictionary<string, object>.
var people = connection.QueryAll<IDictionary<string, object>>("Person");
And below is the equivalent for F# programming language.
let people = connection.QueryAll<IDictionary<string, obj>> "Person";
It is also important that this feature is not limited to the fluent calls. You can also use it in the actual ExecuteQuery operation.
var sql = "SELECT * FROM Person;";
var people = connection.ExecuteQuery<IDictionary<string, object>>(sql);
In the previous version, there is no way for you to reuse the model unless you are working with the dynamics and Anonymous Types. In this release, you can reuse the exact same single model for whatever operations you would like.
Let us say we created a Person model.
public class Person
{
public long Id { get; set; }
public string Name { get; set; }
}
And by doing the code below, it directly queries the Person table.
var people = connection.QueryAll<Person>();
Moving forward, you can use the same model for as long as it defines the characteristics of the records. Below is the code that queries the Supplier and Customer tables using the same Person model.
var customers = connection.QueryAll<Person>("Customer");
var suppliers = connection.QueryAll<Person>("Supplier");
The functionality is also extended to all other operations (i.e.: Insert, Merge, Update, etc).
var person = new Person
{
Name = "John Doe"
};
var customerId = connection.Insert<Person>("Customer", person);
var supplierId = connection.Insert<Person>("Supplier", person);
The importance of this feature is to allow you to eliminate the repetitive models you are creating when designing an application. The ratio of the cases may not be high, but it is quite important in some cases.
This new feature is requested by the community to make the library a completely dynamic ORM. Historically, you can only update specific columns if you are using the dynamics and/or Anonymous Types based calls. See below.
var id = connection.Insert("Customer", new
{
Name = "John Doe",
Address = "New York"
});
The call above inserts a record into the Customer table targeting only the Name and the Address columns. Other columns that are not a part of the Anonymous Type you passed will not be part of the execution.
However, in this release, you can do such a thing even with the actual class models. Simply specify the list of the fields you wish to be covered by the operation in the fields argument.
var customer = new Customer
{
Name = "John Doe",
Address = "New York",
IsActive = true,
CreatedDateUtc = DateTime.UtcNow
};
var fields = Field.Parse<Customer>(e => new
{
e.Name,
e.Address
});
using (var connection = new SqlConnection(connectionString))
{
var id = connection.Insert<Customer>(customer, fields: fields);
}
Among all the properties that the Customer class model has, only the Name and Address columns are being processed.
You can as well do it in literal string like below.
var customer = new Customer
{
Name = "John Doe",
Address = "New York",
IsActive = true,
CreatedDateUtc = DateTime.UtcNow,
UpdatedDateUtc = DateTime.UtcNow
};
using (var connection = new SqlConnection(connectionString))
{
var id = connection.Insert<Customer>(customer,
fields: Field.From("Name", "Address"));
}
The fields argument is now exposed to all the important push and pull operations (i.e.: Query, Merge, Update and all the corresponding batch operations).
This was a very simple yet quite important feature awaited by some of the users of the library. On this release, the Type Result set is now supported.
Historically, if you wish to query a single column from the table, you are required to return it as dynamics or an explicit class model.
Via a dynamic object of type ExpandoObject.
// Returns an IEnumerable<ExpandoObject>
var people = connection.ExecuteQuery("SELECT Name FROM Person;");
Or, with a class model with single property.
public class Person
{
public string Name
}
And do the query like below.
// ExecuteQuery
var people = connection.ExecuteQuery<Person>("SELECT Name FROM Person;");
// Fluent QueryAll
var people = connection.QueryAll<Person>();
Moving forward, you can do it like below.
// String
var names = connection.ExecuteQuery<string>("SELECT Name FROM Person;");
// Long
var ids = connection.ExecuteQuery<long>("SELECT Id FROM Person;");
Without even creating a model, you can directly infer the result to any .NET CLR type.
You can even do it with the enumerations.
public enum Gender
{
Male,
Female
}
And do the query like below.
var sql = "SELECT Gender FROM Person WHERE Name = @Name;";
var gender = connection.ExecuteQuery<Gender>(sql, new { Name = "John Doe" });
Also, as a part of this package release, if you have a table with a single column (which is most likely not happening at all), you can directly query the records like below.
var subjectNames = connection.QueryAll<string>("Subject");
Or just force a fluent call for single field.
var productNames = connection.QueryAll<string>("Product",
fields: Field.From("Name"));
The current release is now fully acknowledging the support for the Table-Valued Parameters (TVP) via User-Defined Types (UDT) from SQL Server.
To do this, simply create a UDT from the database, and then create a stored procedure that accepts that UDT type. You can follow these guidelines from Microsoft.
Let us say, you created a stored procedure named sp_InsertPerson that accepts the UDT named PersonType. The stored procedure is merging the received information into the actual table Person, then returns all the newly inserted records.
To call such thing, simply create a DataTable and set the TableName property equals to the name of the UDT.
var table = new DataTable();
table.TableName = "[dbo].[PersonType]"; // Name of the UDT
// Create the 'table' columns/rows
Then, simply call the stored procedure like below.
using (var connection = new SqlConnection(connectionString))
{
var sql = "EXEC [sp_InsertPerson] @PersonTable = @Table;";
var people = connection.ExecuteQuery<Person>(sql, new { Table = table });
}
Historically, RepoDB’s truncate operation for RepoDb.PostgreSql has not been auto-resetting the identity seed to the beginning. However, in order for the Truncate operation to be identical to the other DB providers, in this package release, we now force the identity seed to reset back to the beginning.
When you call the operation like below.
using (var connection = new NpgsqlConnection(connectionString))
{
connection.Truncate<Person>();
}
The following SQL Statement will be generated.
> TRUNCATE TABLE \"Person\" RESTART IDENTITY ;
Note: This feature update could be considered a breaking change in some cases, specifically when processing the data.
In the previous version, the identity field (and/or Auto-Incremented column) is only being identified if the column is AUTOINCREMENT. See below.
CREATE TABLE IF NOT EXISTS [Person]
(
Id INTEGER PRIMARY KEY AUTOINCREMENT,
Name TEXT,
Age INTEGER,
CreatedDateUtc DATETIME
);
However, referring to the actual Auto-Increment documentation of SQLite, the column INTEGER PRIMARY KEY is also automatically converted to be auto-incremented. Therefore, in this package release, the RepoDb.SqLite is also parsing the INTEGER PRIMARY KEY as an identity field (and/or Auto-Incremented column).
We internally found a major bug in the Merge operation of the RepoDb.SqLite extension library. In the past, it used the INSERT OR REPLACE keyword to merge the changes towards the database. The major problem with this is that the INSERT OR REPLACE does not work for certain columns.
Note: Merge is referring to a functionality of inserting a record in the database if not present, or otherwise updating it.
Let us say you have this class.
public class Person
{
public long Id { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public DateTime CreatedDateUtc { get; set; }
}
And you have the table below.
CREATE TABLE IF NOT EXISTS [Person]
(
Id INTEGER PRIMARY KEY AUTOINCREMENT,
Name TEXT,
Age INTEGER,
CreatedDateUtc DATETIME
);
And you would like to only merge on the Name column with the qualifier Id, like below.
using (var connection = new SQLiteConnection(connectionString))
{
var person = new
{
Id = 10045,
Name = "James Doe"
};
connection.Merge("Person", person);
}
Then, the operation above will not work as expected if the record is already present in the database.
The generated SQL of the above code snippet is below.
> INSERT OR REPLACE INTO [Person] ( Id, Name ) VALUES ( @Id, @Name ); SELECT CAST([Id], INTEGER) AS [Result];
What happened behind the scenes? Even though the other columns (i.e.: Age and CreatedDateUtc columns) are not part of the statement, the INSERT OR REPLACE keyword is setting them to NULL by default.
The solution we introduced for this is to switch the RepoDb.SqLite extension library to use the upsert technique. The upsert keyword stands for Update/Insert.
We implemented it on the client side. Therefore, the logic of identifying the presence of the data will happen within the application (via the Exists operation).
Let us say, you do the call like below.
using (var connection = new SQLiteConnection(connectionString))
{
var person = new
{
Id = 10045,
Name = "James Doe"
};
connection.Merge("Person", person);
}
Then, the library will do the following logic.
if (connection.Exists(person))
{
return connection.Update(person);
}
else
{
return connection.Insert(person);
}
It is important to take note that this change is only for the RepoDb.SqLite extension library, not affecting the other DB providers.
With this update, the MergeAll operation for RepoDb.SqLite has been switched to an atomic operation and is no longer processed via batch operations.
The trigger for this is the SqLiteDbSetting object’s IsUseUpsert property.
public SqLiteDbSetting(bool isExecuteReaderDisposable)
: base()
{
AreTableHintsSupported = false;
AverageableType = typeof(double);
ClosingQuote = "]";
DefaultSchema = null;
IsDirectionSupported = false;
IsExecuteReaderDisposable = isExecuteReaderDisposable;
IsMultiStatementExecutable = true;
IsPreparable = true;
IsUseUpsert = true;
OpeningQuote = "[";
ParameterPrefix = "@";
}
In this section, we will enumerate some of the known breaking changes from the previous releases of this library. It is important for you to spend time reading this.
If you had called the Merge operation with the list of Field objects as the second argument and had not specified the argument name during the calls, then that would trigger a compilation error moving forward (see below).
var people = GetMergeablePeople();
using (var connection = new SqlConnection(connectionString))
{
var qualifiers = Field.Parse<Person>(e => new
{
e.Name,
e.DateOfBirth
});
var mergedRows = connection.Merge<People>(people, qualifiers);
}
Historically, the second argument is the qualifiers; however, in the current release, the fields argument is also placed as the second argument in the overloaded method, therefore invalidating the calls above.
To fix the issue, simply specify the name of the arguments to your call like below.
var qualifiers = Field.Parse<Person>(e => new
{
e.Name,
e.DateOfBirth
});
var mergedRows = connection.Merge<Person>(people, qualifiers: qualifiers);
And if calling for the fields.
var fields = Field.Parse<Person>(e => new
{
e.Id,
e.Name,
e.DateOfBirth,
e.IsActive
});
var mergedRows = connection.Merge<Person>(people, fields: fields);
And if you are using both.
var qualifiers = Field.Parse<Person>(e => new
{
e.Name,
e.DateOfBirth
});
var fields = Field.Parse<Person>(e => new
{
e.Id,
e.Name,
e.DateOfBirth,
e.IsActive
});
var mergedRows = connection.Merge<People>(people, qualifiers: qualifiers, fields: fields);
In the previous version, if you had forced the use of the named argument for whereOrPrimaryKey and/or where when calling any of the push/pull operations, that may have caused a compiler error.
In the current release, we renamed the whereOrPrimaryKey and some of the placements of the where argument into what. The motivation behind this is that what now supports 3 contexts.
Historically, the identity key was not being identified.
So, all the previous calls like below.
using (var connection = new SqlConnection(connectionString))
{
var people = connection.Query<Person>(whereOrPrimaryKey: 10045);
}
This must be refactored to look like the following.
using (var connection = new SqlConnection(connectionString))
{
var people = connection.Query<Person>(what: 10045);
}
Or, simply eliminate the named argument.
using (var connection = new SqlConnection(connectionString))
{
var people = connection.Query<Person>(10045);
}
The update must be done to the other methods that accept the mentioned stale arguments (i.e.: Delete, Update and the others).
The method that is being used to extract the DbDataReader object into a target .NET CLR Type has been completely modified. In the previous version, the method named ToEnumerable with a signature like below is present, together with its corresponding async method.
public IEnumerable<TResult> ToEnumerable<TResult>(DbDataReader reader,
IDbConnection connection = null,
IDbTransaction transaction = null);
However, in the new version, the method signature has been completely modified with the one below.
ToEnumerable<TResult>(DbDataReader reader,
IEnumerable<DbField> dbFields = null,
IDbSetting dbSetting = null)
Also, the async method has been removed as it is not necessary anymore.
The rationale behind this is the following.
The release updates also come with various requests from the community, bug fixes, enhancements and major refactoring. It is important to see the detailed updates from the Releases page and/or by visiting the actual release page for this version at the Github releases page.
We also would like to extend our warm thanks to all the contributors, the collaborators and the issue reporters.
Please do not f
]]>The actual repository can be found at RepoDb.SqlServer.BulkOperations.
BulkDelete (BulkDeleteAsync)BulkInsert (BulkInsertAsync)BulkMerge (BulkMergeAsync)BulkUpdate (BulkUpdateAsync)If you are usually working with large datasets, then this feature is for you. By using this, it will drastically improve the performance of your application, together with increasing your productivity during development.
In addition, it will remove your worries about writing and maintaining the complex and tedious code snippets (implementation complexity) just to achieve your “BASIC” goals with high-performance.
As an author and being an OSS contributor, I am trying to further improve the experience of most .NET Developers when it comes to data accessibility. The features within the library are free and will always be. In return, I only like to hear your thoughts, feedback and experiences about this library. With this, we can help each other to further improve it.
Nothing else, but PERFORMANCE!
Although the same exact goal can be achieved by Batch Operations (i.e.: DeleteAll, InsertAll, MergeAll and UpdateAll), by leveraging the real Bulk Operations functionalities, you are extremely improving the performance of your data manipulation with a huge-percentage gap.
To explain further, when you only use the Batch operations, it only batches the multiple atomic activities and still makes multiple round-trips from your application into the database server. However, when you use the Bulk operations, you are bringing all the data from the application into the database server at once and at the same time ignoring some database specific activities (i.e.: Logging, Audits, Data-Type Checks, Constraints, etc) behind the scene, thus gives you maximum performance during the operation.
It is leveraging the ADO.NET SqlBulkCopy class (both System.Data.SqlClient/Microsoft.Data.SqlClient).
For the BulkInsert operation, it simply calls the WriteToServer method to bring all the data from the application into the database. No additional logic is implied.
For BulkDelete, BulkMerge and BulkUpdate operations, an implied logic and technique has been utilized.
Basically, a pseudo-temporary table will be created in the database under the transaction context. It then re-uses the BulkInsert operation to target that pseudo-temporary table. Through this technique, we brought all the data together from the client application into the database server (at one go).
You as a developer have the opportunity to maximize your own qualifiers by passing them as a list of Field objects. The library will then create a CLUSTERED INDEX on the pseudo-temporary table using your qualifiers.
If you have not passed any qualifiers, the primary key will be used by default. If the primary key is not present, it will use the identity field instead.
Once all the data is in the database pseudo-temporary table, the correct SQL Statement will be used to cascade the changes towards the original table.
See the SQL Statements below.
> DELETE T
> FROM [dbo].[OriginalTable] T
> INNER JOIN [PseudoTempTable] TMP ON TMP.QualiferField1 = T.Field1 AND TMP.QualifierField2 = T.Field2;
> MERGE [dbo].[OriginalTable] T
> USING [PseudoTempTable] S ON S.QualiferField1 = T.Field1 AND S.QualifierField2 = T.Field2
> WHEN NOT MATCHED THEN
> INSERT (...) VALUES (...)
> WHEN MATCHED THEN
> UPDATE
> SET (...);
> UPDATE T
> SET T.Field3 = TMP.Field3
> , T.Field4 = TMP.Field4
> , ...
> FROM [OriginalTable] T
> INNER JOIN [PseudoTempTable] TMP ON TMP.QualiferField1 = T.Field1 AND TMP.QualifierField2 = T.Field2;
The arguments qualifiers and usePhysicalPseudoTempTable are provided for BulkDelete, BulkMerge and BulkUpdate operations.
The argument qualifiers is used to define the qualifier fields to be used in the operation. It usually refers to the WHERE expression of SQL Statements. If not given, the primary or identity column will be used.
The argument usePhysicalPseudoTempTable is used to define whether a physical pseudo-table will be created during the operation. By default, a temporary table (i.e.: #TableName) is used.
RepoDB is automatically setting the value of the options argument to SqlBulkCopyOptions.KeepIdentity when calling the BulkDelete, BulkMerge and BulkUpdate if you have not passed any qualifiers and if your table has an IDENTITY primary key column. The same logic will apply if there is no primary key but has an IDENTITY column defined in the table.
In addition, when calling the BulkDelete, BulkMerge and BulkUpdate operations, the library is creating a pseudo temporary table behind the scene. It requires your user to have the correct privilege to CREATE a table in the database, otherwise a SqlException will be thrown.
To start with, simply install the library. At the Package Manager Console, type the command below.
> Install-Package RepoDb.SqlServer.BulkOperations
You should install the version 1.0.4 (or above). Prior versions only support BulkInsert.
Then, call the bootstrapper once.
SqlServerBootstrap.Initialize();
After that, you can then start calling the Bulk Operations. See the sample code snippets below.
The code snippets below only showcase the BulkDelete via System.Collections.Generic.IEnumerable<T>.
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkDelete<Customer>(customers);
}
Or with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkDelete<Customer>(customers, qualifiers: qualifiers);
}
Or with primary keys
using (var connection = new SqlConnection(connectionString))
{
var primaryKeys = new [] { 10045, ..., 11011 };
var rows = connection.BulkDelete<Customer>(primaryKeys);
}
Or via table-name
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkDelete("Customer", customers);
}
Or via table-name with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkDelete("Customer", customers, qualifiers: qualifiers);
}
The code snippets below only showcase the BulkInsert via System.Collections.Generic.IEnumerable<T>.
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkInsert<Customer>(customers);
}
Or via table-name
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkInsert("Customer", customers);
}
The code snippets below only showcase the BulkMerge via System.Collections.Generic.IEnumerable<T>.
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkMerge<Customer>(customers);
}
Or with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkMerge<Customer>(customers, qualifiers: qualifiers);
}
Or via table-name
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkMerge("Customer", customers);
}
Or via table-name with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkMerge("Customer", customers, qualifiers: qualifiers);
}
The code snippets below only showcase the BulkUpdate via System.Collections.Generic.IEnumerable<T>.
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkUpdate<Customer>(customers);
}
Or with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkUpdate<Customer>(customers, qualifiers: qualifiers);
}
Or via table-name
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var rows = connection.BulkUpdate("Customer", customers);
}
Or via table-name with qualifiers
using (var connection = new SqlConnection(connectionString))
{
var customers = GetCustomers();
var qualifiers = Field.From("LastName", "BirthDate");
var rows = connection.BulkUpdate("Customer", customers, qualifiers: qualifiers);
}
As mentioned in the earlier section, both the
System.Data.Common.DbDataReaderandSystem.DataTableare supported.
Here, we can only advise based on our experience. To be honest, there is no standard of when to use what. It all depends on your situation (i.e.: Network Latency, Data, No. of Columns, etc).
The PROS of using Bulk are maximum performance. However, there are also CONS of usually using it. One of them is it keeps blocking the target table while under the Bulk operations transaction. It can however trigger (or also solve) a deadlock if not handled properly by the developers.
On our end, we usually use the Batch operations if the rows we are manipulating are less than or equal to 1000. Beyond that, we standardize on using the Bulk operations.
By using the RepoDb.SqlServer.BulkOperations features, you will have the following.
The original post was posted on DEV.to by the author.
Thank you for reading this article.
]]>“I am an open source contributor and I am here to share with you what I had done. I worked hard for it to improve the space of data access in .NET. I personally ask your support towards this library. I hope you share it, blog about it and use it.”
All the content of this tutorial is written by me (the author himself). Our knowledge of Dapper is not that deep when compared to our knowledge of RepoDb. So, please allow yourselves to check or comment right away if you think we made this page biased towards RepoDb.
The programming language and database provider we are using on our samples below are C# and SQL Server.
Both libraries are ORM frameworks for .NET. They are both lightweight, fast and efficient. Dapper is a full-fledged micro-ORM whereas RepoDB is a hybrid-ORM.
To avoid the bias on the comparison, we will not cover the features that are present in RepoDB but are absent in Dapper (i.e.:
Cache,Trace,Hints,Extensibility,StatementBuilderandRepositories) (vice-versa). Also, the comparison does not include any other extension libraries of both (i.e.:RepoDb.SqLite,RepoDb.MySql,RepoDb.PostgreSql,Dapper.Contrib,DapperExtensions,Dapper.SqlBuilder, etc).
Let us assume we have the following database tables.
CREATE TABLE [dbo].[Customer]
(
[Id] BIGINT IDENTITY(1,1)
, [Name] NVARCHAR(128) NOT NULL
, [Address] NVARCHAR(MAX)
, CONSTRAINT [PK_Customer] PRIMARY KEY CLUSTERED ([Id] ASC )
)
ON [PRIMARY];
GO
CREATE TABLE [dbo].[Product]
(
[Id] BIGINT IDENTITY(1,1)
, [Name] NVARCHAR(128) NOT NULL
, [Price] Decimal(18,2)
, CONSTRAINT [PK_Product] PRIMARY KEY CLUSTERED ([Id] ASC )
)
ON [PRIMARY];
GO
CREATE TABLE [dbo].[Order]
(
[Id] BIGINT IDENTITY(1,1)
, [ProductId] BIGINT NOT NULL
, [CustomerId] BIGINT
, [OrderDateUtc] DATETIME(5)
, [Quantity] INT
, CONSTRAINT [PK_Order] PRIMARY KEY CLUSTERED ([Id] ASC )
)
ON [PRIMARY];
GO
Let us assume we have the following class models.
public class Customer
{
public long Id { get; set; }
public string Name { get; set; }
public string Address { get; set; }
}
public class Product
{
public long Id { get; set; }
public string Name { get; set; }
public decimal Price { get; set; }
}
public class Order
{
public long Id { get; set; }
public long ProductId { get; set; }
public long CustomerId { get; set; }
public int Quantity { get; set; }
public DateTime OrderDateUtc{ get; set; }
}
Query:
using (var connection = new SqlConnection(connectionString))
{
var customers = connection.Query<Customer>("SELECT * FROM [dbo].[Customer];");
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var customers = connection.ExecuteQuery<Customer>("SELECT * FROM [dbo].[Customer];");
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customers = connection.QueryAll<Customer>();
}
Query:
using (var connection = new SqlConnection(connectionString))
{
var customer = connection.Query<Customer>("SELECT * FROM [dbo].[Customer] WHERE (Id = @Id);", new { Id = 10045 }).FirstOrDefault();
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var customer = connection.ExecuteQuery<Customer>("SELECT * FROM [dbo].[Customer] WHERE (Id = @Id);", new { Id = 10045 }).FirstOrDefault();
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customer = connection.Query<Customer>(e => e.Id == 10045).FirstOrDefault();
}
Execute:
By default, it returns the number of affected rows.
using (var connection = new SqlConnection(connectionString))
{
var customer = new Customer
{
Name = "John Doe",
Address = "New York"
};
var affectedRows = connection.Execute("INSERT INTO [dbo].[Customer] (Name, Address) VALUES (@Name, @Address);", customer);
}
Query:
Returning the identity value.
using (var connection = new SqlConnection(connectionString))
{
var customer = new Customer
{
Name = "John Doe",
Address = "New York"
};
var id = connection.Query<long>("INSERT INTO [dbo].[Customer] (Name, Address) VALUES (@Name, @Address); SELECT CONVERT(BIGINT, SCOPE_IDENTITY());", customer).Single();
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var customer = new Customer
{
Name = "John Doe",
Address = "New York"
};
var id = connection.ExecuteScalar<long>("INSERT INTO [dbo].[Customer] (Name, Address) VALUES (@Name, @Address); SELECT CONVERT(BIGINT, SCOPE_IDENTITY());", customer);
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customer = new Customer
{
Name = "John Doe",
Address = "New York"
};
var id = (long)connection.Insert<Customer>(customer); // or connection.Insert<Customer, long>(customer);
}
Execute:
using (var connection = new SqlConnection(connectionString))
{
var affectedRows = connection.Execute("UPDATE [dbo].[Customer] SET Name = @Name, Address = @Address WHERE Id = @Id;",
new
{
Id = 10045,
Name = "John Doe",
Address = "New York"
});
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var affectedRows = connection.ExecuteScalar<int>("UPDATE [dbo].[Customer] SET Name = @Name, Address = @Address WHERE Id = @Id;",
new
{
Id = 10045,
Name = "John Doe",
Address = "New York"
});
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customer = new Customer
{
Id = 10045,
Name = "John Doe",
Address = "New York"
};
var affectedRows = connection.Update<Customer>(customer);
}
Execute:
using (var connection = new SqlConnection(connectionString))
{
var affectedRows = connection.Execute("DELETE FROM [dbo].[Customer] WHERE Id = @Id;", new { Id = 10045 });
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var affectedRows = connection.ExecuteScalar<int>("DELETE FROM [dbo].[Customer] WHERE Id = @Id;", new { Id = 10045 });
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var affectedRows = connection.Delete<Customer>(10045);
}
Let us assume we have added the Orders (of type IEnumerable<Order>) property on our Customer class.
Customer:
public class Customer
{
public long Id { get; set; }
public string Name { get; set; }
public string Address { get; set; }
public IEnumerable<Order> Orders { get; set; }
}
Order:
public class Order
{
public long Id { get; set; }
public long ProductId { get; set; }
public long CustomerId { get; set; }
public int Quantity { get; set; }
public DateTime OrderDateUtc{ get; set; }
}
Query:
using (var connection = new SqlConnection(connectionString))
{
var sql = "SELECT C.Id, C.Name, C.Address, O.ProductId, O.Quantity, O.OrderDateUtc FROM [dbo].[Customer] C INNER JOIN [dbo].[Order] O ON O.CustomerId = C.Id WHERE C.Id = @Id;";
var customers = connection.Query<Customer, Order, Customer>(sql,
(customer, order) =>
{
customer.Orders = customer.Orders ?? new List<Order>();
customer.Orders.Add(order);
return customer;
},
new { Id = 10045 });
}
QueryMultiple:
using (var connection = new SqlConnection(connectionString))
{
var sql = "SELECT * FROM [dbo].[Customer] WHERE Id = @CustomerId; SELECT * FROM [dbo].[Order] WHERE CustomerId = @CustomerId;";
using (var result = connection.QueryMultiple(sql, new { CustomerId = 10045 }))
{
var customer = result.Read<Customer>().First();
var orders = result.Read<Order>().ToList();
}
}
The JOIN feature is not being supported yet. However, there is an alternative way to do this in RepoDb. It can be done via multi-query that executes packed SELECT-statements in a single call.
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var sql = "SELECT * FROM [dbo].[Customer] WHERE Id = @CustomerId; SELECT * FROM [dbo].[Order] WHERE CustomerId = @CustomerId;";
var extractor = connection.ExecuteQueryMultiple(sql, new { CustomerId = 10045 });
var customer = extractor.Extract<Customer>().FirstOrDefault();
var orders = extractor.Extract<Order>().AsList();
customer.Orders = orders;
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customerId = 10045;
var tuple = connection.QueryMultiple<Customer, Order>(customer => customer.Id == customerId, order => order.CustomerId == customerId);
var customer = tuple.Item1.FirstOrDefault();
var orders = tuple.Item2.AsList();
customer.Orders = orders;
}
Query:
var customers = new List<Customer>();
using (var connection = new SqlConnection(connectionString))
{
var sql = "SELECT C.Id, C.Name, C.Address, O.ProductId, O.Quantity, O.OrderDateUtc FROM [dbo].[Customer] C INNER JOIN [dbo].[Order] O ON O.CustomerId = C.Id;";
var customers = connection.Query<Customer, Order, Customer>(sql,
(customer, order) =>
{
customer = customers.Where(e => e.Id == customer.Id).FirstOrDefault() ?? customer;
customer.Orders = customer.Orders ?? new List<Order>();
customer.Orders.Add(order);
return customer;
});
}
QueryMultiple:
using (var connection = new SqlConnection(connectionString))
{
var sql = "SELECT * FROM [dbo].[Customer]; SELECT * FROM [dbo].[Order];";
using (var result = connection.QueryMultiple(sql, new { CustomerId = 10045 }))
{
var customers = result.Read<Customer>().ToList();
var orders = result.Read<Order>().ToList();
customers.ForEach(customer =>
customer.Orders = orders.Where(o => o.CustomerId == customer.Id).ToList()); // Client memory processing
}
}
Raw-SQL:
using (var connection = new SqlConnection(connectionString))
{
var extractor = connection.ExecuteQueryMultiple("SELECT * FROM [dbo].[Customer]; SELECT * FROM [dbo].[Order];");
var customers = extractor.Extract<Customer>().AsList();
var orders = extractor.Extract<Order>().AsList();
customers.ForEach(customer =>
customer.Orders = orders.Where(o => o.CustomerId == customer.Id).AsList()); // Client memory processing
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customerId = 10045;
var tuple = connection.QueryMultiple<Customer, Order>(customer => customer.Id == customerId, order => order.CustomerId == customerId);
var customers = tuple.Item1.FirstOrDefault();
var orders = tuple.Item2.AsList();
customers.ForEach(customer =>
customer.Orders = orders.Where(o => o.CustomerId == customer.Id).AsList()); // Client memory processing
}
Query:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var identities = connection.Query<long>("INSERT INTO [dbo].[Customer] (Name, Address) VALUES (@Name, @Address); SELECT CONVERT(BIGINT, SCOPE_IDENTITY());", customers);
}
Actually, this is not clear to me:
DbCommand.Execute<Method> multiple times?Please correct me here so I can update this page right away.
Batch operation:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.InsertAll<Customer>(customers);
}
The above operation can be batched by passing a value on the batchSize argument.
Note: You can target a specific column. In addition, the identity values are automatically set back to the entities.
Bulk operation:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.BulkInsert<Customer>(customers);
}
The above operation can be batched by passing a value on the batchSize argument.
Note: This is just an FYI. The operation is using the SqlBulkCopy of ADO.Net. This should not be compared to Dapper’s performance due to the fact that this is a real bulk-operation. This is far faster (extremely fast) when compared to both Dapper (multi-inserts) and RepoDB (InsertAll) operations.
Query:
using (var connection = new SqlConnection(connectionString))
{
var sql = @"MERGE [dbo].[Customer] AS T
USING
(SELECT @Name, @Address) AS S
ON
S.Id = T.Id
WHEN NOT MATCH THEN
INSERT INTO
(
Name
, Address
)
VALUES
(
S.Name
, S.
Address)
WHEN MATCHED THEN
UPDATE
SET Name = S.Name
, Address = S.Address
OUTPUT INSERTED.Id AS Result;";
var customers = GenerateCustomers(1000);
var identities = connection.Query<long>(sql, customers);
}
Here, I have the same question as the previous section.
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.MergeAll<Customer>(customers);
}
The above operation can be batched by passing a value on the batchSize argument.
Note: You can set the qualifier fields. In addition, the identity values are automatically set back to the entities for the newly inserted records.
Query:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.Execute("UPDATE [dbo].[Customer] SET Name = @Name, Address = @Address WHERE Id = @Id;", customers);
}
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.UpdateAll<Customer>(customers);
}
The above operation can be batched by passing a value on the batchSize argument.
Note: You can set the qualifier fields.
ADO.NET:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var table = ConvertToTable(customers);
using (var sqlBulkCopy = new SqlBulkCopy(connection, options, transaction))
{
sqlBulkCopy.DestinationTableName = "Customer";
sqlBulkCopy.WriteToServer(table);
}
}
Note: You can as well pass an instance of DbDataReader (instead of DataTable).
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.BulkInsert<Customer>(customers);
}
Note: You can as well pass an instance of DbDataReader.
using (var connection = new SqlConnection(connectionString))
{
var customers = GenerateCustomers(1000);
var affectedRows = connection.BulkInsert("[dbo].[Customer]", customers);
}
Query:
using (var connection = new SqlConnection(connectionString))
{
var sql = @"WITH CTE AS
(
SELECT TOP (@Rows) ROW_NUMBER() OVER(ORDER BY Name ASC) AS RowNumber
FROM [dbo].[Customer]
WHERE (Address = @Address)
)
SELECT Id
, Name
, Address
FROM
CTE
WHERE
RowNumber BETWEEN @From AND (@From + @Rows);";
using (var connection = new SqlConnection(connectionString))
{
var customers = connection.Query<Customer>(sql, new { From = 0, Rows = 100, Address = "New York" });
}
}
Note: You can as well execute it via (LIMIT) keyword. It is on your preference.
Fluent:
using (var connection = new SqlConnection(connectionString))
{
var customers = connection.BatchQuery<Customer>(e => e.Address == "New York",
page: 0,
rowsPerBatch: 100,
orderBy: OrderField.Parse(new { Name = Order.Ascending }));
}
Query:
using (var sourceConnection = new SqlConnection(SourceConnectionString))
{
var customers = sourceConnection.Query<Customer>("SELECT * FROM [dbo].[Customer];");
using (var destinationConnection = new SqlConnection(DestinationConnectionString))
{
var identities = destinationConnection.Query<long>("INSERT INTO [dbo].[Customer] (Name, Address) VALUES (@Name, @Address); SELECT CONVERT(BIGINT, SCOPE_IDENTITY());", customers);
}
}
Fluent (InsertAll):
using (var sourceConnection = new SqlConnection(SourceConnectionString))
{
var customers = sourceConnection.QueryAll<Customer>();
using (var destinationConnection = new SqlConnection(DestinationConnectionString))
{
var affectedRows = destinationConnection.InsertAll<Customer>(customers);
}
}
Fluent (BulkInsert):
using (var sourceConnection = new SqlConnection(SourceConnectionString))
{
var customers = sourceConnection.QueryAll<Customer>();
using (var destinationConnection = new SqlConnection(DestinationConnectionString))
{
var affectedRows = destinationConnection.BulkInsert<Customer>(customers);
}
}
Fluent (Streaming):
This is the most optimal and recommended call for large datasets. We do not bring the data as class objects in the client application.
using (var sourceConnection = new SqlConnection(SourceConnectionString))
{
using (var reader = sourceConnection.ExecuteReader("SELECT * FROM [dbo].[Customer];"))
{
using (var destinationConnection = new SqlConnection(DestinationConnectionString))
{
var affectedRows = destinationConnection.BulkInsert<Customer>(reader);
}
}
}
Note: Check for collation constraints. It is an ADO.NET thing.
Dynamic:
Query<T>(sql, new { Id = 10045 });
It is always an Equal operation. You control the query through SQL Statement.
Dynamic Parameters:
var parameters = new DynamicParameters();
parameters.Add("Name", "John Doe");
parameters.Add("Address", "New York");
Query<T>(sql, parameters);
Dynamic:
Query<T>(new { Id = 10045 });
Same as Dapper, it is always referring to an Equal operation. You control the query through SQL Statement.
Linq Expression:
Query<T>(e => e.Id == 10045);
QueryField:
Query<T>(new QueryField("Id", 10045));
QueryField(s) or QueryGroup:
var queryFields = new[]
{
new QueryField("Name", "John Doe")
new QueryField("Address", "New York")
};
Query<T>(queryFields); // or Query<T>(new QueryGroup(queryFields));
Query:
using (var connection = new SqlConnection(connectionString))
{
var addresses = new [] { "New York", "Washington" };
var customers = connection.Query<Customer>("SELECT * FROM [dbo].[Customer] WHERE Address IN (@Addresses);", new { Addresses = addresses });
}
ExecuteQuery:
using (var connection = new SqlConnection(connectionString))
{
var addresses = new [] { "New York", "Washington" };
var customers = connection.ExecuteQuery<Customer>("SELECT * FROM [dbo].[Customer] WHERE Address IN (@Addresses);", new { Addresses = addresses });
}
For further explanation, you can visit our documentation.
Query:
using (var connection = new SqlConnection(connectionString))
{
var addresses = new [] { "New York", "Washington" };
var customers = connection.Query<Customer>(e => addresses.Contains(e => e.Address));
}
Linq Expressions, only Dynamics and DynamicParameters.Linq Expressions, Dynamics and QueryObjects.Note: The DynamicParameters is just a subset of QueryObjects. The QueryObjects has much more capability that can further support the Linq Expressions.
Please visit both documentation pages.
Supports all RDBMS data providers.
We only refer to one of the community-approved ORM bencher, the RawDataAccessBencher.
Net Core:
Here is our observation from the official execution results. The official result can be found here.
Performance:
Efficiency:
NetFramework:
RepoDB is the fastest and the most-efficient ORM for fetching both single and set records. The official results can be found here.
Dapper has already been running since 2012 and is being used by StackOverflow.com. It has a huge consumer base and is hugely backed by the community.
We did our best to write one-test per scenario and we have delivered thousands of items (approximately 8K) for both Unit and Integration tests. We would like your help to review them as well.
Below are the links to our test suites.
We (or I as an author) have been challenged that the quality of the software does not depend on the number of tests. However, we strongly believe that spending much effort on writing a test will give confidence to the library consumers (i.e.: .NET community). Practically, it helps us to avoid manual revisits on the already-working features if somebody is doing a PR to us; it prevents the library from any surprising bugs.
Both are of high quality but Dapper is far more mature than RepoDb. We will not contest this!
Proven and is backed hugely by the .NET Community; funded by StackOverflow.com.
Backed by one person and is not funded nor sponsored by any entity. Just starting to expand and asking for more support from the .NET Community.
Both are under the Apache-2.0 license.
We are not experts in legal matters but we are consulting. If any conflict arises regarding the copyright or trademark in front of RepoDB, then that is not yet addressed.
We hope that you somehow consider and revisit this library. It has improved a lot from where it has started.
Dapper is lightweight but will drag you to the most complex level of code development. It is always tedious to write raw-SQLs and it is hard to maintain due to the fact that it is not compiler-friendly. In addition, to accomplish a necessary task, you need to implement necessary features.
RepoDB is a very easy-to-use ORM with enough feature sets that you can play on.
RepoDB is faster than Dapper, enough reason to choose this library if the only factor is performance.
RepoDB is the fastest ORM in .NET. This claim is supported by the official run of the community-approved ORM bencher RawDataAccessBencher.
RepoDB is more efficient than Dapper (same claim as in Performance).
It is easier and faster to develop the code snippets with RepoDb. It has a rich feature set which can be used right away (i.e.: 2nd-Layer Cache, Fluent Methods). It will help you as a developer deliver more code snippets in a fast and clean manner.
In RepoDB, having the necessary features within the space of a micro-ORM will help you a lot in your development.
Features like Bulk & Batch operations, PropertyHandlers, 2nd-Level Cache, Expression Trees, Multi Queries and inline Hints are the most commonly used features. The major pain point is that it is absent in Dapper.
The original post was posted on DEV.to by the author.
]]>I started writing the first code last March 15, 2018. The first official class that has been written is BaseRepository. I developed the initial compiler via IL and found it extremely fast. By that time, I was obsessed with beating Dapper ORM when it comes to performance. I first wrote the following methods (ExecuteQuery, ExecuteNonQuery, ExecuteScalar and ExecuteReader). Then, I added some basic methods like (Insert, Delete, Update and Query). The implementation is very shallow and NOT-tested.
The IL code can be found here, but it is not being used by the library now. We are now using the
Linq-Basedcompiler (can be found here).
The next day, I came to the office early and checked in the RepoDB code snippets in our internal repository. A few minutes after, I broadcasted an email to all the developers of Orsted A/S at Kuala Lumpur, Malaysia without knowing the effect of it.
I received much criticism from various architects the same day (i.e.: re-inventing the wheel, too ambitious, etc).
However, I got good feedback from my direct Team Lead.
As a leading developer of the team by that time, we needed to deliver a solution that will calculate the power production in a real-time manner. I initially developed the solution by reusing the repository-based implementation of our existing project (using the Dapper ORM). I found it hard to develop a very dynamic repository like what BaseRepository and DbRepository can do now. I also encountered a lot of problems when using the Dynamics and ExpandoObject. I gave up when I needed to read data from a View with a different ColumnName as the primary key.
TBH, I am also very good at ADO.NET by mastering it since the start of my career 13 years ago (from today). I also have experience building CRM solutions that have a feature called Selection-Wizard which allows the user to pre-select the columns they would like to query from the complex relationships of the CRM, and provide reports on top of it. I know that I am capable of creating an ORM and with great motivation for new things to play with.
I first posted RepoDB on Reddit 2 years ago. The first post was titled RepoDB - a very fast lightweight ORM and has the perfect spices of Micro-ORM. Faster than Dapper! Rooting for the experts about this., and the post can be found here.
A lot of people joined the discussion and the thread topped the r/csharp in just a few minutes. I saw a lot of up/down votes and some frustrations from the community. Honestly, I do not know how to diligently answer the questions from the community, could be the reason for the frustrations.
I remember one comment that brought me to RawDataAccessBencher ORM bencher of FransBouma. He wanted me to test the library there. The result was that RepoDB was the second slowest ORM and I was totally shocked, actually. I know that the IL I developed is very fast, but by that time, it was only very fast for extracting millions of rows (in a single iteration) (maybe the fastest), but is very slow if you bench it with its execution mean-deviation (includes multiple iterations by batch).
I analyzed the culprits in the code snippets and also tried to optimize the IL. I solved the problem and brought RepoDB as the fastest ORM a month after. The official technique was posted by me on my old blog (can be found here).
It was also 2 years ago (4 months after pushing the first version) that RepoDB was officially accepted by FransBouma to be a part of the RawDataAccessBencher ORM bencher. That is amazing and gave me more spirit to further develop the lib
]]>The docs page has been introduced with the following sections.
The links have been provided but there is no content yet on the pages, but the initial version of our about page has also been introduced.
Lastly, the design of the site has already been put in place (header, footer, footer-links an
]]>