Skip to content

Commit

Permalink
Merge pull request #3 from PandaTechAM/development
Browse files Browse the repository at this point in the history
Random Incrementing Sequence Generation Feature
  • Loading branch information
HaikAsatryan authored Jul 17, 2024
2 parents ecaf089 + 01ba014 commit 58e33c1
Show file tree
Hide file tree
Showing 12 changed files with 505 additions and 177 deletions.
106 changes: 78 additions & 28 deletions Readme.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,34 @@
- [1. Pandatech.EFCore.PostgresExtensions](#1-pandatechefcorepostgresextensions)
- [1.1. Features](#11-features)
- [1.2. Installation](#12-installation)
- [1.3. Usage](#13-usage)
- [1.3.1. Row-Level Locking](#131-row-level-locking)
- [1.3.2. Npgsql COPY Integration](#132-npgsql-copy-integration)
- [1.3.2.1. Benchmarks](#1321-benchmarks)
- [1.3.2.1.1. General Benchmark Results](#13211-general-benchmark-results)
- [1.3.2.1.2. Detailed Benchmark Results](#13212-detailed-benchmark-results)
- [1.3.2.1.3. Efficiency Comparison](#13213-efficiency-comparison)
- [1.3.2.1.4. Additional Notes](#13214-additional-notes)
- [1.4. License](#14-license)

# 1. Pandatech.EFCore.PostgresExtensions
# Pandatech.EFCore.PostgresExtensions

Pandatech.EFCore.PostgresExtensions is an advanced NuGet package designed to enhance PostgreSQL functionalities within
Entity Framework Core, leveraging specific features not covered by the official Npgsql.EntityFrameworkCore.PostgreSQL
package. This package introduces optimized row-level locking mechanisms and an efficient, typed version of the
PostgreSQL COPY operation, adhering to EF Core syntax for seamless integration into your projects.
package. This package introduces optimized row-level locking mechanisms and PostgreSQL sequence random incrementing
features.

## 1.1. Features
## Features

1. **Row-Level Locking**: Implements the PostgreSQL `FOR UPDATE` feature, providing three lock
behaviors - `Wait`, `Skip`, and
`NoWait`, to facilitate advanced transaction control and concurrency management.
2. **Npgsql COPY Integration**: Offers a high-performance, typed interface for the PostgreSQL COPY command, allowing for
2. **Npgsql COPY Integration (Obsolete)**: Offers a high-performance, typed interface for the PostgreSQL COPY command,
allowing for
bulk data operations within the EF Core framework. This feature significantly enhances data insertion speeds and
efficiency.
3. **Random Incrementing Sequence Generation:** Provides a secure way to generate sequential IDs with random increments
to prevent predictability and potential data exposure. This ensures IDs are non-sequential and non-predictable,
enhancing security and balancing database load.

## 1.2. Installation
## Installation

To install Pandatech.EFCore.PostgresExtensions, use the following NuGet command:

```bash
Install-Package Pandatech.EFCore.PostgresExtensions
```

## 1.3. Usage
## Usage

### 1.3.1. Row-Level Locking
### Row-Level Locking

Configure your DbContext to use Npgsql and enable query locks:

Expand Down Expand Up @@ -71,7 +62,66 @@ catch (Exception ex)
}
```

### 1.3.2. Npgsql COPY Integration
### Random Incrementing Sequence Generation

To configure a model to use the random ID sequence, use the `HasRandomIdSequence` extension method in your entity
configuration:

```csharp
public class Animal
{
public long Id { get; set; }
public string Name { get; set; }
}

public class AnimalEntityConfiguration : IEntityTypeConfiguration<Animal>
{
public void Configure(EntityTypeBuilder<Animal> builder)
{
builder.HasKey(x => x.Id);
builder.Property(x => x.Id)
.HasRandomIdSequence();
}
}
```

After creating a migration, add the custom function **above create table** script in your migration class:

```csharp
public partial class PgFunction : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateRandomIdSequence("animal", "id", 5, 5, 10); //Add this line manually
migrationBuilder.CreateTable(
name: "animal",
columns: table => new
{
id = table.Column<long>(type: "bigint", nullable: false, defaultValueSql: "animal_random_id_generator()"),
name = table.Column<string>(type: "text", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("pk_animal", x => x.id);
});
}

/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "animal");
}
}
```
#### Additional notes
- The random incrementing sequence feature ensures the generated IDs are unique, non-sequential, and non-predictable, enhancing security.
- The feature supports only `long` data type (`bigint` in PostgreSQL).


### Npgsql COPY Integration (Obsolete: Use EFCore.BulkExtensions.PostgreSql instead)

For bulk data operations, use the `BulkInsert` or `BulkInsertAsync` extension methods:

Expand All @@ -89,42 +139,42 @@ public async Task BulkInsertExampleAsync()
}
```

#### 1.3.2.1. Benchmarks
#### Benchmarks

The integration of the Npgsql COPY command showcases significant performance improvements compared to traditional EF
Core and Dapper methods:

##### 1.3.2.1.1. General Benchmark Results
##### General Benchmark Results

| Caption | Big O Notation | 1M Rows | Batch Size |
|------------|----------------|-------------|------------|
| BulkInsert | O(log n) | 350.000 r/s | No batch |
| Dapper | O(n) | 20.000 r/s | 1500 |
| EFCore | O(n) | 10.600 r/s | 1500 |

##### 1.3.2.1.2. Detailed Benchmark Results
##### Detailed Benchmark Results

| Operation | BulkInsert | Dapper | EF Core |
|-------------|------------|--------|---------|
| Insert 10K | 76ms | 535ms | 884ms |
| Insert 100K | 405ms | 5.47s | 8.58s |
| Insert 1M | 2.87s | 55.85s | 94.57s |

##### 1.3.2.1.3. Efficiency Comparison
##### Efficiency Comparison

| RowsCount | BulkInsert Efficiency | Dapper Efficiency |
|-----------|----------------------------|---------------------------|
| 10K | 11.63x faster than EF Core | 1.65x faster than EF Core |
| 100K | 21.17x faster than EF Core | 1.57x faster than EF Core |
| 1M | 32.95x faster than EF Core | 1.69x faster than EF Core |

##### 1.3.2.1.4. Additional Notes
##### Additional Notes

- The `BulkInsert` feature currently does not support entity properties intended for `JSON` storage.

- The performance metrics provided above are based on benchmarks conducted under controlled conditions. Real-world
performance may vary based on specific use cases and configurations.

## 1.4. License
## License

Pandatech.EFCore.PostgresExtensions is licensed under the MIT License.
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@
<PackageReadmeFile>Readme.md</PackageReadmeFile>
<Authors>Pandatech</Authors>
<Copyright>MIT</Copyright>
<Version>2.0.1</Version>
<Version>3.0.0</Version>
<PackageId>Pandatech.EFCore.PostgresExtensions</PackageId>
<Title>Pandatech.EFCore.PostgresExtensions</Title>
<PackageTags>Pandatech, library, EntityFrameworkCore, PostgreSQL, For Update, Lock, LockingSyntax, Bulk insert, BinaryCopy</PackageTags>
<Description>The Pandatech.EFCore.PostgresExtensions library enriches Entity Framework Core applications with advanced PostgreSQL functionalities, starting with the ForUpdate locking syntax and BulkInsert function. Designed for seamless integration, this NuGet package aims to enhance the efficiency and capabilities of EF Core models when working with PostgreSQL, with the potential for further PostgreSQL-specific extensions.</Description>
<RepositoryUrl>https://github.com/PandaTechAM/be-lib-efcore-postgres-extensions</RepositoryUrl>
<PackageReleaseNotes>NuGet updates</PackageReleaseNotes>
<PackageReleaseNotes>Random Incrementing Sequence Generation Feature</PackageReleaseNotes>
</PropertyGroup>

<ItemGroup>
Expand All @@ -23,7 +23,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="8.0.2"/>
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="8.0.4" />
</ItemGroup>

</Project>
19 changes: 19 additions & 0 deletions src/EFCore.PostgresExtensions/EntityTypeConfigurationExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
using EFCore.PostgresExtensions.Helpers;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;

namespace EFCore.PostgresExtensions;

public static class EntityTypeConfigurationExtensions
{
public static PropertyBuilder<TProperty> HasRandomIdSequence<TProperty>(
this PropertyBuilder<TProperty> propertyBuilder)
{
var tableName = propertyBuilder.Metadata.DeclaringType.GetTableName();
var pgFunctionName = PgFunctionHelpers.GetPgFunctionName(tableName!);
propertyBuilder.HasDefaultValueSql(pgFunctionName);


return propertyBuilder;
}
}
Loading

0 comments on commit 58e33c1

Please sign in to comment.