BulkSharp
Production-grade .NET 8 library for defining, executing, and tracking bulk data operations from CSV and JSON files.
dotnet add package BulkSharp
Features
Everything you need for reliable bulk data processing at scale
Typed Operations
Define metadata and row types with full validation. Each row is validated and processed individually with detailed error tracking.
Step Pipelines
Break complex processing into ordered steps with per-step retry and exponential backoff. Supports sync, polling, and signal-based async completion.
CSV and JSON
Stream-based parsing via IAsyncEnumerable for memory-efficient processing of large files. Format detected automatically by extension.
Pluggable Storage
File system, in-memory, Amazon S3, or custom providers. Metadata persists via Entity Framework or in-memory stores.
Background Scheduling
Channels-based async scheduler with configurable worker count and backpressure. Immediate mode for testing. Custom schedulers supported.
Blazor Dashboard
Drop-in monitoring UI with operation list, progress tracking, error details, per-row step drill-down, file upload, and REST API.
Define and Run
A bulk operation is a class with an attribute. BulkSharp handles the rest.
[BulkOperation("import-users")]
public class UserImport : IBulkRowOperation<UserMetadata, UserRow>
{
public Task ValidateMetadataAsync(UserMetadata meta, CancellationToken ct)
=> Task.CompletedTask;
public Task ValidateRowAsync(UserRow row, UserMetadata meta, CancellationToken ct)
{
if (!row.Email.Contains('@'))
throw new BulkValidationException("Invalid email");
return Task.CompletedTask;
}
public Task ProcessRowAsync(UserRow row, UserMetadata meta, CancellationToken ct)
{
// Your business logic here
return Task.CompletedTask;
}
}services.AddBulkSharp(builder => builder
.UseFileStorage(fs => fs.UseFileSystem())
.UseMetadataStorage(ms => ms.UseSqlServer(opts =>
opts.ConnectionString = connectionString))
.UseScheduler(s => s.UseChannels(opts =>
opts.WorkerCount = 4)));Packages
Most consumers only need the meta-package. Add optional packages as needed.
Meta-package with DI registration, builders, and sensible defaults
Abstractions, domain models, attributes, and configuration
Processing engine, data formats, storage implementations, and scheduling
Blazor Server monitoring UI with REST API endpoints
SQL Server persistence for operations and row records via EF Core
Amazon S3 (and S3-compatible) file storage provider