Dnp.WorkflowEngine

Dnp.WorkflowEngine is a lightweight, extensible workflow orchestration engine for .NET. It supports JSON-defined workflows, plugin step factories, a Roslyn-based rules engine, durable workflow state persistence, parallel and iterative steps, and built-in activities for HTTP calls, metrics emission, retries, and message-driven triggers (RabbitMQ / SQS). The engine is designed to be embedded in services (including Razor Pages/ASP.NET Core hosts) and extended with custom plugins.

See docs/WorkflowEngine.md for detailed documentation, examples, and operational notes.

---


---

🚀 Key Features

• 🔌 Plugin-based architecture with dynamic step discovery (from plugins folder or scanned in the running assembly)
• 🧩 Declarative workflow definitions in JSON (linear and state-machine modes)
• 🧠 Rule evaluation via Roslyn scripting (compile and run rules against WorkflowContext)
• ✅ Dedicated If activity step for single-condition branching (configurable via step config)
• 🧾 RuleEvaluator step to evaluate named rules stored in the database
• ⏯️ Workflow state persistence and resumable execution via IWorkflowStateRepository
• 🔔 Event storage and background WorkflowEventProcessor to resume suspended workflows on external events
• 🧵 Parallel execution and feedback-loop support
• 🛡️ Advanced activities: ParallelForEach (durable batching), Retry wrapper, EmitMetric (counter/histogram), HttpCall (auth + Polly resiliency), enhanced Switch (regex/glob)
• ⏰ Scheduler with cron-based recurring schedules and a simple management UI
• 🧪 Unit and integration tests (MSTest + Moq, in-memory EF for DB tests)

---

New Activities & Plugins

This repository includes several new steps and capabilities. Brief descriptions and where to find examples:

• ParallelForEach (durable, batched)
• Process collections in parallel with batch commit points and optional persistence for resume.
• Example: Example.Service/Workflows/ParallelForEachAdvancedExampleWorkflow.json.

• Retry wrapper
• Wrap a child step to retry on failure with backoff and jitter.
• Example: Example.Service/Workflows/RetryExampleWorkflow.json (uses Unreliable step in the example plugin).

• EmitMetric (Counter, Histogram)
• Emits metrics via System.Diagnostics.Metrics and stores an aggregated summary in WorkflowState.ContextData for UI inspection.
• Examples: EmitMetricExampleWorkflow.json, EmitHistogramExampleWorkflow.json.

• HttpCall step (dynamic URL, Bearer/Basic auth, Polly resiliency)
• Supports HttpUrlExpression, token expressions, Basic auth, and Polly policies (retry, circuit-breaker, timeout, bulkhead, fallback).
• Examples: HttpCallBearerExampleWorkflow.json, HttpCallBasicExampleWorkflow.json, HttpCallAdvancedCombinedExampleWorkflow.json.

• Switch enhancements
• SwitchCaseConfig now supports IsRegex and IsGlob, enabling powerful pattern matching in Switch steps.

• Scheduler and Schedule UI
• Cron-based recurring schedules implemented via NCrontab.
• Web UI at /Schedules to create, edit and delete schedules.
• API endpoints under /api/schedule for programmatic management.

Configuring additional plugin DLLs (appsettings)

You can instruct the engine to load extra plugin DLLs by listing them in the WorkflowEngine:AdditionalPluginDlls setting in your appsettings JSON. Plugin discovery order and priority:

• First the engine loads DLLs found in the configured PluginFolder (highest priority).
• Next the engine loads any files listed under AdditionalPluginDlls. When a plugin or step name already exists from the plugin-folder DLLs, the plugin-folder implementation is kept and the additional DLL is skipped for that plugin/step.

Paths in AdditionalPluginDlls may be absolute or relative to the host working directory. Example appsettings.json snippet with multiple paths:

{
  "WorkflowEngine": {
    "PluginFolder": "C:\\Workflow\\Plugins",
    "LicenseEmail": "ops@example.com",
    "AdditionalPluginDlls": [
      "./extras/MyCustomPlugin.dll",
      "C:/opt/plugins/ThirdPartyPlugin.dll",
      "..\\shared-plugins\\SharedPlugin.dll"
    ]
  }
}

Notes:

• The loader logs skipped plugins/steps when a name collision occurs (plugin-folder DLLs win).
• For security, only load trusted plugin DLLs. Consider runtime permission checks and signature verification in production.
• If you want additional-DLLs to override plugin-folder DLLs, the registry behavior can be adjusted — contact the code owner to change priority.

---

Web UI & API

The web project Dnp.Workflow.Web includes several UI pages and API controllers for operational tasks:

• /Compensations — metrics and compensation actions
• /Metrics — view metric summaries from workflow states
• /Schedules — manage cron schedules and standalone events

API endpoints of interest:

• GET /api/metrics/{stateId} — read metrics stored in WorkflowState.ContextData
• GET /api/metrics/{stateId}/progress — per-step progress (ParallelForEach)
• GET /api/schedule — list schedules
• POST /api/schedule — create schedule (validates cron expressions)
• PUT /api/schedule/{id} — update schedule
• DELETE /api/schedule/{id} — delete schedule

---

Running the example service

1. Build the solution:

dotnet build

1. Run the Example Service:

cd Example.Service
dotnet run

Open the web UI (run the Dnp.Workflow.Web project) to explore the Schedules and Metrics pages.

---

Running integration tests

Integration tests are disabled by default. To run them locally or in CI:

1. Start required infrastructure (example using Docker Compose for RabbitMQ):

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

1. Enable integration tests by setting environment variable RUNINTEGRATIONTESTS=true.

On Windows (PowerShell):

$env:RUN_INTEGRATION_TESTS = "true"
dotnet test

On Linux/macOS:

export RUN_INTEGRATION_TESTS=true
dotnet test

Integration tests that require external services will mark themselves Inconclusive when the env var is not set.

---

Contributing & Tests

• Add new steps as IWorkflowStep implementations and register them via plugins or the WorkflowRegistry.
• Unit/integration tests are recommended for new features, especially for durable and distributed behaviors.

Run tests:

dotnet test

---

License & Attribution

© Doughnuts Publishing LLC. All rights reserved.