FreeScript
runnerq-rust
by runnerq · version 2026.10.02 · updated 3 days ago
No reviews yet
- Downloads
- 0
- Reviews
- 0
- Size
- 886 KB
- Code check
- Clean
- Licence
- MIT
- Released
- Nov 2026
Durable Rust functions, with pluggable storage
Runner-Q
Durable Rust functions, with pluggable storage.
Features
Backend, Status, Use Case
PostgreSQL, ✅ Built-in, Default. Permanent persistence, SQL-based queries
Redis, ✅ Optional, Use the
Custom, ✅ Supported, Implement
Installation
Quick Start
The public API remains unchanged - you can continue using
Using the Default Redis Backend
Valkey Compatibility
Since Valkey is Redis protocol-compatible, you can use it directly by pointing the URL to your Valkey server:
PostgreSQL Backend
For use cases requiring permanent persistence and SQL-based queries, RunnerQ provides a PostgreSQL backend:
PostgreSQL Backend Features:
Implementing a Custom Backend
You can implement your own backend by implementing the
```rust
use runner_q::storage::{
Storage, QueueStorage, InspectionStorage, StorageError,
QueuedActivity, DequeuedActivity, FailureKind,
};
use runner_q::{QueueStats, ActivitySnapshot, ActivityEvent, DeadLetterRecord};
use async_trait::async_trait;
use std::time::Duration;
use uuid::Uuid;
pub struct MyCustomBackend {
// Your backend state (connection pool, config, etc.)
}
[async_trait]
impl QueueStorage for MyCustomBackend {
async fn enqueue(&self, activity: QueuedActivity) -> Result {
// Implement activity enqueuing
todo!()
}
async fn dequeue(
&self,
worker_id: &str,
timeout: Duration,
activity_types: Option,
) -> Result {
// Implement activity claiming.
// When activity_types is Some, only claim matching types.
todo!()
}
async fn ack_success(
&self,
activity_id: Uuid,
lease_id: &str,
result: Option,
worker_id: Option,
) -> Result {
// Mark activity as completed
todo!()
…
Durable Rust functions, with pluggable storage.
Features
- Pluggable backend system - Trait-based storage abstraction; PostgreSQL is built-in; Redis available via the
runner_q_rediscrate - Priority-based activity processing - Support for Critical, High, Normal, and Low priority levels
- Activity scheduling - Precise timestamp-based scheduling for future execution
- Intelligent retry mechanism - Built-in retry mechanism with exponential backoff
- Dead letter queue - Failed activities are moved to a dead letter queue for inspection
- Concurrent activity processing - Configurable number of concurrent workers
- Graceful shutdown - Proper shutdown handling with signal support
- Activity orchestration - Activities can execute other activities for complex workflows
- Comprehensive error handling - Retryable and non-retryable error types
- Activity metadata - Support for custom metadata on activities
- Built-in observability console - Real-time web UI for monitoring and managing activities
- Worker-level activity type filtering - Isolate workloads by restricting each engine to specific activity types
- Queue statistics - Monitoring capabilities and metrics collection
Backend, Status, Use Case
PostgreSQL, ✅ Built-in, Default. Permanent persistence, SQL-based queries
Redis, ✅ Optional, Use the
runner_q_redis crate for Redis or ValkeyCustom, ✅ Supported, Implement
Storage trait for your own backendInstallation
cargo add runner_qQuick Start
use runner_q::{WorkerEngine, ActivityPriority, ActivityHandler, ActivityContext, ActivityHandlerResult, ActivityError};
use std::sync::Arc;
use async_trait::async_trait;
use serde_json::json;
use serde::{Serialize, Deserialize};
use std::time::Duration;
// Define activity types
**[derive(Debug, Clone)]**
enum MyActivityType {
SendEmail,
ProcessPayment,
}
impl std::fmt::Display for MyActivityType {
fn fmt(&self, f: &mut std::fmt::Formatter|".backend()"| QB
WEB -->|".redis_url()"| RB
WE --> QB
WE --> IB
RB --> QB
RB --> IB
PB --> QB
PB --> IB
Future -.-> QB
Future -.-> IBThe public API remains unchanged - you can continue using
.redis_url() for the default experience, or use .backend() to inject a custom implementation.Using the Default Redis Backend
use runner_q::{WorkerEngine, RedisBackend};
use std::sync::Arc;
// Option 1: Use the simple redis_url API (recommended for most cases)
let engine = WorkerEngine::builder()
.redis_url("redis://localhost:6379")
.queue_name("my_app")
.build()
.await?;
// Option 2: Create a RedisBackend explicitly for more control
let backend = RedisBackend::builder()
.redis_url("redis://localhost:6379")
.queue_name("my_app")
.lease_ms(60_000) // Custom lease duration
.build()
.await?;
let engine = WorkerEngine::builder()
.backend(Arc::new(backend))
.max_workers(8)
.build()
.await?;Valkey Compatibility
Since Valkey is Redis protocol-compatible, you can use it directly by pointing the URL to your Valkey server:
let engine = WorkerEngine::builder()
.redis_url("redis://valkey-server:6379") // Works with Valkey!
.queue_name("my_app")
.build()
.await?;PostgreSQL Backend
For use cases requiring permanent persistence and SQL-based queries, RunnerQ provides a PostgreSQL backend:
**Cargo.toml**
[dependencies]
runner_q = { version = "0.5", features = ["postgres"] }
use runner_q::storage::PostgresBackend;
use std::sync::Arc;
// Create PostgreSQL backend
let backend = Arc::new(
PostgresBackend::new(
"postgres://user:password@localhost/runnerq",
"my_queue"
).await?
);
// Use with WorkerEngine
let engine = WorkerEngine::builder()
.backend(backend)
.max_workers(8)
.build()
.await?;PostgreSQL Backend Features:
- Permanent Persistence - Activities stored indefinitely (no TTL expiration)
- Multi-node Safe - Uses
FOR UPDATE SKIP LOCKEDfor concurrent job claiming - Cross-process Events - PostgreSQL
LISTEN/NOTIFYfor real-time event streaming - Atomic Idempotency - Separate table with
INSERT ... ON CONFLICTfor race-safe key claiming - History Preservation - Never deletes activity records
runnerq_activities- Main activity storagerunnerq_events- Event history timelinerunnerq_results- Activity execution resultsrunnerq_idempotency- Idempotency key mapping
examples/postgres_example.rs for a complete working example, and examples/activity_type_filtering.rs for workload isolation with multiple engines.Implementing a Custom Backend
You can implement your own backend by implementing the
Storage trait (which combines QueueStorage and InspectionStorage):```rust
use runner_q::storage::{
Storage, QueueStorage, InspectionStorage, StorageError,
QueuedActivity, DequeuedActivity, FailureKind,
};
use runner_q::{QueueStats, ActivitySnapshot, ActivityEvent, DeadLetterRecord};
use async_trait::async_trait;
use std::time::Duration;
use uuid::Uuid;
pub struct MyCustomBackend {
// Your backend state (connection pool, config, etc.)
}
[async_trait]
impl QueueStorage for MyCustomBackend {
async fn enqueue(&self, activity: QueuedActivity) -> Result {
// Implement activity enqueuing
todo!()
}
async fn dequeue(
&self,
worker_id: &str,
timeout: Duration,
activity_types: Option,
) -> Result {
// Implement activity claiming.
// When activity_types is Some, only claim matching types.
todo!()
}
async fn ack_success(
&self,
activity_id: Uuid,
lease_id: &str,
result: Option,
worker_id: Option,
) -> Result {
// Mark activity as completed
todo!()
…