background runner

This commit is contained in:
jay brown
2025-08-12 06:37:16 -07:00
parent a8e1095abc
commit 7a693d42f5
13 changed files with 2817 additions and 0 deletions
+159
View File
@@ -0,0 +1,159 @@
# Background Task Runner
A self-contained Go package for running background tasks with periodic execution and on-demand signaling capabilities.
## Purpose
This package provides a robust background task runner that:
- Executes work functions periodically (default: 60 seconds)
- Supports on-demand execution via signals
- Prevents concurrent executions through work coalescing
- Provides graceful shutdown with configurable timeout
- Tracks execution statistics for observability
- Maintains thread-safety for concurrent access
## Core Components
- **Runner**: Main orchestrator that manages the background task lifecycle
- **WorkFunc**: User-defined function signature for task execution
- **Stats**: Thread-safe statistics tracking (cycles, timestamps, errors)
- **Options**: Configuration pattern for customizing behavior
## Usage
### Basic Example
```go
package main
import (
"context"
"log/slog"
"time"
"queryorchestration/internal/backgroundtask"
)
func main() {
// Define configuration
config := map[string]any{
"dbConnection": db,
"apiClient": client,
}
// Define work function
workFunc := func(ctx context.Context, logger *slog.Logger, cfg map[string]any) error {
// Perform background work here
logger.Info("executing background task")
return nil
}
// Initialize runner
runner, err := backgroundtask.Initialize(
config,
slog.Default(),
workFunc,
backgroundtask.WithInterval(30*time.Second), // Optional: custom interval
)
if err != nil {
panic(err)
}
// Start runner
if err := runner.Run(); err != nil {
panic(err)
}
// Signal immediate work (optional)
runner.Signal()
// Graceful shutdown
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
runner.Shutdown(ctx)
}
```
## Key Features
### Work Coalescing
Multiple rapid signals are coalesced into a single execution to prevent resource exhaustion:
```go
runner.Signal() // Triggers work
runner.Signal() // Coalesced if previous work still running
runner.Signal() // Sets pending flag for next execution
```
### Statistics Tracking
Monitor runner performance and health:
```go
stats := runner.Stats()
fmt.Printf("Work cycles: %d\n", stats.WorkCycles())
fmt.Printf("Last error: %v\n", stats.LastErr())
fmt.Printf("Last execution: %v\n", stats.LastStart())
```
### Graceful Shutdown
Ensures clean termination with timeout protection:
```go
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
log.Printf("shutdown timeout: %v", err)
}
```
## Testing
The package includes comprehensive test helpers for unit testing:
```go
// Use provided test configuration
config := backgroundtask.NewTestConfigMap()
testConfig, _ := backgroundtask.GetTestConfig(config)
// Use test work functions
runner, _ := backgroundtask.Initialize(
config,
slog.Default(),
backgroundtask.TestWorkFunc, // Configurable test function
)
// Wait for work cycles in tests
backgroundtask.WaitForWorkCycles(runner, 3, 5*time.Second)
```
## Design Principles
1. **No External Dependencies**: Uses only standard library and slog
2. **Thread-Safe**: All operations safe for concurrent access
3. **Testable**: Fast intervals and test helpers for unit testing
4. **Observable**: Built-in statistics for monitoring
5. **Graceful**: Clean shutdown with context cancellation
6. **Defensive**: Prevents concurrent executions and resource leaks
## Configuration
### Options
- `WithInterval(duration)`: Set custom execution interval (default: 60s)
### WorkFunc Requirements
- Must accept `context.Context` for cancellation
- Must accept `*slog.Logger` for structured logging
- Must accept `map[string]any` for configuration passing
- Should respect context cancellation for graceful shutdown
- Should return meaningful errors for observability
## Error Handling
The runner continues operation even when work functions return errors:
- Errors are logged with details
- Last error is stored in stats for monitoring
- Runner remains functional for next execution
- No panic or crash on work function errors
## Thread Safety
All public methods are thread-safe:
- `Signal()` can be called from any goroutine
- `Stats()` provides consistent snapshots
- `Shutdown()` is idempotent with sync.Once protection
+42
View File
@@ -0,0 +1,42 @@
// Package backgroundtask provides a robust background task runner that executes
// work functions on a periodic schedule or when signaled. The runner supports:
//
// - Periodic execution based on a configurable interval
// - On-demand execution via signal
// - Graceful shutdown with timeout
// - Work coalescing to prevent concurrent executions
// - Statistics tracking for observability
// - Thread-safe operation
//
// Basic usage:
//
// config := map[string]any{
// "myData": getYourData(),
// "settings": getYourSettings(),
// }
// logger := slog.Default()
// workFunc := func(ctx context.Context, logger *slog.Logger, config map[string]any) error {
// // Your work logic here
// return nil
// }
//
// runner, err := backgroundtask.Initialize(config, logger, workFunc,
// backgroundtask.WithInterval(30 * time.Second))
// if err != nil {
// // Handle error
// }
//
// runner.Run()
// // ... do other work ...
// runner.Signal() // Trigger immediate execution
// // ... when shutting down ...
// ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
// defer cancel()
// if err := runner.Shutdown(ctx); err != nil {
// // Handle timeout
// }
//
// Note: This package is currently under development. Some fields and methods
// are defined but not yet implemented, which will cause unused warnings during
// the build process. These will be resolved as implementation progresses.
package backgroundtask
+24
View File
@@ -0,0 +1,24 @@
package backgroundtask
import "errors"
// Package errors that can be returned by the runner.
var (
// ErrStopped is returned when operations are attempted on a stopped runner.
ErrStopped = errors.New("runner is stopped")
// ErrShutdownTimeout is returned when shutdown exceeds the timeout.
ErrShutdownTimeout = errors.New("shutdown timeout exceeded")
// ErrAlreadyRunning is returned when Run is called on an already running runner.
ErrAlreadyRunning = errors.New("runner is already running")
// ErrNilConfig is returned when Initialize is called with a nil config.
ErrNilConfig = errors.New("config cannot be nil")
// ErrNilWorkFunc is returned when Initialize is called with a nil work function.
ErrNilWorkFunc = errors.New("work function cannot be nil")
// ErrInvalidInterval is returned when an invalid interval is specified.
ErrInvalidInterval = errors.New("interval must be between 1ms and 24h")
)
+123
View File
@@ -0,0 +1,123 @@
package backgroundtask_test
import (
"context"
"fmt"
"log/slog"
"sync/atomic"
"time"
"queryorchestration/internal/backgroundtask"
)
func ExampleRunner_basic() {
// Create a simple configuration
type Config struct {
Counter atomic.Int64
}
appConfig := &Config{}
// Create configuration map
config := map[string]any{
"appConfig": appConfig,
}
// Create a work function that increments the counter
workFunc := func(ctx context.Context, logger *slog.Logger, cfg map[string]any) error {
appCfgVal, ok := cfg["appConfig"]
if !ok {
return fmt.Errorf("appConfig not found")
}
c, ok := appCfgVal.(*Config)
if !ok {
return fmt.Errorf("invalid config type")
}
count := c.Counter.Add(1)
logger.Info("work executed", "count", count)
return nil
}
// Initialize the runner with a 100ms interval for quick demonstration
runner, err := backgroundtask.Initialize(
config,
slog.Default(),
workFunc,
backgroundtask.WithInterval(100*time.Millisecond),
)
if err != nil {
panic(err)
}
// Start the runner
if err := runner.Run(); err != nil {
panic(err)
}
// Let it run for a short time
time.Sleep(250 * time.Millisecond)
// Signal immediate work
if err := runner.Signal(); err != nil {
panic(err)
}
// Wait a bit more
time.Sleep(100 * time.Millisecond)
// Shutdown gracefully
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
panic(err)
}
// Check stats
stats := runner.Stats()
fmt.Printf("Work cycles completed: %d\n", stats.WorkCycles())
fmt.Printf("Counter value: %d\n", appConfig.Counter.Load())
// Can also pretty-print all stats (commented out for example test):
// stats.Print(os.Stdout)
// Output:
// Work cycles completed: 4
// Counter value: 4
}
func ExampleRunner_withTestConfig() {
// Use the provided test configuration
config := backgroundtask.NewTestConfigMap()
testConfig, _ := backgroundtask.GetTestConfig(config)
testConfig.WorkDuration = 50 * time.Millisecond
// Initialize with test work function
runner, err := backgroundtask.Initialize(
config,
slog.Default(),
backgroundtask.TestWorkFunc,
backgroundtask.WithInterval(100*time.Millisecond),
)
if err != nil {
panic(err)
}
// Start the runner
if err := runner.Run(); err != nil {
panic(err)
}
// Wait for some work cycles
if !backgroundtask.WaitForWorkCycles(runner, 2, 300*time.Millisecond) {
panic("expected at least 2 work cycles")
}
// Shutdown
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
panic(err)
}
fmt.Printf("Test work executed %d times\n", testConfig.Counter.Load())
// Output will show at least 2 executions
}
+19
View File
@@ -0,0 +1,19 @@
package backgroundtask
import (
"fmt"
"time"
)
// WithInterval sets the interval between periodic work executions.
// The interval must be between MinInterval and MaxInterval.
func WithInterval(d time.Duration) Option {
return func(r *Runner) error {
if d < MinInterval || d > MaxInterval {
return fmt.Errorf("%w: got %v, must be between %v and %v",
ErrInvalidInterval, d, MinInterval, MaxInterval)
}
r.interval = d
return nil
}
}
+254
View File
@@ -0,0 +1,254 @@
package backgroundtask
import (
"context"
"log/slog"
"sync"
"sync/atomic"
"time"
)
// Runner executes a work function periodically or when signaled.
// It provides graceful shutdown, work coalescing, and statistics tracking.
type Runner struct {
config map[string]any // Configuration passed to work function
logger *slog.Logger // Logger for output
workFunc WorkFunc // The function to execute
wakeCh chan struct{} // Channel to signal immediate work
stopCh chan struct{} // Channel to signal shutdown
doneCh chan struct{} // Channel closed when goroutine exits
ticker *time.Ticker // Periodic timer
mu sync.Mutex // Protects started flag
started bool // Whether Run has been called
stopped atomic.Bool // Whether shutdown has been called
working atomic.Bool // Whether work is currently executing
pendingWork atomic.Bool // Whether work was requested during execution
interval time.Duration // Time between periodic executions
stats Stats // Execution statistics
shutdownOnce sync.Once // Ensures shutdown runs once
}
// Initialize creates a new Runner with the provided configuration, logger, and work function.
// Options can be provided to customize the runner's behavior.
func Initialize(config map[string]any, logger *slog.Logger, workFunc WorkFunc, opts ...Option) (*Runner, error) {
if config == nil {
return nil, ErrNilConfig
}
if workFunc == nil {
return nil, ErrNilWorkFunc
}
if logger == nil {
logger = slog.Default()
}
r := &Runner{
config: config,
logger: logger,
workFunc: workFunc,
wakeCh: make(chan struct{}, 1),
stopCh: make(chan struct{}),
doneCh: make(chan struct{}),
interval: DefaultInterval,
}
// Apply options
for _, opt := range opts {
if err := opt(r); err != nil {
return nil, err
}
}
return r, nil
}
// Stats returns a pointer to the current statistics.
// This method is thread-safe and can be called concurrently.
func (r *Runner) Stats() *Stats {
return &r.stats
}
// Run starts the background goroutine that executes the work function.
// This method is idempotent - calling it multiple times has no effect if already running.
// Returns ErrAlreadyRunning if called after shutdown.
func (r *Runner) Run() error {
r.mu.Lock()
defer r.mu.Unlock()
if r.stopped.Load() {
return ErrStopped
}
if r.started {
// Idempotent - already running
return nil
}
r.started = true
r.ticker = time.NewTicker(r.interval)
// Start the worker goroutine
go r.worker()
r.logger.Info("background runner started",
"interval", r.interval,
"config_type", r.config != nil,
)
return nil
}
// Signal requests immediate execution of the work function.
// If the runner is already executing work, the signal is coalesced and
// another execution will occur after the current one completes.
// Returns ErrStopped if the runner has been shut down.
func (r *Runner) Signal() error {
if r.stopped.Load() {
return ErrStopped
}
// Non-blocking send - if channel is full, work is already pending
select {
case r.wakeCh <- struct{}{}:
r.logger.Debug("work signaled")
default:
// Channel full - work already pending
// Set pendingWork flag to ensure we run again after current work
r.pendingWork.Store(true)
r.logger.Debug("work signal coalesced")
}
return nil
}
// Shutdown gracefully stops the runner, waiting for any in-progress work to complete.
// The provided context controls the shutdown timeout. If the context has no deadline,
// a default timeout of MaxShutdownTimeout (60 seconds) is applied.
// This method is idempotent - multiple calls will not cause issues.
func (r *Runner) Shutdown(ctx context.Context) error {
var err error
r.shutdownOnce.Do(func() {
r.logger.Info("shutting down background runner")
// Mark as stopped to prevent new signals
r.stopped.Store(true)
// Stop the ticker
if r.ticker != nil {
r.ticker.Stop()
}
// Signal the worker to stop
close(r.stopCh)
// Apply default timeout if context has no deadline
if _, hasDeadline := ctx.Deadline(); !hasDeadline {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, MaxShutdownTimeout)
defer cancel()
}
// Wait for worker to finish or timeout
select {
case <-r.doneCh:
r.logger.Info("background runner shutdown complete")
case <-ctx.Done():
r.logger.Error("background runner shutdown timeout")
err = ErrShutdownTimeout
}
})
return err
}
// worker is the main goroutine that executes the work function.
func (r *Runner) worker() {
defer close(r.doneCh)
defer r.ticker.Stop()
r.logger.Debug("worker started")
for {
select {
case <-r.wakeCh:
r.logger.Debug("worker woken by signal")
r.tryWork("signal")
case <-r.ticker.C:
r.logger.Debug("worker woken by ticker")
r.tryWork("ticker")
case <-r.stopCh:
r.logger.Debug("worker stopping")
return
}
}
}
// tryWork attempts to execute the work function, handling coalescing and pending work.
func (r *Runner) tryWork(trigger string) {
// Check if already working
if !r.working.CompareAndSwap(false, true) {
// Already working - set pending flag
r.pendingWork.Store(true)
r.logger.Debug("work already in progress, pending flag set", "trigger", trigger)
return
}
// Execute work
r.doWork(trigger)
// Clear working flag
r.working.Store(false)
// Check if there's pending work
if r.pendingWork.CompareAndSwap(true, false) {
r.logger.Debug("processing pending work")
// Recursively process pending work (will only recurse once due to working flag)
r.tryWork("pending")
}
}
// doWork executes the actual work function with proper context and stats tracking.
func (r *Runner) doWork(trigger string) {
// Create context that will be canceled on shutdown
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Create a done channel for this work execution
done := make(chan struct{})
defer close(done)
// Monitor for shutdown
go func() {
select {
case <-r.stopCh:
cancel()
case <-done:
// Work completed
}
}()
// Update stats
r.stats.enterWork()
defer r.stats.exitWork()
startTime := time.Now()
r.stats.setLastStart(startTime)
r.logger.Debug("executing work", "trigger", trigger)
// Execute the work function
err := r.workFunc(ctx, r.logger, r.config)
// Update stats
r.stats.setLastFinish(time.Now())
r.stats.incrementWorkCycles()
if err != nil {
r.stats.setLastErr(err)
r.logger.Error("work function error", "error", err, "duration", time.Since(startTime))
} else {
r.stats.setLastErr(nil)
r.logger.Debug("work completed", "duration", time.Since(startTime))
}
}
+471
View File
@@ -0,0 +1,471 @@
package backgroundtask
import (
"context"
"log/slog"
"sync"
"sync/atomic"
"testing"
"time"
)
// TestSignal_TriggersWork verifies that Signal() triggers work execution
func TestSignal_TriggersWork(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour), // Very long interval to ensure only signal triggers work
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Signal work
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
// Wait for work to complete
if !WaitForWorkCycles(runner, 1, 500*time.Millisecond) {
t.Fatal("Expected at least 1 work cycle after signal")
}
stats := runner.Stats()
if stats.WorkCycles() < 1 {
t.Errorf("Expected at least 1 work cycle, got %d", stats.WorkCycles())
}
if testConfig.Counter.Load() < 1 {
t.Errorf("Expected at least 1 counter increment, got %d", testConfig.Counter.Load())
}
}
// TestTicker_TriggersWork verifies that the ticker triggers work execution
func TestTicker_TriggersWork(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(50*time.Millisecond), // Fast interval for testing
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Wait for ticker-triggered work (should happen at least twice)
if !WaitForWorkCycles(runner, 2, 200*time.Millisecond) {
t.Fatal("Expected at least 2 work cycles from ticker")
}
stats := runner.Stats()
if stats.WorkCycles() < 2 {
t.Errorf("Expected at least 2 work cycles, got %d", stats.WorkCycles())
}
if testConfig.Counter.Load() < 2 {
t.Errorf("Expected at least 2 counter increments, got %d", testConfig.Counter.Load())
}
}
// TestSignal_AfterShutdown verifies that Signal() returns error after shutdown
func TestSignal_AfterShutdown(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(100*time.Millisecond),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
// Shutdown the runner
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Fatalf("Shutdown failed: %v", err)
}
// Signal should return ErrStopped
if err := runner.Signal(); err != ErrStopped {
t.Errorf("Expected ErrStopped after shutdown, got: %v", err)
}
}
// TestSignal_Coalescing verifies that multiple rapid signals are coalesced
func TestSignal_Coalescing(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 100 * time.Millisecond // Make work take some time
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour), // Very long interval
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Send many rapid signals
const signalCount = 20
for i := 0; i < signalCount; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal %d failed: %v", i, err)
}
}
// Wait for work to complete
time.Sleep(500 * time.Millisecond)
// Should have fewer work cycles than signals due to coalescing
stats := runner.Stats()
workCycles := stats.WorkCycles()
if workCycles == 0 {
t.Fatal("Expected some work cycles")
}
// Should be significantly less than signalCount due to coalescing
if workCycles >= signalCount {
t.Errorf("Expected coalescing: work cycles (%d) should be less than signals (%d)", workCycles, signalCount)
}
// But should have at least processed some work
if testConfig.Counter.Load() == 0 {
t.Error("Expected some work to be done")
}
}
// TestWork_NoConcurrency verifies that work functions don't run concurrently
func TestWork_NoConcurrency(t *testing.T) {
var concurrentCount int64
var maxConcurrent int64
// Create a work function that tracks concurrent executions
workFunc := func(ctx context.Context, logger *slog.Logger, config map[string]any) error {
current := atomic.AddInt64(&concurrentCount, 1)
defer atomic.AddInt64(&concurrentCount, -1)
// Track max concurrent
for {
max := atomic.LoadInt64(&maxConcurrent)
if current <= max || atomic.CompareAndSwapInt64(&maxConcurrent, max, current) {
break
}
}
// Simulate some work
time.Sleep(50 * time.Millisecond)
return nil
}
config := map[string]any{"test": true}
runner, err := Initialize(
config,
slog.Default(),
workFunc,
WithInterval(25*time.Millisecond), // Faster than work duration
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
// Let it run for a while to trigger multiple potential concurrent executions
time.Sleep(200 * time.Millisecond)
// Send some signals too
for i := 0; i < 5; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
}
time.Sleep(200 * time.Millisecond)
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Fatalf("Shutdown failed: %v", err)
}
// Should never have more than 1 concurrent execution
if maxConcurrent > 1 {
t.Errorf("Expected max concurrent executions to be 1, got %d", maxConcurrent)
}
// Should have done some work
stats := runner.Stats()
if stats.WorkCycles() == 0 {
t.Error("Expected some work cycles")
}
}
// TestWork_PendingWork verifies that pending work is executed after current work completes
func TestWork_PendingWork(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 100 * time.Millisecond // Make work take time
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour), // Very long interval
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Send first signal to start work
if err := runner.Signal(); err != nil {
t.Fatalf("First signal failed: %v", err)
}
// Wait a bit for work to start
time.Sleep(25 * time.Millisecond)
// Send more signals while work is in progress
for i := 0; i < 3; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal %d failed: %v", i, err)
}
}
// Wait for all work to complete
time.Sleep(500 * time.Millisecond)
// Should have at least 2 work cycles (initial + pending)
stats := runner.Stats()
if stats.WorkCycles() < 2 {
t.Errorf("Expected at least 2 work cycles for pending work, got %d", stats.WorkCycles())
}
if testConfig.Counter.Load() < 2 {
t.Errorf("Expected at least 2 counter increments, got %d", testConfig.Counter.Load())
}
}
// TestWork_ErrorHandling verifies that work function errors are handled gracefully
func TestWork_ErrorHandling(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
ErrorWorkFunc, // Always returns error
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Signal work that will error
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
// Wait for work to complete
if !WaitForWorkCycles(runner, 1, 500*time.Millisecond) {
t.Fatal("Expected at least 1 work cycle")
}
// Check that error is recorded in stats
stats := runner.Stats()
if stats.LastErr() == nil {
t.Error("Expected error to be recorded in stats")
}
// Runner should still be functional after error
if err := runner.Signal(); err != nil {
t.Fatalf("Signal after error failed: %v", err)
}
if !WaitForWorkCycles(runner, 2, 500*time.Millisecond) {
t.Fatal("Expected runner to continue working after error")
}
}
// TestWork_ContextCancellation verifies that work functions receive cancellation signals
func TestWork_ContextCancellation(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 1 * time.Second // Long duration
testConfig.RespectContext = true // Make it respect context
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
// Signal work to start
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
// Let work start
time.Sleep(50 * time.Millisecond)
// Shutdown while work is in progress
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
start := time.Now()
if err := runner.Shutdown(ctx); err != nil {
t.Fatalf("Shutdown failed: %v", err)
}
elapsed := time.Since(start)
// Should shutdown quickly due to context cancellation, not wait for full work duration
if elapsed > 500*time.Millisecond {
t.Errorf("Shutdown took too long (%v), context cancellation may not be working", elapsed)
}
// Work should have been attempted
stats := runner.Stats()
if stats.WorkCycles() == 0 {
t.Error("Expected at least one work cycle to start")
}
}
// TestMultipleSignalsRapidly verifies handling of many rapid signals
func TestMultipleSignalsRapidly(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
FastWorkFunc, // Very fast work function
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Send many signals from multiple goroutines
const numGoroutines = 10
const signalsPerGoroutine = 10
var wg sync.WaitGroup
for i := 0; i < numGoroutines; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < signalsPerGoroutine; j++ {
if err := runner.Signal(); err != nil && err != ErrStopped {
t.Errorf("Signal failed: %v", err)
}
}
}()
}
wg.Wait()
// Wait for work to settle
time.Sleep(100 * time.Millisecond)
// Should have processed some work without crashing
stats := runner.Stats()
if stats.WorkCycles() == 0 {
t.Error("Expected some work cycles")
}
// Max concurrent should still be 1
if stats.MaxConcurrent() > 1 {
t.Errorf("Expected max concurrent to be 1, got %d", stats.MaxConcurrent())
}
}
+173
View File
@@ -0,0 +1,173 @@
package backgroundtask
import (
"fmt"
"io"
"sync"
"sync/atomic"
"time"
)
// Stats provides observability into the runner's operation.
// All methods are thread-safe and can be called concurrently.
type Stats struct {
mu sync.RWMutex
workCycles atomic.Int64 // Total number of work cycles completed
lastStart time.Time // Time when the last work cycle started
lastFinish time.Time // Time when the last work cycle finished
lastErr error // Last error returned from work function
maxConcurrent atomic.Int32 // Maximum concurrent work attempts (should always be 1)
currentConcurrent atomic.Int32 // Current concurrent work executions
}
// WorkCycles returns the total number of work cycles completed.
func (s *Stats) WorkCycles() int64 {
return s.workCycles.Load()
}
// LastStart returns the time when the last work cycle started.
func (s *Stats) LastStart() time.Time {
s.mu.RLock()
defer s.mu.RUnlock()
return s.lastStart
}
// LastFinish returns the time when the last work cycle finished.
func (s *Stats) LastFinish() time.Time {
s.mu.RLock()
defer s.mu.RUnlock()
return s.lastFinish
}
// LastErr returns the last error returned from the work function.
func (s *Stats) LastErr() error {
s.mu.RLock()
defer s.mu.RUnlock()
return s.lastErr
}
// MaxConcurrent returns the maximum number of concurrent work executions observed.
// This should always be 1 if the runner is working correctly.
func (s *Stats) MaxConcurrent() int32 {
return s.maxConcurrent.Load()
}
// CurrentConcurrent returns the current number of concurrent work executions.
// This should be 0 or 1.
func (s *Stats) CurrentConcurrent() int32 {
return s.currentConcurrent.Load()
}
// incrementWorkCycles increments the work cycle counter.
func (s *Stats) incrementWorkCycles() {
s.workCycles.Add(1)
}
// setLastStart updates the last start time.
func (s *Stats) setLastStart(t time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
s.lastStart = t
}
// setLastFinish updates the last finish time.
func (s *Stats) setLastFinish(t time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
s.lastFinish = t
}
// setLastErr updates the last error.
func (s *Stats) setLastErr(err error) {
s.mu.Lock()
defer s.mu.Unlock()
s.lastErr = err
}
// enterWork marks the beginning of a work cycle and tracks concurrency.
func (s *Stats) enterWork() {
current := s.currentConcurrent.Add(1)
// Update max concurrent if needed
for {
max := s.maxConcurrent.Load()
if current <= max {
break
}
if s.maxConcurrent.CompareAndSwap(max, current) {
break
}
}
}
// exitWork marks the end of a work cycle.
func (s *Stats) exitWork() {
s.currentConcurrent.Add(-1)
}
// Reset clears all statistics. Used primarily for testing.
func (s *Stats) Reset() {
s.mu.Lock()
defer s.mu.Unlock()
s.workCycles.Store(0)
s.lastStart = time.Time{}
s.lastFinish = time.Time{}
s.lastErr = nil
s.maxConcurrent.Store(0)
s.currentConcurrent.Store(0)
}
// Print writes a formatted representation of all statistics to the provided writer.
// The output includes work cycles, timing information, error status, and concurrency metrics.
func (s *Stats) Print(w io.Writer) {
s.mu.RLock()
lastStart := s.lastStart
lastFinish := s.lastFinish
lastErr := s.lastErr
s.mu.RUnlock()
workCycles := s.workCycles.Load()
maxConcurrent := s.maxConcurrent.Load()
currentConcurrent := s.currentConcurrent.Load()
fmt.Fprintf(w, "╭─────────────────────────────────────────────╮\n")
fmt.Fprintf(w, "│ Background Task Runner Stats │\n")
fmt.Fprintf(w, "├─────────────────────────────────────────────┤\n")
fmt.Fprintf(w, "│ Work Cycles: %-23d │\n", workCycles)
fmt.Fprintf(w, "│ Current Concurrent: %-23d │\n", currentConcurrent)
fmt.Fprintf(w, "│ Max Concurrent: %-23d │\n", maxConcurrent)
fmt.Fprintf(w, "├─────────────────────────────────────────────┤\n")
// Format timing information
if !lastStart.IsZero() {
fmt.Fprintf(w, "│ Last Start: %-30s │\n", lastStart.Format("2006-01-02 15:04:05.000"))
} else {
fmt.Fprintf(w, "│ Last Start: %-30s │\n", "Never")
}
if !lastFinish.IsZero() {
fmt.Fprintf(w, "│ Last Finish: %-30s │\n", lastFinish.Format("2006-01-02 15:04:05.000"))
if !lastStart.IsZero() {
duration := lastFinish.Sub(lastStart)
fmt.Fprintf(w, "│ Duration: %-30s │\n", duration.String())
}
} else {
fmt.Fprintf(w, "│ Last Finish: %-30s │\n", "Never")
}
fmt.Fprintf(w, "├─────────────────────────────────────────────┤\n")
// Format error information
if lastErr != nil {
errStr := lastErr.Error()
if len(errStr) > 30 {
errStr = errStr[:27] + "..."
}
fmt.Fprintf(w, "│ Last Error: %-30s │\n", errStr)
} else {
fmt.Fprintf(w, "│ Last Error: %-30s │\n", "None")
}
fmt.Fprintf(w, "╰─────────────────────────────────────────────╯\n")
}
+786
View File
@@ -0,0 +1,786 @@
package backgroundtask
import (
"bytes"
"context"
"errors"
"log/slog"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
// TestStats_WorkCycles verifies work cycle counting
func TestStats_WorkCycles(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
FastWorkFunc,
WithInterval(50*time.Millisecond),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
// Check initial stats
stats := runner.Stats()
if stats.WorkCycles() != 0 {
t.Errorf("Expected 0 initial work cycles, got %d", stats.WorkCycles())
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Wait for some work cycles from ticker
if !WaitForWorkCycles(runner, 3, 300*time.Millisecond) {
t.Fatal("Expected at least 3 work cycles from ticker")
}
tickerCycles := runner.Stats().WorkCycles()
// Add some signaled work
for i := 0; i < 5; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal %d failed: %v", i, err)
}
}
// Wait for signal work to complete
time.Sleep(100 * time.Millisecond)
finalStats := runner.Stats()
finalCycles := finalStats.WorkCycles()
// Should have more cycles after signals
if finalCycles <= tickerCycles {
t.Errorf("Expected work cycles to increase after signals: before=%d, after=%d",
tickerCycles, finalCycles)
}
// Should have at least the ticker cycles
if finalCycles < 3 {
t.Errorf("Expected at least 3 work cycles, got %d", finalCycles)
}
}
// TestStats_Timestamps verifies last start/finish timestamp tracking
func TestStats_Timestamps(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 50 * time.Millisecond
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour), // Long interval
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Check initial timestamps (should be zero)
stats := runner.Stats()
if !stats.LastStart().IsZero() {
t.Error("Expected zero LastStart initially")
}
if !stats.LastFinish().IsZero() {
t.Error("Expected zero LastFinish initially")
}
beforeSignal := time.Now()
// Signal work
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
// Wait for work to complete
if !WaitForWorkCycles(runner, 1, 200*time.Millisecond) {
t.Fatal("Expected work to complete")
}
afterWork := time.Now()
stats = runner.Stats()
// Timestamps should be set and reasonable
if stats.LastStart().IsZero() {
t.Error("Expected LastStart to be set after work")
}
if stats.LastFinish().IsZero() {
t.Error("Expected LastFinish to be set after work")
}
// Timestamps should be in reasonable range
if stats.LastStart().Before(beforeSignal) || stats.LastStart().After(afterWork) {
t.Errorf("LastStart timestamp out of range: %v (should be between %v and %v)",
stats.LastStart(), beforeSignal, afterWork)
}
if stats.LastFinish().Before(stats.LastStart()) {
t.Error("LastFinish should be after LastStart")
}
if stats.LastFinish().After(afterWork) {
t.Errorf("LastFinish timestamp after expected: %v > %v",
stats.LastFinish(), afterWork)
}
// Duration should be reasonable
duration := stats.LastFinish().Sub(stats.LastStart())
if duration < testConfig.WorkDuration/2 || duration > testConfig.WorkDuration*3 {
t.Errorf("Work duration seems wrong: %v (expected around %v)",
duration, testConfig.WorkDuration)
}
}
// TestStats_ErrorTracking verifies error tracking in stats
//
//nolint:cyclop // Test function with multiple subtests
func TestStats_ErrorTracking(t *testing.T) {
t.Run("NoErrorInitially", func(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
FastWorkFunc, // No error initially
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Signal successful work
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
if !WaitForWorkCycles(runner, 1, 200*time.Millisecond) {
t.Fatal("Expected work to complete")
}
// Should have no error
stats := runner.Stats()
if stats.LastErr() != nil {
t.Errorf("Expected no error initially, got: %v", stats.LastErr())
}
})
t.Run("ErrorRecording", func(t *testing.T) {
config := NewTestConfigMap()
errorWorkFunc := func(ctx context.Context, logger *slog.Logger, config map[string]any) error {
return errors.New("test error message")
}
errorRunner, err := Initialize(
config,
slog.Default(),
errorWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize error runner: %v", err)
}
if err := errorRunner.Run(); err != nil {
t.Fatalf("Failed to start error runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := errorRunner.Shutdown(ctx); err != nil {
t.Errorf("Error runner shutdown error: %v", err)
}
}()
// Signal error work
if err := errorRunner.Signal(); err != nil {
t.Fatalf("Error runner signal failed: %v", err)
}
if !WaitForWorkCycles(errorRunner, 1, 200*time.Millisecond) {
t.Fatal("Expected error work to complete")
}
// Should have error recorded
errorStats := errorRunner.Stats()
if errorStats.LastErr() == nil {
t.Error("Expected error to be recorded")
} else if errorStats.LastErr().Error() != "test error message" {
t.Errorf("Expected 'test error message', got: %v", errorStats.LastErr())
}
})
t.Run("ErrorClearing", func(t *testing.T) {
config := NewTestConfigMap()
successWorkFunc := func(ctx context.Context, logger *slog.Logger, config map[string]any) error {
return nil
}
successRunner, err := Initialize(
config,
slog.Default(),
successWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize success runner: %v", err)
}
// Set an initial error
successRunner.stats.setLastErr(errors.New("initial error"))
if err := successRunner.Run(); err != nil {
t.Fatalf("Failed to start success runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := successRunner.Shutdown(ctx); err != nil {
t.Errorf("Success runner shutdown error: %v", err)
}
}()
if err := successRunner.Signal(); err != nil {
t.Fatalf("Success runner signal failed: %v", err)
}
if !WaitForWorkCycles(successRunner, 1, 200*time.Millisecond) {
t.Fatal("Expected success work to complete")
}
// Error should be cleared
finalStats := successRunner.Stats()
if finalStats.LastErr() != nil {
t.Errorf("Expected error to be cleared after successful work, got: %v", finalStats.LastErr())
}
})
}
// TestStats_ThreadSafety verifies thread-safe access to stats
func TestStats_ThreadSafety(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
FastWorkFunc,
WithInterval(10*time.Millisecond), // Fast interval
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
var wg sync.WaitGroup
const numReaders = 10
const numWrites = 100
var errors int64
// Start multiple readers
for i := 0; i < numReaders; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < numWrites; j++ {
stats := runner.Stats()
// Access all stats methods concurrently
_ = stats.WorkCycles()
_ = stats.LastStart()
_ = stats.LastFinish()
_ = stats.LastErr()
_ = stats.MaxConcurrent()
_ = stats.CurrentConcurrent()
time.Sleep(1 * time.Millisecond)
}
}()
}
// Also send signals to trigger work and stats updates
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 50; i++ {
if err := runner.Signal(); err != nil && err != ErrStopped {
atomic.AddInt64(&errors, 1)
}
time.Sleep(5 * time.Millisecond)
}
}()
wg.Wait()
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
// Check that we didn't get any signal errors (except possibly ErrStopped at the end)
if errors > 5 { // Allow a few ErrStopped at shutdown
t.Errorf("Too many signal errors during concurrent access: %d", errors)
}
// Final stats should be consistent
finalStats := runner.Stats()
if finalStats.WorkCycles() == 0 {
t.Error("Expected some work cycles")
}
if finalStats.CurrentConcurrent() != 0 {
t.Errorf("Expected current concurrent to be 0 after shutdown, got %d",
finalStats.CurrentConcurrent())
}
}
// TestStats_MaxConcurrent verifies max concurrent tracking
func TestStats_MaxConcurrent(t *testing.T) {
config := map[string]any{"test": true}
var currentConcurrent int64
var maxObserved int64
// Work function that simulates overlapping work attempts
workFunc := func(ctx context.Context, logger *slog.Logger, cfg map[string]any) error {
current := atomic.AddInt64(&currentConcurrent, 1)
defer atomic.AddInt64(&currentConcurrent, -1)
// Track max we observe
for {
max := atomic.LoadInt64(&maxObserved)
if current <= max || atomic.CompareAndSwapInt64(&maxObserved, max, current) {
break
}
}
time.Sleep(50 * time.Millisecond)
return nil
}
runner, err := Initialize(
config,
slog.Default(),
workFunc,
WithInterval(25*time.Millisecond), // Faster than work duration
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
// Initial max concurrent should be 0
stats := runner.Stats()
if stats.MaxConcurrent() != 0 {
t.Errorf("Expected initial max concurrent to be 0, got %d", stats.MaxConcurrent())
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
// Let it run with fast ticking and send some signals
time.Sleep(100 * time.Millisecond)
for i := 0; i < 10; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
}
time.Sleep(200 * time.Millisecond)
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Fatalf("Shutdown failed: %v", err)
}
// Max concurrent should be 1 (no actual concurrency in background task)
finalStats := runner.Stats()
if finalStats.MaxConcurrent() != 1 {
t.Errorf("Expected max concurrent to be 1, got %d", finalStats.MaxConcurrent())
}
// Our test tracking should also show max of 1
if maxObserved != 1 {
t.Errorf("Expected observed max concurrent to be 1, got %d", maxObserved)
}
// Current should be 0 after shutdown
if finalStats.CurrentConcurrent() != 0 {
t.Errorf("Expected current concurrent to be 0 after shutdown, got %d",
finalStats.CurrentConcurrent())
}
}
// TestStats_Reset verifies stats reset functionality
func TestStats_Reset(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
ErrorWorkFunc, // Will set error
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Do some work to populate stats
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
if !WaitForWorkCycles(runner, 1, 200*time.Millisecond) {
t.Fatal("Expected work to complete")
}
// Verify stats are populated
stats := runner.Stats()
if stats.WorkCycles() == 0 {
t.Fatal("Expected work cycles to be > 0")
}
if stats.LastErr() == nil {
t.Fatal("Expected error to be set")
}
if stats.LastStart().IsZero() {
t.Fatal("Expected LastStart to be set")
}
if stats.MaxConcurrent() == 0 {
t.Fatal("Expected MaxConcurrent to be > 0")
}
// Reset stats
runner.stats.Reset()
// Verify stats are cleared
resetStats := runner.Stats()
if resetStats.WorkCycles() != 0 {
t.Errorf("Expected work cycles to be 0 after reset, got %d", resetStats.WorkCycles())
}
if resetStats.LastErr() != nil {
t.Errorf("Expected error to be nil after reset, got %v", resetStats.LastErr())
}
if !resetStats.LastStart().IsZero() {
t.Error("Expected LastStart to be zero after reset")
}
if !resetStats.LastFinish().IsZero() {
t.Error("Expected LastFinish to be zero after reset")
}
if resetStats.MaxConcurrent() != 0 {
t.Errorf("Expected MaxConcurrent to be 0 after reset, got %d", resetStats.MaxConcurrent())
}
if resetStats.CurrentConcurrent() != 0 {
t.Errorf("Expected CurrentConcurrent to be 0 after reset, got %d", resetStats.CurrentConcurrent())
}
}
// TestStats_ConsistentStateAfterWork verifies stats are consistent after work execution
func TestStats_ConsistentStateAfterWork(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
const numCycles = 5
for i := 0; i < numCycles; i++ {
if err := runner.Signal(); err != nil {
t.Fatalf("Signal %d failed: %v", i, err)
}
// Small delay between signals to ensure they're processed
time.Sleep(10 * time.Millisecond)
}
// Wait for all work to complete
if !WaitForWorkCycles(runner, int64(numCycles), 2*time.Second) {
t.Fatalf("Expected all work cycles to complete, got %d", runner.Stats().WorkCycles())
}
// Verify consistency
stats := runner.Stats()
// Work cycles should match what we triggered
if stats.WorkCycles() != int64(numCycles) {
t.Errorf("Expected %d work cycles, got %d", numCycles, stats.WorkCycles())
}
// Counter should also match
if testConfig.Counter.Load() != int64(numCycles) {
t.Errorf("Expected counter to be %d, got %d", numCycles, testConfig.Counter.Load())
}
// Should have no current concurrent work
if stats.CurrentConcurrent() != 0 {
t.Errorf("Expected current concurrent to be 0, got %d", stats.CurrentConcurrent())
}
// Max concurrent should be 1
if stats.MaxConcurrent() != 1 {
t.Errorf("Expected max concurrent to be 1, got %d", stats.MaxConcurrent())
}
// Should have valid timestamps
if stats.LastStart().IsZero() || stats.LastFinish().IsZero() {
t.Error("Expected valid last start/finish timestamps")
}
if stats.LastFinish().Before(stats.LastStart()) {
t.Error("LastFinish should not be before LastStart")
}
}
// TestStats_Print verifies the Print function output
//
//nolint:cyclop // Multiple subtests with setup/teardown logic
func TestStats_Print(t *testing.T) {
t.Run("EmptyStats", func(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
// Print stats before any work
var buf bytes.Buffer
runner.Stats().Print(&buf)
output := buf.String()
// Verify key elements are present
if !strings.Contains(output, "Background Task Runner Stats") {
t.Error("Expected title in output")
}
if !strings.Contains(output, "Work Cycles: 0") {
t.Error("Expected 0 work cycles")
}
if !strings.Contains(output, "Last Start: Never") {
t.Error("Expected 'Never' for last start")
}
if !strings.Contains(output, "Last Error: None") {
t.Error("Expected 'None' for last error")
}
t.Logf("Empty stats output:\n%s", output)
})
t.Run("StatsAfterWork", func(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
TestWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Trigger work
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
if !WaitForWorkCycles(runner, 1, 500*time.Millisecond) {
t.Fatal("Expected work to complete")
}
// Print stats after work
var buf bytes.Buffer
runner.Stats().Print(&buf)
output := buf.String()
// Verify stats reflect work
if !strings.Contains(output, "Work Cycles: 1") {
t.Error("Expected 1 work cycle")
}
if strings.Contains(output, "Last Start: Never") {
t.Error("Expected valid last start time")
}
if strings.Contains(output, "Last Finish: Never") {
t.Error("Expected valid last finish time")
}
if !strings.Contains(output, "Duration:") {
t.Error("Expected duration to be shown")
}
t.Logf("Stats after work output:\n%s", output)
})
t.Run("StatsWithError", func(t *testing.T) {
config := NewTestConfigMap()
runner, err := Initialize(
config,
slog.Default(),
ErrorWorkFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Trigger work that will error
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
if !WaitForWorkCycles(runner, 1, 500*time.Millisecond) {
t.Fatal("Expected work to complete")
}
// Print stats with error
var buf bytes.Buffer
runner.Stats().Print(&buf)
output := buf.String()
// Verify error is shown
if !strings.Contains(output, "Last Error: test error") {
t.Error("Expected error to be shown in output")
}
t.Logf("Stats with error output:\n%s", output)
})
t.Run("LongErrorTruncation", func(t *testing.T) {
config := NewTestConfigMap()
// Create work function with long error message
longErrorFunc := func(ctx context.Context, logger *slog.Logger, cfg map[string]any) error {
return errors.New("this is a very long error message that should be truncated in the pretty print output")
}
runner, err := Initialize(
config,
slog.Default(),
longErrorFunc,
WithInterval(1*time.Hour),
)
if err != nil {
t.Fatalf("Failed to initialize runner: %v", err)
}
if err := runner.Run(); err != nil {
t.Fatalf("Failed to start runner: %v", err)
}
defer func() {
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
if err := runner.Shutdown(ctx); err != nil {
t.Errorf("Shutdown error: %v", err)
}
}()
// Trigger work that will error
if err := runner.Signal(); err != nil {
t.Fatalf("Signal failed: %v", err)
}
if !WaitForWorkCycles(runner, 1, 500*time.Millisecond) {
t.Fatal("Expected work to complete")
}
// Print stats with long error
var buf bytes.Buffer
runner.Stats().Print(&buf)
output := buf.String()
// Verify error is truncated with ellipsis
if !strings.Contains(output, "...") {
t.Error("Expected long error to be truncated with ellipsis")
}
t.Logf("Stats with truncated error output:\n%s", output)
})
}
+218
View File
@@ -0,0 +1,218 @@
package backgroundtask
import (
"context"
"errors"
"log/slog"
"sync/atomic"
"time"
)
// TestConfig holds configuration for test work functions.
type TestConfig struct {
// WorkDuration is how long each work execution should take
WorkDuration time.Duration
// ShouldError indicates if the work function should return an error
ShouldError bool
// ErrorMessage is the error message to return when ShouldError is true
ErrorMessage string
// RespectContext indicates if the work function should respect context cancellation
RespectContext bool
// Counter tracks the number of executions
Counter *atomic.Int64
// MaxExecutions limits the number of executions (0 = unlimited)
MaxExecutions int64
}
// NewTestConfig creates a default test configuration.
func NewTestConfig() *TestConfig {
return &TestConfig{
WorkDuration: 1 * time.Millisecond,
ShouldError: false,
ErrorMessage: "test error",
RespectContext: true,
Counter: &atomic.Int64{},
MaxExecutions: 0,
}
}
// NewTestConfigMap creates a config map with a default test configuration.
// This is a convenience function for testing.
func NewTestConfigMap() map[string]any {
return map[string]any{
"testConfig": NewTestConfig(),
}
}
// GetTestConfig extracts a TestConfig from a config map.
// Returns the config and true if found, nil and false otherwise.
func GetTestConfig(config map[string]any) (*TestConfig, bool) {
cfgVal, ok := config["testConfig"]
if !ok {
return nil, false
}
cfg, ok := cfgVal.(*TestConfig)
return cfg, ok
}
// TestWorkFunc is a configurable work function for testing.
// It increments the counter and simulates work based on the TestConfig stored in the config map.
func TestWorkFunc(ctx context.Context, logger *slog.Logger, config map[string]any) error {
cfg, ok := GetTestConfig(config)
if !ok {
return errors.New("testConfig not found or invalid type")
}
// Check max executions
if cfg.MaxExecutions > 0 {
current := cfg.Counter.Load()
if current >= cfg.MaxExecutions {
logger.Debug("max executions reached", "count", current)
return nil
}
}
// Increment counter
count := cfg.Counter.Add(1)
logger.Debug("test work executing", "count", count)
// Simulate work duration
if cfg.WorkDuration > 0 {
if cfg.RespectContext {
// Use select to respect context cancellation
select {
case <-time.After(cfg.WorkDuration):
// Work completed
case <-ctx.Done():
logger.Debug("test work canceled", "count", count)
return ctx.Err()
}
} else {
// Sleep regardless of context
time.Sleep(cfg.WorkDuration)
}
}
// Return error if configured
if cfg.ShouldError {
logger.Debug("test work returning error", "count", count, "error", cfg.ErrorMessage)
return errors.New(cfg.ErrorMessage)
}
logger.Debug("test work completed", "count", count)
return nil
}
// FastWorkFunc is a minimal work function that just increments a counter.
// Useful for testing rapid executions.
func FastWorkFunc(ctx context.Context, logger *slog.Logger, config map[string]any) error {
cfg, ok := GetTestConfig(config)
if !ok {
return errors.New("testConfig not found or invalid type")
}
count := cfg.Counter.Add(1)
logger.Debug("fast work executed", "count", count)
return nil
}
// SlowWorkFunc simulates a long-running task that respects context cancellation.
// It checks context every 10ms while simulating work.
func SlowWorkFunc(ctx context.Context, logger *slog.Logger, config map[string]any) error {
cfg, ok := GetTestConfig(config)
if !ok {
return errors.New("testConfig not found or invalid type")
}
count := cfg.Counter.Add(1)
logger.Debug("slow work starting", "count", count, "duration", cfg.WorkDuration)
// Check context periodically during long work
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
deadline := time.Now().Add(cfg.WorkDuration)
for time.Now().Before(deadline) {
select {
case <-ticker.C:
// Continue working
case <-ctx.Done():
logger.Debug("slow work canceled", "count", count)
return ctx.Err()
}
}
logger.Debug("slow work completed", "count", count)
return nil
}
// ErrorWorkFunc always returns an error after incrementing the counter.
func ErrorWorkFunc(ctx context.Context, logger *slog.Logger, config map[string]any) error {
cfg, ok := GetTestConfig(config)
if !ok {
return errors.New("testConfig not found or invalid type")
}
count := cfg.Counter.Add(1)
logger.Debug("error work executed", "count", count)
if cfg.ErrorMessage != "" {
return errors.New(cfg.ErrorMessage)
}
return errors.New("error work function failed")
}
// BlockingWorkFunc blocks until context is canceled.
// Useful for testing shutdown behavior.
func BlockingWorkFunc(ctx context.Context, logger *slog.Logger, config map[string]any) error {
cfg, ok := GetTestConfig(config)
if !ok {
return errors.New("testConfig not found or invalid type")
}
count := cfg.Counter.Add(1)
logger.Debug("blocking work started", "count", count)
<-ctx.Done()
logger.Debug("blocking work canceled", "count", count)
return ctx.Err()
}
// WaitForWorkCycles waits for the specified number of work cycles to complete.
// Returns true if the target was reached within the timeout, false otherwise.
func WaitForWorkCycles(runner *Runner, target int64, timeout time.Duration) bool {
deadline := time.Now().Add(timeout)
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
for time.Now().Before(deadline) {
stats := runner.Stats()
if stats.WorkCycles() >= target {
return true
}
<-ticker.C
}
return false
}
// WaitForCondition waits for a condition function to return true.
// Returns true if the condition was met within the timeout, false otherwise.
func WaitForCondition(condition func() bool, timeout time.Duration) bool {
deadline := time.Now().Add(timeout)
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
for time.Now().Before(deadline) {
if condition() {
return true
}
<-ticker.C
}
return false
}
@@ -0,0 +1,329 @@
package backgroundtask
import (
"context"
"log/slog"
"sync/atomic"
"testing"
"time"
)
// TestSlowWorkFunc tests the slow work function
func TestSlowWorkFunc(t *testing.T) {
t.Run("NormalExecution", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 100 * time.Millisecond
start := time.Now()
err := SlowWorkFunc(t.Context(), slog.Default(), config)
elapsed := time.Since(start)
if err != nil {
t.Errorf("Expected no error, got: %v", err)
}
if elapsed < testConfig.WorkDuration/2 {
t.Errorf("Work completed too quickly: %v (expected ~%v)", elapsed, testConfig.WorkDuration)
}
if testConfig.Counter.Load() != 1 {
t.Errorf("Expected counter to be 1, got %d", testConfig.Counter.Load())
}
})
t.Run("ContextCancellation", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 1 * time.Second // Long duration
ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
defer cancel()
start := time.Now()
err := SlowWorkFunc(ctx, slog.Default(), config)
elapsed := time.Since(start)
if err == nil {
t.Error("Expected context cancellation error")
}
if elapsed > 200*time.Millisecond {
t.Errorf("Work took too long after cancellation: %v", elapsed)
}
if testConfig.Counter.Load() != 1 {
t.Errorf("Expected counter to be 1, got %d", testConfig.Counter.Load())
}
})
t.Run("InvalidConfig", func(t *testing.T) {
config := map[string]any{"invalid": "config"}
err := SlowWorkFunc(t.Context(), slog.Default(), config)
if err == nil {
t.Error("Expected error for invalid config")
}
})
}
// TestBlockingWorkFunc tests the blocking work function
func TestBlockingWorkFunc(t *testing.T) {
t.Run("BlocksUntilCancellation", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
defer cancel()
start := time.Now()
err := BlockingWorkFunc(ctx, slog.Default(), config)
elapsed := time.Since(start)
if err == nil {
t.Error("Expected context cancellation error")
}
// Should block for approximately the timeout duration
if elapsed < 40*time.Millisecond || elapsed > 100*time.Millisecond {
t.Errorf("Unexpected blocking duration: %v", elapsed)
}
if testConfig.Counter.Load() != 1 {
t.Errorf("Expected counter to be 1, got %d", testConfig.Counter.Load())
}
})
t.Run("InvalidConfig", func(t *testing.T) {
config := map[string]any{"invalid": "config"}
err := BlockingWorkFunc(t.Context(), slog.Default(), config)
if err == nil {
t.Error("Expected error for invalid config")
}
})
}
// TestWaitForCondition tests the wait for condition helper
func TestWaitForCondition(t *testing.T) {
t.Run("ConditionMet", func(t *testing.T) {
var conditionMet atomic.Bool
// Start a goroutine that will set the condition after a delay
go func() {
time.Sleep(50 * time.Millisecond)
conditionMet.Store(true)
}()
start := time.Now()
result := WaitForCondition(func() bool {
return conditionMet.Load()
}, 200*time.Millisecond)
elapsed := time.Since(start)
if !result {
t.Error("Expected condition to be met")
}
if elapsed > 100*time.Millisecond {
t.Errorf("Condition took too long to be met: %v", elapsed)
}
})
t.Run("ConditionTimeout", func(t *testing.T) {
start := time.Now()
result := WaitForCondition(func() bool {
return false // Never true
}, 50*time.Millisecond)
elapsed := time.Since(start)
if result {
t.Error("Expected condition to timeout")
}
if elapsed < 40*time.Millisecond {
t.Errorf("Timeout too short: %v", elapsed)
}
})
t.Run("ImmediateCondition", func(t *testing.T) {
start := time.Now()
result := WaitForCondition(func() bool {
return true // Always true
}, 100*time.Millisecond)
elapsed := time.Since(start)
if !result {
t.Error("Expected immediate condition to be met")
}
if elapsed > 50*time.Millisecond {
t.Errorf("Immediate condition took too long: %v", elapsed)
}
})
}
// TestHelperFunctions tests additional helper functions for coverage
func TestHelperFunctions(t *testing.T) {
t.Run("NewTestConfigMap", func(t *testing.T) {
config := NewTestConfigMap()
if config == nil {
t.Fatal("Expected config map to be created")
}
testConfig, ok := GetTestConfig(config)
if !ok {
t.Fatal("Expected test config to be in map")
}
if testConfig.Counter == nil {
t.Error("Expected counter to be initialized")
}
if testConfig.WorkDuration != 1*time.Millisecond {
t.Errorf("Expected default work duration to be 1ms, got %v", testConfig.WorkDuration)
}
})
t.Run("GetTestConfigInvalid", func(t *testing.T) {
config := map[string]any{"invalid": "config"}
_, ok := GetTestConfig(config)
if ok {
t.Error("Expected GetTestConfig to return false for invalid config")
}
})
t.Run("GetTestConfigMissing", func(t *testing.T) {
config := map[string]any{}
_, ok := GetTestConfig(config)
if ok {
t.Error("Expected GetTestConfig to return false for missing config")
}
})
t.Run("GetTestConfigWrongType", func(t *testing.T) {
config := map[string]any{"testConfig": "not a TestConfig"}
_, ok := GetTestConfig(config)
if ok {
t.Error("Expected GetTestConfig to return false for wrong type")
}
})
}
// TestErrorWorkFuncVariations tests error work function with different configurations
func TestErrorWorkFuncVariations(t *testing.T) {
t.Run("CustomErrorMessage", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.ErrorMessage = "custom error message"
err := ErrorWorkFunc(t.Context(), slog.Default(), config)
if err == nil {
t.Error("Expected error")
}
if err.Error() != "custom error message" {
t.Errorf("Expected 'custom error message', got: %v", err.Error())
}
if testConfig.Counter.Load() != 1 {
t.Errorf("Expected counter to be 1, got %d", testConfig.Counter.Load())
}
})
t.Run("EmptyErrorMessage", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.ErrorMessage = ""
err := ErrorWorkFunc(t.Context(), slog.Default(), config)
if err == nil {
t.Error("Expected error")
}
if err.Error() != "error work function failed" {
t.Errorf("Expected default error message, got: %v", err.Error())
}
})
}
// TestTestWorkFuncVariations tests the test work function with different configurations
func TestTestWorkFuncVariations(t *testing.T) {
t.Run("MaxExecutionsLimit", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.MaxExecutions = 2
// First execution should work
err := TestWorkFunc(t.Context(), slog.Default(), config)
if err != nil {
t.Errorf("First execution failed: %v", err)
}
// Second execution should work
err = TestWorkFunc(t.Context(), slog.Default(), config)
if err != nil {
t.Errorf("Second execution failed: %v", err)
}
// Third execution should be skipped
err = TestWorkFunc(t.Context(), slog.Default(), config)
if err != nil {
t.Errorf("Third execution failed: %v", err)
}
// Counter should be 2 (third execution was skipped)
if testConfig.Counter.Load() != 2 {
t.Errorf("Expected counter to be 2, got %d", testConfig.Counter.Load())
}
})
t.Run("ErrorConfigured", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.ShouldError = true
testConfig.ErrorMessage = "configured error"
err := TestWorkFunc(t.Context(), slog.Default(), config)
if err == nil {
t.Error("Expected error")
}
if err.Error() != "configured error" {
t.Errorf("Expected 'configured error', got: %v", err.Error())
}
})
t.Run("ContextDisrespected", func(t *testing.T) {
config := NewTestConfigMap()
testConfig, _ := GetTestConfig(config)
testConfig.WorkDuration = 100 * time.Millisecond
testConfig.RespectContext = false
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Millisecond)
defer cancel()
start := time.Now()
err := TestWorkFunc(ctx, slog.Default(), config)
elapsed := time.Since(start)
// Should complete despite context cancellation
if err != nil {
t.Errorf("Expected no error when context is disrespected, got: %v", err)
}
// Should take approximately the full work duration
if elapsed < testConfig.WorkDuration/2 {
t.Errorf("Work completed too quickly: %v", elapsed)
}
})
}
+32
View File
@@ -0,0 +1,32 @@
// Package backgroundtask provides a background task runner that executes work functions
// on a periodic schedule or when signaled. It supports graceful shutdown, work coalescing,
// and provides statistics for observability.
package backgroundtask
import (
"context"
"log/slog"
"time"
)
// WorkFunc defines the signature for work functions that can be executed by the runner.
// The function receives a context that will be canceled on shutdown, a logger for output,
// and a config map containing configuration data with string keys and arbitrary values.
// The function should return an error if the work fails, though errors do not stop the runner.
type WorkFunc func(ctx context.Context, logger *slog.Logger, config map[string]any) error
// Option is a function that configures a Runner during initialization.
// Options are applied in the order they are provided to Initialize.
type Option func(*Runner) error
// DefaultInterval is the default time between periodic work cycles.
const DefaultInterval = 60 * time.Second
// MinInterval is the minimum allowed interval between work cycles.
const MinInterval = 1 * time.Millisecond
// MaxInterval is the maximum allowed interval between work cycles.
const MaxInterval = 24 * time.Hour
// MaxShutdownTimeout is the maximum time to wait for graceful shutdown.
const MaxShutdownTimeout = 60 * time.Second