AWS Durable Execution SDK for Go

A durable function is a Lambda function whose progress the service checkpoints as it runs. The SDK records the result of each operation when the operation completes. An invocation ends when the handler suspends on a timer, on an external signal, or on the function timeout. The service then invokes the function again. On that invocation the SDK replays the recorded results instead of running the completed work again, and the handler continues from the first operation that has no recorded result. The overall orchestration can run for up to a year, because the durable execution spans multiple Lambda invocations and consumes no compute while it waits.

The SDK requires Go 1.24 or later.

Your first durable function

Create a module and add the SDK to it.

mkdir first-durable-function && cd first-durable-function
go mod init example.com/first-durable-function
go get github.com/aws/aws-durable-execution-sdk-go

go get records the latest release. Versioning states what each release guarantees.

Put this in main.go. The handler runs a step, suspends on a two second timer, then runs a second step that reads the first result.

package main

import (
	"fmt"
	"time"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
)

func handler(ctx durable.Context, _ any) (string, error) {
	name, err := durable.Step(ctx, "fetch-name", func(_ durable.StepContext) (string, error) {
		return "world", nil
	})
	if err != nil {
		return "", err
	}
	if err := durable.Wait(ctx, "cooldown", 2*time.Second); err != nil {
		return "", err
	}
	return durable.Step(ctx, "format", func(_ durable.StepContext) (string, error) {
		return fmt.Sprintf("hello, %s", name), nil
	})
}

func main() {
	durable.Start(handler)
}

Build a static Linux binary named bootstrap. That is the file the provided.al2023 runtime executes.

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o dist/bootstrap .

Point a SAM template at the dist directory and give the function a DurableConfig. The DurableConfig is what makes the service checkpoint the function.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: First durable function

Parameters:
  ExecutionRoleArn:
    Type: String
    Description: IAM role the function assumes

Resources:
  FirstDurableFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: first-durable-function
      CodeUri: dist/
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures: [x86_64]
      Timeout: 60
      MemorySize: 128
      Role: !Ref ExecutionRoleArn
      DurableConfig:
        RetentionPeriodInDays: 7
        ExecutionTimeout: 300

The template takes an existing execution role. The role needs permission to write CloudWatch Logs, and it needs the two Lambda API actions the SDK calls, lambda:CheckpointDurableExecution and lambda:GetDurableExecutionState.

The SDK calls these APIs through a Lambda client built from the default AWS config. The client sets a 5 second connect timeout, a 50 second response timeout, and a 55 second total request timeout. Its AWS standard retryer retries server faults, throttling, and connection errors. The SDK adds no retry of its own. A checkpoint call that still fails ends the invocation, and the service invokes the function again. This holds even when handler code catches the error.

sam deploy --template-file template.yaml \
  --stack-name first-durable-function \
  --resolve-s3 --region us-west-2 \
  --parameter-overrides ExecutionRoleArn=arn:aws:iam::111122223333:role/durable-lambda-role

Invoking a durable function starts an execution. A synchronous invoke waits for the execution to finish and returns its result, so the call below returns after about three seconds with "hello, world" in response.json. The invocation metadata on stdout includes the DurableExecutionArn.

aws lambda invoke --function-name first-durable-function \
  --qualifier '$LATEST' --payload '{}' \
  --cli-binary-format raw-in-base64-out response.json

A synchronous invoke is limited to 15 minutes. For a longer execution, invoke with --invocation-type Event. The call then returns at once with status code 202 and the DurableExecutionArn, and the execution runs for up to its ExecutionTimeout. Poll it with the ARN. It reports RUNNING while the timer is pending and SUCCEEDED with the result afterwards.

aws lambda get-durable-execution \
  --durable-execution-arn <DurableExecutionArn> \
  --query '[Status,Result]' --output text

get-durable-execution-history lists the recorded events of an execution. For the handler above it shows the first step, the wait, the end of the first invocation, the completed wait, the second step, and the end of the second invocation.

The handler and the context

A durable handler is a function of two arguments that returns a value and an error.

type Order struct {
	ID string `json:"id"`
}

type Receipt struct {
	ChargeID string `json:"chargeId"`
}

func handler(ctx durable.Context, order Order) (Receipt, error) {
	return Receipt{ChargeID: "ch_" + order.ID}, nil
}

durable.Start(handler) registers the handler with the Lambda runtime and runs one execution per invocation. The event type and the output type are yours to choose. The SDK decodes the event and encodes the output with encoding/json. durable.Wrap returns a raw payload function instead of starting the runtime, for a program that composes its own Lambda entry point.

When the handler returns an error, the execution moves to the FAILED status. The service records the error's type and message, and get-durable-execution returns them in its Error field. A synchronous caller receives the same error object as the payload, with FunctionError set to Unhandled in the invocation metadata. The ErrorType of an error from errors.New or fmt.Errorf is Error. The ErrorType of any other error is the name of its Go type.

durable.Context is the handle to the execution. It exposes ExecutionArn(), RequestID(), InvokedFunctionARN(), Logger(), and IsReplaying(). IsReplaying reports whether the current invocation is replaying recorded results. Every operation takes the context as its first argument.

Determinism

Replay pairs recorded results with operations by position. So the handler must create the same operations in the same order on every invocation. Code between operations must depend only on the event and on results the SDK already recorded. Do not create operations while iterating a map, because Go randomizes map iteration order. Sort the keys into a slice first.

Put nondeterministic work inside a step body. A step body may read the clock, generate a random ID, or call a service. Only its recorded result takes part in replay. durable.StepContext, the argument a step body receives, is a context.Context with Logger() and Attempt(). It exposes no durable operations, so a step cannot create nested operations.

Goroutines

A durable.Context is owned by the goroutine that created it. Calling an operation on it from another goroutine fails with durable.ErrWrongGoroutine. That covers a go statement and an errgroup.Go callback alike. Two goroutines claiming operations on one context would claim them in a scheduling-dependent order, and replay would then pair recorded results with the wrong operations. The check runs in every default build. Build with -tags durablenocheck to remove it.

Use durable.Go to run durable work concurrently. It claims the child operation on the calling goroutine, which keeps the order deterministic, and then starts a goroutine that owns a fresh child context. Use the child context inside the function, never the parent.

The SDK enforces this. A body that runs on the calling goroutine, the body of a Step or of a blocking RunInChildContext, fails an operation on an enclosing context with durable.ErrWrongContext. A body that runs on its own goroutine, the body of Go, RunInChildContextAsync, a Map item, or a Parallel branch, fails the same call with durable.ErrWrongGoroutine, because that goroutine does not own the enclosing context. An operation on a child context kept from a body that already returned, such as a sibling's context, fails with durable.ErrWrongContext when it runs on that context's goroutine. A step body must not call an operation on a captured enclosing context either. Its own argument exposes no durable operations, so such a call fails with durable.ErrWrongContext. Either way the rejected operation claims no operation ID and records nothing. -tags durablenocheck removes both checks.

func handler(ctx durable.Context, _ any) (string, error) {
	fut := durable.Go(ctx, "work", func(child durable.Context) (string, error) {
		return durable.Step(child, "step", func(_ durable.StepContext) (string, error) {
			return "done", nil
		})
	})
	return fut.Result(ctx)
}

Future.Result blocks until the future settles. It returns the suspension signal only when the invocation suspends, which happens once every branch is blocked. Await several futures with a combinator, not with a sequence of Result calls. A sequence that returns on the first error leaves the other futures unawaited. The combinators await every future and record the combined outcome as one operation. The combinators section shows them.

Errors

Return an error from an operation unchanged unless the handler treats it as a business outcome. A non-nil error is one of two things. Either it is a terminal failure of that operation, such as *durable.StepError, or it is the signal that the invocation is suspending. A terminal failure matches its public type with errors.As. The suspension signal matches no public type. So an error that matches no public type must be returned as it is.

A typed failure does not carry the error value the operation body returned. It carries the recorded ErrorType and Message, and the SDK rebuilds it the same way on the first invocation and on replay. So match on ErrorType, not with errors.As against your own error types.

type CardDeclinedError struct{}

func (CardDeclinedError) Error() string { return "card declined" }

func handler(ctx durable.Context, _ any) (string, error) {
	receipt, err := durable.Step(ctx, "charge", func(_ durable.StepContext) (string, error) {
		return "", CardDeclinedError{}
	}, durable.WithRetry(durable.NoRetry()))
	var stepErr *durable.StepError
	switch {
	case err == nil:
		return receipt, nil
	case errors.As(err, &stepErr) && stepErr.ErrorType == "CardDeclinedError":
		return "declined", nil
	default:
		return "", err
	}
}

Operations

Operation	Description
Step	Run a function with automatic checkpointing and configurable retry.
StepAsync	Asynchronous Step returning a *Future.
Wait	Pause execution for a duration without blocking Lambda.
WaitAsync	Asynchronous Wait returning a *Future.
Invoke	Call another durable function and wait for its result.
InvokeAsync	Asynchronous Invoke returning a *Future.
RunInChildContext	Execute a subflow in an isolated child context.
RunInChildContextAsync	Asynchronous RunInChildContext returning a *Future.
Go	Launch a replay-safe concurrent subflow (shorthand for RunInChildContextAsync).
WaitForCondition	Poll a check function until a condition is met or a strategy stops.
CreateCallback	Register a callback and wait for external resolution.
WaitForCallback	Create a callback, invoke a submitter with the callback ID, and wait.
Map	Fan out a function over items with configurable concurrency and completion.
Parallel	Run named branches concurrently with configurable completion.
ParallelMixed	Run named branches of different result types concurrently with configurable completion.
All	Wait for all futures to succeed. Returns results or the first error.
AllSettled	Wait for all futures to settle. Returns all outcomes.
Any	Return the first future to succeed. Errors if all fail.
Race	Return the result of the first future to settle.
Join	Wait for futures of different result types to settle. Returns the first error in argument order.
Select	Run named branches concurrently. Return the first to settle along with its name.
Retry	Retry a function containing durable operations as a unit, suspending between attempts.

Every operation takes a name as its second argument. The name identifies the operation in the execution history. Pass "" for an unnamed operation.

Step

A step is the unit of checkpointing. The SDK records its result with encoding/json, so the result type must round-trip through JSON. Once the service records the result, replay returns it without running the body again.

func handler(ctx durable.Context, _ any) (string, error) {
	return durable.Step(ctx, "fetch-name", func(sc durable.StepContext) (string, error) {
		// Nondeterministic work belongs here. sc is a context.Context, so
		// pass it to AWS SDK calls made inside the step.
		return "world", nil
	})
}

Between the moment the SDK starts a step and the moment the service records the outcome, the function can time out or the runtime can crash. Under the default AtLeastOncePerRetry semantics the SDK runs the body again on resume. So the body may run more than once for one attempt, and it should be idempotent. AtMostOncePerRetry treats the interrupted attempt as a failure instead and consults the retry strategy.

Under AtLeastOncePerRetry the body runs without waiting for the step's START to be recorded. The step still waits for its SUCCEED or FAIL to be recorded before it returns. Under AtMostOncePerRetry the SDK waits for the START to be recorded before the body runs.

WithRetry sets the retry strategy. The default is ExponentialBackoff(), which makes 6 attempts in total, starting 5 seconds apart, doubling each time, capped at 60 seconds, with full jitter. NoRetry() fails on the first error. NewRetryStrategy builds a strategy from a RetryConfig, and LinearBackoff builds one with a constant increment. When the next attempt becomes due while other work of the handler is still running, it runs in the same invocation. When nothing else can make progress, the invocation ends for the delay, so a retrying step does not hold the invocation open. StepContext.Attempt() is the 1-based attempt number.

func handler(ctx durable.Context, _ any) (string, error) {
	strategy, err := durable.NewRetryStrategy(durable.RetryConfig{
		MaxAttempts:  4,
		InitialDelay: time.Second,
		BackoffRate:  2,
	})
	if err != nil {
		return "", err
	}
	return durable.Step(ctx, "flaky-call", func(sc durable.StepContext) (string, error) {
		if sc.Attempt() < 3 {
			return "", fmt.Errorf("transient failure on attempt %d", sc.Attempt())
		}
		return fmt.Sprintf("succeeded on attempt %d", sc.Attempt()), nil
	},
		durable.WithRetry(strategy),
		durable.WithSemantics(durable.AtMostOncePerRetry),
	)
}

When the strategy stops, Step returns a *durable.StepError with the attempt count and the recorded ErrorType and Message of the last attempt.

WithStepSubType sets the subtype a step records in its checkpoints and reports to plugins, in place of Step, so the history can tell one kind of step from another. WithCallbackSubType does the same for a callback. Replay compares the subtype with the checkpoint, so changing it breaks the executions in flight.

Wait

Wait pauses the execution for a duration. The invocation ends only when no other work of the handler can make progress, and the service invokes the function again when the duration elapses. A wait that elapses while other work runs returns in the same invocation. On replay a completed wait returns at once.

The duration must be at least one second. For a shorter duration, Wait returns an error at the call and records no operation, and WaitAsync returns a future that fails with that error. A duration of one second or more is rounded up to whole seconds, so 1.5 seconds waits 2 seconds.

func handler(ctx durable.Context, _ any) (string, error) {
	if err := durable.Wait(ctx, "cooldown", 60*time.Second); err != nil {
		return "", err
	}
	return "waited", nil
}

Return the error of Wait in every case. It never carries a business outcome.

Invoke

Invoke starts another durable function as its own execution and returns its result. When the invoked function completes while other work of the calling handler is still running, Invoke returns in the same invocation. When nothing else can make progress, the calling invocation ends and the execution resumes when the invoked function completes. The output type parameter comes first, so it can be written out while the input type is inferred. The target needs a version or alias qualifier, such as :$LATEST. If the invoked function fails, Invoke returns a *durable.InvokeError.

func handler(ctx durable.Context, orderID string) (string, error) {
	return durable.Invoke[string](ctx, "charge", "payments-function:$LATEST", orderID)
}

WaitForCondition

WaitForCondition runs a check repeatedly. The check receives the state from the previous attempt and returns the new state. The SDK checkpoints the state between attempts and waits for the delay the wait strategy returns. When the next check becomes due while other work of the handler is still running, it runs in the same invocation. When nothing else can make progress, the invocation ends for the delay. The strategy receives the new state and the attempt number. It returns Continue: true with a Delay to poll again, Continue: false to stop with the state, or an Err to fail the operation.

func handler(ctx durable.Context, _ any) (int, error) {
	return durable.WaitForCondition(ctx, "poll-until-ready",
		func(_ durable.StepContext, state int) (int, error) {
			return state + 1, nil
		},
		durable.ConditionConfig[int]{
			InitialState: 0,
			WaitStrategy: func(state int, attempt int) durable.WaitDecision {
				if state >= 3 {
					return durable.WaitDecision{Continue: false}
				}
				return durable.WaitDecision{Continue: true, Delay: time.Second}
			},
		})
}

The service schedules each check in whole seconds. A Delay of 0 is raised to one second. A positive Delay is rounded up to whole seconds, so a positive Delay under one second waits one second. A negative Delay fails WaitForCondition with an error.

NewWaitStrategy builds a strategy from a WaitConfig with a ShouldContinue predicate, an attempt cap, and exponential backoff. WaitStrategy is required. When it is nil, WaitForCondition returns an error at the call, runs no check, and records no operation.

Callbacks

CreateCallback creates a callback and returns it at once. The callback exposes ID(), the identifier to hand to an external system, and Result(ctx), which blocks until that system completes it. The external system completes a callback with the SendDurableExecutionCallbackSuccess or SendDurableExecutionCallbackFailure API. The result is a JSON document. From the CLI, the command below completes the callback with the JSON string "approved". The callback ID also appears in the CallbackStarted event of the execution history.

aws lambda send-durable-execution-callback-success \
  --callback-id <ID> --result '"approved"' \
  --cli-binary-format raw-in-base64-out

func handler(ctx durable.Context, _ any) (string, error) {
	cb, err := durable.CreateCallback[string](ctx, "approval",
		durable.WithCallbackTimeout(time.Hour))
	if err != nil {
		return "", err
	}
	ctx.Logger().Info("awaiting approval", "callbackId", cb.ID())
	return cb.Result(ctx)
}

By default Result returns the submitted bytes unchanged, as the other durable execution SDKs do. The default is durable.RawSerdes, which supports a result type of string, []byte, or json.RawMessage. So the handler above returns "approved" with its quotes, and a submitted 42 returns the string 42. To decode the submitted bytes as JSON into the result type, pass durable.WithCallbackSerdes(durable.JSONSerdes) to the operation, or set durable.WithCallbackDeserializer for the whole handler. WithSerdes does not apply to callback results.

WaitForCallback combines the two halves into one operation. It creates the callback, runs the submitter with the callback ID, and blocks until the result arrives. The submitter runs as a step, so it may call a service, and WithSubmitterRetry retries it with a RetryStrategy. The callback and the WaitForCallback operation both record the submitted bytes unchanged. The result serializer runs once, at the WaitForCallback operation, with the same precedence as for CreateCallback.

func handler(ctx durable.Context, _ any) (string, error) {
	return durable.WaitForCallback[string](ctx, "approval",
		func(sc durable.StepContext, callbackID string) error {
			// Hand callbackID to the approver here, for example by
			// publishing it to a queue.
			sc.Logger().Info("approval requested", "callbackId", callbackID)
			return nil
		},
		durable.WithCallbackTimeout(time.Hour))
}

Both operations take WithCallbackTimeout and WithCallbackHeartbeatTimeout. A timeout fails the operation with a *durable.CallbackTimeoutError. A failure sent by the external system fails it with a *durable.CallbackExternalError.

Child contexts

RunInChildContext runs a function with its own context. The operations inside it are recorded under the child, and the child's overall result is checkpointed. On replay of a completed child, the SDK returns the recorded result without running the function. A result whose serialized form exceeds 256 KiB is not stored. Replay runs the function again and passes the new result through the serdes, as the first run did, so replay returns the same value as the first run.

func handler(ctx durable.Context, _ any) (string, error) {
	return durable.RunInChildContext(ctx, "greet", func(child durable.Context) (string, error) {
		name, err := durable.Step(child, "fetch-name", func(_ durable.StepContext) (string, error) {
			return "world", nil
		})
		if err != nil {
			return "", err
		}
		return "hello, " + name, nil
	})
}

Retry

Retry retries a function that contains durable operations, as one unit. Each attempt runs in its own child context, so a failed attempt's operations are never replayed into the next attempt. The delay between attempts is a Wait, so no compute is consumed while waiting. When the strategy stops, Retry returns a *durable.RetryError with the attempt count and the last attempt's error. The function receives the 1-based attempt number.

func handler(ctx durable.Context, _ any) (string, error) {
	return durable.Retry(ctx, "quote-and-book", func(c durable.Context, attempt int) (string, error) {
		quote, err := durable.Step(c, "fetch-quote", func(_ durable.StepContext) (float64, error) {
			if attempt < 2 {
				return 0, errors.New("quote service unavailable")
			}
			return 104.50, nil
		}, durable.WithRetry(durable.NoRetry()))
		if err != nil {
			return "", err
		}
		return durable.Step(c, "book", func(_ durable.StepContext) (string, error) {
			return fmt.Sprintf("booked at %.2f", quote), nil
		})
	}, durable.ExponentialBackoff())
}

Map and Parallel

Map applies one function to every item. Each item runs in its own child context and receives its index. Parallel runs a fixed set of named branches that share an output type. Both return a BatchResult and accept WithMaxConcurrency and WithCompletion.

func handler(ctx durable.Context, items []string) ([]string, error) {
	results, err := durable.Map(ctx, "process-all", items,
		func(c durable.Context, item string, index int) (string, error) {
			return durable.Step(c, "process", func(_ durable.StepContext) (string, error) {
				return fmt.Sprintf("item-%d:%s", index, item), nil
			})
		},
		durable.WithMaxConcurrency(4))
	if err != nil {
		return nil, err
	}
	return results.Results(), nil
}

func handler(ctx durable.Context, _ any) ([]int, error) {
	results, err := durable.Parallel(ctx, "fan-out", []durable.Branch[int]{
		{Name: "double", Func: func(c durable.Context) (int, error) {
			return durable.Step(c, "compute", func(_ durable.StepContext) (int, error) {
				return 42, nil
			})
		}},
		{Name: "wait-then-value", Func: func(c durable.Context) (int, error) {
			if err := durable.Wait(c, "cooldown", time.Second); err != nil {
				return 0, err
			}
			return 100, nil
		}},
	})
	if err != nil {
		return nil, err
	}
	return results.Results(), nil
}

The default completion policy is fail-fast. The first item failure completes the batch, and Map or Parallel returns a *durable.BatchError carrying the per-item errors. A CompletionConfig changes that. MinSuccessful completes the batch once that many items succeed. ToleratedFailureCount and ToleratedFailurePercentage let the batch continue past failures up to a limit. ShouldComplete decides programmatically from a BatchProgress snapshot. The SDK calls it once before the first item starts, with every count at 0, and again after each item finishes. A ShouldComplete that completes the batch at that first call runs no item and returns an empty result. Items still running when the batch completes early are reported with the status BatchItemStarted.

Map and Parallel return an error only when the batch fails as a unit. That is a breached tolerance, with the reason FAILURE_TOLERANCE_EXCEEDED, or a ShouldComplete that completes the batch as failed, with the reason CUSTOM_COMPLETION_FAILED. The error is a *durable.BatchError carrying the per-item errors when an item failed. A ShouldComplete that fails the batch with no failed item returns a *durable.BatchCompletionError instead, recorded with the ErrorType BatchCompletionError. A failure within a tolerance returns the populated result and a nil error, so if err != nil { return err } keeps the tolerance. Read the failed items from Failed() and Errors(). MIN_SUCCESSFUL_REACHED means the threshold completed the batch with items still unstarted or in flight. When every item ran, the reason is ALL_COMPLETED.

BatchResult reports each item's status through Items, Succeeded(), Failed(), and Started(), the values through Results(), the per-item errors through Errors(), and why the batch ended through Reason.

A batch records its result in the same format as the other durable execution SDKs. A failed item's error is recorded as its type, message, and the chain of errors it wraps. The SDK rebuilds each item error from that record, on the first run and on replay alike. An SDK error type in the chain, such as *durable.StepError, is rebuilt as that type, so errors.As finds it. Its detail fields that the record does not hold, such as StepError.Attempts and CallbackError.CallbackID, are zero. Match an error of your own type on the recorded ErrorType.

Each item has a name. A Map item takes its name from WithItemNamer, and a Parallel branch from Branch.Name. A Map item with no name, or whose namer returns "", is named map-item-<index>, for example map-item-0. A Parallel branch with an empty Name is named parallel-branch-<index>. The name is the name of the item's recorded child context and of the operationName log field inside it. It is also the key that BatchResult.Item and BatchResult.Result look up, so results.Result("map-item-1") returns the value of the second item.

ParallelMixed

ParallelMixed runs branches of different result types concurrently. NewTypedBranch builds a *TypedBranch[T] from a name and a body that returns a T. ParallelMixed takes the branches as a []durable.AnyBranch and returns a BatchResult[json.RawMessage] and an error. TypedBranch.Result reads the branch value from that result as a T.

type Profile struct {
	Name string `json:"name"`
}

func handler(ctx durable.Context, _ any) (string, error) {
	profile := durable.NewTypedBranch("profile", func(c durable.Context) (Profile, error) {
		return durable.Step(c, "load-profile", func(_ durable.StepContext) (Profile, error) {
			return Profile{Name: "Ana"}, nil
		})
	})
	orders := durable.NewTypedBranch("orders", func(c durable.Context) (int, error) {
		return durable.Step(c, "count-orders", func(_ durable.StepContext) (int, error) {
			return 3, nil
		})
	})
	res, err := durable.ParallelMixed(ctx, "load", []durable.AnyBranch{profile, orders},
		durable.WithMaxConcurrency(2))
	if err != nil {
		return "", err
	}
	p, err := profile.Result(res)
	if err != nil {
		return "", err
	}
	n, err := orders.Result(res)
	if err != nil {
		return "", err
	}
	return fmt.Sprintf("%s has %d orders", p.Name, n), nil
}

Every BatchOption applies as in Parallel, and completion and failure follow the Parallel rules above. A ParallelMixed call records the same operations and stores the same payloads as a Parallel over the branch values. Each branch value is marshaled once, by the serdes set with WithTypedBranchSerdes or else by the batch item serdes, and the SDK stores those bytes unchanged. Each item of the result holds its branch's stored bytes in Result, and TypedBranch.Result decodes them into the branch type.

TypedBranch.Result returns the value of a succeeded branch and the item error of a failed branch. For a branch that never started, or that was abandoned when the batch completed early, it returns a *durable.BranchNotCompletedError. It returns an error when the result came from a different ParallelMixed call, or when the branch was never passed to one. WithBatchSummary under ParallelMixed receives BatchResult[json.RawMessage], so its function has the type func(durable.BatchResult[json.RawMessage]) string.

Combinators

StepAsync, WaitAsync, InvokeAsync, RunInChildContextAsync, and Go start an operation and return a *Future. The combinators take futures and record the combined outcome as one operation.

All waits for every future to succeed and returns the values in input order. It fails on the first error.

func handler(ctx durable.Context, _ any) ([]int, error) {
	a := durable.StepAsync(ctx, "a", func(_ durable.StepContext) (int, error) { return 1, nil })
	b := durable.StepAsync(ctx, "b", func(_ durable.StepContext) (int, error) { return 2, nil })
	c := durable.StepAsync(ctx, "c", func(_ durable.StepContext) (int, error) { return 3, nil })
	return durable.All(ctx, "gather", []*durable.Future[int]{a, b, c})
}

AllSettled waits for every future and returns a Settled per future, with either a Value or an Err. It never fails fast.

func handler(ctx durable.Context, _ any) (int, error) {
	good := durable.StepAsync(ctx, "good", func(_ durable.StepContext) (int, error) { return 1, nil })
	bad := durable.StepAsync(ctx, "bad", func(_ durable.StepContext) (int, error) {
		return 0, errors.New("boom")
	}, durable.WithRetry(durable.NoRetry()))
	settled, err := durable.AllSettled(ctx, "collect", []*durable.Future[int]{good, bad})
	if err != nil {
		return 0, err
	}
	failures := 0
	for _, s := range settled {
		if s.Err != nil {
			failures++
		}
	}
	return failures, nil
}

Any returns the value of the first future to succeed, and fails when every future fails. Race returns the outcome of the first future to settle, success or failure. A failure of All, Any, Race, Join, or Select is a *durable.ChildContextError whose ErrorType is "PromiseCombinatorError" and whose cause matches *durable.CombinatorError. Its message is the first error's message, or All promises were rejected for Any. AllSettled does not fail for a failing future. The future's error is in Settled[i].Err. Race and Any with no futures return an error at once.

func handler(ctx durable.Context, _ any) (int, error) {
	primary := durable.StepAsync(ctx, "primary", func(_ durable.StepContext) (int, error) {
		return 0, errors.New("unavailable")
	}, durable.WithRetry(durable.NoRetry()))
	fallback := durable.StepAsync(ctx, "fallback", func(_ durable.StepContext) (int, error) { return 42, nil })
	return durable.Any(ctx, "first-ok", []*durable.Future[int]{primary, fallback})
}

Join takes futures of different result types through the Awaitable interface. It waits for all of them and returns the first error in argument order. Read the values afterwards with Result, which returns at once after Join returned nil.

func handler(ctx durable.Context, _ any) (string, error) {
	charge := durable.StepAsync(ctx, "charge", func(_ durable.StepContext) (string, error) {
		return "ch_123", nil
	})
	reserve := durable.Go(ctx, "reserve", func(c durable.Context) (int, error) {
		return durable.Step(c, "reserve-items", func(_ durable.StepContext) (int, error) {
			return 3, nil
		})
	})
	if err := durable.Join(ctx, "settle", []durable.Awaitable{charge, reserve}); err != nil {
		return "", err
	}
	receipt, _ := charge.Result(ctx)
	reserved, _ := reserve.Result(ctx)
	return fmt.Sprintf("%s reserved %d items", receipt, reserved), nil
}

Select runs named branches and returns the name and value of the first branch to settle. Use it in place of Race when the caller must know which branch won.

func handler(ctx durable.Context, _ any) (string, error) {
	winner, quote, err := durable.Select(ctx, "quote", []durable.Branch[float64]{
		{Name: "primary", Func: func(c durable.Context) (float64, error) {
			return durable.Step(c, "primary-quote", func(_ durable.StepContext) (float64, error) {
				time.Sleep(2 * time.Second)
				return 100.00, nil
			})
		}},
		{Name: "fallback", Func: func(c durable.Context) (float64, error) {
			return durable.Step(c, "fallback-quote", func(_ durable.StepContext) (float64, error) {
				return 104.50, nil
			})
		}},
	})
	if err != nil {
		return "", err
	}
	return fmt.Sprintf("%s quoted %.2f", winner, quote), nil
}

Serialization

The SDK stores operation results as JSON by default. The stored JSON keeps <, > and & literal. WithSerdes sets a different durable.Serdes for the whole handler at construction time, and ConfigureSerdes changes it from inside the handler. Per-operation options such as WithStepSerdes, WithChildSerdes, and WithBatchSerdes override it for one operation. The Serdes interface is untyped. Marshal receives the value as any and Unmarshal fills a pointer passed as any, so one value can serve every result type in a handler. SerdesOf builds a Serdes for one result type from typed marshal and unmarshal functions.

NewFileSystemSerdes stores payloads as files under a base path and records a reference in the checkpoint. In the default FileSystemSerdesModeAlways every value goes to a file. In FileSystemSerdesModeOverflow a value goes to a file only when it exceeds 255 KB. Point the base path at a durable mount such as Amazon EFS, not at the ephemeral /tmp of the function.

A serdes failure is permanent by default. The operation fails with a *durable.SerdesError, which names the operation and the Direction, "marshal" or "unmarshal". A step fails with a *durable.StepError whose ErrorType is "SerdesError". The step retry strategy is not consulted, so the step body runs once. The caller may catch the error. When the handler returns it, the execution settles FAILED. A handler event that does not decode, or a handler result that does not encode, fails the execution the same way, with a SerdesError whose Operation is "execution". To mark a failure transient, return durable.RetryableSerdesError(err) from Marshal or Unmarshal. The SDK then records no outcome for the operation and ends only the current invocation, even when the handler catches the error, and the service invokes the execution again from its last checkpoint.

func handler(ctx durable.Context, _ any) (string, error) {
	upper := durable.SerdesOf(
		func(_ context.Context, _ durable.SerdesContext, v string) ([]byte, error) {
			return []byte(strings.ToUpper(v)), nil
		},
		func(_ context.Context, _ durable.SerdesContext, data []byte) (string, error) {
			return strings.ToLower(string(data)), nil
		},
	)
	return durable.Step(ctx, "shout", func(_ durable.StepContext) (string, error) {
		return "hello", nil
	}, durable.WithStepSerdes(upper))
}

Testing a handler locally

The durable/durabletest package runs a handler in process, without network access or AWS credentials. RunUntilComplete invokes the handler as many times as the execution needs. Between invocations it completes pending timers and step retries, and it returns when the execution finishes or blocks on external action.

The runner checks each checkpoint against the service's limits and rejects the same updates the service rejects, with the service's error code and message. Examples are a step result over 262144 bytes and an Invoke input over 1048576 bytes. It also fails an execution whose handler answers PENDING four times in a row with no pending operation, as the service does. It reports an operation's completion during an invocation the same way the service does. A wait or step retry that becomes due while other work runs, a callback resolved while the handler runs, and an invoke of a registered function are reported in a checkpoint response, so the handler continues in the same invocation.

func TestHandler(t *testing.T) {
	handler := func(ctx durable.Context, n int) (int, error) {
		if err := durable.Wait(ctx, "cooldown", time.Hour); err != nil {
			return 0, err
		}
		return durable.Step(ctx, "add", func(_ durable.StepContext) (int, error) {
			return n + 1, nil
		})
	}

	runner := durabletest.NewLocalRunner(handler)
	result, err := runner.RunUntilComplete(41)
	if err != nil {
		t.Fatal(err)
	}

	if result.Status != durabletest.Succeeded {
		t.Fatalf("status = %s, want SUCCEEDED", result.Status)
	}
	got, err := durabletest.ResultAs[int](result)
	if err != nil {
		t.Fatal(err)
	}
	if got != 42 {
		t.Fatalf("result = %d, want 42", got)
	}
}

The runner methods return an error only when the runner itself fails, for example when the event does not marshal to JSON. The handler's outcome is in the result. A handler that returns an error produces a nil error and a result with Status set to Failed and the recorded error in Error. A run blocked on a callback or an invoke has Status set to Pending, and a run that reaches the invocation cap has CapReached set. ResultAs returns an error for any result that did not succeed, and for a failed handler its message includes the recorded error type and message.

The runners take no test handle, so the same calls work in a plain main package, for example a program you start with go run during local development.

func main() {
	handler := func(ctx durable.Context, name string) (string, error) {
		return durable.Step(ctx, "greet", func(_ durable.StepContext) (string, error) {
			return "hello, " + name, nil
		})
	}

	result, err := durabletest.NewLocalRunner(handler).RunUntilComplete("world")
	if err != nil {
		log.Fatal(err) // the runner failed
	}
	out, err := durabletest.ResultAs[string](result)
	if err != nil {
		log.Fatal(err) // the handler failed, is pending, or the result did not decode
	}
	fmt.Println(out)
}

TestResult also lists every recorded operation in Operations, so a test can assert that a step ran once across replays. The operation records also carry the scheduled end and duration of a wait, the time of a step's next retry, and the error data and stack trace of a failure. TestResult.Error carries the error data and stack trace of a failed execution. SendCallbackSuccess, SendCallbackFailure, and SendCallbackHeartbeat resolve a callback the handler is blocked on. CompleteChainedInvoke and FailChainedInvoke resolve an Invoke. RegisterFunction registers a handler for a function name so that Invoke runs it in process. NewCloudRunner runs the same assertions against a deployed function. Its Run and RunWithArn methods take a context.Context, which bounds the invoke and every poll. The assertion helpers, such as AssertGoldenSignature, take a testing.TB, so benchmarks and fuzz tests can call them.

Logging

Context.Logger() and StepContext.Logger() return a *slog.Logger. By default it writes one JSON object per record to stderr with the field names the other durable execution SDKs use, so one CloudWatch query works across languages.

Field	Present
timestamp, level, message	Always.
requestId, executionArn	Always.
tenantId	When the invocation has one.
operationId, operationName	Inside a child context (RunInChildContext, Go, Map, Parallel, WaitForCallback) or an operation body. operationName only when the operation is named.
attempt	Inside a step body, condition check, or callback submitter.
replay	Only on a record emitted while its context replays, and only under ReplayLogModeEmit (see below). Always true when present.

A field that does not apply in a scope is omitted, never emitted empty. Each scope carries its own identifiers only. A step inside a child context reports the step, not the child.

To use your own logging library, pass its slog.Handler to WithLogHandler. The SDK attaches the fields above through the handler's WithAttrs method as structured attributes, wraps the handler with replay suppression, and adds the fields a plugin returns from Plugin.EnrichLogContext as record attributes. When two sources use the same key, the SDK's own fields win, then the attributes you attach with the record or with Logger.With, then plugin fields. When several plugins return the same key, the plugin registered last wins. Keys are compared by qualified path, so a plugin field under an open slog group collides only with your attributes at that same path.

To choose the handler from inside the handler body, for example from the event payload, call ConfigureLogging. It replaces the handler for the rest of the invocation, on the calling context and on every child context and branch derived from it after the call. The SDK attaches the same fields to the new handler. The change lasts for the current invocation only and does not affect checkpoints or operation ordering, so it is safe to call conditionally.

type Event struct {
	Debug bool `json:"debug"`
}

func handler(ctx durable.Context, event Event) (string, error) {
	if event.Debug {
		if err := durable.ConfigureLogging(ctx, durable.LogConfig{
			Handler: slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelDebug}),
		}); err != nil {
			return "", err
		}
	}
	return "configured", nil
}

Replayed log records

Every line is logged exactly once, in the invocation that first runs it. Code that ran in an earlier invocation is replaying, and its log records are dropped. Code that runs for the first time is live, and its records are written. Context.IsReplaying reports which of the two the code is in.

A context starts an invocation replaying when the checkpoint log holds operations for it. It becomes live, and stays live, at the first of these points.

1. It starts an operation that has no checkpoint.
2. Its code receives the outcome of an operation that the previous invocation did not have. That is an operation that changed while the execution was suspended, or one that completed during this invocation. The code receives an outcome when a blocking operation returns, when Future.Result or Callback.Result returns, or when a combinator, Map, Parallel, or WaitForCallback returns.

An invocation suspends only while an operation the code awaits is still pending. That operation completes while the execution is suspended. So in the next invocation, the code before the await replays, and the code after it is live. Starting an asynchronous operation such as CreateCallback or Go does not make a context live, and neither does the suspension signal.

Result takes the context of the code that reads the outcome. Inside a Go branch, pass the branch's context, even for a future the parent created, so that the branch's own lines are logged.

One case logs a line twice. When an invocation fails, for example by crashing or timing out, the SDK cannot know which lines it wrote. The next invocation logs again the lines the failed invocation ran after it received an outcome. A step body can also run more than once under AtLeastOncePerRetry, and the lines of its StepContext.Logger repeat with the body.

Replay is decided per context. A Go branch that is still replaying stays quiet while a sibling that has reached live execution logs normally.

To see the records of the replayed portion when diagnosing a replay problem, select ReplayLogModeEmit with WithReplayLogMode at construction or with ConfigureLogging inside the handler. Replayed records are then emitted with the field replay set to true. Live records carry no replay field. Expect duplicate lines. Every line written before a suspension appears again, with replay set to true, on each later invocation that replays it. ReplayLogModeSuppress is the default. The top-level replay key belongs to the SDK in every mode. A value you attach under that name with Logger.With or pass with a record is dropped, while the same name inside a group opened with WithGroup is kept. At construction the option is durable.Start(handler, durable.WithReplayLogMode(durable.ReplayLogModeEmit)).

Records the SDK writes

A checkpoint response can withdraw the checkpoint token, which means the service accepts no further checkpoints from the current invocation. When that suspends the invocation, the SDK writes one WARN record through the handler's log handler, with the requestId and executionArn fields. Its message is Checkpoint response contained no CheckpointToken: the service will accept no further checkpoints from this invocation. Suspending; the execution continues on the next invocation. The JavaScript SDK writes the same message, so one CloudWatch query finds it for both. Replay suppression never drops this record, because it reports the state of the invocation. The execution continues on the next invocation. When the withdrawn token answers the checkpoint of the execution's final result, the execution has finished, and the SDK writes no record.

The SDK also traces its own work at slog.LevelDebug. A handler enabled at Debug turns the trace on, whether you pass it to WithLogHandler or to ConfigureLogging. No environment variable is involved. Each record carries requestId, executionArn, and tenantId when the invocation has one.

Message	When	Fields
operation claimed	The SDK claims an operation ID.	operationId, operationSubtype, operationName when named.
replay complete; executing live	A context leaves replay.	operationId and operationName of the child operation, inside a child context.
checkpoint enqueued	The SDK queues a checkpoint request.	updateCount, the number of operation updates.
checkpoint flushed	The service accepts a checkpoint call.	updateCount.
invocation suspending	The invocation ends because an operation is pending.	reason: wait, callback, invoke, retry, condition, or combinator.
operation completed	An operation reaches a terminal state.	operationId, operationSubtype, operationName when named, status (SUCCEEDED or FAILED).

operationSubtype is the subtype the operation records, such as Step, Wait, Callback, ChainedInvoke, RunInChildContext, WaitForCallback, WaitForCondition, Map, MapIteration, Parallel, or ParallelBranch. A trace record written while its context replays follows the replay log mode. Under ReplayLogModeSuppress it is dropped, and under ReplayLogModeEmit it carries replay set to true. An operation completed record is replayed when an earlier invocation already observed the outcome, and live when this invocation is the first to observe it, whatever the replay state of its context. The invocation suspending record, and a checkpoint record for a request the invocation makes itself, report the state of the invocation and are never dropped.

Plugin API

The plugin instrumentation API is experimental. Any part of it may change or be removed in a minor release, as Versioning describes. docs/release-notes.md records each change.

The plugin instrumentation API gives observability and tracing integrations hooks into the lifecycle of an execution. A durable.Plugin is a struct of optional hook functions. Set the ones you need and register the plugin with durable.WithPlugins.

func handler(ctx durable.Context, _ any) (string, error) {
	return durable.Step(ctx, "work", func(_ durable.StepContext) (string, error) {
		return "done", nil
	})
}

func main() {
	tracer := durable.Plugin{
		OnOperationStart: func(_ context.Context, info durable.OperationHookInfo) {
			log.Printf("start %s %s replay=%v", info.Type, info.Name, info.IsReplay)
		},
		WrapOperationAttemptFn: func(ctx context.Context, info durable.AttemptHookInfo, fn func(context.Context) (any, error)) (any, error) {
			start := time.Now()
			defer func() { log.Printf("%s took %s", info.Name, time.Since(start)) }()
			return fn(ctx)
		},
	}
	durable.Start(handler, durable.WithPlugins(tracer))
}

The hooks cover the invocation (OnInvocationStart, OnInvocationEnd, OnOperationChange, WrapInvocation), each operation (OnOperationStart, OnOperationEnd), each attempt of a retryable operation (OnOperationAttemptStart, OnOperationAttemptEnd, WrapOperationAttemptFn), child contexts (WrapChildContextFn), and log records (EnrichLogContext). The wrap hooks receive the wrapped work as a function and may pass it a derived context, which becomes the parent of the context the user code observes. WithPluginChildOperationsDepth bounds how deep in the operation tree hooks are reported.

Each hook has a defined firing time, replay behavior, order relative to the other hooks, and dispatch goroutine. Run go doc github.com/aws/aws-durable-execution-sdk-go/durable Plugin to read them per hook. With one plugin registered a notification hook runs on the goroutine that dispatches the event. With several, each plugin's hook runs on a goroutine the dispatch joins, so the plugins' hooks for one event run in parallel. Hooks of concurrent operations run in parallel too, so a plugin must be safe for concurrent use. Construct Plugin and the hook info types with keyed fields, because fields may be added, and tolerate status values you do not know. The insight module is a complete plugin built on this API.

Deliberate differences from the JavaScript SDK

The behaviors below differ from the JavaScript SDK on purpose. Keep them in mind when you port a handler.

1. The zero value of MaxAttempts selects the default: 3 for RetryConfig, 6 for LinearRetryConfig, and 60 for WaitConfig. It does not mean one attempt. A JavaScript maxAttempts: 0 or maxAttempts: 1 ports to durable.NoRetry() for a retry strategy, and to MaxAttempts: 1 for a WaitConfig.
2. LinearRetryConfig.Jitter defaults to JitterNone, so the default linear delays are exactly 1 s, 2 s, 3 s, 4 s, and 5 s. RetryConfig and WaitConfig default to JitterFull. Both defaults match the JavaScript SDK's presets.
3. Map and Parallel return a plain error for an invalid option: a non-positive WithMaxConcurrency, an invalid WithCompletion, or a failing summary function. The JavaScript SDK ends the execution for the same input.
4. Start and Wrap panic on an invalid handler option, because handler options are read once at program start. The JavaScript SDK does not validate handler options at construction.
5. A WaitForCondition check-function failure returns a *durable.WaitForConditionError in Go, and a StepError in the JavaScript SDK. Python returns a WaitForConditionError, as Go does.
6. All returns the values in input order, and Race returns the outcome of the first future to settle without saying which future won. Both match Promise.all and Promise.race. Use Select when the caller needs the winner's name.
7. Retry returns an error for an invalid argument: a nil function, a nil strategy, or a Context the SDK did not create. The JavaScript SDK throws a TypeError for the same input.
8. A child context waits for its SUCCEED or FAIL to be recorded before it returns. The JavaScript SDK does not wait. The outcome of a combinator, Select, Map, or Parallel can depend on timing, so it must be recorded before later code acts on it.

Examples

examples/ holds one deployable function per pattern. examples/README.md lists them by operation with the terminal status each one reaches. Each is a Lambda function on provided.al2023. examples/build.sh builds them and the SAM template in that directory deploys them.

Versioning

The repository publishes three modules, each with its own version tags.

• The SDK, github.com/aws/aws-durable-execution-sdk-go, holds the durable and durable/durabletest packages. Its tags are vX.Y.Z.
• The insight plugin, github.com/aws/aws-durable-execution-sdk-go/insight, has tags insight/vX.Y.Z.
• The durablelint analyzer, github.com/aws/aws-durable-execution-sdk-go/analysis, has tags analysis/vX.Y.Z.

The SDK follows semantic versioning. Code written against a v1 release keeps compiling and working as documented with every later v1 release. A minor release may add fields to exported structs, so construct structs with keyed fields, as the Go 1 compatibility promise advises.

The plugin API is the one exception. It is experimental, and any part of it may change or be removed in a minor release. It comprises durable.Plugin, durable.WithPlugins, durable.WithPluginChildOperationsDepth, the hook info types, and the plugin status and outcome types and constants. Each of these carries an EXPERIMENTAL marker in its documentation.

The insight and analysis modules stay below v1. So any release of either module may make incompatible changes, and for insight that includes its record schema. Each insight release requires the SDK version it was released with. A later SDK release may change the plugin API in a way that an earlier insight release does not support, so upgrade insight together with the SDK.

docs/release-notes.md records the changes in each release, including every change to the plugin API.

Go versions

The SDK supports the Go releases that the Go project supports. The most recent Go release to leave that support stays supported for six more months. The AWS SDK for Go v2 follows the same policy. A minor release may raise the minimum Go version.

The go line of each module's go.mod names the oldest Go release that builds the module. CI tests that release and the two newest Go releases.

Feedback & Support

• Bug report
• Feature request

Contributing

We welcome contributions and feedback. Please open an issue before submitting a pull request so we can discuss the change. See CONTRIBUTING.md for guidelines.

Acknowledgments

Big thanks to @embano1, whose insightful advice on the early experimental versions caught the bugs before they became bugs, and whose eagle-eyed reviews caught the subtle (and not-so-subtle!) ones before they could bite anyone.

Security

See CONTRIBUTING for information about reporting security issues.

License

Apache-2.0. See LICENSE.