Flow Distributed Workflow System
The Flow server manages job submissions. It provides a GRPC API that is used by clients to submit and manage workflows.
The Flow CLI manages workflows from the command line. You can queue, view, and delete workflows.
NAME: fctl workflows queue - queue workflow USAGE: fctl workflows queue [command options] [arguments...] OPTIONS: --name value, -n value workflow instance name --type value, -t value workflow type --input-file value, -f value file path to use as workflow input --input-workflow-id value, -w value output from another workflow to use as input (can be multiple) --priority value workflow priority (low, normal, urgent) (default: "normal") --parameter value, -p value specify workflow parameters (KEY=VAL) --help, -h show help (default: false)
Redis is used for database storage for workflow metadata, user accounts, and queueing.
Minio is used for workflow input and output storage. Minio is lightweight and works very well with Flow but any compatible S3 system should work.
Flow processors provide the processing logic for queued workflows. Processors use GRPC to access the Flow server which streams workflow events to the corresponding processor type.