Fleets

Definition

A Fleet is a collection of multiple Vessels working together to tackle one larger job. Fleets are the equivalent of a Workflow or DAG in other orchestration tools.

Fleets can contain any number of Vessels and contain any different type of Vessels (including a mixture of coding languages or Blueprints). Vessels can be reused across multiple Fleets and can continue to run independently of the Fleets they belong to.

All Vessels are connected together with paths which check for the final status of the Vessel they are attached to. Paths can check for Success (exit code 0), Errored (exit code 1-255), or Completed (any exit code). By default, paths check for Success.

How Fleets Run

Vessels in a Fleet get run in the order that you choose, starting with one or multiple Vessels that don't have any paths feeding into them. A Fleet can be kicked off with a Schedule Trigger or an On Demand Trigger.

The paths between Vessels can be sequential (one-to-one), branching (one-to-many) or converging (many-to-one).

When an upstream Vessel finishes running, it returns a status code to indicate if the Vessel was successful (exit code of 0) or if it errored (exit code of 1-255). All paths coming out of the finished Vessel are immediately evaluated. If a path evaluates as true, it kicks off all downstream Vessels that are connected to it. If a path evaluates as false, it marks all downstream Vessels as incomplete.

If a Vessel is part of a converging path, it waits until all upstream Vessels have completed. At this point, every upstream Vessel is evaluated to see if the status matches the path condition. If all paths evaluate as true, the downstream Vessel will begin running. If any of the converging paths are evaluated as false, the Vessel will not run.

File Sharing

When a Fleet runs, files generated by upstream Vessels get shared with every downstream Vessel as part of a shared local file system. Files cannot be accessed by any other currently running fleet - only by Vessels in the current Fleet run. In this way, Vessels can be built to run independent, modular tasks off of newly generated data, allowing greater flexibility in how you build Vessels.

This design is the equivalent of running individual scripts one after another on your local computer. Creating two files with the same name in a Fleet will result in the most recent file overwriting the oldest file. Once every Vessel in the Fleet has finished running, all files are immediately wiped from the Shipyard platform.

Screenshots

Editing Vessels in a Fleet

Limitations

While fleets are a powerful way to automate an entire workflow, there are a few limitations to be aware of.

  1. A Fleet cannot be set up with Vessels that live across multiple projects. You'll need to make sure that all of the relevant Vessels live within the same project.

  2. A Fleet cannot contain any paths that would create a loop.

  3. A Fleet must contain two or more Vessels that are all connected together by a path. If you want to have a set of Vessels that aren't connected to the larger group, you will need to create another Fleet and trigger it at the same time.

  4. You are currently unable to kick off a Fleet starting anywhere in the middle, although you can kick off individual Vessels at will. This prevention is in place because we delete data as soon as a Fleet has finished running, whether it was successful or not. If you ran a Vessel starting from the middle and it relied on data created upstream, it would be unable to find that data and immediately fail.

  5. A Fleet can only be created with Vessels that already exist. We currently don't support creating a Vessel while creating a Fleet.

  6. A Fleet cannot have two or more copies of the same Vessel. Once you select a Vessel, it becomes unavailable in the dropdown.

Learn More