Managing Workspaces

A guide to the Workspaces page of Simplebim Cloud, where the work of an account is organised into workspaces. It explains what a workspace is, how workspaces are created, paused and archived, and what the tabs of a workspace do: its connections, its users, its settings and above all its History — the record of every pipeline run and the place where failed runs are handled.

The pipelines inside a workspace — what they are, how they are set up, triggered, validated and run — have their own guide, Using Pipelines. The account-wide connections that workspaces build on are described in Configuring Connections, and the Dataflow Library in Using the Dataflow Library.

Contents

• What a workspace is
• Where the page is
• Who sees and does what
• The page at a glance
• The workspace list
  • Workspace status
  • The Issues column
• Adding a workspace
  • An empty workspace
  • Cloning a workspace
  • From a template
  • Finishing the draft
• The tabs of a workspace
• The Connections tab
  • Account connections
  • Workspace connections
  • Adding and editing a workspace connection
  • Testing and purging workspace connections
• The Pipelines tab
• The History tab
  • The run list
  • Run results
  • Runs needing attention
  • Handling a run
  • The run log
• The Users tab
• The Library tab
• The Settings tab
• Test File Locations
• Flush Pipelines
• Messages
• Troubleshooting

What a workspace is

An account — typically one per company — is divided into workspaces, typically one per project. A workspace is the container for:

• Pipelines, the automated runs of dataflows. Files dropped into a pipeline's trigger folder are processed by the Simplebim Engine and the results are written back to the file storage.
• Workspace connections, optional named folders inside the account's connections that the pipelines of this workspace can use.
• History, the record of every pipeline run in the workspace with its result and log.
• Users, the members of the account who may see and edit this workspace.
• Settings, such as the Simplebim Engine version the workspace runs with.

Workspaces share the account's connections — the authorized access to Dropbox, OneDrive and SharePoint — but nothing else. A pipeline belongs to exactly one workspace.

Where the page is

Workspaces is in the top navigation bar, after Connections. It always shows the workspaces of the account you are currently working in. The page is the working area of Simplebim Cloud — after signing in, most people go straight here.

Who sees and does what

What the page offers depends on your role in the account:

The guide describes the page as an account administrator sees it. Members see the same list and tabs with the editing buttons left out.

The page at a glance

From top to bottom the page has:

The workspace list

The list can be filtered by typing into the boxes under the column headers, sorted by clicking a header, and shows ten workspaces per page. Initially the active workspaces come first, then the paused, then the archived, each group by name. Filters, sorting and column widths are remembered for the rest of your browser session.

Clicking a row expands it. Several rows can be expanded at once, and the expanded row opens on its Pipelines tab, or on the Connections tab when the workspace has no workspace connections yet. A refresh button at the top right of the expanded row reloads that workspace — useful while runs are going on.

Workspace status

The pencil menu in the Status column offers Activate, Pause and, depending on the workspace, Archive or Delete:

Delete is offered instead of Archive for a workspace that has never run a pipeline. Deleting cannot be undone and you are asked to confirm. Once a workspace has any run history it can only be archived, so the history is never lost.

The Issues column

A red badge with a number means that many runs in the workspace need attention: they finished with errors or warnings and nobody has marked them as handled. Click the badge to open the workspace's History tab with the list already filtered to those runs. See Runs needing attention. A 0 means all is well.

Adding a workspace

Add New Workspace opens a dialog with up to three ways of creating the workspace. Choose one and click Create; the new workspace appears at the top of the list as a Draft row, expanded on its Settings tab, and exists only once you Save it there.

An empty workspace

Add Empty Workspace asks for the Workspace Name and creates a workspace with no connections and no pipelines. This is the normal starting point.

Cloning a workspace

Clone Existing Workspace lets you pick a Source Workspace from the account's non-archived workspaces and shows its description. The clone gets the source's name with (copy) appended, its description, administrator e-mail and engine version — all editable in the draft — and copies:

• every workspace connection, with the same account connection but with the folder cleared, so that you point the clone at the folders of the new project;
• every non-archived pipeline with its dataflow, its trigger settings and its file location mappings — again with the folders cleared — and set to Paused.

Connections and folders are cleared on purpose: a cloned project should never write into the original project's folders by accident. After cloning, set the folders of the workspace connections, then the trigger and file location folders of each pipeline, validate, and activate. When the pipelines were built on workspace connections rather than directly on account connections, only the workspace connections need new folders; the pipelines follow.

From a template

Add Workspace from Template appears only when Simplebim has published workspace templates. Pick a Workspace Template; its description is shown. The result is like a clone of the template: connections, pipelines and dataflows are copied, folders are cleared, and connection references are removed as well, because the template belongs to another account. You map the copied connections and folders to your own storage afterwards.

Finishing the draft

The draft's Settings tab shows the notice This workspace has not been created yet. Fill in the required fields and click Save to create it. Fill in at least the Name and the Simplebim Engine Version (see The Settings tab) and click Save. The message The workspace was created successfully. appears, the other tabs become available and the row opens on its Connections tab. Cancel on a draft removes it from the list; nothing was created.

New workspaces start Paused. Activate the workspace from the Status menu once it is set up.

The tabs of a workspace

An expanded workspace shows these tabs:

The Connections tab shows a red warning icon when the account has no connections at all — nothing in the workspace can work until an account administrator adds one.

The Connections tab

Account connections

The upper list shows the account's connections — Name, Type, Folder and Status (Authorized or Unauthorized, with a warning triangle and tooltip when the last test failed). It is the same information as on the Connections page, shown here for reference; nothing can be changed. When the account has no connections the tab says so instead and either links to the Connections page or, for members, lists the account administrators to contact.

Workspace connections

A workspace connection is a named folder inside an account connection, defined once for the workspace. Pipelines can then point their trigger, input and output locations at a workspace connection and give only the relative folder below it — or at an account connection directly. Workspace connections are optional, and their value shows when workspaces are cloned: a pipeline built on workspace connections keeps working in the clone as soon as the clone's workspace connections point at the new project's folders, without touching the pipeline itself. The blue note at the bottom of the tab says the same.

The lower list shows the workspace connections with their Name, the Account Connection they sit in, and their Folder; after a test also a Status column. Each row has an Open in Dropbox / OneDrive / SharePoint ↗ button, which opens the folder in the storage's web interface when the folder was picked with the folder dialog, and a pencil for editing.

Adding and editing a workspace connection

Add New Connection and the pencil open the same dialog:

Click Add or Save. The folder must not already be used by another workspace connection on the same account connection; the dialog refuses with File location folder '…' is already in use.

Testing and purging workspace connections

Test Connections checks every workspace connection in the list and adds a Status column with one of three results:

Purge Connections lists the workspace connections that no pipeline uses and lets you tick and remove them with Purge. A workspace connection that a pipeline points at cannot be removed; the dialog says All connections are in use. when there is nothing to remove.

The Pipelines tab

The Pipelines tab lists the pipelines of the workspace with their status, dataflow, trigger and file locations, and offers Add New Pipeline, Validate and Run. Everything about it is described in Using Pipelines. Two things connect it to this page:

• the Latest Run and Issues columns per pipeline mirror the workspace's own; clicking a pipeline's Issues badge opens the History tab filtered to that pipeline's runs needing attention;
• a pipeline is Active, Paused or Archived independently of the workspace.

The History tab

The History tab is the record of every pipeline run in the workspace — automatic and manual, finished and still running. Read a row from left to right and it tells the story of the run: when this file was dropped into this folder, this pipeline started, ran this long and ended with this result.

The run list

The list can be filtered by typing under the headers, sorted by the sortable columns, and shows ten runs per page.

Run results

Runs needing attention

A finished run with warnings, errors or a failed validation needs somebody to look at it. Such runs are counted in the Issues badges of the workspace and of the pipeline, and are listed by ticking Show only runs needing attention above the list. Clicking an Issues badge opens the History tab with the box already ticked; a pipeline's badge additionally limits the list to that pipeline and shows a Pipeline: name chip that you can close to see all pipelines again.

Once a run is marked as handled it drops out of the count and the filtered list. The handle button next to the result tells the state at a glance: red for an unhandled failure, yellow for unhandled warnings, grey once handled. Hover the result text of a handled run to read who wrote what about it.

Handling a run

Click the handle button of a run. The Handle Pipeline Run dialog shows the pipeline, the trigger file, when the run finished and its result. Tick Handled — A handled run needs no further attention — and write into Comments what was wrong and what was done about it, then Save. The comment is kept with the run and shown in the tooltip of its result. Untick Handled to put the run back on the to-do list.

Handling changes nothing about the run itself; it is bookkeeping for the team.

The run log

📄 Show Log opens the Pipeline Run Log dialog for a finished run:

• Files lists every file the run read and wrote — its type (Trigger, Input, Output), the file location and connection, the file name, size and transfer time.
• Log is the output of the run: the validation of the file locations, the file transfers, the messages of the dataflow and, for a failed run, the error at the end. Copy Log copies it to the clipboard.
• Create Support Ticket appears for failed runs. It opens a small form with a prepared Subject and a Description you fill in; Submit Ticket sends it to Simplebim support together with the workspace, pipeline and run identifiers and the log, so support can see what happened without asking you for it.

A run that never got as far as producing a log says Pipeline run has not finished correctly and it does not have log available. If log is required, please contact support.

The Users tab

Members of the account see a workspace only when they are granted access here; account administrators always see and edit everything, which the tab shows in its second table, Permissions through Account Roles.

The upper table lists the grants: User, Read — always ticked, a grant always includes reading — and Write, which lets the user edit the workspace's settings and workspace connections, handle runs, and manage the grants themselves. Click Add New Permission to add a row, pick the member from the list — it offers the account's members who are not administrators and have no grant yet — tick Write if wanted, and Save. The trash icon removes a grant. Cancel discards unsaved changes. Every row needs a user, and each user at most once; the page tells you otherwise.

A member with a Write grant can also add and remove other members' grants. Creating and configuring pipelines remains an account administrator's task.

The Library tab

In accounts that have the Dataflow Library, each workspace has a library of its own, and this tab shows it. Everyone with access to the workspace can see it; everyone who may edit the workspace may publish into it. What the library is, how the workspace library relates to the account library, and how dataflows are published, compared and used by pipelines is described in Using the Dataflow Library.

The Settings tab

Changing a field shows Unsaved changes and enables Save and Cancel. A missing required field is reported as Missing form values, a bad e-mail address as Invalid email address.

Test File Locations

Test File Locations at the top of the page checks, for every pipeline in every active workspace of the account, that the folders behind its trigger, input and output locations exist and can be reached through their connections. The File Location Test dialog shows Running test… and then Success (n) and Failed (n), with a table of the failures giving the workspace, the pipeline and the connection and path that failed, with the reason when the storage gave one.

Use it after connections were re-authorized, after folders were moved in the storage, or as a periodic health check. A single pipeline is checked more precisely with its own Validate button; see Using Pipelines.

Flush Pipelines

Pipelines normally react to new or changed files in their trigger folders. Flush Pipelines re-arms the triggers of every active pipeline in every active workspace of the whole account, which has two effects:

• a pipeline that moves its trigger files to a handled folder processes every file waiting in its trigger folder that matches its filter — one run per file;
• a pipeline without a handled folder simply starts watching afresh; files already waiting in its folder are not run.

The confirmation says the same: Flushing pipelines re-arms the triggers of all active pipelines in all active workspaces of this account. Pipelines that move their trigger files to a handled folder will run for every file currently in their trigger folder that matches their trigger file name filter. Paused and archived pipelines and paused workspaces are not flushed. Click Flush; the message Flushing of the account's pipelines has been started successfully. appears and the runs show up in the History tabs with the ⟳ icon within a few minutes. Pipelines in paused workspaces are left without a trigger by a flush.

Files that were processed and moved to a handled folder are gone from the trigger folder and are not run again. A file whose earlier run failed is left in the trigger folder with a small marker file next to it and is skipped by a flush, so a broken file does not run forever; remove the marker file or drop the file again to retry it.

Messages

Troubleshooting

I do not see a workspace that exists. You are a member without a grant on it. Ask an account administrator or a user with a Write grant to add you on the workspace's Users tab.

The Connections tab shows a red warning icon. The account has no connections at all. An account administrator adds them on the Connections page; members see the administrators to contact on the tab itself.

The Add New Connection dialog offers no account connection. Only authorized account connections are offered. Authorize them on the Connections page first.

Archive is greyed out. Only an Active workspace can be archived. Activate it first, or leave it Paused.

The menu offers Delete but I wanted to keep the history. Delete is offered only for workspaces that never ran a pipeline, so there is no history to keep.

A cloned workspace's pipelines do nothing. Clones start Paused, with their folders cleared. Set the folders of the workspace connections and of each pipeline's trigger and file locations, validate the pipelines, then activate the pipelines and the workspace.

The Issues badge shows a number but the History tab looks empty. The list is filtered to runs needing attention of one pipeline. Close the Pipeline: name chip or untick Show only runs needing attention to see everything.

A run shows Had Warnings although the dataflow reported none. The run succeeded but the trigger file could not be moved to the handled folder afterwards — the handled folder is missing or not writable. The log says so at the end.

Flush Pipelines did not run my file. Either the pipeline has no handled folder — only pipelines that move their trigger files process waiting files on a flush — or the file does not match the pipeline's trigger filter, the pipeline or its workspace is not Active, or the file has a marker file next to it from an earlier failed run. Run the file manually, or remove the marker or drop the file again.

Test File Locations lists a failure but the pipeline runs fine. The test covers all pipelines of all active workspaces, paused pipelines included. The failing entry may belong to a pipeline nobody uses any more; archive it or fix its folders.

I set the workspace to Paused but files still start runs. Pausing a workspace does not stop its pipelines by itself. Pause the pipelines on the Pipelines tab.