Dataflow Palette

Overview

With Simplebim you can automate repeating BIM tasks so you don’t have to do the same typing, clicking and dragging&dropping with every model in every project.

In Simplebim the solution for automated BIM data processing is called a dataflow, which defines is a sequence of steps you take for getting from the starting point to where you want to go. These steps are run in order, which makes the logic of the dataflow easy to understand.

Dataflows are defined using the Dataflow -palette. By default the Dataflow palette is placed on the left side of the workspace, but since it is a palette like any other, you can move it anywhere in the workspace. Some people for example like to have it on the right side instead of the left side. The only difference to other palettes is that you can have only one Dataflow -palette in a workspace. If you close the Dataflow -palette on purpose or by accident, you can always get it back by resetting the workspace or adding it as a new palette from the ribbon.

The Dataflow -palette has a menu bar on the top and a list of dataflow steps and sections filling up the bottom. The list is divided into two 'columns'. The left column shows the order of the steps and sections, which can be re-ordered via drag&drop from this left side. The right column shows the steps, sections and the step configurations.

Ribbon

Add Palette

Add a dataflow palette to your workspace. Please note that there can be only one dataflow palette per workspace.

Reset Current Workspace

If your dataflow palette gets lost you can always also reset the whole workspace, which brings back the dataflow palette if it is part of the current workspace.

Dataflow Steps

The Dataflow Steps ribbon tab allows quick access to adding steps and section as well as adding most popular steps to your dataflow.

Toolbar

New

Create a new empty dataflow. If you have edited the current dataflow you will be asked if you want to save your changes first.

Open

Open a dataflow stored on your computer as a dataflow file (.sbdf or -sbdt). The menu shows the content of your dataflows folder that you can access from the last menu item "Open My Dataflows -Folder...". You can use a one-level-deep folder structure in this folder for organizing your dataflows.

NOTE: You can use your dataflows folder like any other folder. You can copy dataflow files there and from there, and you can also add, rename and delete sub-folders (remember only one level deep). Any changes you make will take effect immediately, so there is no need to restart Simplebim or reload a model after such changes.

Save

Save the current dataflow to a local file. You can also choose / create a folder for your dataflow. If the dataflow was opened from a local file you can simply click OK to overwrite the previous version.

Share

The whole idea of Dataflows is to automate repetitive tasks – and reuse that automation again and again. Design your Dataflows well, and you can apply them across models, updates, or even entire projects. The more you use them, the more value they bring.

Use Export Current to save the current dataflow anywhere on your own computer.

Use Import as Current to import a dataflow from anywhere on your computer and open it as the current dataflow. At this point the dataflow is not saved yet, but you can save it normally just as if you had created the dataflow yourself. [New in Simplebim 11.0 SR7]

Use Export to save one or more saved dataflows anywhere on your computer.

Use Import to copy one or more dataflows from anywhere on your computer to the Imported sub-folder of your My Dataflows folder.

Add Step

Opens the Dataflow Presets dialog that allows you to add one or more steps into the end of the current dataflow.

There are 4 different types of dataflow steps you can add to your dataflow.

1. Tools are the basic building blocks of dataflows. Tools are software modules that are developed by Datacubist or 3rd party developers and they are installed with the application. Tools essentially define what can be done with dataflows.
2. Installed Dataflows are dataflows that are developed by Datacubist or partners. They are packaging a dataflow and provide a target selection and configuration for this package. This means that for you they behave just like tools.
3. My Dataflows are the dataflows you develop yourself. These dataflows don’t have a target or configuration and when you add them to another of your dataflows, they are not added as one step like the Installed Dataflows, but each step is added as a separate step.
4. Templates are the Simplebim Excel templates from previous Simplebim versions. For backward compatibility these Excel templates can be added as a step into a dataflow.

The Filter helps you find the dataflow step you are looking for.

When you select one step and click Add the new step will first open in the Dataflow Step Configurator -dialog. Here you can configure the step or click OK to configure it later in the Dataflow palette.

When you select multiple steps the new staps are added directly to your dataflow in the dataflow palette.

Add Step via Drag and Drop

Drag and drop IFC files (.ifc, .ifczip) on the Add Step -button in the toolbar to add a new step that merges the dropped files.

Drag and drop IFC files (.xlsx) on the Add Step -button in the toolbar to add a new step that applies the dropped Excel templates.

Add Section

You can optionally use sections in your dataflows for organizing the steps into groups. Sections give longer dataflows visual clarity and make it easier to run a dataflow in in parts. You can also enable/disable a step to enable/disable all steps in that section. When the section is disabled none of its steps are run, but the steps remember their enable/disable status. When you enable the section the steps will regain the status they had before you disabled the section.

Duplicate

Duplicate the currently selected sections and/or steps. You can duplicate either to the end of the current dataflow or below the last selected section/step. If a section is selected then all steps in that section are duplicated.

Delete

Delete the currently selected sections and/or steps If a section is selected then all steps in that section are deleted.

Run

Run all steps of the current dataflow that are not disabled either by disabling the step or the section the step belongs to.

Configure

When Open Dataflow with Sections Expanded is on dataflows are opened with all steps inside sections visible. When this setting is off dataflows are opened with only the sections headers visible and you can expand the sections headers to see the steps in each section. [New in Simplebim 11.0 SR7]

When Show Reload Model Step is on a special dataflow step is shown as the first step in your dataflow. Running this step reloads the currently opened model. When you reload your model the current state of your dataflow is automatically saved into a separate autosave file and opened from that file when the model is open again. This means that  you don't have to save the dataflow before you re-open your model and also the actual dataflow file you are working on is not overwritten. This step is not saved with the dataflow.

Other

Collapse All Step Configurations collapses all configurations of all steps. However, it does not collapse the dataflow sections.

Collapse All Sections collapses all sections hiding the steps in the sections.

Add New Excel Template (Legacy) adds a new Excel template to the User Templates folder. Dataflows are replacing Excel Templates, so you only need to use this function if you are already familiar with Excel templates or doing something that can't yet be done with dataflows.

Clear removes all sections and steps of the current dataflow, but remembers the name and folder of the dataflow.

Working with Steps and Sections

At their core dataflows consist of steps that are run in the order they appear in the dataflow. Optionally sections can be used for organizing the flow, which can be especially useful with long dataflows.

Configuring Steps

Steps usually have a configuration for defining what the step does. The configuration options for different types of steps are documented in the separate documentation of the dataflow or tool the step uses.

Steps can be configured in-place by expanding the step in the Dataflow Palette, or using a separate configuration dialog that opens when you double click or select Edit from the step's menu.

When you configure steps in-place the following logic applies to sub-editors, which are configuration editors that open inside a configuration item. These are for example lists, expressions and queries. Sub-editors can also be nested when you for example open a list editor from inside a query editor.

When you are inside a sub-editor most functionality in the Dataflow palette is disabled until you close (either cancel or accept the changes) the sub-editor. The main toolbar stays disabled until you have closed all sub-editors in all steps. The reason for this is that when a sub-editor is open the dataflow is in a undefined state and can't be run or saved. 

The name of the step can be edited by clinking again on an already selected step or in the configuration dialog. The edited name can be committed by hitting Enter or the ok -icon in the edit box. The edited name can be discarded by hitting Esc or the cancel -icon in the edit box. The description of a step can only be edited in the configuration dialog.

Using Sections

Sections are used primarily for organizing  the dataflow visually. A section starts with the section header and ends with the next section header or the end of the dataflow.

The name of the section can be edited by clinking again on an already selected section or in the configuration dialog. The description of a section can only be edited in the configuration dialog.

Selecting Steps and Sections

Steps can be selected using the same logic found in the Windows File Explorer. Selections can be used for moving, duplicating, deleting, enabling/disabling and running steps.

• Click to select
• Shift+Click to select all steps between the most recently selected and the current one
• Ctrl+Click to add steps to the selection or removing an already selected step from the selection

Selecting a section selects all steps in that section.

Clicking on the empty area after the last step clears the selection. If there is no empty area click on any step or sections and then ctrl + click on the same item again to clear the selection.

Ordering and Duplicating Steps and Sections

Sections and steps can be re-ordered and duplicated via drag&drop. Dragging is always for selected items and if no items are selected, the item from which the drag is started. Please note that dragging is done from the number of the item, not the name. The drag handle on the left of the number gives a visual cue, but dragging can also be done from the number.

The functionality depends on the selection that is being dragged and the options in each case are presented in a menu that appears at drop. The functionality does not depend on the expanded/collapsed state of sections.

Dragging a Single Step

The options are to move or duplicate the dragged step.

Dragging a Single Section

The options are to move or duplicate only the header or the header and all steps in the header. Duplicating or moving only the header is useful when the headers are added after the steps. Moving the header with steps is useful when re-ordering sections and duplicating a section is useful when adding a new section with similar functionality at a specific place in the dataflow.

Dragging Multiple Sections

The options are to move or duplicate the sections headers. For moving and duplicating headers with steps the sections must be moved or duplicated one-by-one.

Dragging a Selection of Sections and Steps

The options are to move or duplicate the selected steps and sections. The operation is always for exactly the selected steps and items. If the selection contains section headers, only the selected steps inside those sections are affected. If the selection has gaps the moved or duplicated are consolidated into one block.

Enabling/Disabling Steps and Sections

When a step is disabled it will not be run when the dataflow is run.

Steps are enabled/disabled individually using the toggle on the right side of the step name.

When multiple steps and/or sections are selected they can all be enabled/disabled using the toggle on any of the selected items.

Disabling a section will disable all steps in the section. However, the individual states of the steps are maintained and used again when the section is enabled.

Running Steps and Sections

Individual steps can always be run, even if the step is disabled, from the Run menu item of the step.

Individual sections can always be run, even if the section is disabled, from the Run menu item of the section.

For other combinations select the steps you wish to run, enable them and run the whole dataflow.

Step Menu

Run Step runs the step, even if it is disabled.

Edit Step opens the step configuration dialog, as does double-clicking the step name.

Duplicate Step - Below creates a duplicate of the step right below the step. 

Duplicate Step - End creates a duplicate of the step at the end of the current dataflow.

Add Section - Above adds a new section header above the step. This is useful when steps are created first and organized into sections afterwards, because the section header can be added directly into the right place.

Delete Step deletes the step.

When more precise control is needed steps can be copied to any position via drag&drop holding down the Ctrl key.

Section Menu

Run Section runs all enabled steps in the section, even if the section is disabled.

Edit Section opens the step configuration dialog, as does double-clicking the section name.

Duplicate Section - Below creates a duplicate of the section and all its steps right below the last step in the section. 

Duplicate Section - End creates a duplicate of the section and all its steps at the end of the current dataflow.

Delete Section Header Only  removes the section header but leaves all steps in the section untouched.

Delete Section Header and Steps deletes the section and all its steps.

Dataflow Step Basics

Simplebim comes with a host of Tools that are the basic building blocks of dataflows. When you add a new step to your dataflow (by clicking on the ‘Add Step’ -button’ or choosing a dataflow step from the ‘Dataflow Steps’ -ribbon tab) you choose which tool the step is using. Alternatively you can use another dataflow as one step in your own dataflow. The definition of a dataflow step typically has two parts: defining the target for the step and configuring the step.

1. Target: First you define which objects are processed by the step or participate in the processing. We call this the ‘Target’ of the step. The target can be for example the whole model or an object class, but the most common target is a group of objects you define. This means that knowing how to create groups is an essential skill for defining dataflows. You can get started by adding objects to the Target via drag&drop, but for defining reusable dataflows you need to get familiar with rule based groups.
2. Configuration: The configuration of a step is simply some questions you have to answer. These questions are different for each type of step.

A very simple dataflow could for example colorize all external walls blue. This dataflow would have two steps. The first step would add a ‘External Walls’ -group using the ‘Add or Modify Group’ -tool. The second step would use the ‘Set Appearance’ -tool with the ‘External Walls’ -group as the target and blue as the configuration for the color. The target is always shown with a green background above the configuration.