Share
Explore
Public Roadmap
Orchestration
Feature
Explanation
Target Delivery Date
Status
Priority
1
Webhook based trigger
Support for initiating workflows via webhook calls, enabling integration with external systems and event-driven architectures.
11/12/2024
Build in-progress
P1
2
Streaming logs
Stream the logs for the jobs that are running instead of waiting for the entire job to be finished
11/30/2024
Build in-progress
P0
3
Slim CI
Implementation of Slim CI (Continuous Integration) practices for dbt, enabling faster and more efficient CI processes by selectively running only the models and tests affected by recent changes.
11/12/2024
Build in-progress
P0
4
‘Next Run’ details
Add 'Next Run' time stamp to Job listing page
11/7/2024
Build in-progress
P0
5
Block - Refresh a Dashboard
Functionality to automatically refresh connected dashboards upon completion of relevant data processing tasks.
10/16/2024
Not started
P2
6
If/else block
Conditional logic within workflows, allowing different execution paths based on specified conditions or data states.
10/10/2024
Not started
P3
7
Export to cloud storage
Export data from warehouse to cloud storage
Not started
P3
8
While/For block
Conditional logic within workflows, allowing different execution paths based on specified conditions or data states.
Not started
P3
9
Run downstream block if one dependency upstream has failed
Option to continue partial workflow execution even if one upstream dependency fails, providing flexibility in error handling.
10/9/2024
Not started
P3
10
Import Dagster repo directly and run in Orchestration as a block
Feature to directly import and execute Dagster repositories within the orchestration platform.
10/3/2024
Not started
P3
11
Backfill support
Functionality to re-run historical data through the workflow, useful for processing retroactive data or fixing past errors.
10/16/2024
Not started
P3
12
Wait for callback actions
Ability to pause workflow execution at specific points, waiting for external actions or human interventions before proceeding.
10/16/2024
Not started
P3
13
Duplicate Pipelines
Add functionality to duplicate pipelines.
Not started
P3
14
Multiple HH and MM combinations on scheduled trigger option
For the scheduled node "Custom Time" configuration, enable the selection of multiple HH and MM combinations (currently limited to just one). (Screenshot attached below)
Not started
P3
15
Retry option for node
Not started
P2
16
Timeout
Timeout of each node
Not started
P2
17
Save code state
To build custom connectors using the Custom Node, introduce the ability to send and receive the last run state of the code. This state, typically a Python dictionary, keeps track of the last run's time offset and can be acc
Not started
P1
18
Schema evolution
One of the biggest challenges when using custom code is efficiently loading data into the warehouse. Typically, this is handled through either a warehouse streaming API or database toolkit libraries like SQLAlchemy or Psycopg. However, to streamline this process, it would be beneficial to introduce a node capable of reading output data (in formats like data frames or JSON) from the previous state. By incorporating schema evolution and a data injection node, this feature could automatically push data into the warehouse. It should offer the flexibility to:
Not started
P3
19
Average run duration for a job on the Job Overview Page
Showing Avg duration is helpful to quickly get on overview of all jobs and to understand overall orchestration
Not started
P1
20
Event based triggers
Ability to start workflows based on specific events or conditions in connected systems, enhancing automation. S3, Azure Blob, Google cloud storage
11/20/2024
Not started
P3
There are no rows in this table
Modeling
Github Roadmap
Python IDE Roadmap
Utilisation Roadmap
Platform Roadmap
Alerts
Teams Roadmap
Architecture Roadmap
Security Roadmap
Billing
BI Roadmap
Warehouse Roadmap
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.