Apache DolphinScheduler 3.4.3 Released: Stronger Security, Stability, and Scheduling Reliability

java dev.to

Apache DolphinScheduler 3.4.3 has officially been released. This release delivers continued improvements across scheduling, permissions, security, performance, and stability, with a focus on real-world production scenarios such as task and workflow instance queries, API access control, sensitive information protection, and task failure recovery. In addition, new missed-scheduling recovery strategies are planned for Apache DolphinScheduler 3.5.0.

As an open-source data orchestration platform designed for complex data workflow scenarios, Apache DolphinScheduler continues to strengthen its scheduling and orchestration capabilities through community collaboration. Version 3.4.3 further improves reliability in areas such as scheduling recovery, enterprise-grade access control, and large-scale task execution, providing a stronger foundation for stable operation in production environments.

View the Apache DolphinScheduler 3.4.3 Release

Stronger Access Control for Improved Platform Security

One of the key areas of improvement in 3.4.3 is access control.

Enterprise data platforms typically involve multiple users, projects, data sources, and workflows. Clear and well-defined permission boundaries are therefore essential to maintaining a secure operating environment.

Version 3.4.3 further improves access control across multiple APIs, covering scenarios such as user list access, data source authorization lists, cluster queries, workflow modifications, and sub-workflow references. Related permissions have also been further standardized and refined.

The release also further restricts access to audit logs for non-administrator users, ensuring that users can only access audit information within the scope of their permissions. In addition, task-level data source access now includes corresponding permission checks, providing stronger controls over data access.

Together, these improvements establish clearer permission boundaries across users, projects, workflows, and data sources, providing more comprehensive security for enterprise environments with multiple users and collaborative workflows.

Enhanced Protection for Sensitive Information

Beyond access control, Apache DolphinScheduler 3.4.3 also introduces targeted improvements to sensitive information protection.

The release removes plaintext passwords from Worker logs and prevents raw passwords from being exposed through parameter validation error messages. These changes reduce the risk of sensitive information leakage in both log output and error messages.

The release also fixes a Sonar Token exposure issue and removes some unused code, further improving the project's overall security and engineering quality.

More Efficient Instance Queries at Scale

As the number of workflows and task instances continues to grow, querying task and workflow instances has become a frequent operation in production environments.

To address this scenario, version 3.4.3 introduces several query optimizations. New indexes, including idx_project_submit_time and idx_project_start_time, have been added for task instance and workflow instance queries. The Mapper query logic has also been optimized to exclude unnecessary large text fields from list queries.

These improvements reduce data retrieval during queries and help lower database load, improving query efficiency and the overall user experience in environments with large numbers of task instances.

Improved Task Failure Recovery and Retry Handling

The ability to recover from task failures directly affects the continuity of data processing. Version 3.4.3 includes multiple fixes covering task failures, retries, and workflow recovery.

The release adds application termination handling during task failover, preventing related applications from continuing to run after a task failure. It also adjusts retry time calculation so that the next execution is scheduled based on endTime + retryInterval.

For scenarios where failed tasks are recreated, 3.4.3 fixes an issue where runtime states were not properly reset. During workflow recovery, the release also ensures that forced-success states are correctly preserved.

In addition, this version fixes issues where Worker Group parameters were not applied as specified when rerunning workflows, as well as abnormal TaskGroup slot handling when tasks were paused or terminated. These fixes further strengthen task recovery in exceptional execution scenarios.

More Robust Workflow Dependencies and Execution Logic

At the workflow execution layer, version 3.4.3 includes multiple fixes related to DAGs and dependent tasks.

The release fixes an issue where DAG.addEdge could accept an edge that creates a cycle in path-convergence scenarios, further strengthening DAG validation and ensuring workflow correctness.

For dependent tasks, 3.4.3 fixes an issue with parsing ALL-type dependencies. It also resolves a problem where a task instance could fail while the workflow instance remained in a running state when using the Continue strategy, enabling workflow status to more accurately reflect actual execution states under complex dependency conditions.

In addition, the release addresses issues related to SQL task data source field persistence, DataX tasks reading Job Definitions from associated resource files, and S3 resource lists containing more than 1,000 records.

Continued Improvements to Kubernetes, Alerts, and Other Capabilities

For cloud-native deployments, version 3.4.3 fixes an issue in the Helm Chart that could result in duplicate app.kubernetes.io/name labels in ConfigMaps and updates the MySQL Helm Chart version.

Alerting capabilities have also been improved. These changes include removing redundant plugin definition table checks during AlertServer startup and adjusting the behavior of Alert Script test notifications. The release also fixes an issue that could cause Kubernetes Alert HTTP tests to fail.

In addition, the release removes some deprecated APIs and unused code, while further improving documentation related to parameter precedence, upgrades, data sources, and configuration. These changes contribute to better maintainability and a smoother user experience.

Missed-Scheduling Recovery Strategies Planned for 3.5.0

Beyond the improvements included in this release, the community continues to advance Apache DolphinScheduler's scheduling capabilities.

The missed-scheduling recovery strategy proposed in DSIP-107 has completed the relevant implementation and has been merged into the development branch. It is planned for inclusion in version 3.5.0.

The feature supports three strategies for handling missed schedules:

  • SKIP_MISSED
  • FIRE_ONCE_NOW
  • FIRE_ALL_MISSED

These strategies provide different ways to handle schedules that were missed, giving users greater control over how scheduled workflows should be recovered after interruptions or other exceptional circumstances.

Continuously Strengthening Enterprise-Grade Data Orchestration

From missed-scheduling recovery and access control to query performance and task failure recovery, Apache DolphinScheduler 3.4.3 focuses closely on the challenges encountered in real-world production environments.

Rather than simply adding new features, this release continues to improve the core qualities required for long-running data workflows, including scheduling reliability, security, query performance, and task stability. The upcoming missed-scheduling recovery strategies will further strengthen scheduling behavior in exceptional scenarios. More granular access controls establish stronger security boundaries for multi-user enterprise environments, while improvements to task recovery, dependency handling, and query performance further enhance platform reliability in complex production environments.

For the complete list of updates, see the Release Notes.

Thanks to Our Contributors

The Apache DolphinScheduler 3.4.3 release would not have been possible without the continued participation and contributions of the community. We would like to thank the following 20 contributors for contributing code, submitting issues, participating in testing and reviews, and helping move the community forward:

det101, wcmolin, liang-wenjie, njnu-seafish, qiuyanjun888, destinyoooo, HomminLee, ruanwenjun, kittimzhe, SbloodyS, eye-gu, vlaborie, nkuprins, nanxiuzi, SEPURI-SAI-KRISHNA, hiSandog, hellodml, zhang-arvin, nikhiln64, yan9651688

Thank you to every contributor for your time, expertise, and continued support. The ongoing collaboration of the global Apache DolphinScheduler community enables the project to continuously improve its scheduling, orchestration, security, and stability capabilities while serving an ever-growing range of data engineering use cases.

We invite you to try Apache DolphinScheduler 3.4.3, explore the latest improvements, and join the community to help advance the open-source data orchestration ecosystem.

Source: dev.to

arrow_back Back to Tutorials