Commit Graph

38 Commits

Author SHA1 Message Date
Adrien de Peretti
516f5a3896 fix: workflow async concurrency (#13769)
* executeAsync

* || 1

* wip

* stepId

* stepId

* wip

* wip

* continue versioning management changes

* fix and improve concurrency

* update in memory engine

* remove duplicated test

* fix script

* Create weak-drinks-confess.md

* fixes

* fix

* fix

* continuation

* centralize merge checkepoint

* centralize merge checkpoint

* fix locking

* rm only

* Continue improvements and fixes

* fixes

* fixes

* hasAwaiting will be recomputed

* fix orchestrator engine

* bump version on async parallel steps only

* mark as delivered fix

* changeset

* check partitions

* avoid saving when having parent step

* cart test

---------

Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2025-10-20 15:29:19 +02:00
Adrien de Peretti
12a96a7c70 chore(): Move peer deps into a single package and re export from framework (#13439)
* chore(): Move peer deps into a single package and re export from framework

* WIP

* update core packages

* update cli and deps

* update medusa

* update exports path

* remove analyze

* update modules deps

* finalise changes

* fix yarn

* fix import

* Refactor peer dependencies into a single package

Consolidate peer dependencies into one package and re-export from the framework.

* update changeset

* Update .changeset/brown-cows-sleep.md

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>

* rm deps

* fix deps

* increase timeout

* upgrade version

* update versions

* update versions

* fixes

* update lock

* fix missing import

* fix missing import

---------

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2025-09-22 18:36:22 +02:00
Adrien de Peretti
fc4d5f0ac9 chore(): Workflow engine timers and notification improvements (#13434)
RESOLVES CORE-1177

**What**
main changes are:
- not blocking execution when notifying
- timers management
- race condition checks improvements
2025-09-08 18:19:55 +00:00
Adrien de Peretti
d7692100e7 chore(orchestration): add support for autoRetry, maxAwaitingRetries, retryStep (#13391)
RESOLVES CORE-1163
RESOLVES CORE-1164

**What**

### Add support for non auto retryable steps.

When marking a step with `maxRetries`, when it will fail it will be marked as temporary failure and then retry itself automatically. Thats the default behaviour, if you now add `autoRetry: false`, when the step will fail it will be marked as temporary failure but not retry automatically. you can now call the workflow engine run to resume the workflow from the failing step to be retried.

### Add support for `maxAwaitingRetries`

When setting `retyIntervalAwaiting` a step that is awaiting will be retried after the specified interval without maximun retry. Now you can set `maxAwaitingRetries` to force a maximum awaiting retry number

### Add support to manually retry an awaiting step

In some scenario, either a machine dies while a step is executing or a step is taking longer than expected, you can now call `retryStep` on the workflow engine to force a retry of the step that is supposedly stucked
2025-09-08 12:46:30 +00:00
Adrien de Peretti
2b89510df3 chore(): only execute race execution checks and publish message only for async workflows (#13396)
* chore(): only execute race execution checks for async workflows

* chore(): workflow redis publish only for async flows

* Create cyan-gorillas-poke.md

* chore(): workflow redis publish only for async flows

* fix negative check

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-09-04 09:12:09 +02:00
Adrien de Peretti
ff152e7ace fix(orchestration): Use the step definition max retries on set step failure (#13319)
* fix(orchestration): Use the step definition max retries on set step failure

* Create sweet-turkeys-wait.md

* allow to force permanent failure

* update changeset
2025-08-28 14:35:31 +02:00
Adrien de Peretti
238e7d53c1 fix(wfe): should notify when finished + add state info (#12982) 2025-07-18 09:20:46 +02:00
Adrien de Peretti
eb83954f23 chore(workflow-engine-*): Align event subscribers management (#12976)
* chore(workflow-engine-*): Align event subscribers management

* Create nervous-eels-build.md
2025-07-16 16:34:47 +02:00
Adrien de Peretti
316a325b63 fix(workflow-engine-*): Cleanup expired executions and reduce redis storage usage (#12795) 2025-06-24 13:32:10 +02:00
Adrien de Peretti
1a78476608 fix(workflow-sdk): Async/nested runAsStep propagation (#12675)
FIXES CLO-524

**What**
Add hidden stepDefinition object as part of the step argument and ensure the runAsStep handlers rely on the latest definition when config is being used on the returned step in order to ensure async configuration propagation and nested configuration
2025-06-10 07:23:12 +00:00
Adrien de Peretti
ab22faaa52 Chore/test runner improvements (#12439)
**What**
Make sure there is no open handles left and that the shutdown function are properly called. Refactor and improve the medusa test runner. Make sure all modules instances are released and cleaned up

**NOTE:**
On a separate PR we can continue the investigation for the memory growing over time while the tests execute
2025-05-14 13:17:41 +00:00
Adrien de Peretti
9d29078b0d fix(workflow-engine-*): q text search (#12435) 2025-05-11 16:21:41 +02:00
Adrien de Peretti
80007f3afd feat(workflows-*): Allow to re run non idempotent but stored workflow with the same transaction id if considered done (#12362) 2025-05-06 17:17:49 +02:00
Adrien de Peretti
8618e6ee38 fix(): Properly handle workflow as step now that events are fixed entirely (#12196)
**What**
Now that all events management are fixed in the workflows life cycle, the run as step needs to leverage the workflow engine if present (which should always be the case for async workflows) in order to ensure the continuation and the ability to mark parent step in parent workflow as success or failure

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-04-17 09:34:19 +00:00
Carlos R. L. Rodrigues
31abba8cde fix(orchestrator): save checkpoint before async step (#12138) 2025-04-10 15:36:36 +00:00
Adrien de Peretti
13e159d8ad fix(workflow-engine-*): Prevent passing shared context reference (#11873)
* fix(workflow-engine-*): Prevent passing shared context reference

* fix(workflow-engine-*): Prevent passing shared context reference

* prevent tests from hanging

* fix event handling

* add integration tests

* use interval for scheduled in tests

* skip tests for now

* Create silent-glasses-enjoy.md

* fix cancel

* changeset

* push multiple aliases

* test multiple field alias

* increase wait time to index on test

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
2025-04-09 10:39:29 +02:00
Carlos R. L. Rodrigues
0625f76cd4 chore(workflow-engine): export cancel method (#11844)
What:
  * Workflow engine exports the method `cancel` to revert a workflow.
2025-03-17 12:59:09 +00:00
Adrien de Peretti
72d2cf9207 fix(workflow-engines): race condition when retry interval is used (#11771) 2025-03-12 09:53:34 -03:00
Carlos R. L. Rodrigues
c8376a9f15 chore(medusa): clear workflow execution (#11200)
CLOSES: SUP-704
2025-02-03 10:47:32 +00:00
Adrien de Peretti
0a077d48e1 chore(workflow-engine): Migrate to DML (#10477)
RESOLVES FRMW-2832
RESOLVES FRMW-2833

**What**
Migrate workflow engines to DML. Alos includes and update to the linkable generation which now takes into account id and primary keys to generate the linkable instead of only primary keys
2024-12-06 13:23:07 +00:00
Carlos R. L. Rodrigues
90ae187e09 fix(workflows-sdk): name for when/then step (#10459) 2024-12-05 15:47:42 -03:00
Carlos R. L. Rodrigues
1eef324af3 fix(orchestration): fix set step failure (#10031)
What:
 - copy data before saving checkpoint
 - removed unused data format function
 - properly handle registerStepFailure to not throw
 - emit onFinish event even when execution failed
2024-11-12 10:06:36 +00:00
Adrien de Peretti
e096feb7d5 chore: Update modules deps (#9286) 2024-09-26 11:14:35 +05:30
Adrien de Peretti
6937d74252 chore(): Move dependencies around (#9278)
RESOLVES FRMW-2716 [1]

**What**
First stab at re organising dependencies in the modules and their usage.
2024-09-24 14:58:04 +00:00
Harminder Virk
9e711720dd chore: upgrade moduleResolution to Node16 (#9269) 2024-09-24 17:19:20 +05:30
Carlos R. L. Rodrigues
adbeb9cc1d fix(workflow-engine-*): pass container to flow (#9180) 2024-09-18 08:44:11 -03:00
Carlos R. L. Rodrigues
ef8dc4087e feat: run nested async workflows (#9119) 2024-09-16 13:06:45 +00:00
Carlos R. L. Rodrigues
eb0bfe9f33 feat(workflows-sdk): log on error (#8666) 2024-08-20 06:23:24 -03:00
Stevche Radevski
4155d0354f fix: Use a global reference to distributed transaction and scheduler (#8462)
* fix: Add storage cleanup on scheduled jobs

* fix: Use a global reference to distributed transaction and scheduler classes
2024-08-06 14:14:51 +02:00
Stevche Radevski
26d600b6db feat: Ensure async workflow executions have access to shared container (#8157)
* feat: Ensure async workflow executions have access to shared container

* fix: Register workflow worker on application start
2024-07-17 12:17:48 +02:00
Adrien de Peretti
b368251ca3 chore: Migrate modules to use the Module util (#7964)
**What**
Apply the `Module` util to each module to handle the export and provide the linkable configuration object.

- Most joiner config that does not have any special config are removed/cleaned up in favor of the autogenerated one or simpler one
- linkable are now available in all modules
- cleaned up the dependencies of the modules

FIXES CORE-2410
2024-07-05 13:00:40 +00:00
Adrien de Peretti
617a5972bf feat: refactor module joiner config and links generation (#7859)
* feat: refactor module joiner config and links generation

* improve typings

* WIP

* WIP

* WIP

* rename type file

* create link config

* finish typings and add utils

* improve links

* WIP typings

* finalize ExportModule utils

* finalize ExportModule utils

* fix: dml tests

* improve and fixes

* simplify typings with id changes

* add toJSON

* multiple fixes and entity builder fixes

* fix currency searchable

* fix tests

* medusa service refactoring

* cleanup

* cleanup and fixes

* make module name optional

* renaming

---------

Co-authored-by: Harminder Virk <virk.officials@gmail.com>
2024-07-03 13:12:49 +02:00
Carlos R. L. Rodrigues
5600e58b7f chore(orchestration): idempotent (#7771) 2024-06-25 10:34:00 -03:00
Adrien de Peretti
48963f55ef Chore/rm main entity concept (#7709)
**What**
Update the `MedusaService` class, factory and types to remove the concept of main modules. The idea being that all method will be explicitly named and suffixes to represent the object you are trying to manipulate.
This pr also includes various fixes in different modules

Co-authored-by: Stevche Radevski <4820812+sradevski@users.noreply.github.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2024-06-19 13:02:16 +00:00
Adrien de Peretti
d2a5201eeb chore: Refactor and improve abstract module service factory (#7688)
* chore: Refactor and improve abstract module service factory

* align naming

* clean up some template args and tests

* partially migrate modules

* partially migrate modules

* migrate more modules

* migrate last modules

* fix typings

* rename interface

* rename interface

* fixes

* fixes

* rm local plain tests
2024-06-13 13:12:37 +02:00
Stevche Radevski
69410162f6 feat: Add support for scheduled workflows (#7651)
We still need to:
But wanted to open the PR for early feedback on the approach
2024-06-10 14:49:52 +00:00
Harminder Virk
11528526fa feat: loosely typed container 2024-05-31 15:22:03 +05:30
Adrien de Peretti
4eae25e1ef chore(): Reorganize modules (#7210)
**What**
Move all modules to the modules directory
2024-05-02 15:33:34 +00:00