Commit Graph

40 Commits

Author SHA1 Message Date
Adrien de Peretti
d7692100e7 chore(orchestration): add support for autoRetry, maxAwaitingRetries, retryStep (#13391)
RESOLVES CORE-1163
RESOLVES CORE-1164

**What**

### Add support for non auto retryable steps.

When marking a step with `maxRetries`, when it will fail it will be marked as temporary failure and then retry itself automatically. Thats the default behaviour, if you now add `autoRetry: false`, when the step will fail it will be marked as temporary failure but not retry automatically. you can now call the workflow engine run to resume the workflow from the failing step to be retried.

### Add support for `maxAwaitingRetries`

When setting `retyIntervalAwaiting` a step that is awaiting will be retried after the specified interval without maximun retry. Now you can set `maxAwaitingRetries` to force a maximum awaiting retry number

### Add support to manually retry an awaiting step

In some scenario, either a machine dies while a step is executing or a step is taking longer than expected, you can now call `retryStep` on the workflow engine to force a retry of the step that is supposedly stucked
2025-09-08 12:46:30 +00:00
Adrien de Peretti
2b89510df3 chore(): only execute race execution checks and publish message only for async workflows (#13396)
* chore(): only execute race execution checks for async workflows

* chore(): workflow redis publish only for async flows

* Create cyan-gorillas-poke.md

* chore(): workflow redis publish only for async flows

* fix negative check

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-09-04 09:12:09 +02:00
Adrien de Peretti
bd206cb250 chore(): improve workflow engine storage (#13345)
* chore(workflow-engines): Improve race condition management

* cleanup

* cleanup

* chore(workflow-engines): Improve race condition management

* chore(workflow-engines): Improve race condition management

* chore(workflow-engines): heartbeat extend TTL

* Refactor chore title for workflow engine improvements

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* chore(): Improve workflow execution db interaction

* update tests

* revert idempotent

* add run_id index + await deletion

* improve saving

* comment

* remove only

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-09-02 11:18:12 +02:00
Carlos R. L. Rodrigues
9412669e65 chore: idempotent cart operations (#13236)
* chore(core-flows): idempotent cart operations

* changeset

* add tests

* revert

* revert route

* promo test

* skip bugs

* fix test

* tests

* avoid workflow name conflict

* prevent nested workflow from being deleted until the top level parent finishes

* remove unused setTimeout

* update changeset

* rm comments

---------

Co-authored-by: adrien2p <adrien.deperetti@gmail.com>
2025-08-28 15:04:00 +02:00
Adrien de Peretti
c5d609d09c fix(orchestration): Prevent workf. cancellation to execute while rescheduling (#12903)
**What**
Currently, when cancelling async workflows, the step will get rescheduled while the current worker try to continue the execution leading to concurrency failure on compensation. This pr prevent the current worker from executing while an async step gets rescheduled

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-07-16 14:44:09 +00:00
Adrien de Peretti
316a325b63 fix(workflow-engine-*): Cleanup expired executions and reduce redis storage usage (#12795) 2025-06-24 13:32:10 +02:00
Adrien de Peretti
7fdbf2a965 fix(workflows-sdk): Miss match context usage within run as step (#12449)
**What**

Currently, runAsStep keep reference of the workflow context that is being run as step, except that the step is composed for the current workflow composition and not the workflow being run as a step. Therefore, the context are currently miss matched leading to wrong configuration being used in case of async workflows.

**BUG**
This fix allow the runAsStep to use the current composition context to configure the step for the sub workflow to be run

**BUG BREAKING**
fix the step config wrongly used to wrap async step handlers. Now steps configured async through .config that returns a new step response will indeed marked itself as success without the need for background execution or calling setStepSuccess (as it was expected originally)

**FEATURE**
This pr also add support for cancelling running transaction, the transaction will be marked as being cancelled, once the current step finished, it will cancel the transaction to start compensating all previous steps including itself

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-05-14 13:28:16 +00:00
Adrien de Peretti
80007f3afd feat(workflows-*): Allow to re run non idempotent but stored workflow with the same transaction id if considered done (#12362) 2025-05-06 17:17:49 +02:00
Adrien de Peretti
28958b2e26 Chore/orchestration storage improvements (#12178)
**What**
Cleanup and improve workflow storage utility
2025-04-18 08:35:23 +00:00
Adrien de Peretti
8618e6ee38 fix(): Properly handle workflow as step now that events are fixed entirely (#12196)
**What**
Now that all events management are fixed in the workflows life cycle, the run as step needs to leverage the workflow engine if present (which should always be the case for async workflows) in order to ensure the continuation and the ability to mark parent step in parent workflow as success or failure

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-04-17 09:34:19 +00:00
Carlos R. L. Rodrigues
31abba8cde fix(orchestrator): save checkpoint before async step (#12138) 2025-04-10 15:36:36 +00:00
Adrien de Peretti
13e159d8ad fix(workflow-engine-*): Prevent passing shared context reference (#11873)
* fix(workflow-engine-*): Prevent passing shared context reference

* fix(workflow-engine-*): Prevent passing shared context reference

* prevent tests from hanging

* fix event handling

* add integration tests

* use interval for scheduled in tests

* skip tests for now

* Create silent-glasses-enjoy.md

* fix cancel

* changeset

* push multiple aliases

* test multiple field alias

* increase wait time to index on test

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
2025-04-09 10:39:29 +02:00
Riqwan Thamir
5ab15a2988 feat(dashboard,js-sdk,admin-shared): add customer addresses + layout change (#11871)
what:

- changes customer layout from 1 layout to 2
- adds ability to create and delete customer addresses
- adds 2 customer widget locations
- adds is_giftcard=false by default to products list

<img width="1663" alt="Screenshot 2025-03-08 at 21 34 02" src="https://github.com/user-attachments/assets/e66f05da-718c-4c25-81ce-67ba0a814ca3" />
2025-03-17 16:16:27 +00:00
Carlos R. L. Rodrigues
0625f76cd4 chore(workflow-engine): export cancel method (#11844)
What:
  * Workflow engine exports the method `cancel` to revert a workflow.
2025-03-17 12:59:09 +00:00
Adrien de Peretti
c73504e0ed fix(workflow-engine-inmemory): Fail trap for integration tests (#11839)
**What**
Jest is patching the event emitter meaning that sometimes it can lead to flacky behaviors and block the test execution if the done callback is never reached. To prevent that from happening, the fail trap will call the done callback after a given time and warn that the test could not be concluded because of jest blocking it
2025-03-15 12:42:38 +00:00
Adrien de Peretti
72d2cf9207 fix(workflow-engines): race condition when retry interval is used (#11771) 2025-03-12 09:53:34 -03:00
Carlos R. L. Rodrigues
c8376a9f15 chore(medusa): clear workflow execution (#11200)
CLOSES: SUP-704
2025-02-03 10:47:32 +00:00
Carlos R. L. Rodrigues
cc73802ab3 chore(order): dml (#10292)
* ../../core/types/src/dml/index.ts

* ../../core/types/src/dml/index.ts

* fix: relationships mapping

* handle nullable foreign keys types

* handle nullable foreign keys types

* handle nullable foreign keys types

* continue to update product category repository

* fix all product category repositories issues

* fix product category service types

* fix product module service types

* fix product module service types

* fix repository template type

* refactor: use a singleton DMLToMikroORM factory instance

Since the MikroORM MetadataStorage is global, we will also have to turn DML
to MikroORM entities conversion use a global bucket as well

* refactor: update product module to use DML in tests

* wip: tests

* WIP product linkable fixes

* continue type fixing and start test fixing

* test: fix more tests

* fix repository

* fix pivot table computaion + fix mikro orm repository

* fix many to many management and configuration

* fix many to many management and configuration

* fix many to many management and configuration

* update product tag relation configuration

* Introduce experimental dml hooks to fix some issues with categories

* more fixes

* fix product tests

* add missing id prefixes

* fix product category handle management

* test: fix more failing tests

* test: make it all green

* test: fix breaking tests

* fix: build issues

* fix: build issues

* fix: more breaking tests

* refactor: fix issues after merge

* refactor: fix issues after merge

* refactor: surpress types error

* test: fix DML failing tests

* improve many to many inference + tests

* Wip fix columns from product entity

* remove product model before create hook and manage handle validation and transformation at the service level

* test: fix breaking unit tests

* fix: product module service to not update handle on product update

* fix define link and joiner config

* test: fix joiner config test

* test: fix joiner config test

* fix joiner config primary keys

* Fix joiner config builder

* Fix joiner config builder

* test: remove only modifier from test

* refactor: remove hooks usage from product collection

* refactor: remove hooks usage from product-option

* refactor: remove hooks usage for computing category handle

* refactor: remove hooks usage from productCategory model

* refactor: remove hooks from DML

* refactor: remove cruft

* order dml

* cleanup

* re add foerign key indexes

* wip

* chore: remove unused types

* wip

* changes

* rm raw

* autoincrement

* wip

* rel

* refactor: cleanup

* migration and models configuration adjustments

* cleanup

* number searchable

* fix random ordering

* fix

* test: fix product-category tests

* test: update breaking DML tests

* test: array assertion to not care about ordering

* fix: temporarily apply id ordering for products

* types

* wip

* WIP type improvements

* update order models

* partially fix types temporarely

* rel

* fix: recursive type issue

* improve type inference breaks

* improve type inference breaks

* update models

* rm nullable

* default value

* repository

* update default value handling

* fix unit tests

* WIP

* toMikroORM

* fix relations

* cascades

* fix

* experimental dml hooks

* rm migration

* serial

* nullable autoincrement

* fix model

* model changes

* fix one to one DML

* order test

* fix addresses

* fix unit tests

* Re align dml entity name inference

* update model table name config

* update model table name config

* revert

* update return relation

* WIP

* hasOne

* models

* fix

* model

* initial commit

* cart service

* order module

* utils unit test

* index engine

* changeset

* merge

* fix hasOne with fk

* update

* free text filter per entity

* tests

* prod category

* property string many to many

* fix big number

* link modules migration set names

* merge

* shipping option rules

* serializer

* unit test

* fix test mikro orm init

* fix test mikro orm init

* Maintain merge object properties

* fix test mikro orm init

* prevent unit test from connecting to db

* wip

* fix test

* fix test

* link test

* schema

* models

* auto increment

* hook

* model hooks

* order

* wip

* orm version

* request return field

* fix return configuration on order model

* workflows

* core flows

* unit test

* test

* base repo

* test

* base repo

* test fix

* inventory move

* locking inventory

* test

* free text fix

* rm timeout mock

* migrate fulfillment values

* v6.4.3

* cleanup

* link-modules update sql

* revert test

* remove fake timers

---------

Co-authored-by: adrien2p <adrien.deperetti@gmail.com>
Co-authored-by: Harminder Virk <virk.officials@gmail.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2025-01-21 08:04:47 -05:00
Adrien de Peretti
0a077d48e1 chore(workflow-engine): Migrate to DML (#10477)
RESOLVES FRMW-2832
RESOLVES FRMW-2833

**What**
Migrate workflow engines to DML. Alos includes and update to the linkable generation which now takes into account id and primary keys to generate the linkable instead of only primary keys
2024-12-06 13:23:07 +00:00
Carlos R. L. Rodrigues
1eef324af3 fix(orchestration): fix set step failure (#10031)
What:
 - copy data before saving checkpoint
 - removed unused data format function
 - properly handle registerStepFailure to not throw
 - emit onFinish event even when execution failed
2024-11-12 10:06:36 +00:00
Harminder Virk
68560787e5 breaking: rename package names to be consistent and under @medusajs scope (#9580) 2024-10-16 22:28:09 +05:30
Adrien de Peretti
1d8939df3a chore(): Allow to register modules through array (#9522) 2024-10-11 15:17:00 +02:00
Adrien de Peretti
6937d74252 chore(): Move dependencies around (#9278)
RESOLVES FRMW-2716 [1]

**What**
First stab at re organising dependencies in the modules and their usage.
2024-09-24 14:58:04 +00:00
Harminder Virk
9f72fb5902 Chore: cleanup workflows SDK (#9244)
Fixes: FRMW-2712
2024-09-23 11:04:38 +00:00
Carlos R. L. Rodrigues
d6ff526820 fix(link-modules): table name (#9151)
FIXES: FRMW-2706
2024-09-16 19:08:42 +00:00
Carlos R. L. Rodrigues
ef8dc4087e feat: run nested async workflows (#9119) 2024-09-16 13:06:45 +00:00
Carlos R. L. Rodrigues
950cf9af79 chore: remove container registration name (#9137) 2024-09-16 06:22:24 -03:00
Carlos R. L. Rodrigues
fdd0543011 chore: joiner config entity property (#9084) 2024-09-11 06:25:25 -03:00
Stevche Radevski
c58a35f0c0 feat: Add support for private files to file module (#8169) 2024-07-18 09:46:10 +02:00
Stevche Radevski
26d600b6db feat: Ensure async workflow executions have access to shared container (#8157)
* feat: Ensure async workflow executions have access to shared container

* fix: Register workflow worker on application start
2024-07-17 12:17:48 +02:00
Adrien de Peretti
b368251ca3 chore: Migrate modules to use the Module util (#7964)
**What**
Apply the `Module` util to each module to handle the export and provide the linkable configuration object.

- Most joiner config that does not have any special config are removed/cleaned up in favor of the autogenerated one or simpler one
- linkable are now available in all modules
- cleaned up the dependencies of the modules

FIXES CORE-2410
2024-07-05 13:00:40 +00:00
Carlos R. L. Rodrigues
9badad24aa feat(workflows-sdk): conditional step (#7912)
* chore: move ModuleRegistrationName to utils

* chore(workflows-sdk): conditional step

* type

* when condition
2024-07-05 10:54:18 +02:00
Carlos R. L. Rodrigues
5600e58b7f chore(orchestration): idempotent (#7771) 2024-06-25 10:34:00 -03:00
Adrien de Peretti
937a632eb6 chore: make module loaders DML aware and auto generate joiner config (#7781)
* chore: make module loaders DML aware and auto generate joiner config

* fixes and cleanup

* improve dml entity check

* add unit tests on load resources

* cleanup deps

* cleanup deps

* cleanup Modules

* finalise

* fix modules-sdk jest

* fix modules-sdk jest

* fix import

* fix import
2024-06-20 18:18:07 +02:00
Adrien de Peretti
48963f55ef Chore/rm main entity concept (#7709)
**What**
Update the `MedusaService` class, factory and types to remove the concept of main modules. The idea being that all method will be explicitly named and suffixes to represent the object you are trying to manipulate.
This pr also includes various fixes in different modules

Co-authored-by: Stevche Radevski <4820812+sradevski@users.noreply.github.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2024-06-19 13:02:16 +00:00
Adrien de Peretti
e0b14519f1 fix: Medusa service base method transaction (#7758)
**What**
Remove transaction decorator from the base medusa service method, the transaction will always be coming from the shared context. It fixes the issue that when you consume a base method directly it will return a proper tuple from the DB instead of the one from the entity map cc @VariableVic 

**NOTE**
This pr also fix some categories issues in the product module which was preventing the tests from working. if @sradevski you could have a look later, in the mean time we can still merge it

FIXES CORE-2342
2024-06-18 10:58:54 +00:00
Adrien de Peretti
afae395fea chore: ensure the event group id is kept for async workflow (#7668)
* chore: ensure the event group id is kept for async workflow

* revert test

* fix tests

---------

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
Co-authored-by: Riqwan Thamir <rmthamir@gmail.com>
2024-06-11 13:20:17 +02:00
Stevche Radevski
3d72002c28 fix: Remove scheduled workflows if they no longer exist when executed (#7673) 2024-06-11 12:57:25 +02:00
Stevche Radevski
69410162f6 feat: Add support for scheduled workflows (#7651)
We still need to:
But wanted to open the PR for early feedback on the approach
2024-06-10 14:49:52 +00:00
Adrien de Peretti
4eae25e1ef chore(): Reorganize modules (#7210)
**What**
Move all modules to the modules directory
2024-05-02 15:33:34 +00:00