Commit Graph

40 Commits

Author SHA1 Message Date
Adrien de Peretti
28958b2e26 Chore/orchestration storage improvements (#12178)
**What**
Cleanup and improve workflow storage utility
2025-04-18 08:35:23 +00:00
Adrien de Peretti
8618e6ee38 fix(): Properly handle workflow as step now that events are fixed entirely (#12196)
**What**
Now that all events management are fixed in the workflows life cycle, the run as step needs to leverage the workflow engine if present (which should always be the case for async workflows) in order to ensure the continuation and the ability to mark parent step in parent workflow as success or failure

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-04-17 09:34:19 +00:00
Carlos R. L. Rodrigues
31abba8cde fix(orchestrator): save checkpoint before async step (#12138) 2025-04-10 15:36:36 +00:00
Adrien de Peretti
13e159d8ad fix(workflow-engine-*): Prevent passing shared context reference (#11873)
* fix(workflow-engine-*): Prevent passing shared context reference

* fix(workflow-engine-*): Prevent passing shared context reference

* prevent tests from hanging

* fix event handling

* add integration tests

* use interval for scheduled in tests

* skip tests for now

* Create silent-glasses-enjoy.md

* fix cancel

* changeset

* push multiple aliases

* test multiple field alias

* increase wait time to index on test

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
2025-04-09 10:39:29 +02:00
Carlos R. L. Rodrigues
0625f76cd4 chore(workflow-engine): export cancel method (#11844)
What:
  * Workflow engine exports the method `cancel` to revert a workflow.
2025-03-17 12:59:09 +00:00
Adrien de Peretti
fc652ea51e fix(workflow-engine-*): scheduled jobs interval (#11800)
**What**
Currently only cron pattern are supported by scheduled jobs, this can lead to issue. for example you set the pattern to execute every hours at minute 0 and second 0 (as it is expected to execute at exactly this constraint) but due to the moment it gets executed we our out of the second 0 then the job wont get executed until the next scheduled cron table execution.

With this pr we introduce the `interval` configuration which allows you the specify a delay between execution in ms (e.g every minute -> 60 * 1000 ms) which ensure that once a job is executed another one is scheduled for a minute later.

**Usage**
```ts
// jobs/job-1.ts
const thirtySeconds = 30 * 1000

export const config = {
  name: "job-1",
  schedule: {
    interval: thirtySeconds
  },
}
```
2025-03-13 15:05:13 +00:00
Carlos R. L. Rodrigues
6053ec3976 chore(workflow-engine-redis): remove repeatable jobs from old queue (#11822)
What:
  * Old  deployments have repeatable jobs registered in a wrong queue. When the `server` instance picks that job, the workflow doesn't exist, it calls to remove the job, which then removes the job from the new queue.
  * This PR cleans up any repeatable job from the queue that is exclusive to handle workflows.
2025-03-12 14:54:10 +00:00
Adrien de Peretti
72d2cf9207 fix(workflow-engines): race condition when retry interval is used (#11771) 2025-03-12 09:53:34 -03:00
Adrien de Peretti
cc8422d3a1 fix(workflow-engine-redis): Split the queues and respect worker mode for job executions (#11740)
**What**
Currently, the workflow engine redis does not make any distinction between worker modes, when starting as server, the engine listen to the queue which contains everything and try to execute the corresponding workflow which does not exists since job workflows are not loaded in server mode. Now, a dedicated queue is created for jobs and the worker is only started if the instance is not in server mode. In order to clean up the old queue, if the old queue trigger a scheduled job then it gets removed from the queue since it will get re added to the new queue by the new worker instances
2025-03-06 11:52:52 +00:00
Carlos R. L. Rodrigues
c8376a9f15 chore(medusa): clear workflow execution (#11200)
CLOSES: SUP-704
2025-02-03 10:47:32 +00:00
Adrien de Peretti
da3906efa4 fix: Unique constraint should account for soft deleted records (#11048)
FIXES FRMW-2878

**What**
Currently, the `one-to-one` unique constraints does not account for deleted record. This prevents from inserting a new record wth the same fk if another one is deleted.

**Caveat**
`hasOne` with FK option is meant to be a special case, for example a many to one - one to many without defining the other side of the relation. In that case we don't handle this behaviour and keep it as it is
2025-01-22 07:42:06 +00:00
Adrien de Peretti
0a077d48e1 chore(workflow-engine): Migrate to DML (#10477)
RESOLVES FRMW-2832
RESOLVES FRMW-2833

**What**
Migrate workflow engines to DML. Alos includes and update to the linkable generation which now takes into account id and primary keys to generate the linkable instead of only primary keys
2024-12-06 13:23:07 +00:00
Carlos R. L. Rodrigues
90ae187e09 fix(workflows-sdk): name for when/then step (#10459) 2024-12-05 15:47:42 -03:00
Carlos R. L. Rodrigues
1eef324af3 fix(orchestration): fix set step failure (#10031)
What:
 - copy data before saving checkpoint
 - removed unused data format function
 - properly handle registerStepFailure to not throw
 - emit onFinish event even when execution failed
2024-11-12 10:06:36 +00:00
Adrien de Peretti
34d57870ad chore: workflow internals improvementss (#9455) 2024-10-10 09:11:56 +02:00
Adrien de Peretti
e096feb7d5 chore: Update modules deps (#9286) 2024-09-26 11:14:35 +05:30
Adrien de Peretti
6937d74252 chore(): Move dependencies around (#9278)
RESOLVES FRMW-2716 [1]

**What**
First stab at re organising dependencies in the modules and their usage.
2024-09-24 14:58:04 +00:00
Harminder Virk
9e711720dd chore: upgrade moduleResolution to Node16 (#9269) 2024-09-24 17:19:20 +05:30
Adrien de Peretti
58167b5dfa feat(index): Index module foundation (#9095)
**What**
Index module foundation

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2024-09-18 19:04:04 +00:00
Carlos R. L. Rodrigues
adbeb9cc1d fix(workflow-engine-*): pass container to flow (#9180) 2024-09-18 08:44:11 -03:00
Carlos R. L. Rodrigues
ef8dc4087e feat: run nested async workflows (#9119) 2024-09-16 13:06:45 +00:00
Carlos R. L. Rodrigues
058378970a chore: dependecies reorg (#8535) 2024-08-23 07:16:04 -03:00
Stevche Radevski
9ba52021d0 fix: Log on error by default in async workflow executions (#8723) 2024-08-22 13:33:11 +02:00
Carlos R. L. Rodrigues
eb0bfe9f33 feat(workflows-sdk): log on error (#8666) 2024-08-20 06:23:24 -03:00
Harminder Virk
19d30df624 refactor: Run migrations and links sync after loading the app (#8582) 2024-08-14 14:09:24 +05:30
Stevche Radevski
4155d0354f fix: Use a global reference to distributed transaction and scheduler (#8462)
* fix: Add storage cleanup on scheduled jobs

* fix: Use a global reference to distributed transaction and scheduler classes
2024-08-06 14:14:51 +02:00
Stevche Radevski
5c09f0813d fix: Upgrade bullmq and adjust to breaking changes in repeatable jobs (#8446) 2024-08-05 13:00:27 +02:00
Stevche Radevski
bcba5b3a6c chore: Make scheduled job tests less likely to fail (#8241) 2024-07-23 16:59:27 +02:00
Stevche Radevski
26d600b6db feat: Ensure async workflow executions have access to shared container (#8157)
* feat: Ensure async workflow executions have access to shared container

* fix: Register workflow worker on application start
2024-07-17 12:17:48 +02:00
Adrien de Peretti
b368251ca3 chore: Migrate modules to use the Module util (#7964)
**What**
Apply the `Module` util to each module to handle the export and provide the linkable configuration object.

- Most joiner config that does not have any special config are removed/cleaned up in favor of the autogenerated one or simpler one
- linkable are now available in all modules
- cleaned up the dependencies of the modules

FIXES CORE-2410
2024-07-05 13:00:40 +00:00
Adrien de Peretti
617a5972bf feat: refactor module joiner config and links generation (#7859)
* feat: refactor module joiner config and links generation

* improve typings

* WIP

* WIP

* WIP

* rename type file

* create link config

* finish typings and add utils

* improve links

* WIP typings

* finalize ExportModule utils

* finalize ExportModule utils

* fix: dml tests

* improve and fixes

* simplify typings with id changes

* add toJSON

* multiple fixes and entity builder fixes

* fix currency searchable

* fix tests

* medusa service refactoring

* cleanup

* cleanup and fixes

* make module name optional

* renaming

---------

Co-authored-by: Harminder Virk <virk.officials@gmail.com>
2024-07-03 13:12:49 +02:00
Carlos R. L. Rodrigues
5600e58b7f chore(orchestration): idempotent (#7771) 2024-06-25 10:34:00 -03:00
Adrien de Peretti
937a632eb6 chore: make module loaders DML aware and auto generate joiner config (#7781)
* chore: make module loaders DML aware and auto generate joiner config

* fixes and cleanup

* improve dml entity check

* add unit tests on load resources

* cleanup deps

* cleanup deps

* cleanup Modules

* finalise

* fix modules-sdk jest

* fix modules-sdk jest

* fix import

* fix import
2024-06-20 18:18:07 +02:00
Adrien de Peretti
48963f55ef Chore/rm main entity concept (#7709)
**What**
Update the `MedusaService` class, factory and types to remove the concept of main modules. The idea being that all method will be explicitly named and suffixes to represent the object you are trying to manipulate.
This pr also includes various fixes in different modules

Co-authored-by: Stevche Radevski <4820812+sradevski@users.noreply.github.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2024-06-19 13:02:16 +00:00
Adrien de Peretti
d2a5201eeb chore: Refactor and improve abstract module service factory (#7688)
* chore: Refactor and improve abstract module service factory

* align naming

* clean up some template args and tests

* partially migrate modules

* partially migrate modules

* migrate more modules

* migrate last modules

* fix typings

* rename interface

* rename interface

* fixes

* fixes

* rm local plain tests
2024-06-13 13:12:37 +02:00
Stevche Radevski
3d72002c28 fix: Remove scheduled workflows if they no longer exist when executed (#7673) 2024-06-11 12:57:25 +02:00
Stevche Radevski
69410162f6 feat: Add support for scheduled workflows (#7651)
We still need to:
But wanted to open the PR for early feedback on the approach
2024-06-10 14:49:52 +00:00
Harminder Virk
11528526fa feat: loosely typed container 2024-05-31 15:22:03 +05:30
Adrien de Peretti
63623422fe chore(): Autoload module resources (#7291)
**What**
- automatically build and consume connection and container loader if not exported by the module
- therefore load the services and repositories automatically, including baseRepository
- automatically build run and revert migrations if not provided
- cleaup modules to remove extra unnecessary bits and pieces
- remove the `initializeFactory` in favor of using `medusaApp`

Should drastically improve the module building DX by removing a lot of boilerplate to handle by the user, that plus the base entity should simplify quite a lot the flow cc @shahednasser 

**Note**
I had to choose a way to identify connection and container loader from the exported loader from the module. I decided to go with named function `connectionLoader` and `containerLoader`, also, now the factories will return named function so if the user use the factories we are providing to build those loaders, the function will also be named and identified
2024-05-13 12:12:36 +00:00
Adrien de Peretti
4eae25e1ef chore(): Reorganize modules (#7210)
**What**
Move all modules to the modules directory
2024-05-02 15:33:34 +00:00