Commit Graph

85 Commits

Author SHA1 Message Date
Adrien de Peretti
fe49b567d6 chore: Backend HMR (expriemental) (#14074)
**What**

 This PR introduces experimental Hot Module Replacement (HMR) for the Medusa backend, enabling developers to see code changes reflected immediately without restarting the server. This significantly improves the development experience by reducing iteration time.

### Key Features

  - Hot reload support for:
    - API Routes
    - Workflows & Steps
    - Scheduled Jobs
    - Event Subscribers
    - Modules
  - IPC-based architecture: The dev server runs in a child process, communicating with the parent watcher via IPC. When HMR fails, the child process is killed and restarted, ensuring
  clean resource cleanup.
  - Recovery mechanism: Automatically recovers from broken module states without manual intervention.
  - Graceful fallback: When HMR cannot handle a change (e.g., medusa-config.ts, .env), the server restarts completely.


### Architecture
```mermaid
  flowchart TB
      subgraph Parent["develop.ts (File Watcher)"]
          W[Watch Files]
      end

      subgraph Child["start.ts (HTTP Server)"]
          R[reloadResources]
          R --> MR[ModuleReloader]
          R --> WR[WorkflowReloader]
          R --> RR[RouteReloader]
          R --> SR[SubscriberReloader]
          R --> JR[JobReloader]
      end

      W -->|"hmr-reload"| R
      R -->|"hmr-result"| W
```

### How to enable it

Backend HMR is behind a feature flag. Enable it by setting:

```ts
  // medusa-config.ts
  module.exports = defineConfig({
    featureFlags: {
      backend_hmr: true
    }
  })
```

or

```bash
export MEDUSA_FF_BACKEND_HMR=true
```

or

```
// .env
MEDUSA_FF_BACKEND_HMR=true
```

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-12-08 08:48:36 +00:00
Adrien de Peretti
d51ae2768b chore(workflow-engine-*): cleanup and improvements (#13789)
**What**
Cleanup recent work on workflows
2025-10-23 10:50:24 +00:00
Adrien de Peretti
516f5a3896 fix: workflow async concurrency (#13769)
* executeAsync

* || 1

* wip

* stepId

* stepId

* wip

* wip

* continue versioning management changes

* fix and improve concurrency

* update in memory engine

* remove duplicated test

* fix script

* Create weak-drinks-confess.md

* fixes

* fix

* fix

* continuation

* centralize merge checkepoint

* centralize merge checkpoint

* fix locking

* rm only

* Continue improvements and fixes

* fixes

* fixes

* hasAwaiting will be recomputed

* fix orchestrator engine

* bump version on async parallel steps only

* mark as delivered fix

* changeset

* check partitions

* avoid saving when having parent step

* cart test

---------

Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2025-10-20 15:29:19 +02:00
Adrien de Peretti
8734866eb1 fix(): Transform map (#13655)
**What**
It seems that for some reason the weak map fail in some scenario, but after investigation, the usage of map would not have a bad impact as it will be released after the Distributed transaction if finished. Therefore, falling back to Map instead

FIXES https://github.com/medusajs/medusa/issues/13654

NOTE: Waiting for the user feedback as he is also using node 18. We also use the exact same pattern in all our core flows without issues 🤔
2025-10-02 15:54:11 +00:00
Adrien de Peretti
76aa4a48b3 fix(): workflows concurrency (#13645) 2025-10-02 11:11:38 -03:00
Adrien de Peretti
12a96a7c70 chore(): Move peer deps into a single package and re export from framework (#13439)
* chore(): Move peer deps into a single package and re export from framework

* WIP

* update core packages

* update cli and deps

* update medusa

* update exports path

* remove analyze

* update modules deps

* finalise changes

* fix yarn

* fix import

* Refactor peer dependencies into a single package

Consolidate peer dependencies into one package and re-export from the framework.

* update changeset

* Update .changeset/brown-cows-sleep.md

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>

* rm deps

* fix deps

* increase timeout

* upgrade version

* update versions

* update versions

* fixes

* update lock

* fix missing import

* fix missing import

---------

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
2025-09-22 18:36:22 +02:00
Adrien de Peretti
0079464f32 fix(): temporary transform cached data (#13467)
**What**
Properly store key obj reference for weak map usage
2025-09-11 07:31:01 +00:00
Adrien de Peretti
8e5c22a8e8 chore(): Improve workflows sdk tooling (#13421)
RESOLVES CORE-1171

**What**
 - Reduced async overhead for objects with mixed sync/async properties
 - Lower memory pressure from eliminated promise allocations
 - Faster primitive value processing with early returns
 - Better concurrency through selective batching of truly async operations
 - Event loop friendly behavior preventing artificial delays
 - Reduced memory allocation from eliminated duplicate processing
 - Decreased GC pressure from WeakMap-based caching instead of map

**note**
Now, `resolveValue`always treat every resoltion as sync operation unless it is not, meaning we do not create promise overhead when not necessary and only when actually treating with promises
2025-09-08 16:24:10 +00:00
Adrien de Peretti
d7692100e7 chore(orchestration): add support for autoRetry, maxAwaitingRetries, retryStep (#13391)
RESOLVES CORE-1163
RESOLVES CORE-1164

**What**

### Add support for non auto retryable steps.

When marking a step with `maxRetries`, when it will fail it will be marked as temporary failure and then retry itself automatically. Thats the default behaviour, if you now add `autoRetry: false`, when the step will fail it will be marked as temporary failure but not retry automatically. you can now call the workflow engine run to resume the workflow from the failing step to be retried.

### Add support for `maxAwaitingRetries`

When setting `retyIntervalAwaiting` a step that is awaiting will be retried after the specified interval without maximun retry. Now you can set `maxAwaitingRetries` to force a maximum awaiting retry number

### Add support to manually retry an awaiting step

In some scenario, either a machine dies while a step is executing or a step is taking longer than expected, you can now call `retryStep` on the workflow engine to force a retry of the step that is supposedly stucked
2025-09-08 12:46:30 +00:00
Adrien de Peretti
55a35e4721 chore(workflows-sdk, utils): Prevent unnecessary serialization (#13413)
* chore(workflows-sdk, utils): Prevent unnecessary serialization

* Create poor-mugs-cheat.md
2025-09-04 16:19:35 +02:00
Carlos R. L. Rodrigues
9412669e65 chore: idempotent cart operations (#13236)
* chore(core-flows): idempotent cart operations

* changeset

* add tests

* revert

* revert route

* promo test

* skip bugs

* fix test

* tests

* avoid workflow name conflict

* prevent nested workflow from being deleted until the top level parent finishes

* remove unused setTimeout

* update changeset

* rm comments

---------

Co-authored-by: adrien2p <adrien.deperetti@gmail.com>
2025-08-28 15:04:00 +02:00
Adrien de Peretti
ff152e7ace fix(orchestration): Use the step definition max retries on set step failure (#13319)
* fix(orchestration): Use the step definition max retries on set step failure

* Create sweet-turkeys-wait.md

* allow to force permanent failure

* update changeset
2025-08-28 14:35:31 +02:00
Carlos R. L. Rodrigues
e413cfefc2 feat(utils): define file config (#13283)
** What
 - Allow auto-loaded Medusa files to export a config object.
 - Currently supports isDisabled to control loading.
 - new instance `FeatureFlag` exported by `@medusajs/framework/utils`
 - `feature-flags` is now a supported folder for medusa projects, modules, providers and plugins. They will be loaded and added to `FeatureFlag`

** Why
 - Enables conditional loading of routes, migrations, jobs, subscribers, workflows, and other files based on feature flags.

```ts
// /src/feature-flags

import { FlagSettings } from "@medusajs/framework/feature-flags"

const CustomFeatureFlag: FlagSettings = {
  key: "custom_feature",
  default_val: false,
  env_key: "FF_MY_CUSTOM_FEATURE",
  description: "Enable xyz",
}

export default CustomFeatureFlag
```

```ts
// /src/modules/my-custom-module/migration/Migration20250822135845.ts

import { FeatureFlag } from "@medusajs/framework/utils"

export class Migration20250822135845 extends Migration {
  override async up(){ }
  override async down(){ }
}

defineFileConfig({
  isDisabled: () => !FeatureFlag.isFeatureEnabled("custom_feature")
})
```
2025-08-26 12:22:30 +00:00
Carlos R. L. Rodrigues
1bd455bc7b fix(workflows-sdk): fix step name config used before default (#12926) 2025-07-22 09:53:24 -03:00
Riqwan Thamir
9a62f359f1 fix(core-flows,workflows-sdk): compensate account holders only when its created (#12825)
* fix(core-flows,workflows-sdk): compensate account holders only when its created

* chore: remove only
2025-06-26 12:30:08 +02:00
Adrien de Peretti
1a78476608 fix(workflow-sdk): Async/nested runAsStep propagation (#12675)
FIXES CLO-524

**What**
Add hidden stepDefinition object as part of the step argument and ensure the runAsStep handlers rely on the latest definition when config is being used on the returned step in order to ensure async configuration propagation and nested configuration
2025-06-10 07:23:12 +00:00
Carlos R. L. Rodrigues
490bd7647f fix(core-flows): complete cart improvements (#12646)
* fix(core-flows): use cartId as transactionId and acquire lock to complete cart

* fix cart update compensation
2025-05-30 14:15:08 +01:00
Harminder Virk
117fc25aea feat: run workflow hooks inside a when/then block (#11963)
* feat: run workflow hooks inside a when/then block

* fix conditionals and add test

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
2025-05-23 09:52:18 -03:00
Adrien de Peretti
7fdbf2a965 fix(workflows-sdk): Miss match context usage within run as step (#12449)
**What**

Currently, runAsStep keep reference of the workflow context that is being run as step, except that the step is composed for the current workflow composition and not the workflow being run as a step. Therefore, the context are currently miss matched leading to wrong configuration being used in case of async workflows.

**BUG**
This fix allow the runAsStep to use the current composition context to configure the step for the sub workflow to be run

**BUG BREAKING**
fix the step config wrongly used to wrap async step handlers. Now steps configured async through .config that returns a new step response will indeed marked itself as success without the need for background execution or calling setStepSuccess (as it was expected originally)

**FEATURE**
This pr also add support for cancelling running transaction, the transaction will be marked as being cancelled, once the current step finished, it will cancel the transaction to start compensating all previous steps including itself

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-05-14 13:28:16 +00:00
Adrien de Peretti
091041f2da test(core-flows): Ensure productVariantUpdated failure compensate the workflow (#12381) 2025-05-09 13:03:22 +02:00
Adrien de Peretti
80007f3afd feat(workflows-*): Allow to re run non idempotent but stored workflow with the same transaction id if considered done (#12362) 2025-05-06 17:17:49 +02:00
Harminder Virk
c2fe3f1520 fix: steps to return undefined and still chain the config method (#12255)
Fixes: FRMW-2946
2025-04-22 10:57:15 +00:00
Carlos R. L. Rodrigues
e180253d60 feat(orchestration): skip on permanent failure (#12027)
What:
 - Added step config `skipOnPermanentFailure`. Skip all the next steps when the current step fails. If a string is used, the workflow will resume from the given step.
 - Fix `continueOnPermanentFailure` to continue the execution of the flow when a step fails.
 
```ts
createWorkflow("some-workflow", () => {
  errorStep().config({
    skipOnPermanentFailure: true,
  })
  nextStep1() // skipped
  nextStep2() // skipped
})


createWorkflow("some-workflow", () => {
  errorStep().config({
    skipOnPermanentFailure: "resume-from-here",
  });
  nextStep1(); // skipped
  nextStep2(); // skipped
  nextStep3().config({ name: "resume-from-here" }); // executed
  nextStep4(); // executed
});
```
2025-04-17 12:49:58 +00:00
Adrien de Peretti
8618e6ee38 fix(): Properly handle workflow as step now that events are fixed entirely (#12196)
**What**
Now that all events management are fixed in the workflows life cycle, the run as step needs to leverage the workflow engine if present (which should always be the case for async workflows) in order to ensure the continuation and the ability to mark parent step in parent workflow as success or failure

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
2025-04-17 09:34:19 +00:00
Adrien de Peretti
13e159d8ad fix(workflow-engine-*): Prevent passing shared context reference (#11873)
* fix(workflow-engine-*): Prevent passing shared context reference

* fix(workflow-engine-*): Prevent passing shared context reference

* prevent tests from hanging

* fix event handling

* add integration tests

* use interval for scheduled in tests

* skip tests for now

* Create silent-glasses-enjoy.md

* fix cancel

* changeset

* push multiple aliases

* test multiple field alias

* increase wait time to index on test

---------

Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
Co-authored-by: Carlos R. L. Rodrigues <rodrigolr@gmail.com>
2025-04-09 10:39:29 +02:00
Harminder Virk
b1b3b48474 feat: Set pricing context hook (#11891) 2025-03-28 15:17:52 +05:30
Harminder Virk
cab6f3a8ad feat: add support for accessing step results via context (#11907)
Fixes: FRMW-2934

This PR adds support for accessing the results of a hook using the `hook.getResult` method.
2025-03-21 10:15:38 +00:00
Harminder Virk
ff3885500c fix: hooks auto-generated types to have precise input options (#11886) 2025-03-18 15:41:01 +05:30
Adrien de Peretti
72d2cf9207 fix(workflow-engines): race condition when retry interval is used (#11771) 2025-03-12 09:53:34 -03:00
Adrien de Peretti
84f991192e feat(workflows-sdk): Allow when then in parallelize (#11756)
**What**
Update typings to allow using when then inside parallelize
2025-03-06 13:23:37 +00:00
Shahed Nasser
61a0e4f3cf chore(workflows-sdk): fix examples in TSDocs (#11689) 2025-03-03 12:20:21 +02:00
Adrien de Peretti
c250de7919 chore(): Prevent sub workflow events release early + redis unlink (#11641)
**What**
- Prevent event release when a workflow is run as step and finish
- Use `unlink` instead of `del` when removing keys from redist to push the execution to async thread
2025-02-27 10:33:30 +00:00
Adrien de Peretti
7d8f6cf39f fix(): Workflow cancellation + gracefully handle non serializable state (#10674)
FIXES FRMW-2852

**What**
A workflow distributed transaction expect any response and error to be serializable. When it is not the case, the distributed transaction might fail during the save checkpoint that occurs for async steps. This can lead to unexpected behaviour.

With this pr, we introduce a way to handle non serialazable object in a more sustainable manner, this means the following:

- If a workflow throw any non serialazable error (e.g AWS error that contains full IncomingMessage object that related to network communication, think of req/res) then we identify that this object is not serialzable and we clean up the object to make it serializable without loosing the main information, add a new error to the workflow to informed of this issue and can be handled by the user.
- If a response is not serializable (which should not happen at this point because it is handled before by the value resolver), in that case, we wont be able to reuse that response to continue the workflow which means that the workflow is in a non runnable state. In that case we throw a specific error stating that a non serializable context is being provided

**second what**
This pr refactor the `runAsStep` to add better support for workflow cancelation, especially async ones
2025-01-05 13:30:17 +00:00
Shahed Nasser
7c76ee24cb chore: fix documentation links in TSDocs (#10511) 2024-12-09 18:52:02 +02:00
Carlos R. L. Rodrigues
90ae187e09 fix(workflows-sdk): name for when/then step (#10459) 2024-12-05 15:47:42 -03:00
Carlos R. L. Rodrigues
1f3754a75e chore(workflows-sdk): add unit test (#10419) 2024-12-04 10:32:03 +00:00
Carlos R. L. Rodrigues
aeb5b43692 feat(workflows-sdk): add response to permanent failure (#10177) 2024-11-20 17:29:37 -03:00
Adrien de Peretti
10da8b379f chore(workflows-sdk): Display each error individually instead of packed (#10072)
**What**
Instead of displaying a single string prepared with all the errors, log each error individually
2024-11-15 11:27:52 +00:00
Carlos R. L. Rodrigues
1eef324af3 fix(orchestration): fix set step failure (#10031)
What:
 - copy data before saving checkpoint
 - removed unused data format function
 - properly handle registerStepFailure to not throw
 - emit onFinish event even when execution failed
2024-11-12 10:06:36 +00:00
Adrien de Peretti
c0368cca28 fix(workflows-sdk): Paralellize steps rollback issue with config (#9921)
* fix(workflows-sdk): Paralellize steps rollback issue with config

* Create tall-starfishes-travel.md
2024-11-05 08:54:04 +01:00
Shahed Nasser
92bbd7953b chore: update links to v2 docs in source code and comments (#9732) 2024-10-24 15:18:38 +03:00
Adrien de Peretti
7c5415ba3a fix(workflows-sdk):transaction id inheritence (#9507) 2024-10-10 17:08:42 +02:00
Adrien de Peretti
34d57870ad chore: workflow internals improvementss (#9455) 2024-10-10 09:11:56 +02:00
Adrien de Peretti
ad3524ffbe feat: Add useQueryStep (#9384)
* feat: Query step replacement

* fix

* fix types
2024-10-07 12:07:06 +02:00
Harminder Virk
d98f22c7d6 Feat: Move container bindings declaration merging within the framework (#9467) 2024-10-04 15:47:06 +05:30
Harminder Virk
48e00169d2 breaking: move shared HTTP utils to the framework (#9402)
Fixes: FRMW-2728, FRMW-2729

After this PR gets merged the following middleware will be exported from the `@medusajs/framework/http` import path.

- applyParamsAsFilters
- clearFiltersByKey
- applyDefaultFilters
- setContext
- getQueryConfig
- httpCompression
- maybeApplyLinkFilter
- refetchEntities
- unlessPath
- validateBody
- validateQuery

Co-authored-by: Adrien de Peretti <25098370+adrien2p@users.noreply.github.com>
2024-10-03 09:42:00 +00:00
Carlos R. L. Rodrigues
8155c4e9ee fix(workflows-sdk): when then return value (#9427) 2024-10-02 07:40:39 -03:00
Adrien de Peretti
02629625ec feat(orchestration): Provide hint in workflows error (#9400)
* feat(orchestration): Provide hint in workflows error

* remove log

* fix tests

* improve stack

* fix type

* formatting

---------

Co-authored-by: Riqwan Thamir <rmthamir@gmail.com>
2024-10-02 11:54:07 +02:00
Shahed Nasser
20943902f9 chore: update imports in tsdocs (#9379) 2024-10-01 11:04:03 +02:00
Harminder Virk
9f72fb5902 Chore: cleanup workflows SDK (#9244)
Fixes: FRMW-2712
2024-09-23 11:04:38 +00:00