* feat(index): add filterable fields to link definition
* rm comment
* break recursion
* validate read only links
* validate filterable
* gql schema array
* link parents
* isInverse
* push id when not present
* Fix ciruclar relationships and add tests to ensure proper behaviour (part 1)
* log and fallback to entity.alias
* cleanup and fixes
* cleanup and fixes
* cleanup and fixes
* fix get attributes
* gql type
* unit test
* array inference
* rm only
* package.json
* pacvkage.json
* fix link retrieval on duplicated entity type and aliases + tests
* link parents as array
* Match only parent entity
* rm comment
* remove hard coded schema
* extend types
* unit test
* test
* types
* pagination type
* type
* fix integration tests
* Improve performance of in selection
* use @@ to filter property
* escape jsonPath
* add Event Bus by default
* changeset
* rm postgres analyze
* estimate count
* new query
* parent aliases
* inner query w/ filter and sort relations
* address comments
---------
Co-authored-by: adrien2p <adrien.deperetti@gmail.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
**What**
First iteration to prevent events from overwhelming the systems.
- Group emitted event ids when possible instead of creating a message per id which leads to reduced amount of events to process massively in cases of import for example
- Update the index engine to process event data in batches of 100
- Update event handling by the index engine to be able to upsert by batch as well
- Fix index engine build config for intermediate listeners inferrence
* fix(): Allow to query deleted records from graph API
* fix(): Allow to query deleted records from graph API
* handle empty q value
* update staled at sync
* rename integration tests file
* Create strong-houses-marry.md
* try to fix flacky tests
* fix pricing context
* update changeset
* update changeset
* fix import
* skip test for now
---------
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
**What**
- Add index engine feature flag
- apply it to the `store/products` end point as well as `admin/products`
- Query builder various fixes
- search capabilities on full data of every entities. The `q` search will be applied to all involved joined table for selection/where clauses
Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
What:
- `query.index` helper. It queries the index module, and aggregate the rest of requested fields/relations if needed like `query.graph`.
Not covered in this PR:
- Hydrate only sub entities returned by the query. Example: 1 out of 5 variants have returned, it should only hydrate the data of the single entity, currently it will merge all the variants of the product.
- Generate types of indexed data
example:
```ts
const query = container.resolve(ContainerRegistrationKeys.QUERY)
await query.index({
entity: "product",
fields: [
"id",
"description",
"status",
"variants.sku",
"variants.barcode",
"variants.material",
"variants.options.value",
"variants.prices.amount",
"variants.prices.currency_code",
"variants.inventory_items.inventory.sku",
"variants.inventory_items.inventory.description",
],
filters: {
"variants.sku": { $like: "%-1" },
"variants.prices.amount": { $gt: 30 },
},
pagination: {
order: {
"variants.prices.amount": "DESC",
},
},
})
```
This query return all products where at least one variant has the title ending in `-1` and at least one price bigger than `30`.
The Index Module only hold the data used to paginate and filter, and the returned object is:
```json
{
"id": "prod_01JKEAM2GJZ14K64R0DHK0JE72",
"title": null,
"variants": [
{
"id": "variant_01JKEAM2HC89GWS95F6GF9C6YA",
"sku": "extra-variant-1",
"prices": [
{
"id": "price_01JKEAM2JADEWWX72F8QDP6QXT",
"amount": 80,
"currency_code": "USD"
}
]
}
]
}
```
All the rest of the fields will be hydrated from their respective modules, and the final result will be:
```json
{
"id": "prod_01JKEAY2RJTF8TW9A23KTGY1GD",
"description": "extra description",
"status": "draft",
"variants": [
{
"sku": "extra-variant-1",
"barcode": null,
"material": null,
"id": "variant_01JKEAY2S945CRZ6X4QZJ7GVBJ",
"options": [
{
"value": "Red"
}
],
"prices": [
{
"amount": 20,
"currency_code": "CAD",
"id": "price_01JKEAY2T2EEYSWZHPGG11B7W7"
},
{
"amount": 80,
"currency_code": "USD",
"id": "price_01JKEAY2T2NJK2E5468RK84CAR"
}
],
"inventory_items": [
{
"variant_id": "variant_01JKEAY2S945CRZ6X4QZJ7GVBJ",
"inventory_item_id": "iitem_01JKEAY2SNY2AWEHPZN0DDXVW6",
"inventory": {
"sku": "extra-variant-1",
"description": "extra variant 1",
"id": "iitem_01JKEAY2SNY2AWEHPZN0DDXVW6"
}
}
]
}
]
}
```
Co-authored-by: Adrien de Peretti <25098370+adrien2p@users.noreply.github.com>
Closes: FRMW-2892, FRMW-2893
**What**
Wired up the building block that we merged previously in order to manage data synchronization. The flow is as follow
- On application start
- Build schema object representation from configuration
- Check configuration changes
- if new entities configured
- Data synchronizer initialize orchestrator and start sync
- for each entity
- acquire lock
- mark existing data as staled
- sync all data by batch
- marked them not staled anymore
- acknowledge each processed batch and renew lock
- update metadata with last synced cursor for entity X
- release lock
- remove all remaining staled data
- if any entities removed from last configuration
- remove the index data and relations
Co-authored-by: Carlos R. L. Rodrigues <37986729+carlos-r-l-rodrigues@users.noreply.github.com>
* ../../core/types/src/dml/index.ts
* ../../core/types/src/dml/index.ts
* fix: relationships mapping
* handle nullable foreign keys types
* handle nullable foreign keys types
* handle nullable foreign keys types
* continue to update product category repository
* fix all product category repositories issues
* fix product category service types
* fix product module service types
* fix product module service types
* fix repository template type
* refactor: use a singleton DMLToMikroORM factory instance
Since the MikroORM MetadataStorage is global, we will also have to turn DML
to MikroORM entities conversion use a global bucket as well
* refactor: update product module to use DML in tests
* wip: tests
* WIP product linkable fixes
* continue type fixing and start test fixing
* test: fix more tests
* fix repository
* fix pivot table computaion + fix mikro orm repository
* fix many to many management and configuration
* fix many to many management and configuration
* fix many to many management and configuration
* update product tag relation configuration
* Introduce experimental dml hooks to fix some issues with categories
* more fixes
* fix product tests
* add missing id prefixes
* fix product category handle management
* test: fix more failing tests
* test: make it all green
* test: fix breaking tests
* fix: build issues
* fix: build issues
* fix: more breaking tests
* refactor: fix issues after merge
* refactor: fix issues after merge
* refactor: surpress types error
* test: fix DML failing tests
* improve many to many inference + tests
* Wip fix columns from product entity
* remove product model before create hook and manage handle validation and transformation at the service level
* test: fix breaking unit tests
* fix: product module service to not update handle on product update
* fix define link and joiner config
* test: fix joiner config test
* test: fix joiner config test
* fix joiner config primary keys
* Fix joiner config builder
* Fix joiner config builder
* test: remove only modifier from test
* refactor: remove hooks usage from product collection
* refactor: remove hooks usage from product-option
* refactor: remove hooks usage for computing category handle
* refactor: remove hooks usage from productCategory model
* refactor: remove hooks from DML
* refactor: remove cruft
* order dml
* cleanup
* re add foerign key indexes
* wip
* chore: remove unused types
* wip
* changes
* rm raw
* autoincrement
* wip
* rel
* refactor: cleanup
* migration and models configuration adjustments
* cleanup
* number searchable
* fix random ordering
* fix
* test: fix product-category tests
* test: update breaking DML tests
* test: array assertion to not care about ordering
* fix: temporarily apply id ordering for products
* types
* wip
* WIP type improvements
* update order models
* partially fix types temporarely
* rel
* fix: recursive type issue
* improve type inference breaks
* improve type inference breaks
* update models
* rm nullable
* default value
* repository
* update default value handling
* fix unit tests
* WIP
* toMikroORM
* fix relations
* cascades
* fix
* experimental dml hooks
* rm migration
* serial
* nullable autoincrement
* fix model
* model changes
* fix one to one DML
* order test
* fix addresses
* fix unit tests
* Re align dml entity name inference
* update model table name config
* update model table name config
* revert
* update return relation
* WIP
* hasOne
* models
* fix
* model
* initial commit
* cart service
* order module
* utils unit test
* index engine
* changeset
* merge
* fix hasOne with fk
* update
* free text filter per entity
* tests
* prod category
* property string many to many
* fix big number
* link modules migration set names
* merge
* shipping option rules
* serializer
* unit test
* fix test mikro orm init
* fix test mikro orm init
* Maintain merge object properties
* fix test mikro orm init
* prevent unit test from connecting to db
* wip
* fix test
* fix test
* link test
* schema
* models
* auto increment
* hook
* model hooks
* order
* wip
* orm version
* request return field
* fix return configuration on order model
* workflows
* core flows
* unit test
* test
* base repo
* test
* base repo
* test fix
* inventory move
* locking inventory
* test
* free text fix
* rm timeout mock
* migrate fulfillment values
* v6.4.3
* cleanup
* link-modules update sql
* revert test
* remove fake timers
---------
Co-authored-by: adrien2p <adrien.deperetti@gmail.com>
Co-authored-by: Harminder Virk <virk.officials@gmail.com>
Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
What:
* removes resouces type "shared" or "isolated" from internal modules.
* modules can have an isolated database connection by providing a database config as part of their options on `medusa-config`
CLOSES: FRMW-2593
**What**
Align the index engine API to be similar to the Query API
## Example
```ts
// Benefit from the same level of typing like the remote query
const { data, metadata } = await indexEngine.query<'product'>({
fields: [
"product.*",
"product.variants.*",
"product.variants.prices.*",
],
filters: {
product: {
variants: {
prices: {
amount: { $gt: 50 },
},
},
},
},
pagination: {
order: {
product: {
variants: {
prices: {
amount: "DESC",
},
},
},
},
},
})
```
