Shahed Nasser
@@ -0,0 +1,489 @@
|
||||
export const metadata = {
|
||||
title: `${pageNumber} Index Module`,
|
||||
}
|
||||
|
||||
# {metadata.title}
|
||||
|
||||
In this chapter, you'll learn about the Index Module and how you can use it.
|
||||
|
||||
<Note type="soon" title="Experimental">
|
||||
|
||||
The Index Module is experimental and still in development, so it is subject to change. Consider whether your application can tolerate minor issues before using it in production.
|
||||
|
||||
</Note>
|
||||
|
||||
## What is the Index Module?
|
||||
|
||||
The Index Module is a tool to perform highly performant queries across modules, for example, to filter linked modules.
|
||||
|
||||
While modules share the same database by default, Medusa [isolates modules](../../modules/isolation/page.mdx) to allow using external data sources or different database types.
|
||||
|
||||
So, when you retrieve data across modules using Query, Medusa aggregates the data coming from diffeent modules to create the end result. This approach limits your ability to filter data by linked modules. For example, you can't filter products (created in the Product Module) by their brand (created in the Brand Module).
|
||||
|
||||
The Index Module solves this problem by ingesting data into a central data store on application startup. The data store has a relational structure that enables efficiently filtering data ingested from different modules (and their data stores). So, when you retrieve data with the Index Module, you're retrieving it from the Index' data store, not the original data source.
|
||||
|
||||
|
||||
|
||||
### Ingested Data Models
|
||||
|
||||
Currently, only the following core data models are ingested into Index when you install it:
|
||||
|
||||
- `Product`
|
||||
- `ProductVariant`
|
||||
- `Price`
|
||||
- `SalesChannel`
|
||||
|
||||
Consequently, you can only index custom data models if they are linked to an ingested data model. You'll learn more about this in the [Ingest Custom Data Models](#how-to-ingest-custom-data-models) section.
|
||||
|
||||
Future versions may add more data models to the list.
|
||||
|
||||
---
|
||||
|
||||
## How to Install the Index Module
|
||||
|
||||
To install the Index Module, run the following command in your Medusa project to install its package:
|
||||
|
||||
```bash npm2yarn
|
||||
npm install @medusajs/index
|
||||
```
|
||||
|
||||
Then, add the Index Module to your Medusa configuration in `medusa-config.ts`:
|
||||
|
||||
```ts title="medusa-config.ts"
|
||||
module.exports = defineConfig({
|
||||
// ...
|
||||
modules: [
|
||||
// ...
|
||||
{
|
||||
resolve: "@medusajs/index",
|
||||
},
|
||||
],
|
||||
})
|
||||
```
|
||||
|
||||
Finally, run the migrations to create the necessary tables for the Index Module in your database:
|
||||
|
||||
```bash
|
||||
npx medusa db:migrate
|
||||
```
|
||||
|
||||
The index module only ingests data when you start your Medusa server. So, to ingest the [currently supported data models](#ingested-data-models), start the Medusa application:
|
||||
|
||||
```bash npm2yarn
|
||||
npm run dev
|
||||
```
|
||||
|
||||
The ingestion process may take a while if your product catalog is large. You'll see the following messages in the logs:
|
||||
|
||||
```bash
|
||||
info: [Index engine] Checking for index changes
|
||||
info: [Index engine] Found 7 index changes that are either pending or processing
|
||||
info: [Index engine] syncing entity 'ProductVariant'
|
||||
info: [Index engine] syncing entity 'ProductVariant' done (+38.73ms)
|
||||
info: [Index engine] syncing entity 'Product'
|
||||
info: [Index engine] syncing entity 'Product' done (+18.21ms)
|
||||
info: [Index engine] syncing entity 'LinkProductVariantPriceSet'
|
||||
info: [Index engine] syncing entity 'LinkProductVariantPriceSet' done (+33.87ms)
|
||||
info: [Index engine] syncing entity 'Price'
|
||||
info: [Index engine] syncing entity 'Price' done (+22.79ms)
|
||||
info: [Index engine] syncing entity 'PriceSet'
|
||||
info: [Index engine] syncing entity 'PriceSet' done (+10.72ms)
|
||||
info: [Index engine] syncing entity 'LinkProductSalesChannel'
|
||||
info: [Index engine] syncing entity 'LinkProductSalesChannel' done (+11.45ms)
|
||||
info: [Index engine] syncing entity 'SalesChannel'
|
||||
info: [Index engine] syncing entity 'SalesChannel' done (+7.00ms)
|
||||
```
|
||||
|
||||
### Enable Index Module Feature Flag
|
||||
|
||||
Since the Index Module is still experimental, the `/store/products` and `/admin/products` API routes will use the Index Module to retrieve products only if the Index Module's feature flag is enabled. By enabling the feature flag, you can filter products by their linked data models in these API routes.
|
||||
|
||||
To enable the Index Module's feature flag, add the following line to your `.env` file:
|
||||
|
||||
```env
|
||||
MEDUSA_FF_INDEX_ENGINE=true
|
||||
```
|
||||
|
||||
If you send a request to the `/store/products` or `/admin/products` API routes, you'll receive the following response:
|
||||
|
||||
```json
|
||||
{
|
||||
"products": [
|
||||
// ...
|
||||
],
|
||||
"count": 2,
|
||||
"estimate_count": 2,
|
||||
"offset": 0,
|
||||
"limit": 50
|
||||
}
|
||||
```
|
||||
|
||||
Notice the `estimate_count` property, which is the estimated total number of products in the database. You'll learn more about it in the [Pagination](#apply-pagination-with-the-index-module) section.
|
||||
|
||||
---
|
||||
|
||||
## How to Use the Index Module
|
||||
|
||||
The Index Module adds a new `index` method to [Query](../query/page.mdx) and it has the same API as the `graph` method.
|
||||
|
||||
For example, to filter products by a sales channel ID:
|
||||
|
||||
export const basicHighlights = [
|
||||
["17", "id", "Filter products by their linked sales channel ID."]
|
||||
]
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={basicHighlights}
|
||||
import {
|
||||
MedusaRequest,
|
||||
MedusaResponse,
|
||||
} from "@medusajs/framework/http"
|
||||
|
||||
export const GET = async (
|
||||
req: MedusaRequest,
|
||||
res: MedusaResponse
|
||||
) => {
|
||||
const query = req.scope.resolve("query")
|
||||
|
||||
const { data: products } = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "sales_channels.*"],
|
||||
filters: {
|
||||
sales_channels: {
|
||||
id: "sc_123"
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
res.json({ products })
|
||||
}
|
||||
```
|
||||
|
||||
This will return all products that are linked to the sales channel with the ID `sc_123`.
|
||||
|
||||
The `index` method accepts an object with the same properties as the `graph` method's parameter:
|
||||
|
||||
- `entity`: The data model's name, as specified in the first parameter of the `model.define` method used for the data model's definition.
|
||||
- `fields`: An array of the data model’s properties, relations, and linked data models to retrieve in the result.
|
||||
- `filters`: An object with the filters to apply on the data model's properties, relations, and linked data models that are ingested.
|
||||
|
||||
---
|
||||
|
||||
## How to Ingest Custom Data Models
|
||||
|
||||
Aside from the [core data models](#ingested-data-models), you can also ingest your own custom data models into the Index Module. You can do so by defining a link between your custom data model and one of the core data models, and setting the `filterable` property in the link definition.
|
||||
|
||||
<Note>
|
||||
|
||||
Read-only links are not supported by the Index Module.
|
||||
|
||||
</Note>
|
||||
|
||||
For example, assuming you have a Brand Module with a Brand data model (as explained in the [Customizations](../../../customization/custom-features/module/page.mdx)), you can ingest it into the Index Module using the `filterable` property in its link definition to the Product data model:
|
||||
|
||||
export const filterableHighlights = [
|
||||
["12", "filterable", "Inject Brand by setting its filterable properties."]
|
||||
]
|
||||
|
||||
```ts title="src/links/product-brand.ts" highlights={filterableHighlights}
|
||||
import BrandModule from "../modules/brand"
|
||||
import ProductModule from "@medusajs/medusa/product"
|
||||
import { defineLink } from "@medusajs/framework/utils"
|
||||
|
||||
export default defineLink(
|
||||
{
|
||||
linkable: ProductModule.linkable.product,
|
||||
isList: true,
|
||||
},
|
||||
{
|
||||
linkable: BrandModule.linkable.brand,
|
||||
filterable: ["id", "name"],
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
The `filterable` property is an array of property names in the data model that can be filtered using the `index` method. When the `filterable` property is set, the Index Module will ingest into its data store the custom data model.
|
||||
|
||||
But first, you must run the migrations to sync the link, then start the Medusa application:
|
||||
|
||||
```bash npm2yarn
|
||||
npx medusa db:migrate
|
||||
npm run dev
|
||||
```
|
||||
|
||||
You'll then see the following message in the logs:
|
||||
|
||||
```bash
|
||||
info: [Index engine] syncing entity 'LinkProductProductBrandBrand'
|
||||
info: [Index engine] syncing entity 'LinkProductProductBrandBrand' done (+3.64ms)
|
||||
info: [Index engine] syncing entity 'Brand'
|
||||
info: [Index engine] syncing entity 'Brand' done (+0.99ms)
|
||||
```
|
||||
|
||||
You can now filter products by their brand, and vice versa. For example:
|
||||
|
||||
```ts title="src/api/custom/products/route.ts"
|
||||
import {
|
||||
MedusaRequest,
|
||||
MedusaResponse,
|
||||
} from "@medusajs/framework/http"
|
||||
|
||||
export const GET = async (
|
||||
req: MedusaRequest,
|
||||
res: MedusaResponse
|
||||
) => {
|
||||
const query = req.scope.resolve("query")
|
||||
|
||||
const { data: products } = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "brand.*"],
|
||||
filters: {
|
||||
brand: {
|
||||
name: "Acme"
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
res.json({ products })
|
||||
}
|
||||
```
|
||||
|
||||
This will return all products that are linked to the brand with the name `Acme`. For example:
|
||||
|
||||
```json title="Example Response"
|
||||
{
|
||||
"products": [
|
||||
{
|
||||
"id": "prod_123",
|
||||
"brand": {
|
||||
"id": "brand_123",
|
||||
"name": "Acme"
|
||||
},
|
||||
// ...
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Apply Pagination with the Index Module
|
||||
|
||||
Similar to Query's `graph` method, the Index Module accepts a `pagination` object to paginate the results.
|
||||
|
||||
For example, to paginate the products and retrieve `10` products per page:
|
||||
|
||||
export const paginationHighlights = [
|
||||
["14", "metadata", "Pagination details."],
|
||||
["23", "pagination", "Apply pagination to the retrieved records."]
|
||||
]
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={paginationHighlights}
|
||||
import {
|
||||
MedusaRequest,
|
||||
MedusaResponse,
|
||||
} from "@medusajs/framework/http"
|
||||
|
||||
export const GET = async (
|
||||
req: MedusaRequest,
|
||||
res: MedusaResponse
|
||||
) => {
|
||||
const query = req.scope.resolve("query")
|
||||
|
||||
const {
|
||||
data: products,
|
||||
metadata
|
||||
} = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "brand.*"],
|
||||
filters: {
|
||||
brand: {
|
||||
name: "Acme"
|
||||
}
|
||||
},
|
||||
pagination: {
|
||||
take: 10,
|
||||
skip: 0,
|
||||
}
|
||||
})
|
||||
|
||||
res.json({ products, ...metadata })
|
||||
}
|
||||
```
|
||||
|
||||
The `pagination` object accepts the following properties:
|
||||
|
||||
- `take`: The number of items to retrieve per page.
|
||||
- `skip`: The number of items to skip before retrieving the items.
|
||||
|
||||
When the `pagination` property is set, the `index` method will also return a `metadata` property. `metadata` is an object with the following properties:
|
||||
|
||||
- `skip`: The number of items skipped.
|
||||
- `take`: The number of items retrieved.
|
||||
- `estimate_count`: The estimated total number of items in the database matching the query. This value is retrieved from the PostgreSQL query planner rather than using a `COUNT` query, so it may not be accurate for smaller data sets.
|
||||
|
||||
For example, this is the response returned by the above API route:
|
||||
|
||||
```json title="Example Response"
|
||||
{
|
||||
"products": [
|
||||
// ...
|
||||
],
|
||||
"skip": 0,
|
||||
"take": 10,
|
||||
"estimate_count": 100
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## index Method Usage Examples
|
||||
|
||||
The following sections show examples of how to use the `index` method in different scenarios.
|
||||
|
||||
### Retrieve Linked Data Models
|
||||
|
||||
Retrieve the records of a linked data model by passing in fields the data model's name suffixed with `.*`.
|
||||
|
||||
For example:
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={[["3"]]}
|
||||
const { data: products } = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "brand.*"],
|
||||
})
|
||||
```
|
||||
|
||||
This will return all products with their linked brand data model.
|
||||
|
||||
### Use Advanced Filters
|
||||
|
||||
When setting filters on properties, you can use advanced filters like `$ne` and `$gt`. These are the same advanced filters accepted by the [listing methods generated by the Service Factory](!resources!/service-factory-reference/tips/filtering).
|
||||
|
||||
For example, to only retrieve products linked to a brand:
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={[["9"]]}
|
||||
const {
|
||||
data: products,
|
||||
} = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "brand.*"],
|
||||
filters: {
|
||||
brand: {
|
||||
id: {
|
||||
$ne: null
|
||||
}
|
||||
}
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
You use the `$ne` operator to filter products that are linked to a brand.
|
||||
|
||||
Another example is to retrieve products whose brand name starts with `Acme`:
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={[["9"]]}
|
||||
const {
|
||||
data: products,
|
||||
} = await query.index({
|
||||
entity: "product",
|
||||
fields: ["*", "brand.*"],
|
||||
filters: {
|
||||
brand: {
|
||||
name: {
|
||||
$like: "Acme%"
|
||||
}
|
||||
}
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
This will return all products whose brand name starts with `Acme`.
|
||||
|
||||
### Use Request Query Configuations
|
||||
|
||||
API routes using the `graph` method can configure default [query configurations](../query/page.mdx#request-query-configurations), such as which fields to retrieve, while also allowing clients to override them using query parameters.
|
||||
|
||||
The `index` method supports the same configurations. For example, if you add the request query configuration as explained in the [Query documentation](../query/page.mdx#request-query-configurations), you can use those configurations in the `index` method:
|
||||
|
||||
```ts title="src/api/custom/products/route.ts" highlights={[["17"]]}
|
||||
import {
|
||||
MedusaRequest,
|
||||
MedusaResponse,
|
||||
} from "@medusajs/framework/http"
|
||||
|
||||
export const GET = async (
|
||||
req: MedusaRequest,
|
||||
res: MedusaResponse
|
||||
) => {
|
||||
const query = req.scope.resolve("query")
|
||||
|
||||
const {
|
||||
data: products,
|
||||
metadata
|
||||
} = await query.index({
|
||||
entity: "product",
|
||||
...req.queryConfig,
|
||||
filters: {
|
||||
brand: {
|
||||
name: "Acme"
|
||||
}
|
||||
},
|
||||
})
|
||||
|
||||
res.json({ products, ...metadata })
|
||||
}
|
||||
```
|
||||
|
||||
You pass the `req.queryConfig` object to the `index` method, which will contain the fields and pagination properties to use in the query.
|
||||
|
||||
### Use Index Module in Workflows
|
||||
|
||||
In a workflow's step, you can resolve `query` and use its `index` method to retrieve data using the Index Module.
|
||||
|
||||
For example:
|
||||
|
||||
export const workflowHighlights = [
|
||||
["11", "query", "Resolve query in a workflow step."],
|
||||
["13", "index", "Use the `index` method to retrieve data."],
|
||||
]
|
||||
|
||||
```ts title="src/workflows/custom-workflow.ts" highlights={workflowHighlights}
|
||||
import {
|
||||
createStep,
|
||||
createWorkflow,
|
||||
StepResponse,
|
||||
WorkflowResponse,
|
||||
} from "@medusajs/framework/workflows-sdk"
|
||||
|
||||
const retrieveBrandsStep = createStep(
|
||||
"retrieve-brands",
|
||||
async ({}, { container }) => {
|
||||
const query = container.resolve("query")
|
||||
|
||||
const { data: brands } = await query.index({
|
||||
entity: "brand",
|
||||
fields: ["*", "products.*"],
|
||||
filters: {
|
||||
products: {
|
||||
id: {
|
||||
$ne: null,
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
return new StepResponse(brands)
|
||||
}
|
||||
)
|
||||
|
||||
export const retrieveBrandsWorkflow = createWorkflow(
|
||||
"retrieve-brands",
|
||||
() => {
|
||||
const retrieveBrands = retrieveBrandsStep()
|
||||
|
||||
return new WorkflowResponse(retrieveBrands)
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
This will retrieve all brands that are linked to at least one product.
|
||||
Reference in New Issue
Block a user