The regular way Elovate handles API requests, is by sending data entity by entity. When working with lots of changes, this can take a long time.
To speed up this process, Elovate supports the bulk endpoints.
Reference to Adobe documentation:
https://developer.adobe.com/commerce/webapi/rest/use-rest/asynchronous-web-endpoints/
Bulk endpoints in Magento
Bulk enpoints are introduced from Magento 2.3.0 and up. However, in order to use this, RabbitMQ is needed. Since Magento 2.4.0, bulk endpoints are available without RabbitMQ and runs through MySQL.
In short:
Available since: Magento 2.3.0
RabbitMQ no longer mandatory from: Magento 2.4.2 (2.4.0/2.4.1 work too, but require patch MDVA-31590 or equivalent custom fixes for reliability)
To use bulk endpoints, make sure you run Magento 2.4.2 or higher.
Before enabling bulk endpoints in your Elovate account, make sure to test the bulk endpoints in your Magento 2 store first!
Testing bulk endpoints
The easiest way to test the bulk endpoints, is as follows (your development agency can run these tests for you, or can tell you if this is enabled). Open Postman (or your preferred API testing tool) and use a POST request: https://www.[store-url].nl/rest/all/async/bulk/V1/products
Important: you will see 'all' in the API url. This means that Magento will send the data to global. If you do not add this, Magento will fall back to your default storeview. This can result in data being set on store view level unwanted. Always make sure to check this carefully.
If you want to send data to a specific storeview, replace 'all' for your storecode, for example 'english'.
Add the following body (change SKU to and existing product SKU and change to your needs):
[
{
"product": {
"sku": "product-sku-001",
"name": "Product Name 1 - BULK test",
"custom_attributes": [
{
"attribute_code": "description",
"value": "Product Description 1"
},
{
"attribute_code": "short_description",
"value": "Short Description 1"
}
]
}
}
]
You will get the following response:
{
"bulk_uuid": "f6b48f01-3c1f-4053-87be-10f548dcd34e",
"request_items": [
{
"id": 0,
"data_hash": "c0f2bca4f248809e0f2cbfb58da4a6f28614d3035dfb723114a5078815bfe1b5",
"status": "accepted"
}
],
"errors": false
}
The message above shows that the update is schedule for process. The Magento cron will take care of processing this request.
Note: your Magento cron must be enabled. Every minute, your cron will process the pending messages. Without an active cron, the bulk operations will not be processed.
Magento can handle multiple updates in one API call this way. Elovate uses the bulk API for products and categories. Elovate sends updates in batches of 100 entries at a time. How fast these are processed within your Magento application is dependant on your hosting setup.
Verify within your database
If you want to see if your message is processed, you can find this message in your database.
Look for the table 'queue_message'
Note the 'id' of your message
Look for the table 'queue_message_status' and lookup the ID noted
The message has a status. If the message has been processed, this will have the status: 4.
Verify within your backend
Once your message has been processed, the changes will reflect in your Magento backend.
Questions about the bulk API
If you have any questions about the bulk API within Elovate, or you want to enable the bulk endpoints for your account, send us a message!