Daisycon product feeds are generated live every time you request a feed URL. Large or unnecessary downloads slow down product feeds for all publishers. This article explains how to set up your downloads so you get up-to-date product data with as little load as possible.
The rate limit is already active. Other rules in this article may be enforced technically in the future, so it's best to set up your downloads this way now.
Product feed update frequency
This differs per product feed. Most product feeds are updated once a day, some more often. The exact time differs per campaign.
Recommended way to download product feeds
We recommend this setup for every product feed you use:
-
Download only what you need, in parts. Use filters, the
fieldsparameter and pagination. -
Next time, check first if the feed changed. Send the
If-Modified-Sinceheader. If you get a304, stop here. -
If the feed changed, fetch only the changes. Use
update=allandupdate_since.
Each step is explained below.
Download only what you need, in parts
- Limit the products with filters, for example one brand or a price range.
- Limit the data with the
fieldsparameter, see Generate a product feed url manually. - Always use pagination, for example 1000 products per request. A single request stops after about 5 minutes, so a large feed without pagination ends up incomplete.
Check if a product feed changed
Every feed response contains a Last-Modified header. Save this value and send it back in an If-Modified-Since header the next time you request the same feed URL.
Your request header: If-Modified-Since: Sun, 6 Sep 2026 12:34:06 GMT Feed unchanged: 304 Not Modified (no content, nothing to process) Feed changed: 200 OK (continue with the next step)
Fetch only the changes
When a feed changed, don't download all products again. Add update=all and the time of your last successful sync to the feed URL:
&update=all&update_since=2026-09-06%2014:34:06
Please note: update_since uses Dutch time (CET/CEST), while Last-Modified is in GMT. In this example, 12:34:06 GMT is 14:34:06 Dutch summer time.
You then only get new, changed and removed products. Read more in Filtering product feeds.
Product feed rate limits
You can send up to 30 feed requests per minute per IP address. Other users on the same network or shared hosting count as well.
When you go over the limit, you get a 429 Too Many Requests error. What to do:
- Wait the number of seconds in the
Retry-Afterheader before you try again. Retrying immediately keeps you blocked. - Request pages one after another, never at the same time.
- Spread your downloads over the day instead of starting all feeds at once.
Make product feed downloads faster
- Send the header
Accept-Encoding: gzip. This makes downloads up to 80% smaller. - Only use
rawdata=truewhen you need it, as it makes a feed a lot bigger.