I am using version 8.x-3.5 and below is my configuration.

Maximum links in a sitemap: 2000
Sitemap generation max duration: 20 Second

And i am generating default variant with 67715 items. It take hrs. to just generate 100 items. To generate 67715 items. it might take huge number of days. Kindly help me on this.

Thanks in advance.

Comments

hasmimeraj created an issue. See original summary.

gbyte’s picture

Doesn't sound right, as 67k items is a relatively small site.

It take hrs. to just generate 100 items.

How are you generating? Are you generating through cron, is cron running every few minutes?
Try generating through the UI, Drush and cron and compare the three in terms of speed. How long does it take to generate through the UI?

proxiss’s picture

Same here, it's far too slow. I get about 600 entries per 20 seconds of cron. So it takes about an hour to re-generate - and it re-generates all the time when one indexed item changes...

gbyte’s picture

@proxiss See questions above.

gbyte’s picture

Status: Active » Fixed

Please try the latest version of the module, it should be a better performer. If you find it does not help, please give me additional info,

gbyte’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

christiana.k’s picture

Hello,
I'm also experiencing the same issue. Very slow sitemap generation. I have arround 400,000 links in the sitemap.
My settings are as follows:
- Maximum links in sitemap: 5000
- Sitemap generation max duration: 40 sec
- Remove duplicates: true (I don't know if this has an impact on performance)

It's now been 3 days and the sitemap has not managed to regenerate. It has completed 86% and moving up really slowly.
Simple XML Sitemap cron job runs every 15 minutes.
Through the UI the results are similar.

I was wondering if the following are feasible / might help:
1. Doing incremental sitemap generation. I think currently (correct me if I'm wrong) the sitemap is regenerated at its entirety even if a few new links are added.
2. Limit sitemap links for some content types to only recent items. Our site is a news portal so most links are links to articles. Is there a way to limit the articles indexed only to the ones published (or created) in the last month?

it-cru’s picture

@christiana.k: If it is an option for you, I would set Remove duplicates to FALSE, this will improve performance a little bit.

Do you generate your sitemap via drush command or via web interface? What also could help is to run it via drush command on a web instance which only used for drush commands and get no impact from traffic of website visitors.

Maybe also #3180776: Improve queuing performance by implementing a batch process could be interesting for you to test.

christiana.k’s picture

The sitemap generation is run through a cron schedule.
I have increased the frequency of the cron job and also the maximum duration.
I have also removed the option "Exclude duplicate links".
I will see how this improves the sitemap generation process and will come back with an update.

alexpott’s picture

The current dev release has a tonne of performance improvements - particularly #3203715: Process multiple entities with a single queue item - if you change the number of entities being processed by run you'll see a big speed up.