Wiki source code of Performance

Version 105.2 by Vincent Massol on 2018/09/17

Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 Here are some tips to increase XWiki's performance.
6
7 = Clustering =
8
9 If you need high availability or if the load on your XWiki instance is too high you can [[configure XWiki in a cluster>>Documentation.AdminGuide.Clustering.WebHome]] to spread the load.
10
11 = Standalone Solr =
12
13 By default XWiki use an embedded instance of Solr for easy of use but if you find yourself struggling with very slow search you should probably give external Solr instance a try. You can use ##debug=true## in the URL of the search to check how long is taken in Solr itself to know if improving Solr speeed would really change anything (the issue might come from the UI on XWiki side too).
14
15 See [[Performance Guide in Solr module documentation>>extensions:Extension.Solr Search API#HPerformances]].
16
17 = Slow random number generation on UNIX =
18
19 The library used for random number generation in Oracle's JVM relies on ##/dev/random## by default for UNIX platforms.
20
21 Although ##/dev/random## is more secure, it's possible to use ##/dev/urandom## if the default JVM configuration instead.
22
23 To determine if your operating system exhibits this behavior, try displaying a portion of the file from a shell prompt:
24
25 {{code language="bash"}}
26 head -n 1 /dev/random
27 {{/code}}
28
29 If the command returns immediately, you can use ##/dev/random## as the default generator for Oracle's JVM. If the command does not return immediately, use on of the following solutions to use ##/dev/urandom##:
30
31 == JVM setup ==
32
33 1. Open the ##$JAVA_HOME/jre/lib/security/java.security## file in a text editor.
34 1. Change the line:
35 {{code language="properties"}} securerandom.source=file:/dev/random{{/code}}
36 to read:
37 {{code language="properties"}} securerandom.source=file:/dev/urandom{{/code}}
38 1. Save your change and exit the text editor.
39
40 == Command line parameter ==
41
42 The same effect can be obtained using ##-Djava.security.egd=file:/dev/urandom## in the Java command line (usually in the application server configuration).
43
44 = Gzip compression and caching of static pages =
45
46 HTTP compression is a capability that can be built into web servers and web clients to improve transfer speed and bandwidth utilization. HTTP data is compressed before it is sent from the server: compliant browsers will announce what methods are supported to the server before downloading the correct format; browsers that do not support compliant compression method will download uncompressed data.
47
48 Many application servers (Tomcat, etc.) and HTTP proxies (Apache HTTPd, Nginx, etc.) support it.
49
50 == In Apache HTTP Server ==
51
52 The recommended solution is to set up an Apache Web Server in front of your servlet container and install/configure the following modules:
53
54 * [[mod-deflate>>http://httpd.apache.org/docs/2.0/mod/mod_deflate.html]]
55 * [[mod-expires>>http://httpd.apache.org/docs/2.0/mod/mod_expires.html]]
56 * [[mod-proxy-ajp>>http://httpd.apache.org/docs/2.2/mod/mod_proxy_ajp.html]] (note that this depends on [[mod-proxy>>http://httpd.apache.org/docs/2.2/mod/mod_proxy.html]] that you also need to install)
57
58 Modify your Apache configuration file to load the different modules:
59
60 {{code language="none"}}
61 LoadModule expires_module /usr/lib/apache2/modules/mod_expires.so
62 LoadModule deflate_module /usr/lib/apache2/modules/mod_deflate.so
63 LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
64 # Depends: proxy
65 LoadModule proxy_ajp_module /usr/lib/apache2/modules/mod_proxy_ajp.so
66 {{/code}}
67
68 //Alternatively you can run the following commands as root (sudo)//
69
70 {{code language="bash"}}
71 a2enmod deflate
72 a2enmod proxy_ajp
73 a2enmod expires
74 {{/code}}
75
76 and configure your different modules as described below:
77
78 === Mod Deflate Configuration ===
79
80 {{code language="none"}}
81 vwwwpro-1:~# cat /etc/apache2/conf.d/deflate
82 <Location />
83 # Insert filter
84 SetOutputFilter DEFLATE
85
86 # Netscape 4.x has some problems...
87 BrowserMatch ^Mozilla/4 gzip-only-text/html
88
89 # Netscape 4.06-4.08 have some more problems
90 BrowserMatch ^Mozilla/4.0[678] no-gzip
91
92 # MSIE masquerades as Netscape, but it is fine
93 # BrowserMatch bMSIE !no-gzip !gzip-only-text/html
94
95 # NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
96 # the above regex won't work. You can use the following
97 # workaround to get the desired effect:
98 BrowserMatch bMSI[E] !no-gzip !gzip-only-text/html
99
100 # Don't compress images
101 SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
102
103 # Make sure proxies don't deliver the wrong content
104 #Header append Vary User-Agent env=!dont-vary
105 </Location>
106 {{/code}}
107
108 //On debian apache2 the config file for deflate is located under /etc/apache2/mods-enabled/deflate.conf//
109
110 === Mod Expire Configuration ===
111
112 {{code language="none"}}
113 vwwwpro-1:~# cat /etc/apache2/conf.d/expires
114 <Location /xwiki/skins/>
115 ExpiresActive on
116 ExpiresDefault "access plus 1 day"
117 </Location>
118
119 <Location /xwiki/bin/skin/>
120 ExpiresActive on
121 ExpiresDefault "access plus 1 day"
122 </Location>
123 {{/code}}
124
125 === Mod Proxy AJP Configuration ===
126
127 {{code language="none"}}
128 ProxyRequests Off
129 <Proxy *>
130 Order deny,allow
131 Allow from all
132 </Proxy>
133 ProxyPreserveHost On
134 ProxyPass /xwiki ajp://192.168.1.181:8009/xwiki
135 {{/code}}
136
137 where ##ajp:~/~/192.168.1.181:8009/xwiki## is the internal address of your Servlet container where XWiki is running.
138
139 If you use Tomcat(7) you need to enable the ajp connector in the /etc/tomcat7/server.xml. Comment out the following line with adding {{code}}<!-- -->{{/code}}. //Maybe give a comment why you did it.//
140
141 {{code}}
142 <!-- disable to use ajp connector instead
143 <Connector port="8080" protocol="HTTP/1.1"
144 connectionTimeout="20000"
145 URIEncoding="UTF-8"
146 redirectPort="8443" />
147 -->
148 {{/code}}
149
150 Uncomment the following line by removing the {{code}}<!-- -->{{/code}} and add {{code}}URIEncoding="UTF-8"{{/code}} to it.Maybe give a comment too.
151
152 {{code}}
153 <!-- Activate ajp connector for apache proxy_ajp -->
154 <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8"/>
155 {{/code}}
156
157 = Memory =
158
159 You need to configure your Servlet container so that XWiki has enough memory. You'll need to tune the value to your need. You should check the logs and see if there are any "out of memory" errors. Here are some good default values:
160
161 * For Java 8 (i.e. XWiki >= 8.1). Notice that there's no permgen anymore in Java 8:
162 ** Small and medium installs: A minimum of 1024MB (##-Xmx1024m)##
163 ** Large installs: 2048MB or beyond (##-Xmx2048m).##
164 * For Java 7 (i.e. XWiki < 8.1)
165 ** Small installs: A minimum of 512MB of heap memory and 196MB of permGen (##-Xmx512m -XX:MaxPermSize=196m##)
166 ** Medium installs: 1024MB for the heap and 196MB of permGen (##-Xmx1024m -XX:MaxPermSize=196m##)
167 ** Large installs: 2048MB (or beyond) for the heap and 196MB of permGen (##-Xmx2048m -XX:MaxPermSize=196m##).
168
169 {{info}}
170 You should not increase the memory beyond what you need because increasing it means that there's more Objects in memory at any time and the automatic JVM Garbage Collector has to work harder to clean itself, which can results in performance degradation in XWiki (since a full GC will pause the application for a longer time).
171 {{/info}}
172
173 {{warning}}
174 Note that storing attachments with the default (in database) storage mechanism is very memory intensive. See the [[administrators guide to attachments>>Documentation.AdminGuide.Attachments]] for more information about memory cost and the alternative filesystem based attachment store.
175
176 Also note that uploading a lot of pages can trigger out of memory (OOM) errors due to scheduled watchlist jobs. For example uploading 1 million pages will trigger OOM errors even when the JVM is configured to run with 2GB of heap space. For such kind of load we recommend to disable (unschedule) the Watchlist jobs (in ##/xwiki/bin/view/Scheduler/##) before uploading the pages. To track the progress on this issue, please see [[XWIKI-10594>>https://jira.xwiki.org/browse/XWIKI-10594]].
177 {{/warning}}
178
179 {{warning}}
180 If you use HSQLDB as the wiki database, be aware that the full content of the database is stored in memory and thus the memory requirements are higher. See [[HSQLDB installation page>>Documentation.AdminGuide.Installation.InstallationWAR.InstallationHSQL.WebHome]] for more details.
181 {{/warning}}
182
183 For your information here are the values used for the xwiki.org site:
184
185 {{code}}
186 CATALINA_OPTS="-server -Xms800m -Xmx1600m -XX:MaxPermSize=222m -Dfile.encoding=utf-8 -Djava.awt.headless=true -XX:+UseParallelGC -XX:MaxGCPauseMillis=100"
187 {{/code}}
188
189 = Sizing =
190
191 To give you an idea about what you need to run XWiki on, [[XWiki SAS>>https://xwiki.com]] has the following configuration for its cloud instances:
192
193 * 2GB of RAM,
194 * 2 cores,
195 * 16GB disk size by default
196
197 = Database Indexes =
198
199 Make sure you've set [[Database indexes>>Documentation.AdminGuide.Performances.Database Administration.WebHome]]. This is especially important when you start having lots of documents.
200
201 = Large number of users =
202
203 When you have large number of users it's recommended to turn on implicit All Group, i.e. to consider that all users are members of XWiki.XWikiAllGroup by default in the configuration. This is achieved by editing the ##xwiki.cfg## file and setting:
204
205 {{code language="none"}}
206 xwiki.authentication.group.allgroupimplicit=1
207 {{/code}}
208
209 Then you should remove all the XObjects from the XWikiAllGroup page but you should keep the page since otherwise you won't be able to set permissions for this group. This will prevent XWiki from having to load all that page's XObjects representing the users (thousands of them if you have thousands of users).
210
211 Also make sure that the ##XWikiAllGroup## is listed in the ##xwiki.users.initialGroups## property (it's there by default if you haven't touched that property):
212
213 {{code language="none"}}
214 #-# List of groups that a new user should be added to by default after registering. Comma-separated list of group
215 #-# document names.
216 # xwiki.users.initialGroups=XWiki.XWikiAllGroup
217 {{/code}}
218
219 = Robots.txt =
220
221 If your wiki is open on the Internet, it'll be crawled by search robots (like GoogleBot, etc). They will call all the URLs and especially the ones that are resource hungry like exports (PDF/RTF). You need to protect against this. To do so configure a ##robots.txt## file and put it in your webserver configuration.
222
223 Some example:
224
225 {{code language="none"}}
226 User-agent: *
227 # Prevent bots from executing all actions except "view" since:
228 # 1) we don't want bots to execute stuff in the wiki!
229 # 2) we don't want bots to consume CPU and memory
230 # (for example to perform exports)
231 # Note: You may want to allow /download/ if you wish to have
232 # attachments indexed.
233 Disallow: /xwiki/bin/viewattachrev/
234 Disallow: /xwiki/bin/viewrev/
235 Disallow: /xwiki/bin/pdf/
236 Disallow: /xwiki/bin/tex/
237 Disallow: /xwiki/bin/edit/
238 Disallow: /xwiki/bin/create/
239 Disallow: /xwiki/bin/inline/
240 Disallow: /xwiki/bin/preview/
241 Disallow: /xwiki/bin/save/
242 Disallow: /xwiki/bin/saveandcontinue/
243 Disallow: /xwiki/bin/rollback/
244 Disallow: /xwiki/bin/deleteversions/
245 Disallow: /xwiki/bin/cancel/
246 Disallow: /xwiki/bin/delete/
247 Disallow: /xwiki/bin/deletespace/
248 Disallow: /xwiki/bin/undelete/
249 Disallow: /xwiki/bin/reset/
250 Disallow: /xwiki/bin/register/
251 Disallow: /xwiki/bin/propupdate/
252 Disallow: /xwiki/bin/propadd/
253 Disallow: /xwiki/bin/propdisable/
254 Disallow: /xwiki/bin/propenable/
255 Disallow: /xwiki/bin/propdelete/
256 Disallow: /xwiki/bin/objectadd/
257 Disallow: /xwiki/bin/commentadd/
258 Disallow: /xwiki/bin/commentsave/
259 Disallow: /xwiki/bin/objectsync/
260 Disallow: /xwiki/bin/objectremove/
261 Disallow: /xwiki/bin/attach/
262 Disallow: /xwiki/bin/upload/
263 Disallow: /xwiki/bin/download/
264 Disallow: /xwiki/bin/temp/
265 Disallow: /xwiki/bin/downloadrev/
266 Disallow: /xwiki/bin/dot/
267 Disallow: /xwiki/bin/svg/
268 Disallow: /xwiki/bin/delattachment/
269 Disallow: /xwiki/bin/skin/
270 Disallow: /xwiki/bin/jsx/
271 Disallow: /xwiki/bin/ssx/
272 Disallow: /xwiki/bin/login/
273 Disallow: /xwiki/bin/loginsubmit/
274 Disallow: /xwiki/bin/loginerror/
275 Disallow: /xwiki/bin/logout/
276 Disallow: /xwiki/bin/charting/
277 Disallow: /xwiki/bin/lock/
278 Disallow: /xwiki/bin/redirect/
279 Disallow: /xwiki/bin/admin/
280 Disallow: /xwiki/bin/export/
281 Disallow: /xwiki/bin/import/
282 Disallow: /xwiki/bin/get/
283 Disallow: /xwiki/bin/distribution/
284 Disallow: /xwiki/bin/imagecaptcha/
285 Disallow: /xwiki/bin/unknown/
286 Disallow: /xwiki/bin/webjars/
287 # Don't index sandbox content since it's sample content
288 Disallow: /xwiki/bin/view/Sandbox/
289 # Don't index Admin space since it contains Admin stuff.
290 # Note that the Admin space is protected by permissions
291 # anyway but this acts as a safety net to not have private
292 # info leaked on the internet ;)
293 Disallow: /xwiki/bin/view/Admin/
294 # Don't index Stats data (just because it's not useful and
295 # those pages are a bit CPU intensive)
296 Disallow: /xwiki/bin/view/Stats/
297 # Don't index Panels data (because we don't want it
298 # indexed on the internet)
299 Disallow: /xwiki/bin/view/Panels/
300 {{/code}}
301
302 Other example:
303
304 {{code}}
305 [...]
306 # It could be also useful to block certain spaces from crawling,
307 # especially if this spaces doesn't provide new content
308 Disallow: /xwiki/bin/view/Main/
309 Disallow: /xwiki/bin/view/XWiki/
310 # On the other hand you would like to have your recent (public) changes included
311 Allow: /xwiki/bin/view/Main/Dashboard
312 {{/code}}
313
314 **Note:**
315
316 For Tomcat6 the placement of the ##robots.txt## file should be within the //$TOMCAT/webapps/ROOT// folder and should have permission 644 applied.
317
318 {{code}}
319 -rw-r--r-- 1 root www 1478 Jan 8 15:52 robots.txt
320 {{/code}}
321
322 In order to test if the ##robots.txt## file is either accessable or working as desired use this [[checker>>http://www.frobee.com/robots-txt-check]].
323
324 = Statistics =
325
326 {{info}}
327 This is no longer true starting with XE 1.4M2 since statistics are now put on a queue and written in a different thread in the database in one go, thus reducing the overhead to a maximum.
328 {{/info}}
329
330 The statistics module is off by default since it's quite database intensive. If you don't need it you should turn it off.
331
332 = Document Cache =
333
334 You can tune the Document cache in the ##xwiki.cfg## configuration file. The value depends on how much memory you have. The higher the better. A good reasonable value is 1000.
335
336 {{code language="none"}}
337 xwiki.store.cache.capacity=1000
338 {{/code}}
339
340 = Cache Macro =
341
342 It's possible to perform selective content caching by using the [[Cache Macro>>extensions:Extension.Cache Macro]].
343
344 = LESS CSS Performances =
345
346 [[LESS>>extensions:Extension.LESS Module]] is a preprocessor used to generate CSS files for skins and skin extensions. See the [[Performances section>>extensions:Extension.LESS Module#HPerformances]] of the LESS module documentation to learn more about how to optimize its cache for performances, and to set the appropriate number of simultaneous compilations your server can handle.
347
348 = Rendering cache =
349
350 Some pages are complex to render (they may aggregate outside data for example or do complex and slow queries). For theses pages you can use rendering cache.
351
352 == Configuration based ==
353
354 Pages can be cached (i.e. their rendered content cached) to speed up displaying. The configuration is done in ##xwiki.properties## with the following configuration options:
355
356 {{code language="none"}}
357 #-# [Since 2.4M1]
358 #-# Indicate if the rendering cache is enabled.
359 #-# Default value is false.
360 # core.renderingcache.enabled=true
361
362 #-# [Since 2.4M1]
363 #-# A list of Java regex patterns matching full documents reference.
364 # core.renderingcache.documents=wiki:Space\.Page
365 # core.renderingcache.documents=wiki:Space\..*
366
367 #-# [Since 2.4M1]
368 #-# The time (in seconds) after which data should be removed from the cache when not used.
369 #-# Default value is 300 (5 min).
370 # core.renderingcache.duration=300
371
372 #-# [Since 2.4M1]
373 #-# The size of the rendering cache. Not that it's not the number of cached documents but the number of cached results.
374 #-# (For a single document several cache entries are created, because each action, language and request query string
375 #-# produces a unique rendering result)
376 #-# Default value is 100.
377 # core.renderingcache.size=100
378 {{/code}}
379
380 You can force a page to refresh using ##refresh=1## in the URL.
381
382 Since 6.2 it's also possible to programmatically refresh any document cache using ##com.xpn.xwiki.internal.cache.rendering.RenderingCache## component:
383
384 {{code language="java"}}
385 @Inject
386 private RenderingCache renderingCache;
387
388 ...
389
390 renderingCache.flushWholeCache();
391 renderingCache.flushCache(new DocumentReference("xwiki", "MySpace", "MyCachedDocument"));
392 {{/code}}
393
394
395 = Merge the CSS files =
396
397 In order to reduce the number of requests and files that are downloaded from the browser or client, it could help to merge all XWiki CSS files into a single one. See the [[Merge CSS Script>>extensions:Extension.Merge CSS]].
398
399 = Set up NginX =
400
401 If you experience //__heavy loads__// on your wiki, you could try using NginX.
402
403 NginX is used to fetch static content: images, javascript, styles, etc, but it can also be used as a reverse-proxy to pass requests down to the web container (e.g. Tomcat on port 8080).
404
405 Unlike Apache, which instantiates a new process per every static file, NginX uses the same process to fetch all the static data, and thus gives you extra perfomance "for free".
406
407 For more info on setting up NginX check [[this guide>>Documentation.AdminGuide.Installation.NginX.WebHome]].
408
409 = Backlinks =
410
411 While a pretty neat feature, keeping track of the backlinks has a medium impact on the document saving time and a minor impact on the document loading time. If you feel that your wiki does not need backlinks, you can safely disable them with the following line in ##xwiki.cfg##:
412
413 {{code language="none"}}
414 xwiki.backlinks=0
415 {{/code}}
416
417 = Versioning =
418
419 One of the key features of any wiki system, versioning greatly affects the database size and the document update time. If you are sure your wiki does not need to keep track of all the changes and you will never need to revert documents to a previous version, then you can add the following line in ##xwiki.cfg##:
420
421 {{code language="none"}}
422 xwiki.store.versioning=0
423 {{/code}}
424
425 = Custom Mapping =
426
427 In some cases you may not want to rely on XWiki's generic database schema for storing XClass data and instead you'd like to provide your own optimized table. For these use cases you can use [[Custom Mapping>>platform:DevGuide.CustomMapping]].
428
429 = LDAP =
430
431 == Disable LDAP sub groups search ==
432
433 By default when loading a LDAP group, each member is searched and loaded to figure out if it's a group or not (and then load the sub group members, etc). Since **7.2** If you know there is not sub group in your LDAP groups you can disable it and speed up quite a lot big groups handling using ##xwiki.authentication.ldap.group_sync_resolve_subgroups## property in ##xwiki.cfg## configuration file.
434
435 = Performance tree =
436
437 Since 7.1 it's possible to directly get a tree of time spent in each step of the request by using [[debug mode>>dev:Community.Debugging#HDebugmode]].
438
439 = Navigation Tree =
440
441 The Navigation Panel and other navigation trees can have some small performance issues under very high volumes. Here ares some base measures we did on some developer laptops to give you some ideas of the performance you should expect:
442
443 Measure set 1:
444
445 (% width="100%" class="table-bordered" %)
446 |=(% width="20%" %)DB|=(% width="10%" %)Levels|=(% width="10%" %)Spaces|=(% width="10%" %)Pages|=(% width="15%" %)Requests|=Time
447 |MySQL|5|6887|4049|31|180ms
448 |HSQLDB|5|6762|4063|27|138ms
449 |MySQL|1|2514|4962|27|331ms
450 |HSQLDB|1|2377|4718|21|**3.15s**
451
452 Measure set 2:
453
454 (% width="100%" class="table-bordered" %)
455 |=(% width="20%" %)DB|=(% width="10%" %)Levels|=(% width="10%" %)Spaces|=(% width="10%" %)Pages|=(% width="15%" %)Requests|=Time
456 |Oracle|5|6943|4106|20|119ms
457 |Oracle|1|2493|4982|20|153ms
458
459 Measure set 3:
460
461 (% width="100%" class="table-bordered" %)
462 |=(% width="20%" %)DB|=(% width="10%" %)Levels|=(% width="10%" %)Spaces|=(% width="10%" %)Pages|=(% width="15%" %)Requests|=Time
463 |Oracle|1|2485|4991|20|151ms
464 |PostgreSQL|1|2494|4991|20|125ms
465
466 Measure set 4 (XWiki 10.5, Intel i7 CPU, SSD Storage):
467
468 (% width="100%" class="table-bordered" %)
469 |=(% width="20%" %)DB|=(% width="10%" %)Levels|=(% width="10%" %)Spaces|=(% width="10%" %)Pages|=(% width="15%" %)Requests|=Time
470 |PostgreSQL 10|5|6847|4052|25|74ms
471
472 = Legacy =
473
474 == Monitor plugin ==
475
476 More a developer-oriented feature, XWiki can monitor its own code, reporting the time spent for each sub-component activated during a request. While the monitoring code isn't time consuming, it increases the memory consumption a bit, and the create/start/stop/log/destroy calls are spread all around the code, so you will save a lot of method calls by disabling this. You can do that by setting the following line in ##xwiki.cfg##:
477
478 {{code language="properties"}}
479 xwiki.monitor=0
480 {{/code}}
481
482 == 1.0 rendering cache using velocity in document content itself ==
483
484 You can add the following to their content to cache them after they are rendered. Note that the document is refreshed whenever the content of the document changes, and the cache takes into account the URL, so it is pretty safe to add a long cache duration for all documents that don't contain scripts gathering data from the wiki. For example to cache the rendered content for 60 seconds you would add:
485
486 {{code language="none"}}
487 $context.setCacheDuration(60)
488 {{/code}}
489
490 Since 1.5M2, you can set the default rendering cache duration for all pages in ##xwiki.cfg##:
491
492 {{code language="none"}}
493 ## cache all rendered documents for one hour
494 xwiki.rendering.defaultCacheDuration=3600
495 {{/code}}
496
497 Setting the default cache duration to a large value, and manually disabling the cache in dynamic pages would really speed up the wiki, since the rendering is one of the most time consuming processes.
498
499 == Wiki syntax features for XWiki Syntax 1.0 ==
500
501 If you're using XWiki Syntax 1.0 and if you don't plan to use all of the markup features, like the strikethrough filter, the automatic http links filter, the SVG, Laszlo or style macros, you can disable them in ##xwiki-core-*.jar/META-INF/services/com.xpn.xwiki.render.*##. The wiki rendering is the most costly operation in the rendering process, so any disabled feature counts.
502
503 Now this will have no effect if you're using another syntax, like XWiki Syntax 2.x.

Get Connected