Foswiki Upgrade Guide

This guide covers upgrading from a previous version of Foswiki or TWiki (such as Cairo or TWiki4.0) to Foswiki 1.1 (References to Cairo are for TWiki versions older than TWiki 4.0, such as TWikiRelease04Sep2004)

On this page:

Overview

Foswiki is a fork from TWiki 4.2.3. Upgrades from all TWiki versions and earlier Foswiki versions are supported. Foswiki is designed to be 100% compatible with the content of TWiki sites, using the same markup language and supporting the same plugin API (through the use of a TWikiCompatibilityPlugin), thereby enabling a smooth transition from TWiki to Foswiki.

Upgrade requirements

Upgrading from TWiki to Foswiki

You are strongly advised to read the Foswiki:System.ReleaseNotes01x00 (also available in an HTML file in the root of your installation), which contains a list of changes from TWiki 4.2.3 to Foswiki 1.0. A number of system topics and macros (formerly known as TWiki variables) have been renamed, to better describe their purpose and, where appropriate, to rebrand to the Foswiki name.

Upgrading to a new patch release

To upgrade to a new patch release — for example, from Foswiki 1.1.0 to 1.1.2 — an upgrade package can be used that will not overwrite any of your customizations.

For patch releases you will find a brief upgrade procedure on the download page for the release. Follow this procedure to upgrade to the patch release. It may contain important steps that are unique to each patch release (for example, some configure settings may need to be changed).

If you use the Foswiki PageCaching feature, be sure to refresh the cache after upgrading to a new Foswiki version.

Upgrade procedure: upgrading to a new major or minor version, or upgrading from TWiki

The following is a high level view of the upgrade procedure:

  1. Prepare for all upgrade steps.
  2. Install the new Foswiki version and configure it with the same settings as the old version.
  3. Install any additional extensions (Plugins) used by your old installation. Make sure to use the latest Foswiki versions.
  4. Copy all the non-default webs from the old installation to the new one.
  5. Copy the users, groups, and site customizations from the old installation to the Main web in the new installation, including all user topics.
  6. Apply preferences from the old installation.
  7. Apply your site customizations: skin, logos, menu bars, forms for personal information, and so forth.
  8. Validate your Wiki applications and other key functionality.
  9. Switch your production site from the old installation to the new installation.
    Managing caches of static .js, .css files: some parts of Foswiki use URI versioning to help web browsers obtain a current version of these files when they change, but currently other parts do not. You may need to get your users to clear their browser's cache after upgrading your site to a new major or minor release (usually not necessary for patch upgrades). An alternative approach is to manage your web server configuration to strategically set appropriate expiry times for static files (see performance supplement)

More details for each step appear in the following sections. The steps may need to be modified or otherwise tailored with specifics for your installation. In particular, you must take care to preserve any special configuration or customizations you have made, especially if you have modified any of the default software files or system topics that are contained within the installation package.

For purposes of discussion, the following conventions are used: After the upgrade, in the new installation, the Main web is used for user topics and site preferences, and the System web is used to hold documentation and default preferences.

The configure page mentioned in this document is accessible via your web browser at http://yourdomain/<newwiki>/bin/configure .

Prepare for all upgrade steps

Read the Foswiki:System.ReleaseNotes01x00 and learn about the differences between your old installation and the new release to which you are upgrading. Take note of any areas that affect your site and what special steps you may need to take.

Check that all the extensions (plugins, contribs, skins) used by your old installation are available with the new release. Familiarize yourself with any new behaviour that you will have to adapt to or any configuration changes you will have to perform.

If you are using authentication, prepare a test plan to verify that your authentication mechanism is working correctly. Make sure you are able to test logins by a sufficient sample of users to cover all categories of users of your site. For example, users of various groups may need to be tested. In particular, ensure you test that non-admin users cannot access topics restricted to admins.

Identify all essential Wiki topics and Wiki applications that must be fully functional upon completion of the upgrade. Prepare a test plan to verify their functionality. If you are using access controls, ensure that the test plan will adequately test all categories and groups of users of your site.

If your testing will require a test environment to be set up, ensure that it is ready, with any required support infrastructure (for example, testbed authentication servers). If you need to be able to login with different users in different categories and groups, ensure that you have the required login information ready, or you have testers from those groups available to perform the required test cases.

Download the Foswiki distribution from the following location: http://foswiki.org/Download — if you are installing your extensions manually, also download them from the repository where they are stored.

Installation

Follow the installation instructions in INSTALL.html, located in the root of the new installation. Install the new release in a new directory. Do not install on top of the old release.

Test your newly-installed Foswiki site and ensure that its basic functionality works: viewing and editing topics (you can try creating and editing a topic in the Sandbox web).

To make it easier to follow the subsequent steps, you can view this upgrade guide using your new Foswiki site by entering System.UpgradeGuide into the "Jump" text box on the top right of any topic. By doing this instead of using the UpgradeGuide.html file from the distribution, you will be able to use the embedded hyperlinks to jump directly to the referenced pages.

Install extensions

Install all of the extensions that were installed in your old site. In particular, start with any extensions required for the authentication and authorization methods you use (if any). You can use the Find more extensions button in the Extensions section of the configure page to install and configure extensions from the Foswiki:Extensions repository. You can also install extensions manually; see the instructions on the extension's web page from where you obtained the extension (for Foswiki extensions, on foswiki.org).

Check the plugin topics from your old TWiki/Foswiki installation and transfer the plugin settings to the Main.SitePreferences topic in your new Foswiki site, prefixing each setting with the name of the plugin in uppercase followed by an underscore. For example, to copy over the DEFAULT_TYPE setting from the CommentPlugin topic in the old site to the new site, copy the value to a COMMENTPLUGIN_DEFAULT_TYPE setting in the Main.SitePreferences topic in the new site.

Commonly-customized plugin settings include the following:

Activate, and if required, configure the installed extensions in configure.

Copy content from non-default webs in old installation to the new installation

If you are upgrading from TWiki Cairo or earlier, it may be necessary to unlock the rcs files in data and pub directories from the old installation using the following shell commands:

Copy your local webs over to the data and pub directories of the new installation. Do not copy the default webs: <old_system_web> (by default, either System or TWiki), Main, Trash, Sandbox, _default, and _empty.

Copy users, user topics, and site customizations to Main web

Copy all topics and attachments from <old_users_web>: copy all files from <oldwiki>/data/<old_users_web>/ to <newwiki>/data/Main/, and copy all files from <oldwiki>/pub/<old_users_web>/ to <newwiki>/pub/Main/ . Do not overwrite any topics already present in the <newwiki>/data/Main/ directory.

For upgrades from an older Foswiki installation:

For upgrades from a TWiki installation:

Copy over any topics and attachments you want to preserve from the Sandbox web in the old installation: copy the desired files from <oldwiki>/data/Sandbox/ to <newwiki>/data/Sandbox and from <oldwiki>/pub/Sandbox/ to <newwiki>/pub/Sandbox . Some pages you may wish to preserve are the WebHome topic and the WebLeftBar topic (if you had created it in the old wiki installation). The Sandbox web often contains work-in-progress topics that users will want to keep.

Make sure the data and pub directories, as well as the files within them, are readable and writeable by the web server user.

Execute your test plans for authentication and authorization. Test that users that you have transferred from the old installation can login with any problems, and that access controls work appropriately: check that users are able to view and edit pages for which they have access, and are denied permission to view or edit pages for which they do not have access. Also check that pages restricted to the admin group are not accessible by non-admin users, and that administrators continue to have access.

Apply preferences from old installation

If you have not already set your desired site-wide preferences, as described in the section "Set Foswiki Preferences" in the InstallationGuide, then set your preferences. The location of your site preferences is specified in the Miscellaneous settings pane of the configure page, in the {LocalSitePreferences} setting (visible when Expert mode is enabled) — the default location is Main.SitePreferences. Copy any customized preferences from the site preferences topic in your old installation to the site preferences topic in the new installation. (Note you may have already copied over your customized preferences when you transfered the contents of the <old_users_web> web.)

If, in your old installation, you customized the default preferences in <old_system_web>.DefaultPreferences, then transfer your customizations from this topic to the site preferences topic instead (i.e. the topic specified in your {LocalSitePreferences} setting), so that your customizations will not get overwritten on the next upgrade.

If you are upgrading from TWiki, note that the default location of the default preferences in TWiki is <old_system_web>.TWikiPreferences, and the default location of the site preferences is Main.TWikiPreferences. Transfer any customized preferences from these topics to the site preferences topic in your new installation.

Apply additional site customizations

Modify skin with customizations for your site

If you did not already customize the appearance of your new installation, as described in the section "Customize the appearance of your Foswiki site" in the InstallationGuide, then reapply the customizations from your old installation to the new one. Ensure you transfer over any skin templates — .tmpl files, or topics referred to using VIEW_TEMPLATE or EDIT_TEMPLATE preferences — you need. Also ensure you transfer any style sheets or Javascript files required.

If you are upgrading from the Cairo version of TWiki: note that the skins from this release do not work well with Foswiki. Starting from TWiki 4.0.2, the default PatternSkin has been fairly stable and so your customizations should continue to work.

Customize pages for managing personal information

In your new installation, default copies of the following topics were installed:

If you customized these topics in your old installation, transfer the changes to these topics in the new installation. Use the corresponding files in the <oldwiki>/<old_system_web>/ directory as a reference.

Validate your Wiki applications and other key functionality

Execute your test plan to validate the Wiki applications and other key functionality that need to be up and running after the upgrade.

Switch your production site from the old installation to the new installation

If you had been running your old installation in parallel with the new one during a test phase, then disable your old installation, and repeat the step "Copy content from non-default webs in old installation to the new installation".

Change your web server configuration so that the new installation is accessible to all of your users, and so the old installation is no longer accessible.

Change your web server configuration so that the new installation is accessible using the same URL prefix as your old installation. For purposes of discussion, assume that your old installation is accessible from http://yourdomain/wiki/. You can use one of the following approaches to make the new installation accessible using the same URL prefix:

Re-execute your test plan to verify that your newly-upgraded site is accessible to your users, and that all authentication and authorization mechanisms work as expected (including denying access to those who are not authorized).

Re-execute your test plan to verify that your Wiki applications and other key functionality work as intended.

Additional steps when upgrading from TWiki Cairo

Favicon

The favicon is now configurable as a site preference or per-web preference. See System.DefaultPreferences for a description of the FAVICON preference. To set it for your site, add the FAVICON preference to your site preferences topic, Main.SitePreferences (or the topic you configured in your {LocalSitePreferences} setting in the Miscellaneous settings pane on the configure page). To set it for a web, add the FAVICON preference to the WebPreferences topic for that web.

Important changes since TWiki 4.0.5

Supported Perl version

To use Foswiki, you must use Perl 5.8 or higher. Foswiki no longer supports Perl version 5.6.x (the minimum version required for TWiki 4.0.5).

Template spec changed

Until TWiki 4.0.5, any text inside template definition blocks for SkinTemplates (formerly called TWikiTemplates) — that is, between %TMPL:DEF{"block"}% and %TMPL:END% — was stripped of leading and trailing white space, including newlines. This made it difficult for skin developers to include a newline before or after a block of text.

From TWiki 4.1.0 onwards, and in Foswiki 1.0, this has changed so that white space is no longer stripped. Skins like PatternSkin and NatSkin have been updated so that they work with the new behavior. If you use an older skin or have written your own you will most likely need to make some adjustments.

In general, if you get mysterious blank lines in your skin, the newline after the %TMPL:DEF{"block"}% needs to be removed: the content of the block must follow on the same line as the TMPL:DEF.

CommentPlugin templates also must be modified to remove extraneous newlines — in particular, any newline immediately after the TMPL:DEF. See the System.CommentPluginTemplate for examples of how comment template definitions should look like in TWiki 4.1.x.

Example: a CommentPlugin template that adds a row to a table. With versions of TWiki prior to 4.1, the following syntax can be used: 

%TMPL:DEF{OUTPUT:tabletest}%%POS:BEFORE%
|%URLPARAM{"comment"}%| -- %WIKIUSERNAME% - %DATE% |
%TMPL:END%

Starting in TWiki 4.1 and continuing with Foswiki 1.0, the newline before the start of the table row must be removed: 

%TMPL:DEF{OUTPUT:tabletest}%%POS:BEFORE%|%URLPARAM{"comment"}%| -- %WIKIUSERNAME% - %DATE% |
%TMPL:END%

Important changes for the upgrader - since TWiki 4.1.0

New location for session and other temporary files

The directory for passthrough and session files have been replaced by a common directory for temporary files used by Foswiki. Previously the two configure settings {PassthroughDir} and {Sessions}{Dir} were set by default to /tmp. These config settings have been eliminated: Foswiki creates a tmp directory and other temporary directories under the directory defined by the configure setting {WorkingDir}.

Important changes for the upgrader - since TWiki 4.1.2

New WYSIWYG editor

Foswiki now ships with a new WYSIWYG editor based on TinyMCE that replaces the Kupu-based editor. When TinyMCEPlugin is enabled, the Edit button will initiate WYSIWYG editing mode. A Raw Edit link has been added for users to edit the topic markup directly.

The WYSIWYG button has been removed.

NEWTOPICLINKSYMBOL removed

The NEWTOPICLINKSYMBOL preference that was deprecated in TWiki 4.1 is no longer supported. To control the appearance of new links, set the NEWLINKFORMAT preference in Main.SitePreferences (see System.DefaultPreferences for more information on NEWLINKFORMAT).

UserForm and NewUserTemplate customization

When a new user registers on Foswiki his user topic is created based on the NewUserTemplate and UserForm in the Main web, if they exist. If the topic does not exist in the Main web, then the default version from System is used. Thus on upgrades, any customizations you made to NewUserTemplate or UserForm in the Main web will be preserved.

In previous TWiki versions, only System.NewUserTemplate and System.UserForm are used; you cannot override them by creating your own version in the Main web.

WikiUsers no longer distributed

The Main.WikiUsers topic contains all the registered users. It is not included in the Foswiki distribution, so that your list of users will not be overwritten on upgrades. When the first user is registered in Foswiki, the Main.WikiUsers topic is created on demand.

New working directory

The configuration setting {WorkingDir} in the General path settings pane on the configure page defines the location of a working directory for various subdirectories for use by Foswiki. The default value for this setting is the working subdirectory within the installation root directory for the Foswiki installation.

The subdirectories within the working directory include the following:

Note: Remember to restrict access to this new directory when you upgrade.

If you have your own scheduled task to delete obsolete session files, note they are now located in the working/tmp/ subdirectory.

New internal admin login

Foswiki has a new internal admin login feature that lets you login as a temporary administrator, using "admin" as your user name and the password for the configure page. The internal admin username can be configured in the {AdminUserLogin} setting in the Security setup pane on the configure page.

In order to add an initial user to the default {SuperAdminGroup}, Main.AdminGroup, you must login as the internal admin user using the link on the Main.AdminGroup page.

Important changes for the upgrader - since Foswiki 1.0.4

An additional security feature has been added to Foswiki 1.0.5 so that saving data now requires the http method POST. This means that it is no longer possible to store data via an "<a href=..." link or img tag. It also means that if you have an application with an HTML form that creates new topics you must specify in the form tag method="post". This change is done to further tighten the security of Foswiki. 

<form name="new" action="%SCRIPTURLPATH{"save"}%/Sandbox/" method="post">
    ...
</form>

The template webs _default and _empty are frequent targets for spammers because these webs are normally not monitored very carefully by the community around a Foswiki installation. These webs are now write protected so on an administrator can edit topics in these webs. When an administrator creates a new web it is important to remember to remove the access restriction from the WebPreferences in the new web.

Important changes for the upgrader - since Foswiki 1.0.5

Foswiki 1.0.6 introduces a major security enhancement, protecting against Cross-Site Request Forgery. The extra safe "double submit" algorithm, as recommended by the Open Web Application Security Project (OWASP) project has been used. This is the same algorithm used by several major banks and other security-conscious institutions, and requires that users have Javascript enabled.

See Foswiki:Support.SecurityFeatures for more information on these, and other, Foswiki security features. This is recommended reading for all Foswiki administrators.

If these new countermeasures against CSRF creates problems for your users and your applications, you can alter the way the protection works or disable it. These are the configure parameters you need to visit (note that some are expert settings).

Important changes for the upgrader - since Foswiki 1.0.x

ADDTOZONE instead of ADDTOHEAD

Foswiki 1.1.0 introduces the concept of zones, realised by the ADDTOZONE and RENDERZONE macros. Although ADDTOHEAD will continue to work, there may be problems with Foswiki applications that use ADDTOHEAD to add <script>... markup to the page where dependencies have been required which are now added with %ADDTOZONE{"script"....

Foswiki treats the head and script zones of a page separately and won't be able to order these dependencies properly unless the {MergeHeadAndScriptZones} option is set in configure. This feature will be removed in a future version of Foswiki.

See also Foswiki:Development.UpdatingExtensionsScriptZone, VarADDTOZONE and Foswiki:Development.UsingADDTOZONE

JQueryPlugin

Foswiki 1.1.x ships with JQueryPlugin by default. If you are upgrading from a 1.0.x site that used previous versions of JQueryPlugin, care needs to be taken to ensure that your existing jQuery code is compatible with the new JQueryPlugin.

JavascriptFiles and BehaviourContrib

If you have written wiki applications or plugins that assumed the presence of certain Foswiki JS libraries, such as behaviour (no longer shipped with Foswiki), foswikilib.js, foswikiPref.js, foswikiString.js etc. these should now be explicitly required into the page using 
%TMPL:P{"JavascriptFiles/foswikiPref"}%
as instructed at Foswiki:Development.UsingADDTOZONE

TinyMCEPlugin TINYMCEPLUGIN_INIT changes

TinyMCEPlugin's configuration scheme has been reorganised. If you had a customised TINYMCEPLUGIN_INIT preference variable on Foswiki 1.0.x, it will need to be removed, re-built, replaced with a dedicated topic section or replaced with newly introduced preference variables for Foswiki 1.1.x. See upgrade advice for more information.

Logging

Foswiki 1.1 uses a new configurable location for logs (a pluggable logging API has been introduced):
Old location New location
data/warnYYYYMM working/logs/error.log
data/logYYYYMM working/logs/events.log
data/configure.log working/logs/configure.log

The currently used log file is always the same name. They are renamed to somehthing resembling the old scheme on a monthly basis automatically.

For more detailed information on new and deprecated features, especially API changes, refer to the release notes

Managing caches of static files during upgrade

Managing caches of static .js, .css files: some parts of Foswiki use URI versioning to help web browsers obtain a current version of these files when they change, but currently other parts do not. You may need to get your users to clear their browser's cache after upgrading your site to a new major or minor release (usually not necessary for patch upgrades). An alternative approach is to manage your web server configuration to strategically set appropriate expiry times for static files (see performance supplement)

Important changes for the upgrader - since Foswiki 1.1.4

Due to fixes to the authentication session cookies when using http and https, your users may need to login again. This applies to both TemplateLogin and ApacheLogin.

Related Topics: AdminDocumentationCategory, Foswiki:Support.InstallationGuide, Foswiki:Support.InstallingOnSpecificPlatforms, Foswiki:Support.ApacheConfigGenerator, Foswiki:Support.SettingFileAccessRightsLinuxUnix