r2 - 31 Jan 2007 - 16:53:41 - TWikiAdminGroupYou are here: TWiki >  TWiki Web  > PublishContrib

PublishContrib

Generates a static view of a web, as HTML files on disc, or as a PDF, or as a zip or tgz archive file, or by uploading directly to an FTP server.

Previously known as GenHTMLAddOn, and then PublishAddOn, this is the original publishing extension for TWiki.

NOTE: This version is TWiki 4.0 specific. Download the previous version if you want to use this extension with an earlier version of TWiki.

When TWiki generates a view, it does so dynamically i.e. there's a CGI script that runs, processes some files, and generates HTML that is displayed by the browser. There are circumstances in which this may not be desirable or even possible. For example:

  1. TWiki is used to create documentation which has to be bundled into a product release
  2. Published versions of TWiki pages must be read-only
  3. The TWiki server is inaccessible to the audience (e.g. on the other side of a corporate firewall)

PublishContrib provides support for the generation of stand-alone HTML from a TWiki web. It will generate fully rendered versions of a set of TWiki pages together with any attached files.

Features

  • All standard TWiki tags are interpreted
  • Plugins are called
  • Unresolved links to non-existent topics are silently ignored
  • Topic links internal to the TWiki are translated to relative links
  • Powerful support for choosing what content gets published
  • Any links to the 'pub' areas of topics in the web are automatically resolved and the referenced files copied
  • Any links to images outside the TWiki are resolved, and the image is stored in the output (requires LWP)
  • Output in HTML or PDF. HTML can be compressed in different archive formats.
  • Full support for hierarchical webs
  • Multiple instances (e.g. dev, test, prod) can be specified
  • Templates (such as viewprint) can also be generated, and the links are fixed up appropritately
  • Able to publish HTML and referenced files directly to a remote server via ftp

Usage

Publish Form

The easiest way to publish a web is from this topic, by filling in the following form.

The output is generated in a directory designated during installation. The progress messages printed during documentation generation tell you exactly where the output is.

You can also create a permanent topic in a web to help with the publishing process.

DescriptionValueParameter Name
Web to publish web
Comma-separated list of wildcard patterns that match the names of topics to include inclusions
Comma-separated list of wildcard patterns that match the names of topics to exclude exclusions
A regular expression that will cause a topic to be excluded if the RE matches the topic content filter
Select skin for published HTML
You are recommended to pick text, or plain, or a print skin. Your installation may also offer a special export or publish skin.
skin
Output format
WARNING The rendered data can get pretty big, and the process itself puts a heavy load on the server, especially when using compression on large webs.
format
FTP options (only relevant if Output format is ftp)

The FTP output format generates a sitemap.xml, and can also generate default.htm, index.html and google site verification files.

The format generator was written by TWiki:Main.SvenDowideit, who neglected to document it, so these option descriptions are 'best guess'.

Destination FTP server destinationftpserver
Path to upload to on server destinationftppath
FTP username destinationftpusername
FTP Password destinationftppassword
Google file googlefile
Name of topic to use to generate default.htm, index.html defaultpage
Relative URL used in sitemap relativeurl
Other extra options Some output generators support extra options (e.g. PDF; you can add htmldoc command-line parameters here e.g --linkstyle underline)
genopt

Wildcard Patterns

Wildcard patterns are well known to people who are used to command lines on computers, but may be unfamiliar to the Windows generation. A wildcard is a special string that you can put into a filename so that it matches a whole range of files:
String What it does Example What the example matches
* Matches any string, including an empty string. *Cheese* Every topic with "Cheese" somewhere in the name (but not "cheese")
? Matches any single character. Example1? Example10 and Example 1X but not example1
[...] Matches any one of the enclosed characters. A pair of characters separated by a hyphen denotes a range expression; any character that sorts between those two characters, inclusive, using the current locale's collating sequence and character set, is matched. If the first character following the [ is a ^ then any character not enclosed is matched. A - may be matched by including it as the first or last character in the set. A ] may be matched by including it as the first character in the set.
Within [ and ], character classes can be specified using the syntax [:class:], where class is one of the following classes defined in the POSIX.2 standard: alnum, alpha, ascii, blank, cntrl, digit, graph, lower, print, punct, space, upper, word, xdigit. A character class matches any character belonging to that class. The word character class matches letters, digits, and the character _.
B[aeiou]g Bag, Bog, Big, Beg, Bug

Regular Expressions

A perl regular expression. You can use a simple string here, which will be matched exactly, or you can read up on perl regular expressions on the web.

Using a Publish Topic

You can create a publish topic in a web that contains all the details needed to publish that web. This is just a topic with a series of standard TWiki variable settings in it. You can use the PublishWeb topic in this web as a template for your own topics.

Alternatively you can just take a copy of the form in this topic, paste it into your own topic, and change the defaults.

Publishing from the command line

TWiki-4 allows you to call any TWiki script from the command line, and the publish script is no exception. Just cd to the bin directory, and perl publish. Parameters are passed as name=value pairs, for example:
perl publish web=Book exclusions='Web*' format=file
perl publish web=Book inclusions=WebBook format=pdf genopt='--book --duplex --toclevels=5'
The available parameter names are shown in the example above, in the 'Name' column.

Controlling which parts of a topic get published

You can control what gets published from a topic using %STARTPUBLISH% and %STOPPUBLISH% control tags:
  • If %STARTPUBLISH% is the first control tag seen in the file, everything before it will be ignored.
  • Everything between %STOPPUBLISH% and the next %STARTPUBLISH% (or the end of the topic) will be ignored.
  • %STARTPUBLISH% and %STOPPUBLISH% will be visible in the viewed topic, so you can easily see what will be published from the topic.
Note: the old <nopublish> tag is deprecated and should be replaced in topics

Another good trick is to set up a special "publishing" web. Create topics in the web that %INCLUDE the topics from other webs that you want to publish. You can use STARTSECTION and ENDSECTION to highlight what you want published. This way the "publishing" web gives you a view of exactly what will be in the published output, without the need for special publishing tags.

Known problems

  • Doesn't handle revision info tags in templates
  • Plugin tags that create complex HTML (such as TWiki:Plugins/TWikiDrawPlugin DRAWING tags) are not handled correctly.
  • Links to excluded topics still get generated as (broken) links

Installation Instructions

Dependencies

Note: If you want to generate PDF files, you will need an installation of htmldoc. htmldoc is available from http://www.easysw.com/htmldoc/ for free, but you are strongly recommended to buy the commercial version. Your support for OSS projects helps make OSS possible.

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

  • If you have TWiki 4.1 or later, and Perl 5.8, you can install from the configure interface (Go to Plugins->Find More Extensions)
    • The webserver user has to have permission to write to all areas of your installation for this to work.
  • If you have a permanent connection to the internet (and Perl 5.8), you are recommended to use the automatic installer script
    • Just download the PublishContrib_installer perl script and run it.
  • Notes:
    • The installer script will:
      • Automatically resolve dependencies,
      • Copy files into the right places in your local install (even if you have renamed data directories),
      • check in new versions of any installed files that have existing RCS histories files in your existing install (such as topics).
      • If the $TWIKI_PACKAGES environment variable is set to point to a directory, the installer will try to get archives from there. Otherwise it will try to download from twiki.org or cpan.org, as appropriate.
      • (Developers only: the script will look for twikiplugins/PublishContrib/PublishContrib.tgz before downloading from TWiki.org)
    • If you don't have a permanent connection, you can still use the automatic installer, by downloading all required TWiki archives to a local directory.
      • Point the environment variable $TWIKI_PACKAGES to this directory, and the installer script will look there first for required TWiki packages.
        • $TWIKI_PACKAGES is actually a path; you can list several directories separated by :
      • If you are behind a firewall that blocks access to CPAN, you can build a local CPAN mini-mirror, as described at http://twiki.org/cgi-bin/view/Codev/BuildingDakar#CPAN_local_minimirror
  • If you don't want to use the installer script, or have problems on your platform (e.g. you don't have Perl 5.8), then you can still install manually:
    1. Download and unpack one of the .zip or .tgz archives to a temporary directory.
    2. Manually copy the contents across to the relevant places in your TWiki installation.
    3. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
    4. Manually edit LocalSite.cfg to set any configuration variables.
    5. Run configure and enable the module, if it is a plugin.
    6. Repeat from step 1 for any missing dependencies.

Additional install steps

Run configure and complete the installation in the PublishContrib section.

PDF output

  1. install htmldoc from http://www.easysw.com/htmldoc/

Note that htmldoc can also be used to generate PostScript by using the -t option in the Other extra options field above. See the htmldoc man pages for details.

.tgz (tar) output

  1. Install Archive::Tar and everything it depends on

.zip output

  1. Install Archive::Zip and everything it depends on

Info

This add-on started as the TWiki:Plugins/GenHTMLAddon, written by TWiki:Main/CrawfordCurrie at Motorola. It was then rewritten by TWiki:Main/EricScouten, and then fixed and enhanced by TWiki:Main/CrawfordCurrie (http://c-dot.co.uk). It has been further extended by TWiki:Main/SvenDowideit and TWiki:Main/MartinCleaver.

Authors: TWiki:Main/CrawfordCurrie, TWiki:Main/EricScouten, TWiki:Main.SvenDowideit, TWiki:Main.MartinCleaver
Dependencies:
NameVersionDescription
File::Spec>0Required. Used to analyse URL paths.
File::Copy>0Required. Used to move files around.
File::Path>0Required to manipulate directories.
LWP>0Optional. Used to include images referenced by absolute URLs
Archive::Zip>=0Optional. Required to generate .zip output
Archive::Tar>=0Optional. Required to generate .tgz output
Net::FTP>0Optional. Required for ftp publishing.
htmldocOptional. Required to generate .pdf output
Version: 12824
Change History:  
12824 Added support for new internal api - no user changes
12708 Added UI for FTP. Added .spec file. Fixed Bugs:Item3515 and Bugs:Item2725
12028 Michael Daum - create a new TWiki object for every topic, don't reuse the current one (Bugs:Item3139)
10412 Correction to the correction for anchors.
10345 Correction to support anchors in URLs properly
10242 Martin Cleaver - changes to allow generation of viewprint and viewxxx when specified by TEMPLATE; multiple INSTANCE (dev/test/prod); (Bugs:Item2269)
10063 Bugfix Bugs:Item2216
10006 Crawford Currie - fixed problem where it was failing to remove <base> tags completely (Bugs:Item2200)
9986 Crawford Currie - added doc on usage from command line, corrected sense of topicsearch filter (Bugs:Item2120, Bugs:Item2121), renamed parameters (old ones are still valid), corrected handling of empty web refs (Bugs:Item2128), deprecated nopublish html-style tag in favour of PublishWebPlugin-compatible style (though with richer semantics) (Bugs:Item2196)
9823 Crawford Currie - added support for hierarchical webs, and inclusion of external images.
9773 Crawford Currie - added tgz and pdf support
9725 Michael Daum - fixed rewriting urls; fixed nested resources issue; creating a new prefs object for each topic
9713 Corrected form action so it uses up the right web preferences
9695 Michael Daum - recursively archive resources imported by css files; fixed several html errors in the PublishContrib and PublishWeb topics; removed hardcoded reference to print.pattern
8959 TWiki-4 version. Also supports publishing to a file area, making TWiki easier to use as a CMS (see also TWiki:Plugins/PublishWebPlugin, which does almost the same thing frown )
6276 Item196 bugfix for HTTP_HOST, as described in the Dev topic for the contrib
5566 Changed interface to support wildcards, and lightened the plugin by replacing a lot of files with simpler ways of doing things.
5499 Added Compress::Zlib dependency, as requested by Brad Taylor
27 Apr 2005 1.301 Crawford Currie - fixed minor issues highlighted by Bruce Dillahunty and Scott Claridge
11 Apr 2005 1.3 Crawford Currie - reworked the interface and code to work better
13 October 2004 1.200 Crawford Currie - Cairo compatible
7 Jan 2003 1.1 Initial version
Home: http://TWiki.org/cgi-bin/view/Plugins/PublishContrib
Feedback: http://TWiki.org/cgi-bin/view/Plugins/PublishContribDev
Appraisal: http://TWiki.org/cgi-bin/view/Plugins/PublishContribAppraisal

Related Topics: TWikiPreferences, TWikiPlugins

Copyright

This code is a development of the Architectures and System Platforms group of Motorola Inc. and is protected by the following copyrights:
  • Copyright © 2001 Motorola. All Rights Reserved.
  • Copyright © 2002-2003, Eric Scouten.
  • Copyright © 2004-2006 Crawford Currie http://c-dot.co.uk
  • Copyright © 2006 Martin Cleaver http://www.cleaver.org

The 2005 functionality improvements were sponsored by Wind River Systems

The pdf and tgz output formats were made possible by Sabio Labs

License

As required for the publication of all extensions to TWiki, the software is published under the terms of the GNU General Public License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details, published at http://www.gnu.org/copyleft/gpl.html

Edit | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r2 < r1 | More topic actions
 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
Note: Please contribute updates to this topic on TWiki.org at TWiki:TWiki.PublishContrib