doc/developers-guide/producers.xml

   1 <chapter id="producer_framework">
   2 <title>The producer framework</title>
   3 Please read the presentation of the <glossterm linkend="producers">producers</glossterm> concept
   4 for an introduction.
   5
   6 <section><title>How to use the <filename>producers.xml</filename> file</title>
   7   [FIXME: this should be in the admin's guide, not here]
   8
   9 <para>
  10 Please check the standard <filename>producers.xml</filename> file
  11 for two fully commented, real-world examples :
  12 the <code>&lt;nodedefinition name="Language"&gt;</code> node
  13 and the <code>&lt;producer name="articles"&gt;</code> node.
  14 </para>
  15
  16   <section><title>Introduction</title>
  17
  18   <para>
  19   Mir allows admins to fully configure the producer tasks and set up
  20   arbitrary producers through the <filename>producers.xml</filename> file.
  21   </para>
  22
  23   <para>
  24   Producers consist of "nodes". Every node has a specific function.
  25   For example, it is possible to use a node to generate a file out of a template.
  26   Or it is possible to use a node to enumerate over a collection of articles.
  27   </para>
  28
  29
  30
  31   <para>
  32   A producer is defined using a Producer tag:
  33   <programlisting>
  34     &lt;producer name="content"/&gt;
  35   </programlisting>
  36   This would define a producer named <code>content</code>.
  37
  38   </para><para>
  39   In a producer, <emphasis>verbs</emphasis> must be defined.
  40   Verbs are sub-tasks of a producer.
  41
  42   <programlisting>
  43
  44     &lt;producer name="content"&gt;
  45       &lt;verbs&gt;
  46         &lt;verb name="new"&gt;
  47         &lt;/verb&gt;
  48         &lt;verb name="all"&gt;
  49         &lt;verb&gt;
  50       &lt;/verbs&gt;
  51     &lt;producer name="content"/&gt;
  52   </programlisting>
  53
  54   This would define a producer with verbs named <code>all</code> and <code>new</code>.
  55
  56   </para><para>
  57   And also the specific nodes and their relationship should be specified:
  58
  59   <programlisting>
  60
  61     &lt;producer name="content"&gt;
  62       &lt;verbs&gt;
  63         &lt;verb name="new"&gt;
  64         &lt;/verb&gt;
  65         &lt;verb name="all"&gt;
  66         &lt;verb&gt;
  67       &lt;/verbs&gt;
  68       &lt;body&gt;
  69         &lt;Generate
  70             generator="/producer/startpage.template"
  71             destination="${config.storageRoot}/index.shtml"/&gt;
  72       &lt;/body&gt;
  73     &lt;/producer&gt;
  74   </programlisting>
  75
  76   we will later learn that this producer generates a single file.
  77
  78   </para><para>
  79   Producers can be made to do different things for different verbs:
  80
  81   <programlisting>
  82
  83     &lt;producer name="content"&gt;
  84       &lt;verbs&gt;
  85         &lt;verb name="new"&gt;
  86           &lt;Set key="count" value="3"/&gt;
  87         &lt;/verb&gt;
  88         &lt;verb name="all"&gt;
  89           &lt;Set key="count" value="5"/&gt;
  90         &lt;verb&gt;
  91       &lt;/verbs&gt;
  92       &lt;body&gt;
  93         &lt;Generate
  94             generator="/producer/startpage.template"
  95             destination="${config.storageRoot}/index.shtml"/&gt;
  96       &lt;/body&gt;
  97     &lt;/producer&gt;
  98   </programlisting>
  99
 100   if a producer is called with a specific verb, first the nodes of that verb
 101   are processed, and only thereafter the body: in our case, if the producer
 102   <code>content</code> is called with verb <code>new</code>, first the variable <code>count</code>
 103
 104   is set to <code>3</code>, and after that, a file is generated.
 105
 106 </para>
 107   </section>
 108 <section><title>Node arguments</title>
 109   <para>
 110   Nodes can have arguments.
 111   Arguments that amount to integer values can be direct expressions.
 112   Arguments that amount to text values can be enriched with expressions between ${}.
 113
 114   </para>
 115   <para>
 116   Some examples:
 117   <programlisting>
 118     &lt;Log message="This article has the following title: ${content.title}"/&gt;
 119
 120   </programlisting>
 121   <code>Log</code> has 1 mandatory argument, <code>message</code>. This argument should be text,
 122         but can be enriched with expressions enclosed by ${ and }. In this example,
 123         the title of an article is logged.
 124
 125   <programlisting>
 126     &lt;Set key="age" value="34+22*(3+2)"/&gt;
 127   </programlisting>
 128   <code>Set</code> has 2 mandatory argument: <code>key</code> and <code>value</code>.
 129         The key arugment is a fixed text,
 130         the value argument is a direct expression, in this case of arithmetic nature.
 131
 132   </para>
 133
 134   </section>
 135 <section><title>Expressions</title>
 136
 137
 138   <para>
 139   Expressions, either direct expressions, or expressions between ${}, can contain the following
 140   constructions:
 141     </para>
 142
 143   <para>
 144   <table border="1">
 145     <title>Expressions</title>
 146
 147     <tr><td>a string literal</td>       <td><code>'hello'</code></td></tr>
 148     <tr><td>a numeric literal</td>      <td><code>2138</code></td></tr>
 149
 150     <tr><td>a variable reference</td>   <td><code>content.title</code></td></tr>
 151     <tr><td>arithmetic operators</td>   <td><code> 3 + 4 *(2+5-2)</code></td></tr>
 152     <tr><td>string operators</td>   <td><code> 'hello' ++ ' ' ++ 'Mir'</code></td></tr>
 153
 154     <tr><td>boolean operators</td>      <td><code>(content.id==3) or (content.id in (5,7,2,8) and (content.title!='hello')</code></td></tr>
 155   </table>
 156
 157   </para>
 158   </section>
 159 <section><title>Node types and statements</title>
 160     <para>
 161     Here's  a list of different node types that can be used inside
 162   producer definitions.
 163 Currently, there exists only one statement (&lt;nodedefinition&gt;), that
 164 is declared outside producer definitions.
 165
 166
 167     Here an overview:
 168
 169     </para>
 170     <table border="1">
 171     <title>Reference for producer node types and statements</title>
 172
 173       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Set</code></td></tr>
 174       <tr><td>Purpose</td>                    <td>Alter a variable's using a free expression</td></tr>
 175       <tr><td colspan="2">
 176               Arguments</td></tr>
 177       <tr><td>key</td>                        <td>The variable</td></tr>
 178
 179       <tr><td>value</td>                      <td>The expression to set the variable to</td></tr>
 180       <tr>
 181         <td>Example</td>
 182         <td>
 183           <code>
 184             &lt;Set key="data.result" value="3 + 5 * (5-2)"/&gt;
 185
 186           </code>
 187         </td>
 188       </tr>
 189
 190
 191       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Define</code></td></tr>
 192       <tr><td>Purpose</td>                    <td>Alter a variable's using a string</td></tr>
 193       <tr><td colspan="2">
 194
 195               Arguments</td></tr>
 196       <tr><td>key</td>                        <td>The variable</td></tr>
 197       <tr><td>value</td>                      <td>The string to set the variable to</td></tr>
 198       <tr>
 199         <td>Example</td>
 200
 201         <td>
 202           <code>
 203             &lt;Define key="filename" value="/var/www/${content.id}.shtml"/&gt;
 204           </code>
 205         </td>
 206       </tr>
 207
 208
 209
 210       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>If</code></td></tr>
 211
 212       <tr><td>Purpose</td>                    <td>Create a conditional part of a producer</td></tr>
 213       <tr><td colspan="2">
 214               Arguments</td></tr>
 215       <tr><td>condition</td>                  <td>The expression to test</td></tr>
 216       <tr><td colspan="2">
 217               Sub tags</td></tr>
 218
 219       <tr><td>then</td>                       <td>The part to process if the expression evaluates to true</td></tr>
 220       <tr><td>else</td>                      <td>The part to process if the expression evaluates to false</td></tr>
 221
 222
 223       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>nodedefinition</code>&nbsp;&nbsp;(statement)</td></tr>
 224
 225       <tr><td>Purpose</td>                    <td>Acts as a "function" (or "macro") that can be "called" elsewhere in a producer.
 226 More precisely, it is a way to define a new producer node type
 227 inside the  <filename>producers.xml</filename> file.
 228 (check the <code>Language</code> node in that file for an example).
 229 </td></tr>
 230       <tr><td colspan="2">
 231               Arguments</td></tr>
 232       <tr><td>name</td>                  <td>The name of the newly created node ("function")</td></tr>
 233       <tr><td colspan="2">
 234               Sub tags</td></tr>
 235
 236       <tr><td>parameters</td>                 <td>a list describing the arguments the "function" must be given </td></tr>
 237       <tr><td>definition</td>                 <td>the actual body of the function, containing the code that should be executed. Note that this may contain a <code>&lt;sub/&gt;</code> tag that is replaced by the child nodes of the calling node. </td></tr>
 238
 239
 240
 241
 242       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Log</code></td></tr>
 243
 244       <tr><td>Purpose</td>                    <td>Log a message in the producer log</td></tr>
 245       <tr><td colspan="2">
 246               Arguments</td></tr>
 247       <tr><td>message</td>                    <td>The message to log</td></tr>
 248
 249
 250
 251       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Enumerate</code></td></tr>
 252       <tr><td>Purpose</td>                    <td>Enumerate over the results of a query</td></tr>
 253       <tr><td colspan="2">
 254               Arguments</td></tr>
 255       <tr><td>key</td>                        <td>The variable name that receives the enumerated record</td></tr>
 256
 257       <tr><td>table</td>                      <td>The table that is used to enumerate over</td></tr>
 258       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 259       <tr><td>order (optional)</td>           <td>The order in which the results of the query are enumerated.</td></tr>
 260       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 261
 262       <tr><td>limit (optional)</td>           <td>The maximum number of records to enumerate</td></tr>
 263       <tr><td colspan="2">
 264               Sub tags</td></tr>
 265       <tr><td colspan="2">
 266               This node can have subnodes that will be processed for every enumerated record</td></tr>
 267 <!--
 268       <tr>
 269         <td>Remarks</td>
 270         <td>
 271         </td>
 272       </tr>
 273       <tr>
 274         <td>Example</td>
 275         <td>
 276           <code>
 277             &lt;Generate generator="/producer/content.template" destination="/var/www/${content.date.formatted.yyyy}/${content.date.formatted.MM}/${content.id}.shtml"/&gt;
 278           </code>
 279         </td>
 280       </tr>
 281  -->
 282
 283
 284
 285
 286       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>List</code></td></tr>
 287
 288       <tr><td>Purpose</td>                    <td>Store the results of a query into a variable</td></tr>
 289       <tr><td colspan="2">
 290               Arguments</td></tr>
 291       <tr><td>key</td>                        <td>The variable name that receives the result list</td></tr>
 292       <tr><td>table</td>                      <td>The table that is used to select from</td></tr>
 293
 294       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 295       <tr><td>order (optional)</td>           <td>The order in which the results of the query are put into the list.</td></tr>
 296       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 297       <tr><td>limit (optional)</td>           <td>The maximum size of the list</td></tr>
 298
 299
 300
 301       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Batch</code></td></tr>
 302       <tr><td>Purpose</td>                    <td>Divide the results of a query into batches</td></tr>
 303       <tr><td colspan="2">
 304               Arguments</td></tr>
 305
 306       <tr><td>key</td>                        <td>The variable name that receives the batch</td></tr>
 307       <tr><td>infokey</td>                    <td>The variable name that receives meta information on the batches</td></tr>
 308       <tr><td>table</td>                      <td>The table that is used to select from</td></tr>
 309       <tr><td>batchsize</td>                  <td>The size of a batch (the first batch however varies in size)</td></tr>
 310
 311       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 312       <tr><td>order (optional)</td>           <td>The order in which the results of the query are put into the list.</td></tr>
 313       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 314       <tr><td>process (optional)</td>         <td>The maximum number of batches to process</td></tr>
 315
 316       <tr><td>minbatchsize (optional)</td>    <td>The minimal size of the first batch</td></tr>
 317       <tr><td colspan="2">
 318               Sub tags</td></tr>
 319       <tr><td>batches</td>                    <td>The part to process for every batch</td></tr>
 320       <tr><td>batchlist</td>                  <td>The part to process once with the meta info</td></tr>
 321
 322
 323
 324
 325
 326
 327       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Generate</code></td></tr>
 328       <tr><td>Purpose</td>                    <td>Generate a page using a generator (i.e. an abstraction of a template)</td></tr>
 329       <tr><td colspan="2">
 330
 331               Arguments</td></tr>
 332
 333       <tr><td>generator</td>                  <td>the generator to use</td></tr>
 334       <tr><td>destination</td>                <td>the specification of the destination.</td></tr>
 335       <tr><td>parameters</td>                 <td>Additional configuration info for the generator (for
 336                                                   freemarker this now only contains the wanted encoding,
 337                                                   empty for the default).</td></tr>
 338
 339       <tr>
 340         <td>Remarks</td>
 341         <td>
 342            This node is used to have an actual page generated.
 343            The generator parameter usually is the name of a template.
 344            The destination is the file to be generated.
 345            Variable references are possible in all arguments, and, especially for the
 346                  destination attribute, widely used.
 347         </td>
 348       </tr>
 349       <tr>
 350         <td>Example</td>
 351
 352         <td>
 353           <code>
 354             &lt;Generate generator="/producer/content.template" destination="/var/www/${content.date.formatted.yyyy}/${content.date.formatted.MM}/${content.id}.shtml"/&gt;
 355           </code>
 356         </td>
 357       </tr>
 358
 359
 360
 361       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>GenerateMedia</code></td></tr>
 362       <tr><td>Purpose</td>                    <td>
 363 The generateMedia node instructs the media handler associated with the media to
 364 "reproduce" the media. In practice this can mean generate an icon from an image,
 365 writing an image from the database to the web root, or create an m3u file or so.
 366 Media handling is limited at this moment, but  a serious
 367 redesign is planned right after 1.1
 368 </td></tr>
 369       <tr><td colspan="2">
 370
 371               Arguments</td></tr>
 372
 373       <tr><td>key</td>                  <td>FIXME???</td></tr>
 374
 375
 376
 377       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>DeleteFile</code></td></tr>
 378       <tr><td>Purpose</td>                    <td>Delete a file</td></tr>
 379       <tr><td colspan="2">
 380               Arguments</td></tr>
 381       <tr><td>filename</td>                   <td>The filename of the file to delete</td></tr>
 382
 383
 384
 385
 386
 387       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>SetFileDate</code></td></tr>
 388       <tr><td>Purpose</td>                    <td>Set a file's date</td></tr>
 389       <tr><td colspan="2">
 390               Arguments</td></tr>
 391
 392       <tr><td>filename</td>                   <td>The filename</td></tr>
 393       <tr><td>date</td>                       <td>The date to use</td></tr>
 394
 395
 396
 397
 398       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Resource</code></td></tr>
 399       <tr><td>Purpose</td>                    <td>Make a message resource bundle available. In other words, this defines a function that can be used in expressions to reference a bundle. </td></tr>
 400       <tr><td colspan="2">
 401
 402               Arguments</td></tr>
 403       <tr><td>key</td>                        <td>The name of the function.</td></tr>
 404       <tr><td>bundle</td>                     <td>The bundle to use</td></tr>
 405       <tr><td>language (optional)</td>        <td>The specific language to use</td></tr>
 406       <tr>
 407         <td>Example</td>
 408         <td>
 409           <code>
 410             &lt;Resource bundle="bundles.producer" key="lang" language="en"/&gt;
 411           </code>
 412           This makes the english language producer bundle available
 413           through the "lang()" function. One
 414           can then  use expressions like  ${lang("page.title")} to
 415           refer to a that value.
 416         </td>
 417       </tr>
 418
 419
 420
 421
 422
 423       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Execute</code></td></tr>
 424       <tr><td>Purpose</td>                    <td>Execute a script</td></tr>
 425       <tr><td colspan="2">
 426               Arguments</td></tr>
 427
 428       <tr><td>command</td>                    <td>The command to execute</td></tr>
 429
 430
 431
 432
 433
 434
 435       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>ModifyContent</code></td></tr>
 436       <tr><td>Purpose</td>                    <td>Modify a field of an article</td></tr>
 437       <tr><td colspan="2">
 438
 439               Arguments</td></tr>
 440       <tr><td>key</td>                        <td>The variable containing the article</td></tr>
 441       <tr><td>field</td>                      <td>The field to modify</td></tr>
 442       <tr><td>value</td>                      <td>The value to set the field to</td></tr>
 443
 444
 445
 446       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>MarkContent</code></td></tr>
 447       <tr><td>Purpose</td>                    <td>Mark an article as produced</td></tr>
 448       <tr><td colspan="2">
 449               Arguments</td></tr>
 450
 451       <tr><td>key</td>                        <td>The variable containing the article</td></tr>
 452
 453       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>IndexContent</code></td></tr>
 454       <tr><td>Purpose</td>                    <td>adds/updates content to the search index (search engine)</td></tr>
 455       <tr><td colspan="2">
 456               Arguments</td></tr>
 457
 458       <tr><td>key</td>                        <td>The variable containing the article (FIXME????)</td></tr>
 459       <tr><td>pathToIndex</td>                        <td>(FIXME????)</td></tr>
 460
 461       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>UnIndexContent</code></td></tr>
 462       <tr><td>Purpose</td>                    <td>removes content from the search index (search engine)</td></tr>
 463       <tr><td colspan="2">
 464               Arguments</td></tr>
 465
 466       <tr><td>key</td>                        <td>The variable containing the article (FIXME????)</td></tr>
 467       <tr><td>pathToIndex</td>                        <td>(FIXME????)</td></tr>
 468
 469     </table>
 470 </section>
 471 </section>
 472
 473 <section><title>How the producer framework is implemented</title>
 474
 475 <!-- <para>A Producer is a set of tasks, scripted in xml. Producers allow -->
 476 <!-- mir installations to have their own actions that can be called for -->
 477 <!-- instance when a new article is posted.  Originally producers were -->
 478 <!-- mostly used to generate pages, but they are used for a lot of -->
 479 <!-- other tasks such as pulling rss feeds for the global wire on -->
 480 <!-- indymedia.org. Producers are added and configured through the -->
 481 <!-- <filename>producers.xml</filename> file.</para> -->
 482
 483
 484 <para>The xml nodes contained within a <code>&lt;producer&gt;</code>
 485 tag  in the <filename>producers.xml</filename> file define
 486 a small  program.
 487 This program  (or script) may  contain constructs
 488 such as  <code>if</code> clauses, loops  and variables... The  program is parsed  into a
 489 tree  of <classname>ProducerNode</classname>s (figure).   The root  of this  tree is  defined  in a
 490 <classname>NodedProducer</classname> (which  is the  only class that  currently implements
 491 the  <classname>Producer</classname>  interface).   When  the producer  is  executed,  the
 492 <function>produce()</function> methods of each  node are recursively called, effectively
 493 executing the program as it was scripted. </para>
 494
 495 <figure><title>A producer in the <filename>producer.xml</filename> file is parsed into a tree of <classname>ProducerNodes</classname></title>
 496 <mediaobject>
 497 <imageobject>
 498 <imagedata fileref="figures/producer-node-tree-example.eps" format="EPS"></imagedata></imageobject>
 499 <imageobject>
 500 <imagedata fileref="figures/producer-node-tree-example.png" format="PNG"></imagedata></imageobject>
 501 <textobject>
 502 </textobject>
 503 </mediaobject>
 504 </figure>
 505
 506 <para>
 507 </para>
 508 </section>
 509 </chapter>