doc/developers-guide/producers.xml

   1 <chapter id="introduction">
   2 <title>The producer framework</title>
   3
   4 <section><title>How to use the <filename>producers.xml</filename> file</title>
   5   [FIXME: this should be in the user's guide, not here]
   6
   7   <section><title>Introduction</title>
   8
   9   <para>
  10   With Mir, it is now possible to set up arbitrary producers using an xml file.
  11   </para>
  12
  13   <para>
  14   Producers consist of "nodes". Every node has a specific function.
  15   For example, it is possible to use a node to generate a file out of a template.
  16   Or it is possible to use a node to enumerate over a collection of articles.
  17   </para>
  18
  19
  20
  21   <para>
  22   A producer is defined using a Producer tag:
  23
  24   <programlisting>
  25     &lt;producer name="content"/&gt;
  26   </programlisting>
  27
  28   would define a producer named <code>content</code>.
  29
  30   </para><para>
  31
  32   In a producer, <emphasis>verbs</emphasis> must be defined.
  33   Verbs are sub-tasks of a producer.
  34
  35   <programlisting>
  36
  37     &lt;producer name="content"&gt;
  38       &lt;verbs&gt;
  39         &lt;verb name="new"&gt;
  40         &lt;/verb&gt;
  41         &lt;verb name="all"&gt;
  42         &lt;verb&gt;
  43       &lt;/verbs&gt;
  44     &lt;producer name="content"/&gt;
  45   </programlisting>
  46
  47   would define a producer with verbs named <code>all</code> and <code>new</code>.
  48
  49   </para><para>
  50   And also the specific nodes and their relationship should be specified:
  51
  52   <programlisting>
  53
  54     &lt;producer name="content"&gt;
  55       &lt;verbs&gt;
  56         &lt;verb name="new"&gt;
  57         &lt;/verb&gt;
  58         &lt;verb name="all"&gt;
  59         &lt;verb&gt;
  60       &lt;/verbs&gt;
  61       &lt;body&gt;
  62         &lt;Generate
  63             generator="/producer/startpage.template"
  64             destination="${config.storageRoot}/index.shtml"/&gt;
  65       &lt;/body&gt;
  66     &lt;/producer&gt;
  67   </programlisting>
  68
  69   we will later learn that this producer generates a single file.
  70
  71   </para><para>
  72   Producers can be made to do different things for different verbs:
  73
  74   <programlisting>
  75
  76     &lt;producer name="content"&gt;
  77       &lt;verbs&gt;
  78         &lt;verb name="new"&gt;
  79           &lt;Set key="count" value="3"/&gt;
  80         &lt;/verb&gt;
  81         &lt;verb name="all"&gt;
  82           &lt;Set key="count" value="5"/&gt;
  83         &lt;verb&gt;
  84       &lt;/verbs&gt;
  85       &lt;body&gt;
  86         &lt;Generate
  87             generator="/producer/startpage.template"
  88             destination="${config.storageRoot}/index.shtml"/&gt;
  89       &lt;/body&gt;
  90     &lt;/producer&gt;
  91   </programlisting>
  92
  93   if a producer is called with a specific verb, first the nodes of that verb
  94   are processed, and only thereafter the body: in our case, if the producer
  95   <code>content</code> is called with verb <code>new</code>, first the variable <code>count</code>
  96
  97   is set to <code>3</code>, and after that, a file is generated.
  98
  99 </para>
 100   </section>
 101 <section><title>Node arguments</title>
 102   <para>
 103   Nodes can have arguments.
 104   Arguments that amount to integer values can be direct expressions.
 105   Arguments that amount to text values can be enriched with expressions between ${}.
 106
 107   </para>
 108   <para>
 109   Some examples:
 110   <programlisting>
 111     &lt;Log message="This article has the following title: ${content.title}"/&gt;
 112
 113   </programlisting>
 114   <code>Log</code> has 1 mandatory argument, <code>message</code>. This argument should be text,
 115         but can be enriched with expressions enclosed by ${ and }. In this example,
 116         the title of an article is logged.
 117
 118   <programlisting>
 119     &lt;Set key="age" value="34+22*(3+2)"/&gt;
 120   </programlisting>
 121   <code>Set</code> has 2 mandatory argument: <code>key</code> and <code>value</code>.
 122         The key arugment is a fixed text,
 123         the value argument is a direct expression, in this case of arithmetic nature.
 124
 125   </para>
 126
 127   </section>
 128 <section><title>Expressions</title>
 129
 130
 131   <para>
 132   Expressions, either direct expressions, or expressions between ${}, can contain the following
 133   constructions:
 134     </para>
 135
 136   <para>
 137   <table border="1">
 138     <tr><td>a string literal</td>       <td><code>'hello'</code></td></tr>
 139     <tr><td>a numeric literal</td>      <td><code>2138</code></td></tr>
 140
 141     <tr><td>a variable reference</td>   <td><code>content.title</code></td></tr>
 142     <tr><td>arithmetic operators</td>   <td><code> 3 + 4 *(2+5-2)</code></td></tr>
 143     <tr><td>string operators</td>   <td><code> 'hello' ++ ' ' ++ 'Mir'</code></td></tr>
 144
 145     <tr><td>boolean operators</td>      <td><code>(content.id==3) or (content.id in (5,7,2,8) and (content.title!='hello')</code></td></tr>
 146   </table>
 147
 148   </para>
 149   </section>
 150 <section><title>Node types and statements</title>
 151     <para>
 152     Here's  a list of different node types that can be used inside
 153   producer definitions.
 154 Currently, there exists only one statement (&lt;nodedefinition&gt;), that
 155 is declared outside producer definitions.
 156
 157
 158     Here an overview:
 159
 160     </para>
 161     <table border="1">
 162
 163       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Set</code></td></tr>
 164       <tr><td>Purpose</td>                    <td>Alter a variable's using a free expression</td></tr>
 165       <tr><td colspan="2">
 166               Arguments</td></tr>
 167       <tr><td>key</td>                        <td>The variable</td></tr>
 168
 169       <tr><td>value</td>                      <td>The expression to set the variable to</td></tr>
 170       <tr>
 171         <td>Example</td>
 172         <td>
 173           <code>
 174             &lt;Set key="data.result" value="3 + 5 * (5-2)"/&gt;
 175
 176           </code>
 177         </td>
 178       </tr>
 179
 180
 181       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Define</code></td></tr>
 182       <tr><td>Purpose</td>                    <td>Alter a variable's using a string</td></tr>
 183       <tr><td colspan="2">
 184
 185               Arguments</td></tr>
 186       <tr><td>key</td>                        <td>The variable</td></tr>
 187       <tr><td>value</td>                      <td>The string to set the variable to</td></tr>
 188       <tr>
 189         <td>Example</td>
 190
 191         <td>
 192           <code>
 193             &lt;Define key="filename" value="/var/www/${content.id}.shtml"/&gt;
 194           </code>
 195         </td>
 196       </tr>
 197
 198
 199
 200       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>If</code></td></tr>
 201
 202       <tr><td>Purpose</td>                    <td>Create a conditional part of a producer</td></tr>
 203       <tr><td colspan="2">
 204               Arguments</td></tr>
 205       <tr><td>condition</td>                  <td>The expression to test</td></tr>
 206       <tr><td colspan="2">
 207               Sub tags</td></tr>
 208
 209       <tr><td>then</td>                       <td>The part to process if the expression evaluates to true</td></tr>
 210       <tr><td>else</td>                      <td>The part to process if the expression evaluates to false</td></tr>
 211
 212
 213       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>nodedefinition</code>&nbsp;&nbsp;(statement)</td></tr>
 214
 215       <tr><td>Purpose</td>                    <td>Acts as a "function" (or "macro") that can be "called" elsewhere in a producer.
 216 More precisely, it is a way to define a new producer node type
 217 inside the  <filename>producers.xml</filename> file.
 218 (check the <code>Language</code> node in that file for an example).
 219 </td></tr>
 220       <tr><td colspan="2">
 221               Arguments</td></tr>
 222       <tr><td>name</td>                  <td>The name of the newly created node ("function")</td></tr>
 223       <tr><td colspan="2">
 224               Sub tags</td></tr>
 225
 226       <tr><td>parameters</td>                 <td>a list describing the arguments the "function" must be given </td></tr>
 227       <tr><td>definition</td>                 <td>the actual body of the function, containing the code that should be executed. Note that this may contain a <code>&lt;sub/&gt;</code> tag that is replaced by the child nodes of the calling node (FIXME is this true???)  </td></tr>
 228
 229
 230
 231
 232       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Log</code></td></tr>
 233
 234       <tr><td>Purpose</td>                    <td>Log a message in the producer log</td></tr>
 235       <tr><td colspan="2">
 236               Arguments</td></tr>
 237       <tr><td>message</td>                    <td>The message to log</td></tr>
 238
 239
 240
 241       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Enumerate</code></td></tr>
 242       <tr><td>Purpose</td>                    <td>Enumerate over the results of a query</td></tr>
 243       <tr><td colspan="2">
 244               Arguments</td></tr>
 245       <tr><td>key</td>                        <td>The variable name that receives the enumerated record</td></tr>
 246
 247       <tr><td>table</td>                      <td>The table that is used to enumerate over</td></tr>
 248       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 249       <tr><td>order (optional)</td>           <td>The order in which the results of the query are enumerated.</td></tr>
 250       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 251
 252       <tr><td>limit (optional)</td>           <td>The maximum number of records to enumerate</td></tr>
 253       <tr><td colspan="2">
 254               Sub tags</td></tr>
 255       <tr><td colspan="2">
 256               This node can have subnodes that will be processed for every enumerated record</td></tr>
 257 <!--
 258       <tr>
 259         <td>Remarks</td>
 260         <td>
 261         </td>
 262       </tr>
 263       <tr>
 264         <td>Example</td>
 265         <td>
 266           <code>
 267             &lt;Generate generator="/producer/content.template" destination="/var/www/${content.date.formatted.yyyy}/${content.date.formatted.MM}/${content.id}.shtml"/&gt;
 268           </code>
 269         </td>
 270       </tr>
 271  -->
 272
 273
 274
 275
 276       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>List</code></td></tr>
 277
 278       <tr><td>Purpose</td>                    <td>Store the results of a query into a variable</td></tr>
 279       <tr><td colspan="2">
 280               Arguments</td></tr>
 281       <tr><td>key</td>                        <td>The variable name that receives the result list</td></tr>
 282       <tr><td>table</td>                      <td>The table that is used to select from</td></tr>
 283
 284       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 285       <tr><td>order (optional)</td>           <td>The order in which the results of the query are put into the list.</td></tr>
 286       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 287       <tr><td>limit (optional)</td>           <td>The maximum size of the list</td></tr>
 288
 289
 290
 291       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Batch</code></td></tr>
 292       <tr><td>Purpose</td>                    <td>Divide the results of a query into batches</td></tr>
 293       <tr><td colspan="2">
 294               Arguments</td></tr>
 295
 296       <tr><td>key</td>                        <td>The variable name that receives the batch</td></tr>
 297       <tr><td>infokey</td>                    <td>The variable name that receives meta information on the batches</td></tr>
 298       <tr><td>table</td>                      <td>The table that is used to select from</td></tr>
 299       <tr><td>batchsize</td>                  <td>The size of a batch (the first batch however varies in size)</td></tr>
 300
 301       <tr><td>selection (optional)</td>       <td>The condition (where clause) of the query.</td></tr>
 302       <tr><td>order (optional)</td>           <td>The order in which the results of the query are put into the list.</td></tr>
 303       <tr><td>skip (optional)</td>            <td>The number of records to skip</td></tr>
 304       <tr><td>process (optional)</td>         <td>The maximum number of batches to process</td></tr>
 305
 306       <tr><td>minbatchsize (optional)</td>    <td>The minimal size of the first batch</td></tr>
 307       <tr><td colspan="2">
 308               Sub tags</td></tr>
 309       <tr><td>batches</td>                    <td>The part to process for every batch</td></tr>
 310       <tr><td>batchlist</td>                  <td>The part to process once with the meta info</td></tr>
 311
 312
 313
 314
 315
 316
 317       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Generate</code></td></tr>
 318       <tr><td>Purpose</td>                    <td>Generate a page using a generator (i.e. an abstraction of a template)</td></tr>
 319       <tr><td colspan="2">
 320
 321               Arguments</td></tr>
 322
 323       <tr><td>generator</td>                  <td>the generator to use</td></tr>
 324       <tr><td>destination</td>                <td>the specification of the destination.</td></tr>
 325       <tr><td>parameters</td>                 <td>Additional configuration info for the generator (for
 326                                                   freemarker this now only contains the wanted encoding,
 327                                                   empty for the default).</td></tr>
 328
 329       <tr>
 330         <td>Remarks</td>
 331         <td>
 332            This node is used to have an actual page generated.
 333            The generator parameter usually is the name of a template.
 334            The destination is the file to be generated.
 335            Variable references are possible in all arguments, and, especially for the
 336                  destination attribute, widely used.
 337         </td>
 338       </tr>
 339       <tr>
 340         <td>Example</td>
 341
 342         <td>
 343           <code>
 344             &lt;Generate generator="/producer/content.template" destination="/var/www/${content.date.formatted.yyyy}/${content.date.formatted.MM}/${content.id}.shtml"/&gt;
 345           </code>
 346         </td>
 347       </tr>
 348
 349
 350
 351
 352       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>DeleteFile</code></td></tr>
 353       <tr><td>Purpose</td>                    <td>Delete a file</td></tr>
 354       <tr><td colspan="2">
 355               Arguments</td></tr>
 356       <tr><td>filename</td>                   <td>The filename of the file to delete</td></tr>
 357
 358
 359
 360
 361
 362       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>SetFileDate</code></td></tr>
 363       <tr><td>Purpose</td>                    <td>Set a file's date</td></tr>
 364       <tr><td colspan="2">
 365               Arguments</td></tr>
 366
 367       <tr><td>filename</td>                   <td>The filename</td></tr>
 368       <tr><td>date</td>                       <td>The date to use</td></tr>
 369
 370
 371
 372
 373       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Resource</code></td></tr>
 374       <tr><td>Purpose</td>                    <td>Make a message resource bundle available</td></tr>
 375       <tr><td colspan="2">
 376
 377               Arguments</td></tr>
 378       <tr><td>key</td>                        <td>The variable in which the bundle will be stored</td></tr>
 379       <tr><td>bundle</td>                     <td>The bundle to use</td></tr>
 380       <tr><td>language (optional)</td>        <td>The specific language to use</td></tr>
 381
 382
 383
 384
 385
 386       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>Execute</code></td></tr>
 387       <tr><td>Purpose</td>                    <td>Execute a script</td></tr>
 388       <tr><td colspan="2">
 389               Arguments</td></tr>
 390
 391       <tr><td>command</td>                    <td>The command to execute</td></tr>
 392
 393
 394
 395
 396
 397
 398       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>ModifyContent</code></td></tr>
 399       <tr><td>Purpose</td>                    <td>Modify a field of an article</td></tr>
 400       <tr><td colspan="2">
 401
 402               Arguments</td></tr>
 403       <tr><td>key</td>                        <td>The variable containing the article</td></tr>
 404       <tr><td>field</td>                      <td>The field to modify</td></tr>
 405       <tr><td>value</td>                      <td>The value to set the field to</td></tr>
 406
 407
 408
 409       <tr><td>Name</td>                       <td bgcolor="#eea8a8"><code>MarkContent</code></td></tr>
 410       <tr><td>Purpose</td>                    <td>Mark an article as produced</td></tr>
 411       <tr><td colspan="2">
 412               Arguments</td></tr>
 413
 414       <tr><td>key</td>                        <td>The variable containing the article</td></tr>
 415
 416
 417
 418     </table>
 419 </section>
 420 </section>
 421
 422 <section><title>How the producer framework is implemented</title>
 423
 424 <para>A Producer is a set of tasks, scripted in xml. Producers allow
 425 mir installations to have their own actions that can be called for
 426 instance when a new article is posted.  Originally producers were
 427 mostly used to generate pages, but they are used for a lot of
 428 other tasks such as pulling rss feeds for the global wire on
 429 indymedia.org. Producers are added and configured through the
 430 <filename>producers.xml</filename> file.</para>
 431
 432
 433 <para>The xml nodes contained within a <code>&lt;producer&gt;</code>
 434 tag  in the <filename>producers.xml</filename> file define
 435 a small  program.
 436 This program  (or script) may  contain constructs
 437 such as  <code>if</code> clauses, loops  and variables... The  program is parsed  into a
 438 tree  of ProducerNodes (figure).   The root  of this  tree is  defined  in a
 439 NodedProducer (which  is the  only class that  currently implements
 440 the  Producer  interface).   When  the Producer  is  executed,  the
 441 <function>produce</function> methods of each  node are recursively called, effectively
 442 executing the program as it was scripted. </para>
 443
 444 <figure><title>A producer in the <filename>producer.xml</filename> file is parsed into a tree of <classname>ProducerNodes</classname></title>
 445 <mediaobject>
 446 <imageobject>
 447 <imagedata fileref="figures/producer-node-tree-example.eps" format="EPS"></imagedata></imageobject>
 448 <imageobject>
 449 <imagedata fileref="figures/producer-node-tree-example.png" format="PNG"></imagedata></imageobject>
 450 <textobject>
 451 </textobject>
 452 </mediaobject>
 453 </figure>
 454
 455 <para>
 456 </para>
 457 </section>
 458 </chapter>