[mir.git] / doc / LONG.INSTALL.mir

MIR INSTALLATION HOWTO

Last updated: $Date: 2002/10/25 20:22:10 $
----------------------------------------------------------------

Here is a short installation-howto of Mir, somewhat more explicit than
the earlier version.  Some Debian-specific instructions are given.  


prerequisites: 

- tomcat 4.0.4 (4.0.3 and below have some bad bugs) or above (3.3 works too as
  of 04.04.2002, but this could change)
  tomcat is available from http://jakarta.apache.org/tomcat/
- apache 1.3.x. with mod_jk.so. As far as I can tell the connector for 2.x is
  still rather undocumented. http://httpd.apache.org
- postgres 7.1.x or 7.2.x. http://www.postgresql.org
- ant (a java-based make) 
- jaxp-1.1 (a SAX 2.0 compliant XML parser, comes with ant >= 1.4)
- the JAI image framework (Java Advanced Imaging) versin 1.1.1 . get it from 
  java.sun.com. ** NOTE: because JAI uses a native acceration library (a .so)
  it must be placed in tomcat's "lib" (i.e $TOMCAT_HOME/common/lib) directory and
  not under the default webapps/Mir/WEB-INF/lib directory **
- A good reading of Tomcat, Apache and Postgresql documentation if you are not
  familiar with any of them. The documentation is available at:
  http://jakarta.apache.org/tomcat/tomcat-4.0-doc/index.html,
  http://httpd.apache.org/docs/ and http://www.postgresql.org respectively.
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
0. To set all this up on Debian:

make sure you have stable, testing, and unstable listings in
/etc/apt/sources.list.  THen do all the following as root or using sudo!

If you already have tomcat, and it's working well with apache, then
skip directly to (c).  If it's not working well, then make sure you
purge _all_ conf files & so forth ( though it never hurts to back them
up first).  "apt-get remove --purge" may not get rid of everything; try 

locate tomcat
and then delete all the files you find (again, if you're unsure what
they are, then back them up first).


(a) Install Java 
apt-get install j2sdk1.3 j2sdk1.3-doc

JAVA_HOME=/usr/lib/j2sdk1.3; export JAVA_HOME
PATH=$PATH:$JAVA_HOME/bin; export PATH
(add these lines to your .bashrc as well)

(b) Installing Tomcat and Apache

As root, do the following:
apt-get install apache 
 configure apache by editing httpd.conf:
* enable shtml includes:
  - add LoadModule includes_module /usr/lib/apache/1.3/mod_include.so
  - make sure your directory contains "Options Includes"
* Determine if you need to modify any apache mime-mappings
  - The web-server host must recognize the .m3u, .pls and other file extensions
    and send the proper "audio/x-mpegurl" and "audio/x-scpls" mime-types
    respectively.  If the web server is apache, it's easy, just
    add:
    
    audio/x-mpegurl                 m3u
    audio/x-scpl                    pls
    
    to the file pointed to by the "TypesConfig" command in your apache config
    file. Or add and equivalent AddType command to your httpd.conf.  Of course
    this assumes that the mod_mime is loaded.


Now deal with Tomcat:
apt-get install -t unstable tomcat4 tomcat4-webapps tomcat4-admin 
        --> this gets you tomcat4.1.  You can work with tomcat 4.0.4
        as well, but a couple of specific configuration elements are
        different.  To get 4.0.4, run the following:
        apt-get install -t testing tomcat4 tomcat4-webapps
apt-get install apachelib-java

Tomcat will be installed in /usr/share/tomcat4, so you should put 

CATALINA_HOME=/usr/share/tomcat4; export CATALINA_HOME
TOMCAT_HOME=$CATALINA_HOME; export TOMCAT_HOME

in your .bashrc; you might as well execute the ocmmand right now as
well.  

I found it somewhat difficult to get Tomcat and Apache working
   together, until I emailed the Debian package maintainer, who told
   me that he had purposely removed the mod_jk.conf-auto function from
   Tomcat4.1!  This is because Debian encourages Tomcat users to
   switch to mod_webapp instead.  You may want to try this option
   out.  I didn't, because I was able to get it working using the
   JkMount option.  So in /etc/apache/httpd.conf, look for a section
   like this at the end:  
        <IfModule mod_jk.c>                                                           
        # The following line is for apacheconfig - DO NOT REMOVE!         
        JkWorkersFile /etc/tomcat/jk/workers.properties                   
        Include /var/lib/tomcat/conf/mod_jk.conf                          
        </IfModule>                                                       
                                                                        
   This has been insrted by libapache-java to try to get mod_jk to
   work. unfortunately it doesn't work with Tomcat 4.1, so replace it
   with the following:

        <IfModule mod_jk.c>                                               
        # The following line is for apacheconfig - DO NOT REMOVE!         
        JkWorkersFile /etc/tomcat/jk/workers.properties                   
        JkMount /*.jsp ajp13                                              
        JkMount /Mir ajp13                                                
        JkMount /Mir/* ajp13                                              
        JkMount /servlet ajp13                                            
        jkMount /examples/* ajp13                                                 
        #Include /var/lib/tomcat/conf/mod_jk.conf                         
        </IfModule>                                                       
restart tomcat(as root):
/usr/share/tomcat4/bin/shutdown.sh
/usr/share/tomcat4/bin/startup.sh
restart apache(as root):
apachectl restart

In your browser, check to see if each pogram is working on its own:
http://localhost should give you the debian apache start page
http://localhost:8180 should give you the tomcat start page

now check to see if mod_jk is allowing the two programs to connect:
http://localhost/examples/servlets/index.html should give you the
Tomcat example servlets.  

Once this is working, you should enable the tomcat manager
application:  
cd /usr/share/tomcat4/conf
open tomcat-users.xml in a text editor.  Add a line like this:

<user username="name" password="password" roles="standard,manager,admin"/>

You should now be able to click on the "Manager" link on the Tomcat
home page, log in, and use the Manager functions.  This is extremely
convenient when you're eloading Mir multiple times later on!

(c) install postgres:
apt-get install postgresql postgresql-client postgresql-doc
--> the new postgres package should work fine with Mir.  Make sure you
enable UNICODE, and if asked about JDBC (you shouldn't be) make sure
to enable it as well. if you would like to try a graphical frontend
for the database, find one using

apt-cache show postgresql

fmailiarize yourself with the psql interface, and the function of the
"postgres" user.

(d) install JAI
go to:
http://java.sun.com/products/java-media/jai/downloads/download.html

and download the "CLASSPATH for Linux" version of the JAI package
(1.1.01 works fine, later versions have not been tested by me) to 
/usr/share/tomcat4/common/lib

now:

cd /usr/share/tomcat4/common/lib

gunzip "name_of_file.tar.gz" (don't use the quotes: replace with the
real name of the file you downloaded!)

tar -xvf "name_of_file.tar" 

read the instructions in
/usr/share/tomcat4/common/lib/jai_whatever_version_you_have/INSTALL-jai.txt

Now add the JAI files to your CLASSPATH and LSD_LIBRARY_PATH:  
JAIHOME=$TOMCAT_HOME/common/lib/jai-1_1_1_01/lib; export JAIHOME
CLASSPATH=$JAIHOME/jai_core.jar:$JAIHOME/jai_codec.jar:$JAIHOME/mlibwrapper_jai.jar:$CLASSPATH;
export CLASSPATH
LD_LIBRARY_PATH=.:$JAIHOME:$CLASSPATH; export LD_LIBRARY_PATH

--> make sure to replace "jai-1_1_1_01" in the above with the
version-number of your copy of JAI
--> as above, it's a good idea to add these lines to your .bashrc

---------------------------------------------------------------


(1) CONFIGURE MIR!

(a).  get the latest version:

cd to the directory where you would like to install the mir package

cd /absolute/path

CVS LOGIN:

  cvs -d :pserver:anonymous@mir.indymedia.org:/var/lib/cvs login
        password: anonymous

CVS CHECKOUT:

        cvs -d :pserver:anonymous@mir.indymedia.org:/var/lib/cvs co mir 


(b) customize the config: 

        cd mir/etc
        cp config.properties-dist config.properties 

now customize config.properties for your needs.  config.properties is
pretty well-documented, but read it carefully.  I had to recompile
several times before I got all the pathnames right.
 

(c) configure the perms.sh file if neccessary -- IMPORTANT! READ THIS!
We provide a script that sets all files' and direcories' permissions to
a quite reasonable state. This script gets automagically called by
ant after compilationl. The most important thing you have to do after
compiling Mir is to ensure that the log files -- especially 
dbentity.log -- are not readable by users that could compromise 
system security, because all passwords and the like will be logged here.
        
        cp perms.sh-dist perms.sh

Now, change the install directory and group in perms.sh

        edit perms.sh 


(d) Customize the templates
you need to edit the templates before compiling, or not all of your
changes will show up in the produced site.  If this is too
intimidating, don't worry -- you can always recompile!  It only takes
about a minute...

(e) check web.xml!
--> tomcat 4.1 has a different method of composing servlet URL's, so
if you are uasing 4.1, you wil need to make some minor changes to
mir/etc/web.xml

look for sections like this: 
  <servlet-mapping>
        <servlet-name>
            Mir
        </servlet-name>
        <url-pattern>
            /Mir
        </url-pattern>
    </servlet-mapping>

the url-pattern has to change; so replace the above with:
  <servlet-mapping>
        <servlet-name>
            Mir
        </servlet-name>
        <url-pattern>
            /servlet/Mir
        </url-pattern>
    </servlet-mapping>
make sure to do the same for OpenMir and OutputMir.


-------------------------------------------------------------
-------------------------------------------------------------
IMPORTANT!  It is a good idea to skip ahead to (2) and create the
Postgres database at this point.  Then cd back to the mir directory,
and continue.  
-------------------------------------------------------------
-------------------------------------------------------------

(f) compile. Make sure all the environment variables are set as
   indicated in section (0), or build.xml will not run properly:

su
ant

  This should take about a minute.  If you don't get an error, run
   perms.sh:
        sudo perms.sh


Do this as root so the permissions script is able to set
the permissions and owners correctly.
   

(g) Get Tomcat to recognize Mir.
Link in the webapps directory of tomcat to the install directory
(unless you changed the setting in config.properties, the 
directory is called "Mir" and is located in the same directory in which 
you installed the "mir" directory). (Here and in the rest of this document,
we assume you called the link "Mir", but this could be named anything.)
        cd /path/to/tomcat/webapps (tomcat-4.0.x/webapps)
        ln -s /path/to/Mir Mir

dynamically reload Mir: 

http://localhost:8180/manager/stop?path=/Mir
if you're not using the Debian installation, you need to use:
http://localhost:8080/manager/stop?path=/Mir

if you're using Tomcat 3.3, you need to restart Tomcat (consult the
docs for how to do that).  


(h) Copy any dynamic library files ending with ".so" (so far only the JAI native
acceleration library found in the JAI package tarball or zip from sun) to your
$JAVA_HOME/jre/lib/i386 directory (where the other ".so" files live). Or, you
can skip the whole thing and live without "native" acceleration for image
manupulation.

(2) CREATE AND CONLFIGURE THE MIR DATABASE!

Review mir/etc/config.properties.  Look carefully at the entries you
put in the DATABASE SETUP section, especially the Username, Password,
Host, and Name variables.  If you don't know anything about Postgres,
look over the INSTALL.postgresql document, which is very helpful (but
remember, you don't need to install postgresql from scratch
anymore if you're on Debian woody!).  Please do not use the "postgres"
user in config.properties -- this is a serious security risk.  Choose
a username,a password, and a database name, then put those values in
config.properties if you haven't already.  

(a) Create the new database

We create and configure the database as posgresql user "postgres":

   createdb -U postgres --encoding=unicode Mir 
--> here and elsewhere, replace "Mir" with the name you already chose
for your database


(b) create an unprivileged database user for Mir
First, connect to the database as the database's superuser. 

        psql -U postgres Mir

Now we create the actual user. Please choose a password that is hard to 
guess instead of "joshua". Good passwords have characters and numerals in
it, have no link to its owner (like being her birthday, age, name of her 
husband, dog, child, car, favourite beer brand). A good password looks like
this: "8ncx4un".
    
    CREATE USER Mir WITH PASSWORD 'joshua' NOCREATEDB NOCREATEUSER;
--> again, don't just copy this into the psql command line -- replace
"Mir" and "joshua" with your username and password from config.properties

now exit: 
\q


(c) create base table
cd back to the mir home directory.  now execute the following:

        psql -Upostgres -f dbscripts/create_pg.sql Mir
now quit psql:
        \q
back in the shell, execute the dbscripts:
    for i in dbscripts/help*.sql ; do psql -Upostgres -f $i Mir ; done
    for i in dbscripts/populate*.sql ; do psql -Upostgres -f $i Mir ; done

(d) Grant the required permissions to the new user
First, make your new user into the database administrator for the new
database (instead of postgres):
psql -U postgres Mir
     select * from pg_database;
     select * from pg_user;
you'll see a display like this:
   usename    | usesysid | usecreatedb | usetrace | usesuper | usecatupd |  passwd  | valuntil
--------------+----------+-------------+----------+----------+-----------+----------+----------
 postgres     |        1 | t           | t        | t        | t         | ******** |
 matt         |      100 | t           | f        | t        | t         | ******** |
 mir          |      101 | f           | f        | f        | f         | ******** |

followed by

  datname  | datdba | encoding | datistemplate | datallowconn | datlastsysoid | datvacuumxid | datfrozenxid | datpath
-----------+--------+----------+---------------+--------------+---------------+--------------+--------------+---------

 template1 |      1 |        5 | t             | t            |         16554 |        11258 |   3221236731 |

 template0 |      1 |        5 | t             | f            |         16554 |           49 |           49 |

 Mir       |      1 |        5 | f             | t            |         16554 |        10805 |   3221236278 |
 
note the "usesysid" column for your new user, and the datname for your
new database.  now execute the following line, obviousjly using the
variable you just noted:

update pg_database set datdba=USESYSID_FROM_PG_USER where datname=DATABASENAME

now exit psql: 
\q

save the following lines to a file called set.permissions:

select 'grant all on '||relname||' to Mir;'
from pg_class
where relname not like 'pg%'
order by relname;

-->be sure to replace "Mir" with your username!!!

(e) Apply neccessary changes to config.properties

--> if you folowed the instructions above, you shouldn't have to do
this, but doublecheck!
Please open config.properties and look for the lines that begin with
"Database.". The interesting properties are "Username", "Password", "Host"
and "Name". Change these properties so that they reflect the settings you
used to create the database and the user.

You should make sure that no copy of config.properties (neither in mir nor
in Mir/src nor in Mir/WEB-INF/classes nor in the directory tree you compiled
Mir from) is world-readable. Else you wouldn't have to install a password,
anyway.

(f) Setup PostgreSQL so that all connections have to pass a password

In /etc/postgresql/pg_hba.conf you should make sure that nobody can
use the database without a password, by inserting these lines:

local        all                                           password
host         all         127.0.0.1     255.0.0.0           password
host         all         0.0.0.0       0.0.0.0             reject

make sure you comment out all other permissions lines in this file!

This means: All local connections (i.e. psql without "-h hostname" option)
have to authenticate themselves with a password. All connections from
localhost (127.0.0.1) have to supply a password, too. All other connections
are rejected. This line doen't have to be there if you have a properly
configured firewall but even if you do have one, it adds to the security in
case an attacker penetrates the firewall by some hack.

If you can't access PostgreSQL after this for any reason, try and change
"password" in /etc/postgresql/pg_hba.conf to "trust". This should disable
any authentication method and make the database accessible again. Please use 
this setting only temporarily because anybody who can access the PostgreSQL
server could take over the database completely this way. After you fixed
your password setting, switch the setting back to "password".
You may want to change your PostgreSQL password from time to time to make
database takeover harder. Rememer: Security is a process.

----------------------------------------------------------------

AT THIS POINT YOU SHOULD HAVE A WORKING INSTAATION OF MIR!  CHECK TO
MAKE SURE -- DON'T BOTHER WITH THE REST TILL YOIU'VE MADE MIR WORK ONCE!


Goodies:

(3) Add the dupe prevention trigger to the database:
        cd mir/dbscripts/dupetrigger
        
        There, read INSTALL and follow the instructions.
        

(4) Tweak mime-type extensions mappings in etc/web.xml file.

*** Note the defaults should be o.k for most installations ***

Add or remove any mime types you wish to support. This is used to figure
out the mime-type when (broken browsers?) browsers don't send the mime-type
in the content-type header field when uploading a media file. Note add the
moment you still have to add these to the media_type SQL table as well which
maps the mime-types to the correct mediaHandler class. See the comments in
the MirMedia class in javadoc for more details.

11. restart tomcat 

13. configure apache

edit http.conf:
* set the document root to the same directory as in the mir config file
* enable shtml includes:
  - add LoadModule includes_module /usr/lib/apache/1.3/mod_include.so
  - make sure your directory contains "Options Includes"
* Determine if you need to modify any apache mime-mappings
  - The web-server host must recognize the .m3u, .pls and other file extensions
    and send the proper "audio/x-mpegurl" and "audio/x-scpls" mime-types
    respectively.  If the web server is apache, it's easy, just
    add:
    
    audio/x-mpegurl                 m3u
    audio/x-scpl                    pls
    
    to the file pointed to by the "TypesConfig" command in your apache config
    file. Or add and equivalent AddType command to your httpd.conf.  Of course
    this assumes that the mod_mime is loaded.

that's it :)

now the admin-application is accesable via:  

        http://host/Mir/servlet/Mir 

and the openposting-servlet via  
        
        http://host/Mir/servlet/OpenMir

standard login is admin/indymedia. See the webdb_users SQL table to change/add
users or passwords.


SEARCHING

The Mir code offers no internal search facilities, rather, the design
expects the use of an external program to crawl and index the static
site.  One (recommended) tool for doing this is htdig
(http://htdig.org), which generates static databases of the site
content and then accesses those databases through a very fast CGI
program written in C.  In the scripts directory, a perl CGI script 
which wraps calls to htsearch is provided (scripts/search.pl) which
will allow searching based off of media type.  (This is possible
because the standard templates will include META keywords like
hasAudio, hasVideo, etc.)  

UPGRADING

see the UPGRADING.mir file.

TROUBLESHOOTING

You can give these a try if anything goes wrong:

+ Restart Tomcat. Especially after compiling the sources Tomcat has to be
  restarted.

+ Check file permissions and ownership. Try and run perms.sh.

----------------------------------------------------------------

$Date: 2002/10/25 20:22:10 $ - the Mir coders