README.deploy

= PUPPI DEPLOY INSTRUCTIONS
Documentation and examples related to the puppi actions: deploy, rollback and init

== SYNOPSIS (COMMAND LINE)
Shell command to launch a deploy:
        puppi deploy <project_name>
        puppi deploy <project_name> [-f] [-i] [-t] [-d yes|full] [-r yes|no|fail] [-p "parameter=value parameter2=value2"]

Shell command to launch a rollback:
        puppi rollback <project_name> 

Shell command to launch the first deploy:
        puppi init <project_name>


== EXAMPLES (cli)
puppi deploy myapp : Deploys myapp with the standard logic/parameters defined in Puppet
puppi deploy myapp -f : Deploys myapp and doesn't stop in case of Critical errors
puppi deploy myapp -i : Deploys myapp in interactive mode. Confirmation is asked for each step
puppi deploy myapp -t : Test mode. Just show the commands that would be executed
puppi deploy myapp -d full : Deploys myapp with full debugging output
puppi deploy myapp -i -o "version=1.1 source_url=http://dev.example42.com/code/my_app/" : Deploys
  myapp in interactive mode and sets some custom options that override the standard Puppet params.
  Note that these parameters change according to the script you use (and the scripts must honour
  this override in order to make this option work).

puppi init myapp : Make the first deploy of "myapp". Can be optional and may require a subsequent 
                   puppi deploy myapp

puppi rollback myapp : Rollbacks myapp to a previous archived state. User is asked to choose which
                       deploy to override. Note that by default you have 5 backups to rollback from.
                       Older backups are automatically removed.
puppi rollback myapp latest : Automatically rollbacks to the latest saved state (unattended).

== EXAMPLES (puppet)
Here follow some sample defines you can use out of the box in your manifests once you include puppi.
Get a war from $source and deploy it in $deploy_root:
        puppi::project::war { "myapp":
            source           => "http://repo.example42.com/deploy/prod/myapp.war",
            deploy_root      => "/store/tomcat/myapp/webapps",
        }

Get a tarball from $source, unpack it in $deploy_root, restart service called apache and send a mail
to $report_mail (you can have a comma separated list of destination addresses):
        puppi::project::tar { "mysite":
            source           => "rsync://repo.example42.com/deploy/prod/release.tgz",
            init_script      => "apache",
            deploy_root      => "/var/www/html/",
            report_email     => "sysadmins@example42.com",
            enable           => "true", # Redundant
        }

Get a tarfile with a .sql query file and apply to to you local Mysql with the credentials provided:
        puppi::project::mysql { "myweb_sql":
            source          => "http://repo.example42.com/deploy/prod/database.sql.gz",
            mysql_user      => "root",
            mysql_password  => "MyVerySecretPassword",
            mysql_host      => "localhost",
            mysql_database  => "myweb",
            report_email    => "sysadmins@example42.com,dbadmins@example42.com",
            enable          => "true",
        }

Get a list of files from $source, retrieve the actual files from $source_baseurl and place them
in $deploy_root
        puppi::project::files { "gfxupdates":
            source           => "http://deploy.example42.com/prod/website2/list.txt",
            source_baseurl   => "http://design.example42.com/website2/gfx/",
            deploy_root      => "/var/www/html/gfx",
            report_email     => "sysadmins@example42.com,designers@example42.com",
            enable           => "true",
        }

Deploy from a Nexus repository (retrieve maven-metadata.xml from dir specified in $source), get the war 
(version is achieved from the "release" tag in the xml) and deploy it in $deploy_root and then restart tomcat.
        puppi::project::maven { "supersite":
            source           => "http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/",
            deploy_root      => "/usr/local/tomcat/supersite/webapps",
            init_script      => "tomcat",
            report_email     => "sysadmins@example42.com",
            enable           => "true",
        }

Get the maven-metadata.xml from a Nexus repository and deploy:
- The release war in $deploy_root
- A configurations tarball tagged with the Maven qualifier $config_suffix in $config_root
- A static files tarball tagged with the Maven qualifier $document_suffix in $document_root
        puppi::project::maven { "supersite":
            source           => "http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/",
            deploy_root      => "/usr/local/tomcat/supersite/webapps",
            config_suffix    => "cfg",
            config_root      => "/srv/htdocs/supersite",
            document_suffix  => "css",
            document_root    => "/srv/htdocs/supersite",
            init_script      => "tomcat",
            report_email     => "sysadmins@example42.com",
            enable           => "true",
        }

The same deploy Nexus repository with some more options:
- A Source dir to be used to deploy static files when issuing "puppi init supersite"
- A block from a loadbalancer IP (managing different sites addresess)
- Some more elaborate rsync exclusion rules
- A backup retention of 3 archives (instead of the default 5)
        puppi::project::maven { "supersite":
            source           => "http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/",
            deploy_root      => "/usr/local/tomcat/supersite/webapps",
            config_suffix    => "cfg",
            config_root      => "/srv/htdocs/supersite",
            document_suffix  => "css",
            document_root    => "/srv/htdocs/supersite",
            document_init_source => "rsync://backup.example42.com/initdir/supersite/",
            firewall_src_ip  => $site ? {
                dr      => "192.168.101.1/30",
                main    => "192.168.1.1/30",
            },
            backup_rsync_options => "--exclude .snapshot --exclude /doc_root/autojs/*** --exclude /doc_root/autocss/*** --exclude /doc_root/xsl",
            backup_retention => "3",
            init_script      => "tomcat",
            report_email     => "sysadmins@example42.com",
            enable           => "true",
        }

An elaborated war deploy: get from $source, execute /usr/local/bin/mychecks.sh as root before the deploy
(or better with sequence number 39), in /data/tomcat/myapp/webapps as user pippo, stop and start
tomcat-myapp but also monit and puppet and backup passing  $backup_rsync_options to rsync:
        puppi::project::war { "myapp":
            source                  => "http://repo.example42.com/deploy/prod/myapp.war",
            deploy_root             => "/store/tomcat/myapp/webapps",
            predeploy_customcommand => "/usr/local/bin/mychecks.sh",
            predeploy_priority      => "39",
            predeploy_user          => "root",
            backup_rsync_options    => "--exclude logs/",
            user                    => "pippo",
            init_script             => "tomcat-myapp",
            deploy_root             => "/data/tomcat/myapp/webapps",
            report_email            => "sysadmins@example42.com",
            disable_services        => "monit puppet",
        }

An example of usage of the generic builder define to deploy a zip file, with an example custom
post deploy command executed as root (as all puppi commands, if not specified otherwise)
        puppi::project::builder { "cms":
            source                   => "http://repo.example42.com/deploy/cms/cms.zip",
            source_type              => "zip",
            user                     => "root",
            deploy_root              => "/var/www",
            postdeploy_customcommand => "chown -R www-data /var/www/files",
            postdeploy_user          => "root",
            postdeploy_priority      => "41",
            report_email             => "sysadmins@example42.com",
            enable                   => "true",
        }

These are just examples, possibilities are many, refer to the docs in puppi/manifests/project/*.pp
to have a full list of the available options.


== BASIC PUPPI DEFINES
The basic defines related to a deployment procedure are:
puppi::project  - Creates the main project structure. One or more different deployment projects
                  can exist on a node.
puppi::initialize - Creates a single command to be placed in the init sequence.
                  It's not required for every project.
puppi::deploy   - Creates a single command to be placed in the deploy sequence.
                  More than one is generally needed for each project.
puppi::rollback - Creates a single command to be placed in the rollback sequence.
                  More than one is generally needed for each project.

These defines have generally a standard structure and similar arguments.
Every one is reversable (enable => false) but you can wipe out the whole /etc/puppi directory
to have it rebuilt from scratch. Here is an example for a single deploy command:
        puppi::deploy { "Retrieve files":       # The $name of the define is used in the file name
            command  => "get_curl.sh",          # The name of the general-use script to use
            argument => "file:///storage/file", # The argument(s) passed to the above script
            priority => "10",                   # Lower priority scripts are executed first
            user     => "root",                 # As what user we run the script
            project  => "my-web.app",           # The name of the project to use
        }
This define creates a file named /etc/puppi/projects/${project}/deploy/${priority}-${name}
Its content is, simply:
su - ${user} -c "export project=${project} && /etc/puppi/scripts/${command} ${arguments}"


== DEPLOY PROCEDURES DEFINES
The puppi module provides some examples of deploy workflows they are in the puppi/manifests/project
directory and contain simple to use defines that contain one puppi::project and several puppi::deploy and
puppi::rollback defines to design a specific workflow using the builtin commands present in
puppi/files/scripts.
Note that if you need to design your own deploy procedure you have different options:
- Verify if you can reuse the existing ones, using optional arguments as pre/postdeploy_commands
- Use the existing ones as a base to make you own custom defines, reusing parts of their logic 
  and the builtin commands (puppi/files/scripts/*) they use
- Write your own commands (in whatever language) and integrate them in your own procedure.

Here follow the main and most reusable deploy workflows defines available in puppi/manifests/project/.
The have generally a set of common arguments that make you manage if to stop and restart specific 
services, if you want to isolate your server from a loadbalancer during the deploy procedure, what to backup,
how many backup copies to mantain, if to send a report mail to specific addresses and if you need
to run custom commands during the standard procedure.
For all of the you have to specify a source from where to get the source files (http/ftp/rsync/file..) 
a directory where to place them and the user that has to own the deploy files.
Full documentation is available in the relevant .pp files
puppi::project::tar - Use this to retrieve and deploy a simple tar file
puppi::project::war - Use this to retrieve and deploy a war
puppi::project::files - Use this to deploy one or more files based on a provided list
puppi::project::dir - Use this to syncronize a remote directory to a local deploy one
puppi::project::mysql - Use this to retrive and apply a .sql file for mysql schema updates 
puppi::project::maven - Use this to deploy war and tar files generated via Maven released on Nexus or similar
                        A good source of Open Source Java artifacts is http://www.artifact-repository.org/
puppi::project::builder - This is a general purpose define that incorporates most the of cases provided
                          by the above procedures


== BUILTIN COMMANDS
The puppi/files/scripts directory in the module contains some general usage "native" bash scripts
that can be used in custom deployments. They are generally made to work together according to a
specific logic, which is at the base of the deploy procedures defines in puppi/manifests/project/
but you're free to write your own scripts, in whatever language, according to your needs.

The default scripts are engineered to follow these steps for a deployment:
- Remote files are downloaded in /tmp/puppi/$project/store or directly in the predeploy
  directory: /tmp/puppi/$project/deploy
- If  necessary the downloaded files are expanded in one or more predeploy directories
  (default:/tmp/puppi/$project/deploy).
- Runtime configuration entries might be saved in /tmp/puppi/$project/config
- Files are eventually backed from the deploy directory (Apache' docroot, Tomcat webapps or whatever)
  to the archive directory (/var/lib/puppi/archive/$project). Older backups are deleted (by default
  there's a retention of 5 backups).
- Files are copied from the predeploy directory to the deploy dir.
- Relevant services are eventually stopped and started

The most used common scripts are (they might have different arguments, some of them are quite simple):
get_file.sh - Retrieves a file via ssh/http/rsync/svn and places it in a temp dir (store or predeploy)
deploy.sh - Copies the files in the predeploy dir to deploy dir
archive.sh - Backups and restores files in deploy dir
service.sh - Stops or starts one or more services
wait.sh - Waits for the presence or absence of a file, for the presence of a string in a file or a
          defined number or seconds.
get_metadata.sh - Extracts metadata from various sources in order to provide info to other scripts
                - These info are save in the Runtime configuration file (/tmp/puppi/$project/config)
report_mail.sh - Sends a mail with the report of the operations done

General functions, used by both the puppi command and these scripts, are in puppi/files/scripts/functions