README-GLUE2

This is a description of the support for GLUE 2.0 in the glue-service-provider
package; see the README file for GLUE 1 and general information.


Overview of the GLUE 2 providers
================================

The provider glite-info-service-glue2-beta is a direct drop-in substitute for
the GLUE 1 provider glite-info-service, which reads the same configuration
file and writes essentially the same information in the GLUE 2 format. This
provider was intended only for initial testing and will not be developed
further; it will be removed from the distribution at a later date.

The beta provider above was originally called glite-info-service-glue2, and
some services may still call it under that name. In the current distribution,
glite-info-service-glue2 is just a wrapper script which calls the current
GLUE 2 provider glite-info-glue2-simple; this should be backward-compatible
but supports more GLUE 2 features.

glite-info-glue2-service is a script which creates a GLUE 2 Service
object using information passed via its arguments. This is more
of a text processor than a true info provider and is intended to be
called from a higher-level script, although it could be called
directly for very simple cases.

glite-info-glue2-endpoint is a provider to generate GLUE 2 Endpoint
and AccessPolicy objects. This uses a configuration file which is similar
to, and backward-compatible with, the file used for the GLUE 1 provider,
but supports extra parameters to enable the new GLUE 2 attributes
to be published. This allows essentially all GLUE 2 features to
be used.

glite-info-glue2-simple is a wrapper script which calls the service and
endpoint scripts mentioned above. It only supports simple services
with a single Endpoint, but this covers the majority of the current
uses for the GLUE 1 provider. It is backward-compatible with the
GLUE 1 provider in the sense that it can be called with the same
arguments and the same configuration file, but is a lot more flexible.
In particular it allows a second configuration file to specify attributes
in the Service object.

glite-info-glue2-multi is similar to glite-info-glue2-simple but allows
for Services with multiple Endpoints, by passing a configuration file
per Endpoint.

glite-info-glue2-voms is a dedicated provider for VOMS servers. It generates
an Endpoint per VO for VOMS and optionally voms-admin, using a single
configuration file for each.

No providers currently exist for the Share, Manager, Resource or Activity
objects as these are not normally needed for generic services, but it would be
relatively easy to adapt glite-info-glue2-service to generate them if
necessary.

Extended configuration files for the various supported services are provided
as necessary; GLUE 1 configuration files can also be used but do not cover
the full range of GLUE 2 attributes.

These providers could potentially be used as a skeleton for Computing and
Storage services, but these would need substantial further development
which is beyond the scope of a generic provider.

The rpm also contains test configuration files in the etc directory,
which can be used to verify the functionality of the providers and can
also be used as an example to develop configurations for new services. To
perform such a test, make the etc directory the cwd, add the src
directory to the PATH, and run the following commands:

glite-info-glue2-simple glite-info-glue2-test.conf.template,glite-info-glue2-service-test.conf.template

glite-info-glue2-multi glite-info-glue2-test.conf.template,glite-info-glue2-test.conf.template glite-info-glue2-service-test2.conf.template

The ldif output for the current and previous versions of each information
provider is also available (in files matching etc/*.ldif*) for comparison,
although note that some elements of the attributes, e.g. hostnames and
timestamps, are expected to vary.


Detailed descriptions
=====================

glite-info-glue2-simple
-----------------------

This is the main GLUE 2 information provider, equivalent to glite-info-service
for GLUE 1. It calls glite-info-glue2-service and glite-info-glue2-endpoint
to generate the actual objects. It should normally called from a small wrapper
script in the BDII provider directory. The call signature is:

glite-info-glue2-simple <endpoint-config-file>[,<service-config-file>] [<site-ID> [<service-ID> [<endpoint-ID>]]]

endpoint-config-file: the configuration file used in glite-info-glue2-endpoint
and described in detail in the description of that script.

service-config-file: an optional configuration file in a similar format to
the endpoint configuration, to allow various GLUE2Service attributes to be
published. This is decribed further below.

site-ID: The unique name for the hosting site, i.e. the GLUE2DomainID. This
can be omitted only if it is specified in the service-config-file.

service-ID: The unique ID for the Service (GLUE2ServiceID). If not specified
as an argument (the usual case) it is constructed as described below. 

endpoint-ID: The unique ID for the Endpoint (GLUE2EndpointID). If omitted this
is generated as explained in the description of glite-info-glue2-endpoint.

As for the Endpoint provider, the Service configuration file is a list of
key=value pairs, with the values being executed as commands with the text
written to stdout used as the attribute value. All keys are optional,
and a null string is equivalent to an omitted key. Unknown keys are ignored,
as are lines not containing an = and lines starting with a #, and leading
and trailing white space is also ignored. If a key occurs more than once
the last definition takes precedence; multi-valued attributes are supported
via a multi-line output string with individual attribute values seperated
by newlines.

The Service configuration keys are as follows:

get_site_id: A command which returns the unique name for the hosting site,
i.e. the GLUE2DomainID. An explicit argument takes precedence over the
configuration file. There is no default; if the DomainID is undefined
the provider exits with an error message without producing any LDIF. Example:

get_site_id = echo UKI-SOUTHGRID-RALPP

get_service_id: A command which returns the unique name for the Service,
i.e. the GLUE2ServiceID. An explicit argument takes precedence over the
configuration file. If no explicit ID is specified (the usual case) an
ID is constructed of the form "<hostname>_<typeguess>_<checksum>", where
hostname is the fully-qualified hostname of the machine on which the
provider is running and checksum is the cksum of the Endpoint configuration
file, which in most cases should suffice to ensure a globally unique ID.
typeguess is added to aid human-readability, e.g. in an LDAP browser,
and is taken from the name of the Endpoint configuration file assuming the
standard naming style of glite-info-*-<type>.conf, or "unknown" if the name
does not match that format. Example:

get_service_id = echo myservice.example.com_myservice_1

get_capability: A command to return a comma-separated list of Capability
attributes to be added to the ones included in the Endpoint, if the
Service has Capabilities which are not directly provided by the Endpoint.
Example:

get_capability = echo an.additional.capability

get_type: A command to return the ServiceType attribute. If this is omitted
the ServiceType will be the same as the EndpointInterfaceName. Example:

get_type = echo org.glite.Argus

get_qualitylevel: The ServiceQualityLevel attribute is normally the same as
the EndpointQualityLevel, but this allows the value to be explicitly
specified if necessary. Example:

get_qualitylevel = echo testing

get_complexity: The complexity is normally 1:0:0, but this allows the value
to be explicitly specified if necessary (see the description of
glite-info-glue2-service for an explanation of the format). Example:

get_complexity = echo 1:1:0

get_statusinfo: This can return any number of strings separated by newlines,
which will be printed verbatim as StatusInfo attributes. These are defined
to be URLs but the format is not enforced. Example:

get_statusinfo = echo http://myservice.example.com/monitor

get_otherinfo: This can return any number of strings separated by newlines,
which will be printed verbatim as OtherInfo attributes. Example:

get_otherinfo = echo  -e "Some information\nSome more information"

glite-info-glue2-multi
----------------------

This is largely the same as glite-info-glue2-simple, but deals with the
case of a Service with multiple Endpoints, each of which has its own
configuration file. It can in fact also deal with the "simple" case but is
not directly compatible with the GLUE 1 provider. It is called like:

glite-info-glue2-multi <endpoint-config-file>[,<endpoint-config-file>[,...]] [<site-ID>] [<service-config-file>] [<service-ID> [<endpoint-ID>[ <endpoint-ID>[ ...]]]]

where the arguments have the same meanings as for the -simple provider
described above. If the second or third argument can be opened as a file
name it is assumed to be a configuration file, otherwise it is taken to be
the site name or Service ID respectively. If any Endpoint IDs are specified
the Service ID must also be specified. They are associated with the 
Endpoints in the same order as the configuration files, but need not match
in number.

The ServiceType for multi-Endpoint Services should normally be specified
in the Service configuration file, but if not it defaults to the
InterfaceName of the first Endpoint. The Service Capability, Complexity and
QualityLevel attributes are derived from the Endpoints as defined in the
schema unless modified by the Service configuration.

glite-info-glue2-voms
---------------------

VOMS servers have a dedicated information provider as the publication is more
complex than the generic providers can manage. It publishes one Endpoint
per supported VO for voms itself and optionally for voms-admin. The list of
supported VOs can be determined from the directory structure on the server
and hence do not need to be configured explicitly. The provider is called with:

glite-info-glue2-voms <voms-config-file>[,<voms-admin-config-file>] <site-ID> [<service-ID>]

The configuration files are in the usual format as described under
glite-info-glue2-endpoint, and should use the environment variable
GLITE_INFO_SERVICE_VO to determine the VO name - the Endpoint provider will
be called once per VO with the variable set appropriately, using the voms
configuration file and then the voms-admin file if supplied.

There is currently no provision for a Service configuration file. By default
the Service ID is constructed as described under glite-info-glue2-simple
but with the "typeguess" set to VOMS. The Endpoint IDs are constructed
including the VO name to ensure uniqueness. The Service Capability,
Complexity and QualityLevel attributes are derived from the Endpoints as
defined in the schema, and the Type is set to "VOMS".

glite-info-glue2-service
------------------------

This is a simple script which massages its arguments into a valid GLUE2Service
object, which it prints to stdout. All the information is passed in the
arguments; this is not especially elegant, but the Service object is fairly
simple and this covers everything which is likely to be needed. Several of
the arguments are comma-separated lists, which implies that the list elements
should not contain commas, but in practice that should not be a significant
restriction. There is no support for Extension objects, but they can easily
be appended after this script has been called. The script is called like:

glite-info-glue2-service <site-ID> <service-ID> <service-type> <endpoint-info> [<other-info> [<other-info>] ...]

site-ID: The unique name for the hosting site, i.e. the GLUE2DomainID.

service-ID: The unique ID for this Service (GLUE2ServiceID). This is optionally
followed by a comma-separated list of IDs of Services to which this Service
is related (GLUE2ServiceServiceForeignKey).

service-type: The GLUE2ServiceType, optionally followed by a comma-separated
list of GLUE2ServiceCapability attributes.

endpoint-info: a string summarising the Endpoints contained in the Service.
This string in turn has the format:

<quality-level>[,<complexity>[,<status-info>[,<status-info>[,...]]]]

quality-level: The highest QualityLevel among the Endpoints
(GLUE2ServiceQualityLevel).

complexity: A string of the form <e>[:<s>[:<r>]] (default e=1, s=0, r=0) where
e, s and r are the counts of the numbers of Endpoint types, Shares and
Resources as defined for the GLUE2ServiceComplexity attribute. 

status-info: Any number of strings which will be printed verbatim as
GLUE2ServiceStatusInfo attributes.

other-info: Any number of strings may be passed as extra arguments, which are
simply printed verbatim as GLUE2EntityOtherInfo attributes.

The Validity attribute is currently hardwired to 3600, i.e. 1 hour, which
seems a reasonable default for a dynamic object. (If this script is used
to generate a static object at configuration time the Validity should be
much longer, e.g. 31536000 = 1 year.) The CreationTime is set
as the provider is run, and the Name is of the form <site>-<type>, where
<type> is the final component in the ServiceType, assuming a dot-separated
format.

For diagnostic purposes the information provider name and version and the
name of the host on which it ran are printed as OtherInfo items, e.g.:

GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.1
GLUE2EntityOtherInfo: InfoProviderHost=host.name

To provide a basic level of support for Computing and Storage Services,
if the Type is org.glite.ce or org.glite.se the provider adds

objectClass: GLUE2ComputingService

or

objectClass: GLUE2StorageService

respectively.

glite-info-glue2-endpoint
-------------------------

This is the principal information provider, broadly equivalent to the GLUE 1
service provider and using a very similar (and backward-compatible)
configuration file; see the README file for general information. It generates
a GLUE2Endpoint object and an attached GLUE2AccessPolicy object. Usually it
will be called from a wrapper script which also generates the parent
GLUE2Service object (note that the BDII requires parent objects to
exist for the LDAP tree to be valid). It is called like:

glite-info-glue2-endpoint <config-file> <service-ID> [<endpoint-ID>]

where the config-file format is described below, service-ID is the unique
ID of the parent Service object and endpoint-ID is an optional ID
of the Endpoint object. If the latter is not supplied an ID of the form
"<service-ID>_<endpoint-type>_<checksum>" is used, where endpoint-type is the
GLUE2EndpointInterfaceName (equivalent to GlueServiceType in GLUE 1) and
checksum is the cksum of the config-file.

Most schema attributes are supported by the provider, but some are not
configurable at present; these attributes may be made configurable in
future if a use-case is identified:

The Validity attributes are currently hardwired to 3600, i.e. 1 hour, which
seems a reasonable default for a dynamic object. The CreationTime is set as
the provider is run. The EndpointName is of the form "<type> endpoint for
Service <service-ID>", where <type> is the final component in the
InterfaceName, assuming a dot-separated format. The TrustedCA attribute is
hardwired to "IGTF". The IssuerCA is extracted from the host certificate
(if any) using the openssl tool - the certificate is assumed to be in the
standard location (/etc/grid-security/hostcert.pem). Downtime publication
is not currently supported as this is managed via the GOC DB in EGI; the
provider publishes only the hardwired attribute:

GLUE2EndpointDownTimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/

The PolicyScheme for the AccessPolicy object is hardwired to
"org.glite.standard", and the Name of the object is "Access control rules
for Endpoint <endpoint-ID>". There is no support for configurable OtherInfo
attributes or Extension objects related to the AccessPolicy.

For diagnostic purposes the information provider name and version and the
name of the host on which it ran are printed as OtherInfo items, e.g.:

GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=host.name

If a file called /etc/emi-version is present on the node it is assumed to
contain the overall EMI middleware version string, and additional
OtherInfo lines are printed like:

GLUE2EntityOtherInfo: MiddlewareName=EMI
GLUE2EntityOtherInfo: MiddlewareVersion=1.8.0-1

To provide a basic level of support for Computing and Storage Services,
if the Type is org.glite.ce.* or SRM the provider adds

objectClass: GLUE2ComputingEndpoint

or

objectClass: GLUE2StorageEndpoint

respectively.

As for the GLUE 1 provider the configuration file is a list of key=value pairs,
with most values being executed as commands with the text written to stdout
used as the attribute value after performing some basic sanity checks.
A null string will usually result in no attribute being printed. All the
GLUE 1 values are supported so a GLUE 1 configuration file can be used,
but there are several new attributes for GLUE 2 so a new file will usually
be desirable. One change is that the GLUE 1 provider requires all keys
to be supplied even if they produce a null value, wherereas the GLUE 2
provider allows some attributes to be omitted if a sensible default exists.

Unknown keys are ignored, as are lines not containing an = and lines
starting with a #, and leading and trailing white space is also ignored.
If a key occurs more than once the last definition takes precedence;
multi-valued attributes are supported via a multi-line output string
with individual attribute values seperated by newlines. If the
configuration is invalid no objects are generated; otherwise the objects
will always be syntactically correct even if some attribute values are
unobtainable. String attributes have leading and trailing whitespace stripped
and are limited to 240 characters to prevent excessive output; truncated
string have "...4444" appended (4444 being a "well-known" error indicator
in GLUE 1 usage).

The configuration keys are as follows:

init (default null): This allows a setup command which will be executed before
anything else. Environment variables can be set for subsequent commands by
writing lines of the form X=Y to stdout. A non-zero return code from the
init command will abort the provider without producing any output, e.g. if
the service is not installed on the node. init scripts are typically called
glite-info-service-<service> where <service> is the name of the service.
Example:

init = glite-info-service-wmproxy

service_type (mandatory): This is a string rather than a command, and should
be taken from the official list of types (= InterfaceNames) at:
https://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/ServiceTypes.
This is currently the same as the GLUE 1 ServiceType: see
http://glueschema.forge.cnaf.infn.it/V12/ServiceType. Example:

service_type = org.glite.wms.WMProxy

get_otherinfo (default null): This can return any number of strings separated
by newlines, which will be printed verbatim as OtherInfo attributes. Example:

get_otherinfo = echo  -e "Some information\nSome more information"

get_endpoint (mandatory): A command to return the endpoint URL. This is not
validated other than by stripping any embedded whitespace. A null string
is replaced with "http://unknown.invalid:4444/". Example:

get_endpoint = echo https://$WMPROXY_HOST:$WMPROXY_PORT/glite_wms_wmproxy_server

get_capabilities (default null): A command which returns a list of
newline-separated strings to be printed as Capability attributes. These
are simply printed verbatim, but should follow the values defined in the
schema document: http://glue20.web.cern.ch/glue20/#b5. Example:

get_capabilities = echo -e "executionmanagement.jobdescription\nexecutionmanagement.jobmanager"

get_technology (default null): A command which returns a string to be printed
as the Technology attribute. This is printed verbatim apart from
removing any embedded whitespace, but should follow the values defined in
the schema document: http://glue20.web.cern.ch/glue20/#b15. If the value
is null but there are any WSDL attributes (see below) the value "webservice"
is printed. Example:

get_technology = echo webservice

get_version (mandatory): A command to return the service interface version. The
canonical form is x.y.z, but unlike GLUE 1 there is no required format
although any embedded whitespace is removed. GLUE 2 also allows the attribute
to be multi-valued if the interface supports multiple protocol versions.
The attribute is technically optional but the provider requires something
to be specified -  "echo 1" can be used as a minimal default. Example:

get_version = echo 1.0.0

get_interfaceextensions (default null): A command which returns a list of
newline-separated strings to be printed as InterfaceExtension attributes. These
are supposed to be URIs but no validation is performed.

get_interfaceextensions = echo http://www.nordugrid.org/schemas/a-rex

get_WSDL (default null): A command which returns a list of newline-separated
strings to be printed as WSDL attributes. These are supposed to be URLs
pointing to the WSDL description of the interface (if a web service) but
no validation is performed other than removing any embedded whitespace.
Example:

get_WSDL = echo https://$CREAM_HOST:$CREAM_PORT/ce-monitor/services/CEMonitor?wsdl 

WSDL_URL (default "nohttp://nothing.to.see.here/"): This is a fixed string
giving a WSDL URL explicitly, for compatibility with the GLUE 1
configuration. Any string not starting with "http" is treated as null. If
both keys are present the values are merged.

get_profiles (default null):  A command which returns a list of
newline-separated strings to be printed as SupportedProfile attributes. These
are supposed to be URIs but no validation is performed. Example:

get_profiles = echo http://www.ws-i.org/Profiles/BasicProfile-1.0.html

get_semantics (default null):: A command which returns a list of
newline-separated strings to be printed as Semantics attributes. These are
supposed to be URLs pointing to a human-readable description of the interface
(e.g. a manual) but no validation is performed other than removing any
embedded whitespace. Example:

get_semantics = echo http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guide.pdf

semantics_URL (default "nohttp://nothing.to.see.here/"): This is a fixed string
giving a Semantics URL explicitly, for compatibility with the GLUE 1
configuration. Any string not starting with "http" is treated as null. If
both keys are present the values are merged.

get_implementor (default null, may become mandatory in future): A command to
return a free-format string printed as the Implementor attribute. Example:

get_implementor = echo gLite

get_implementationname (default null, may become mandatory in future): A
command to return a free-format string printed as the ImplementationName
attribute. Example:

get_implementationname = echo WMS

get_implementionversion (default null, may become mandatory in future): A
command to return a free-format string printed as the ImplementationVersion
attribute. Note that unlike the InterfaceVersion this attribute is
single-valued. Example:

get_implementationversion = rpm -q glite-wms-wmproxy --queryformat '%{version}\n';

get_qualitylevel (default 4 => "production", may become mandatory in future): A
command to return an integer from 1 to 4 representing the QualityLevel
(mapped to the closed enumeration "development", "testing", "pre-production",
"production"). Any other value will be printed verbatim preceded by
"UNDEFINEDVALUE: Level ". Example:

get_qualitylevel = echo 4

get_status (mandatory): A command to check the state of the service.
The return code is used to determine the standard GLUE HealthState values:
0 = ok, 1 = critical, 2 = warning, 3 = unknown, other = Other. Any text
written to stdout will appear as the HealthStateInfo attribute, truncated
before the first control code if any and with any newlines replaced by
spaces. Unlike the corresponding GLUE 1 attribute the string may be null
in which case the attribute is omitted, but this is not recommended.
Typically the status check will use the standard init.d status method
on one or more underlying services, but more elaborate checks are
possible as long as they are fast enough (ideally less than a second).

There are two helper scripts provided for use with the get_status command:

glite-info-service-status is a wrapper around the standard "/sbin/service
<service> status" command, where <service> is a service name passed as an
argument (any further arguments are appended to the command). The script
translates the return codes to suitable values assuming that they follow
the standard init.d conventions, sets LANG=C to attempt to avoid localised
strings, and strips pids from the output if they follow the standard format
(a list of numbers between parentheses preceded by "pid"). 

glite-info-service-test provides a simple way to put a service into a warning
state. If an environment variable GLITE_INFO_SERVICE_STATUS_<SERVICE>
is defined, where <SERVICE> is passed an argument to the script, it prints
the value of the variable to stdout and sets the return code to 2.

A typical get_status command is therefore something like:

get_status = glite-info-service-test BDII && glite-info-service-status bdii

get_servingstate (default 4 => "production", may become mandatory in future): A
command to return an integer from 1 to 4 representing the ServingState
(mapped to the closed enumeration "closed", "draining", "queueing",
"production"). Any other value will be printed verbatim preceded by
"UNDEFINEDVALUE: State ". Example:

get_servingstate = echo 4

get_starttime (mandatory): A command to return the service start time as a Unix
timestamp, e.g. from a stat on a pid file. A null string (e.g. resulting from
a stat on a non-existent file) will result in the attribute being omitted,
indicating that the service is not running at all; a value of 0 or a
non-numeric string implies the Unix epoch (1st January 1970) which can be
treated as an error indicator. Example:

get_starttime = perl -e '@st=stat($ENV{WMPROXY_PID_FILE});print "@st[10]\n";'

get_extensions (default null): This is a command which can supply arbitrary
additional information as key/value pairs, written to stdout as one "Key=Value"
line per Extension object. Keys are not allowed to contain whitespace or
"=" characters and may not be null strings, but (unlike GLUE 1) there may
be multiple Extensions with the same Key. A null Value will be replaced with
UNDEFINEDVALUE as the schema does not allow the attribute to be omitted.
Example:

get_extensions = echo -n DN= && openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject | cut -d = -f 2-

get_data (default null): This is a synonym for get_extensions for compatibility
with GLUE 1 configuration files. If both keys are present the values are
merged. Note that in many cases an OtherInfo string may be more convenient
(there is no equivalent attribute in GLUE 1).

get_acbr (default null): A command to return access-control information
(PolicyRule attributes), in the standard GLUE 1 AccessControlBaseRule
format, with one entry per line written to stdout. In the simplest case
this will be the same list as the "owners" with VO names prefixed by "VO:".
The interpretation of this information may be service-specific. Note that
embedded white space is allowed, e.g. if the Rule is a DN. A null string
yields a Rule with the reserved word ALL, meaning that there is no
authorisation control. (If necessary, to explicitly indicate that no access
is allowed use a rule of NONE.) Example:

get_acbr = echo -e "VO:dteam\nVO:ops\nVOMS:/atlas/Role=production"

get_owner (default null): A command to return a list of "owners", typically
the names of VOs which are served by the service. In GLUE 2 these are
conceptually treated as the unique IDs of AdminDomain objects, although
there is no guarantee that such objects will actually exist; hence no
whitespace is allowed. These should be written to stdout, one owner per line.
The list may be null if the service is a generic one usable by anyone.
There is no general requirement for the owner and acbr strings to correspond
although it will often be the case. Example:

get_owner = echo -e "dteam\nops\natlas"