gLite Data Management File Transfer Service (Oracle)

1. Release Description

This release contains the gLite Data Management File Transfer Service for Oracle module v. 1.0.0. The following sections provide additional information about the release content, the module dependencies, the know bugs and issues and a list of bugs closed since the previous release. For information about installing and using the gLite Data Management File Transfer Service, please refer to the gLite Installation and User Guides.

2. Changes in this Release

The name of the module has changed from Local Transfer Service to File Transfer Service since it only contains the File Transfer Service and not the File Placement Service that will be released as a separate module. The main changes in this release are:

From this release, the FTS service runs by default as a secure web service on the https port 8443 and requires the installation of the gLite Security Utilities. In order to open the non-secure http port 8080, the following new parameter must be set to true in the configuration:

<allow.unsecure.port
      description="Enable using the unsecure port 8080. It can be true or false.
      Example: false"
      value="true"/>

The configuration script checks if the FTS database is already installed before trying to load the database schema to prevent destroying an existing database. If the database must be reinstalled from scratch the --remove option of the configuration script glite-file-transfer-service.py must be used first to drop the database

All R-GMA service publishing instances in the configuration template now have default values except the new vo.name parameter that has to be set from ‘changeme’ to a valid VO name for each service instance (one instance per service type per VO)

Bug fixes

3. Release contents

The gLite Data Management File Transfer Service v. 1.0.0 is composed of the following gLite components:

Component name	Version	File
org.glite.deployment.file-transfer-service-oracle	1.0.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/installers/glite-file-transfer-service-oracle_installer.sh ../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-file-transfer-service-oracle-config-1.0.0-1.noarch.rpm apt-get install glite-file-transfer-service-oracle-config
org.glite.deployment.config	1.1.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-config-1.1.1-3.noarch.rpm
org.glite.data.transfer-fts	2.2.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-data-transfer-fts-2.2.0-2.noarch.rpm
org.glite.data.catalog-api-java	2.0.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-data-catalog-api-java-2.0.0-6.noarch.rpm
org.glite.data.common	1.2.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-data-common-1.2.1-1.noarch.rpm
org.glite.data.transfer-common	2.2.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-data-transfer-common-2.2.0-2.noarch.rpm
org.glite.data.transfer-interface	2.9.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/noarch/RPMS/glite-data-transfer-interface-2.9.1-4.noarch.rpm
org.glite.data.transfer-url-copy	1.1.5	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-data-transfer-url-copy-1.1.5-1.i386.rpm
org.glite.data.srm-api-c	2.9.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-data-srm-api-c-2.9.0-1.i386.rpm
org.glite.data.transfer-agents	1.8.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-data-transfer-agents-1.8.1-1.i386.rpm
org.glite.data.config-service	1.1.7	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-data-config-service-1.1.7-1.i386.rpm
org.glite.data.transfer-cli	1.3.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-data-transfer-cli-1.3.1-1.i386.rpm
org.glite.security.util-java	1.0.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-security-util-java-1.0.1-2.noarch.rpm
org.glite.security.trustmanager	1.6.4	../../../../../../../glite-web/egee/packages/R1.1/R20050430/bin/rhel30/i386/RPMS/glite-security-trustmanager-1.6.4-1.noarch.rpm

4. Dependencies

The gLite File Transfer Service module v. 1.0.0 has the following dependencies:

Component name	Version	RPM file name
gLite Security Utilities	1.0.1	../../../../../../../glite-web/egee/packages/R1.1/R20050430/installers/glite-security-utils_installer.sh
gLite R-GMA Service Publisher	4.2.0	../../../../../../../glite-web/egee/packages/R1.1/R20050430/installers/glite-security-utils_installer.sh
gLite C/C++ Essentials	1.1.1	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/glite-essentials-cpp-1.1.1-1_EGEE.i386.rpm
GPT	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/gpt-VDT1.2.2rh9-1.i386.rpm
VDT Globus Essentials	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/vdt_globus_essentials-VDT1.2.2rh9-1.i386.rpm
CGSI_gSOAP_2.6	1.1.8	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/CGSI_gSOAP_2.6-1.1.8-2.slc3.i386.rpm
perl-Expect.pm	1.01	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/perl-Expect.pm-1.01-9.i386.rpm
myproxy	1.14	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/myproxy-1.14-EGEE.i386.rpm
Tomcat	5.0.28	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/tomcat5-5.0.28-6_EGEE.i386.rpm
Java JRE/SDK	1.4.2	http://java.sun.com/j2se/1.4.2/download.html
Oracle InstantClient	10.1.0.3	http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/linuxsoft.html

The Oracle InstantClient libraries and executables are not freely redistributable and are not included with this release. The four RPMS have to be downloaded from the Oracle web site and installed on the system before installing the gLite File Transfer Service module.

5. Known bugs and issues

This release has the following known bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw .

A number of Command Line issues are known: submission does not work properly in secure mode, status details are not printed properly (two values are swapped), some error messages are misleading.
BDII support: BDII is not integrated into the usage of the FTS yet.
VO management: the FTS currently needs to be run per channel per VO. Clearly, it is desirable to run only per channel, supporting many VOs including VO policy management.
Resilience towards failure should be improved
The FTS has never been tested with dCache and DPM. It may just work, but most probably there are open issues.
No removal procedure is provided with this release apart from the removal of the RPMS. Any account, group or other resource created during the module configuration must be manually cleaned.

Known open bugs:

Bug number	Description
#6933	FTS: Should support Channel management
#6935	Install guide mentions RGMA-Server in LTS Section
#7182	LTS deployment module does not set the log file name
#7353	FTS: if XML config file is badly formatted, the glite_config_service tool hangs on start
#7502	FTS: Channel management HALTED state should be added
#7506	FTS: requirement 1.2 should be met: proper retry
#7508	FTS: now needs to implement the protocol option passing as defined in interface
#7509	FTS: update security per Channel
#7516	FTS: log rotation on service
#7517	FTS: rename FilePlacement ot FileTransfer, fps -> fts
#7581	FTS: add support for error categories
#7596	FTS: Commandline tools should be tested fully against service
#7738	FTS: clean up SRM after transfer failure
#7739	FTS: time out on Gridftp if it hangs on network failure
#7740	FTS: glite-url-copy dumps core on some SRM failures
#8333	FTS: service is not able to parse correctly the "grid-mapfile" with the administrators of the service
#8336	FTS: testing methods do not work
#8345	gLite I/O and gLite FTS security models conflict
#8364	FTS: Accept not fully qualified SURLs on the server side
#8365	FTS: Command Lines need service resolution from BDII
#8366	FTS: Transfer Agent for a channel is per-VO
#8367	FTS: Handling of full stage pools not optimal
#8368	FTS: Improve retry and channel halting mechanisms
#8369	FTS: Validation of FTS for DB debployment
#8370	FTS: Application Server deployment
#8371	FTS: Usage of MySQL
#8372	FTS: Security enhancements for the portal
#8373	FTS: Certification with dCache and DPM
#8374	FTS: Usage of SRM Copy
#8375	FTS: Daemon watchdog scripts
#8376	FTS: per transfer logs need better management
#8377	FTS: Trace facility
#8378	FTS: Cleanup of jobs in terminal states in the database
#8379	FTS: MyProxy version is not compatible with the VDT version on LCG-2
#8380	FTS: Interaction with network monitoring
#8381	FTS: Service Level monitoring
#8382	FTS: Split deployment model
#8383	FTS: Knowledge of the service portal needed
#8384	FTS: Not clear if single file to multiple destinations is done well
#8385	FTS: Multi hop means no async transfers
#8386	FTS: No inter-VO scheduling
#8486	FTS: submit commandline doesn't accept myproxy password
#8503	FTS: detailed transfer-status commandline doesn't work correctly
#8504	FTS: failure message for channel-list could be better
#8538	FTS: channel-set -S Active does not work.

Bugs fixed in this or previous releases but not yet tested:

Bug number	Description
#6931	FTS: Should use user proxy for transfer
#6932	FTS: Should support Oracle for its back end database
#7113	FTS: glite-url-copy - if SRM.put fails, the code loops forever
#7318	FTS: Transfer agent sometimes mis-counts the number of active transfers
#7352	FTS: forked glite-url-copy should chdir somewhere it can write
#7354	FTS: glite_config_service waits for all child transfer processes to finish before stopping
#7355	FTS: transfer-url-copy sometimes dies on gridFTP
#7358	FTS: channel allocation fails if the SURL doesn't contain a port number
#7361	FTS: transfer agent should detect a bad Oracle connection, and try to reconnect
#7466	FTS: transfer-url-copy returns 0 for size and bytes written
#7503	FTS: Add TURL:TURL to DB
#7568	FTS: url-copy sometimes core dumps if SURL format is bad
#7594	FTS: should migrate all components to FileTransfer interface
#7595	FTS: Need to test installation of -fts module

6. Bugs closed since last release

This release fixes the following bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw

Bug number	Description
#7205	LTS deployment script says it cannot download the site config file
#8502	FTS: submit commandline doesn;t pass credential correctly.

7. Previous Releases

7.1. Release 0.1.3

7.1.1. Release Description

This release contains the gLite Data Management Local Transfer Service module v. 0.1.3. The following sections provide additional information about the release content, the module dependencies, the know bugs and issues and a list of bugs closed since the previous release. For information about installing and using the gLite Data Management Local Transfer Service, please refer to the gLite Installation and User Guides.

7.1.2. Changes in this Release

Implemented remote site configuration file mechanism
Stork is not used anymore as transfer service, url-copy is used instead
io-client, io-resolve-fireman and io-resolve-fr:
renamed executables from glite_* to glite-*
Do not set group name; catalog server does it.
catalog-interface:
Not generating unused classes into the WSDL.
catalog-service-fr:
Added storageindex port-type
Configuration updates.
catalog-service-fr-mysql:
Added storageindex port-type (catalog-service-storageindex can be still deployed as a standalone service)
transfer-common, transfer-fps, transfer-ds:
Renamed *-schema-stork.sql to *-schema-transfer.sql
transfer-fps:
Transfer worker thread is switched off (transfer-agents take over the work).
transfer-url-copy:
Bugfixes, daemonization.

7.1.3. Release contents

The gLite Data Management Local Transfer Service v. 0.1.3 is composed of the following gLite components:

Component name	Version	File
org.glite.deployment.data-local-transfer-service	0.1.3	../../../../../../../glite-web/egee/packages/I20050204/installers/glite-data-local-transfer-service_installer.sh ../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-local-transfer-service-config-0.1.3-1.noarch.rpm
org.glite.deployment.config	0.7.2	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-config-0.7.2-1.noarch.rpm
org.glite.data.transfer-fps	1.0.8	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-transfer-fps-1.0.8-1.noarch.rpm
org.glite.data.catalog-api-java	2.0.0	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-catalog-api-java-2.0.0-2.noarch.rpm
org.glite.data.common	1.1.4	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-common-1.1.4-2.noarch.rpm
org.glite.data.transfer-common	1.0.5	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-transfer-common-1.0.5-2.noarch.rpm
org.glite.data.transfer-interface	2.0.0	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/noarch/RPMS/glite-data-transfer-interface-2.0.0-3.noarch.rpm
org.glite.data.transfer-url-copy	1.0.1	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/i386/RPMS/glite-data-transfer-url-copy-1.0.1-1.i386.rpm
org.glite.data.srm-api-c	1.0.0	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/i386/RPMS/glite-data-srm-api-c-1.0.0-4.i386.rpm
org.glite.data.transfer-agents	1.1.1	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/i386/RPMS/glite-data-transfer-agents-1.1.1-1.i386.rpm
org.glite.data.config-service	1.1.4	../../../../../../../glite-web/egee/packages/I20050204/bin/rhel30/i386/RPMS/glite-data-config-service-1.1.4-1.i386.rpm

7.1.4. Dependencies

The gLite Workload Management System module has the following dependencies:

Component name	Version	RPM file name
gLite Security Utilities	0.6.0	../../../../../../../glite-web/egee/packages/I20050204/installers/glite-security-utils_installer.sh
gLite C/C++ Essentials	1.1.1	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/glite-essentials-cpp-1.1.1-1_EGEE.i386.rpm
MySQL-server	4.0.20	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/MySQL-server-4.0.20-0.i386.rpm
MySQL-client	4.0.20	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/MySQL-client-4.0.20-0.i386.rpm
Tomcat	5.0.28	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/tomcat5-5.0.28-4_EGEE.i386.rpm
GPT	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/gpt-VDT1.2.2rh9-1.i386.rpm
VDT Globus Essentials	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/vdt_globus_essentials-VDT1.2.2rh9-1.i386.rpm
CGSI_gSOAP_2.6.slc3	1.1.7	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/CGSI_gSOAP_2.6.slc3-1.1.7-2.i386.rpm

7.1.5. Known bugs and issues

This release has the following known bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw .

Bug number	Description	Fixed in this or previous releases but not yet tested
#6011	Wrong gsoap Exception type in SOAPFault
#6390	AXIS configuration in glite-data-transfer-fps is missing	X
#6425	DM service installer creates directory "glite-ce"	X
#6468	Perl interfaces to the Fireman and Transfer loose parameters in the functions
#6537	Can't install SRM and gLite on the same box	X

No removal procedure is provided with this release apart from the removal of the RPMS. Any account, group or other resource created during the module configuration must be manually cleaned.

7.1.6. Bugs closed since last release

This release fixes the following bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw

Bug number

Description