gLite Data Management File Transfer Agents

1. Release Description

This release contains the gLite Data Management File Transfer Agents module v. 2.0.0. The following sections provide additional information about the release content, the module dependencies, the know bugs and issues and a list of bugs closed since the previous release. For information about installing and using the gLite Data Management File Transfer Agents, please refer to the gLite Installation and User Guides.

2. Changes in this Release

The main changes in this release are:

This version of the File Transfer Agents module supports both Oracle and MySQL backends
The interface was changed: the FPS interface was merged into the FTS interface, but FTS stayed backward compatible with R1.3.
There is a new FileTransferStats interface to ease the query of statistical information of the transfers
Agent running information is published into the DB, and so agent startup has more checks to avoid clashes in a distributed setup (if there is no other agent for the channel, if there is a channel)
Preparations for the 'agent development kit'.
SRMcopy support

3. Release contents

The gLite Data Management File Transfer Agents v. 2.0.0 is composed of the following gLite components:

Component name	Description	Version	File
glite-config	gLite configuration scripts	1.4.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-config-1.4.2-1.noarch.rpm
glite-data-catalog-api-c	gLite Data Catalogs C/C++ API	2.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-catalog-api-c-2.0.0-3.i386.rpm
glite-data-catalog-cli	Data Catalog CLI	1.7.1	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-catalog-cli-1.7.1-2.i386.rpm
glite-data-catalog-interface	gLite Catalog Interfaces	2.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-data-catalog-interface-2.0.0-6.noarch.rpm
glite-data-config-service	GLite Data Config Service	1.2.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-config-service-1.2.2-2.i386.rpm
glite-data-srm-api-c	gLite Data SRM C/C++ API	1.1.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-srm-api-c-1.1.0-1.i386.rpm
glite-data-srm-cli	Data SRM CLI	1.1.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-srm-cli-1.1.0-2.i386.rpm
glite-data-transfer-agents	gLite Data Transfer Agents	2.0.4	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-transfer-agents-2.0.4-2.i386.rpm
glite-data-transfer-url-copy	gLite Data Transfer URL Copy	1.4.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-transfer-url-copy-1.4.2-1.i386.rpm
glite-data-util-c	Data C util libraries	1.1.4	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-data-util-c-1.1.4-1.i386.rpm
glite-file-transfer-agents-config	gLite File Transfer Agents configuration files	2.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-file-transfer-agents-config-2.0.0-1.noarch.rpm
glite-rgma-api-c	C API for R-GMA	4.1.13	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-rgma-api-c-4.1.13-1.i386.rpm
glite-rgma-api-cpp	C++ API for R-GMA	4.1.15	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-rgma-api-cpp-4.1.15-1.i386.rpm
glite-rgma-api-java	Java API for R-GMA	4.1.5	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-api-java-4.1.5-1.noarch.rpm
glite-rgma-base	R-GMA basic configuration and documentation	4.1.19	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-base-4.1.19-1.noarch.rpm
glite-rgma-common-config	gLite rgma common configuration items installation	5.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-common-config-5.0.0-1.noarch.rpm
glite-rgma-servicetool	R-GMA service tool	4.1.19	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-servicetool-4.1.19-3.noarch.rpm
glite-rgma-servicetool-config	gLite R-GMA servicetool installation	5.1.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-servicetool-config-5.1.0-1.noarch.rpm
glite-rgma-stubs-servlet-java	Java client implementation stubs for R-GMA	4.1.13	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-rgma-stubs-servlet-java-4.1.13-1.noarch.rpm
glite-security-trustmanager	The java certificate path checkin for proxy certs in SSL with plugins for tomcat and axis.	1.7.3	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-security-trustmanager-1.7.3-1.noarch.rpm
glite-security-util-java	The java utilities library for security	1.1.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-security-util-java-1.1.2-2.noarch.rpm
glite-security-utils-config	gLite Security Utilities configuration files	1.0.4	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-security-utils-config-1.0.4-1.noarch.rpm
glite-security-voms-api-c		1.6.3	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-security-voms-api-c-1.6.3-0.i386.rpm
glite-service-discovery-api-c	C API for Service Discovery.	2.0.1	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-service-discovery-api-c-2.0.1-0.i386.rpm
glite-service-discovery-api-java		2.0.1	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-service-discovery-api-java-2.0.1-1.noarch.rpm
glite-service-discovery-bdii-c	BDII based Service Discovery	2.0.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-service-discovery-bdii-c-2.0.2-1.i386.rpm
glite-service-discovery-cli	Service Discovery CLI	2.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-service-discovery-cli-2.0.0-1.i386.rpm
glite-service-discovery-config	gLite Service Discovery installation	1.1.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-service-discovery-config-1.1.0-1.noarch.rpm
glite-service-discovery-file-c	File based Service Discovery	2.0.0	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-service-discovery-file-c-2.0.0-1.i386.rpm
glite-service-discovery-rgma-c	C API for Service Discovery.	2.0.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/i386/RPMS/glite-service-discovery-rgma-c-2.0.2-0.i386.rpm
glite-service-discovery-rgma-java		2.0.2	../../../../../../../glite-web/egee/packages/R1.4/R20050916/bin/rhel30/noarch/RPMS/glite-service-discovery-rgma-java-2.0.2-1.noarch.rpm

4. Dependencies

The gLite File Transfer Agents module v. 2.0.0 has the following dependencies:

Component name	Version	RPM file name
gLite C/C++ Essentials	1.1.1	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/glite-essentials-cpp-1.1.1-1_EGEE.i386.rpm
GPT	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/gpt-VDT1.2.2rh9-1.i386.rpm
VDT Globus Essentials	VDT 1.2.2	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/vdt_globus_essentials-VDT1.2.2rh9-1.i386.rpm
CGSI_gSOAP_2.6	1.1.8	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/CGSI_gSOAP_2.6-1.1.8-2.slc3.i386.rpm
perl-Expect.pm	1.01	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/perl-Expect.pm-1.01-9.i386.rpm
myproxy	1.14	../../../../../../../glite-web/egee/packages/externals/bin/rhel30/RPMS/myproxy-1.14-EGEE.i386.rpm
Java JRE/SDK	1.4.2	http://java.sun.com/j2se/1.4.2/download.html
Oracle InstantClient	>= 10.1.0.4	http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/linuxsoft.html

The Oracle InstantClient libraries and executables are not freely redistributable and are not included with this release. The four RPMS have to be downloaded from the Oracle web site and installed on the system before installing the gLite File Transfer Agents module.

5. Known bugs and issues

This release has the following known bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw .

The gLite I/O and gLite FTS security models conflict. This is being discussed in several coordination bodies. A proposal to harmonize them has been already produced and circulated as part of the updated gLite Architecture document
No removal procedure is provided with this release apart from the removal of the RPMS. Any account, group or other resource created during the module configuration must be manually cleaned.

Known open bugs:

Bug number	Description
#8374	FTS: Usage of SRM Copy
#8906	FTS: auto-generated quattor template has a couple of bugs
#8947	FTS: Throughput phase: need a post-transfer agent to remove files from disk
#9803	FTS: FTA logrotate does not rotate logs
#9830	FTS: FTA log file names could be improved for clarity
#10279	FTS: glite-transfer-submit produce out of memory if the job description file is empty
#10569	FTS: FTA Service error: VO 'egtest' agent not running.
#10588	FTS: FTA canceled job does not want to die...

Bugs fixed in this or previous releases but not yet tested:

Bug number	Description
#9200	FTS: glite-url-copy uses both jar and tar
#9201	FTS: glite-url-copy-cleanlog usage message issues
#9223	FTS: glite-url-copy print out perf markers in blocking mode?
#9224	FTS: Can't copy large files to dCache
#9232	FTS: glite-url-copy doesn't timeout if file never transitions from pending
#9240	FTS: glite-url-copy does not distinguish in FINAL message on timeouts
#9309	FTS: dCache returns error on gridftp call, but glite-url-copy doesn't see it?
#9310	FTS: two FINAL messages if advisoryDelete fails against dCache
#9338	FTS: agent shoud fill in DN field when it updates a table
#9398	FTS: mysterious failure on PUT when it apparently completed OK
#9399	FTS: blocking glite-url-copy segfaults or buffer overrruns sometimes on srm-putdone error
#9793	FTS: FTA wrong default value for vo-agent
#9813	FTS: 1.3: url-copy: SRM get failure seg faults.
#9944	FTS: FTA logrotate is not configured
#9946	FTS: FTA+FTS configuration on the same node does not work using site config files
#10055	FTS: FTA global user parameter file-transfer.VONAME should be remove
#10060	FTS: FTA 1.3: does not retry submission after the retry interval time
#10235	FTS: must be able to stop individual instances without stopping the others
#10317	FTS: FTA installation script doesn't allow to ovewrite parameters defined in glite-global.cfg.xml
#10329	FTS: FTA doesn't retry transfers when urlcopy doesn't return a reason class
#10362	cannot submit fts jobs using delegated proxies
#10440	FTS: ChannelAgent doesn't always use client delegated credentials for srmcopy
#10490	FTS: FTA channel agent with MySQL backend don't start
#10498	FTS: FTA missing MySQL port FTA configuration parameter
#10534	FTS: FTA MySQL: agents get the wrong type
#10557	FTS: transfers with SRM copy fail
#10582	FTS: FTA SRM copy log file are way too verbose...

6. Bugs closed since last release

This release fixes the following bugs and issues. Bug numbers refer to the gLite Bug Tracking system database hosted on the CERN Savannah system at https://savannah.cern.ch/bugs/?group=jra1mdw

Bug number	Description
#7113	FTS: glite-url-copy - if SRM.put fails, the code loops forever
#7318	FTS: Transfer agent sometimes mis-counts the number of active transfers
#7352	FTS: forked glite-url-copy should chdir somewhere it can write
#7355	FTS: transfer-url-copy sometimes dies on gridFTP
#7361	FTS: transfer agent should detect a bad Oracle connection, and try to reconnect
#7466	FTS: transfer-url-copy returns 0 for size and bytes written
#7739	FTS: time out on Gridftp if it hangs on network failure
#7740	FTS: glite-url-copy dumps core on some SRM failures
#8366	FTS: Transfer Agent for a channel is per-VO
#8373	FTS: Certification with dCache, DPM and new castor SRM
#8375	FTS: Daemon watchdog scripts
#8376	FTS: per transfer logs need better management
#8382	FTS: Split deployment model
#8386	FTS: No inter-VO scheduling
#8485	FTS: Deployment script should not set X509_USER_CERT/KEY
#8783	FTS: agents EnableHold property is not read correctly
#8826	FTS: if setDone fails on the destination, glite-url-copy flags as successful
#8878	FTS: Occasional infinite loops in glite-url-copy process
#8884	FTS: if SRMs hang, we should timeout
#8966	FTS: srm failures are not always detected and a NULL TURL is returned
#8968	FTS: if SRM get or put retries reach the max timeout, it didn't report as failed
#8969	FTS: don't fail hard if SRM putdone fails on source
#9951	FTS: FTA wrong log file name for vo agent
#10592	FTS: FTA transfers do not use delegation by default
#10625	FTS: web service installer also installs agents