Date 05 03 2005 Release notes for LCG-2_4_0 ============================ This release includes major changes compared to the 2_3 series of releases. As you can see here: http://grid-deployment.web.cern.ch/grid-deployment/releaseNoteLCGDetailed-2.4.0.txt a large number of outstanding bugs have been fixed. In this release note we would like to give you some overview of the components that have been modified. In addition we would like to communicate a few general matters to the sites. The order in which we report these items doesn't reflect their relative priorities. Please take the time to read these notes. General Comments ================= - We no longer distribute JAVA ------------------------------------------ As has been mentioned on several occasions the practice to include the JAVA RPMs in our repositories was at best questionable from a legal standpoint. Of course, the middleware still requires JAVA to be installed. The installation guides contain instructions on how to download the software from SUN and install it. This is certainly not a step forward to more comfort, but we see no way to avoid it. - The release procedure has changed ---------------------------------------------------------- 2-4-0 is the first of the new fixed date releases. For those who didn't follow the development and discussions in this area, here it is in a nutshell: Every three months (1st April, 1st July, 1st October ....) a major release will be made. This release can contain new services, new clients and might require the installation of new RPMs and additional configuration changes. --> It is expected that these major releases are put into operation by the sites within 3 weeks after the release came out. <-- In between there will be a release every month, if needed. The sites are not required to install these releases, however, we expect them to provide space for dteam on the shared file system used for user software installation to make the client libs available to the VOs. The central services can get fully backward compatible upgrades at any time. This will affect the CICs only. In addition at any time a security upgrade might be needed. We expect the sites to follow those in a timely manner. - Shared space for user level software installation -------------------------------------------------------------------- We have observed that most VOs can't make efficient use of sites that do not provide some shared space for the installation of user level software. It would be really helpful if you could consider this for your site. Please contact us if you have any questions concerning this. On several sites that already provide this to the VOs, this service is not provided for the dteam VO. This now becomes more important, since we would like to use the mechanism described in the workload management section to keep the client libs recent. - Test Pilots ----------------- To make the upgrades smoother we now test deploy first with a few sites that volunteered. (Thanks for the help) Some Major Changes =================== The work load management system -------------------------------------------------- Beside various bug fixes and performance improvements, the system now includes an interface to file catalogues that conform to the DLI. A mechanism has been put in place that enables the user to choose between multiple version of the middleware offered by a site. The selection is done by a simple JDL statement. Details on how this works for users can be found in the LCG User Guide. This feature will allow users to use the latest versions of the middleware even on sites that can't follow the monthly updates. New Data Management Software ---------------------------------------------- -dCache: dCache is an SRM enabled disk pool manager provided by DESY and FNAL. For an overview of the system please refer to: http://www.dcache.org A first version of an installation and operations manual is issued parallel to this release. Currently there is no simple migration path from an existing classic SE to a dCache SE, but a site can choose dCache for a new SE. dCache is a highly flexible, tunable system. SA1 can only support a limited number of standard configurations, due to resource limitations. For more complex setups, please contact the dCache team. A draft of the documentation can be found here: http://grid-deployment.web.cern.ch/grid-deployment/documentation/public/gis/dCache-SiteAdmin-Guide/pdf/dCache4SiteAdmins.pdf LFC: The LCG File Catalogue, already tested by the people active in ARDA and by DESY, is supported by this release. It gives access to a more performant secure catalogue. The client libs are part of the WNs and UIs. Before we include detailed installation guides for the server we would like to gain some experience with typical deployment scenarios. If your site, or VO, is interested in setting up an LFC server, please contact the deployment team at CERN (support-lcg-deployment@cern.ch). This catalogue will interoperate with the RB via the DLI interface. Migration scripts from the RLS to the LFC are available on request. A first version of LCF admin guide can be found here: https://edms.cern.ch/file/579088/1/LFC-Administrator-Guide.pdf -DPM: One of the interesting capabilities of the lightweight Disk Pool Manager is that it can convert an existing, populated Classic SE into an scalable SRM-SE without moving the data. The server is in the final stage of testing and will be released in the very near future. We mention the DPM already here to help smaller sites in planning the migration from Classic SEs to SRM enabled storage. VOMS ===== This release includes support for VOMS on the CE and the classic SE. On other services a VOMS proxy is accepted as a standard grid proxy. On all nodes the gridmapfiles can be populated with information provided by the VOMS servers and LDAP-VO servers. This eases the interoperation with other grid infrastructures. New BDII with Web based Config. ========================== The new version of the BDII has a re-engineered database swap and update mechanism that improves the BDIIs performance and availability under load. Very soon a service to allow VO specific configuration of the BDII will be opened for public use. This service, located at the GOC at RAL, has the following features: - sites will reflect the operational state as published in the GOC DB - VOs can (if they want) apply filters on the sites to be used for their jobs, depending on the status of the site defined by the test matrix of the daily Site Functional Tests, run by the CIC-on-duty team. - VOs can in addition white- and blacklist sites While for performance reasons it is still possible to have private BDIIs for VOs, all BDIIs can rely on the identical configuration. This should help to stop the confusion that was observed in the past when different BDIIs have been used at submission and runtime. New improved R-GMA =================== The new version of R-GMA is based on the version of R-GMA in gLite, but is mostly backwards compatible. This version is expected to be more stable and have improved performance. One service that will make use of the new R-GMA is edg-rgma-gin, which in this release has been activated and will export to R-GMA all the information that is produced by the Generic Information Provider and the Grid Ice sensors. Improved GridICE ============== The latest version provided by the INFN team has been included. New version of APEL ================= The R-GMA based accounting tool now supports PBS and LSF. Improved EIS user tools =================== These tools are targeted at the endusers and ease the access to the information system. lcg-infosites helps especially in finding SEs and their matching CEs. It is a replacement for the very inefficient "edg-rm printInfo" command lcg-info for user friendly ldap browsing Improved YAIM ============= Based on the sites' constructive feedback from the previous releases a large number of improvements to the scripts and configuration have been made. We also made it more flexible to ease the localization for sites with non standard layouts. Infrastructure for job level monitoring ============================= New components have been included for job level monitoring. This will allow users and monitoring tools to follow the status of jobs, including their CPU consumption and other indicators. The following two components provide the basis for this. It should be pointed out that the functionality is independent of the batch system and doesn't add additional load to the services. lcg-mon-wn: Will publish every 5mins some information about the running job on a WN into R-GMA. It is started by the job wrapper. lcg-mon-job-status: Publishes job state changes from the Logging and Bookkeeping server into the R-GMA. Tank & Spark ============ The RPMs for this tool for user level software installation are included in this release, the configuration and documentation can be found here: http://grid-deployment.web.cern.ch/grid-deployment/cgi-bin/index.cgi?var=eis/docs This can be an alternative for sites that have problems providing a shared file system for the user software.