Maintaining TRSF file sets
In this newsletter we discuss how to maintain TRSF file sets between development, pre-production and production environments. A TRSF file set consists of the TRSF files that Virtel uses. These are the distributed SAMP and HTML VSAM files and any number of SCENARIO files that may have been set up to support Virtel studio developers. For our scenario we simulate a typical development / production IT infrastructure. The development VIRTEL, SPVIR1A, is run and maintained at an IT centre located in Paris (ZAMVS1) whilst the production VIRTEL, SPVIR2A, is located in Zagreb (ZAMVS2). We look at how a developer can manage TRSF directories and promotes files through to production with the use of the Virtel ADMIN tool and the ADMINVWM scenario.
Developers view
Developers will be using tools like Virtel Studio and Virtel Screen Redesigner to populate templates and scenarios into their development TRSF file set. At some point some of these elements will to be moved into a production environment. This can be achieved in a control and secure way through the use of the administration and directory management tools provide by the scenario ADMINVWM. In our example a transitory set of TRSF files, known as the pre-production TRSF file set, will be used between by the developers to move work from development to production. A developer will move web elements from their developing TRSF file set to the pre-production TRSF file set in preparation to moving them into production. The final stage of promoting work to production from the pre-production TRSF file set is controlled by operations. See the section called “Operations View” for more information on how this phase of the process is achieved.
Defining the ADMINVWM transaction
Before we can use the ADMINVWM administration tool we must define it to VIRTEL within the SPVIR1A VIRTEL task. The Virtel transaction definition to provide the ADMINVWM tool is defined as follows:-
Once this has been defined to VIRTEL we can access the tool with the following URL:-
http://192.168.170.30:41001/ADMVWM.htm+ADMINVWM
This will present the developer with the Administrations tool panel and will provide administration functions from which directory management functions can be performed.
In this newsletter we are only concerned with the Directories management function. We will not discuss any of the other administration tools. Using the Directories management function we can copy web elements (Scenarios, Templates, HTML files etc.) from one directory to another. We can also delete members as well as define new directories. Note that new directories require setting up before than can be used by this tool. Developers can now selectively copy elements from their development TRSF to the pre-production TRSF and give them new names. This may well be pertinent to maintaining different releases of a web element with the development and pre-production TRSF structure. For example the web element #DOPSSO.MYTEST.V0.390 may become #DOPSSO.V1.S90 in production.
Setting up a pre-production TRSF file set means deciding on a group of pre-production directions that will be used as intermediate files. These files will only contain elements that are required to be promoted into the production system. The intermediate files should be directory names and VSAM cluster names that identify them as such. It is assumed that the developer has already set up Virtel Studio and any relevant development infrastructure.
Allocating a pre-production VSAM file and allocating it to SPVIR1A
Before using a directory in Virtel we have to allocate a VSAM TRSF file and initialize it. The following JCL can be used:-
//*----------------------------------------------------------------*
//* DEFINITION OF A VSAM TRSF FILE *
//*----------------------------------------------------------------*
//* STEP 1 : DEFINITION DU FICHIER CUSTTRSF *
//*----------------------------------------------------------------*
//STEP1 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE (SP000.SPVIR1A.SCENARIO.PROD.TRSF) CLUSTER PURGE
SET MAXCC = 0
DEFINE CLUSTER (-
NAME (SP000.SPVIR1A.SCENARIO.PROD.TRSF) -
VOL (SPT308) -
SHR (2) -
KEYS (16 0) -
RECSZ (100 32758) -
RECORDS (2500 1000)) -
DATA (NAME(SP000.SPVIR1A.SCENARIO.PROD.TRSF.DATA) -
CISZ(4096)) -
INDEX (NAME(SP000.SPVIR1A.SCENARIO.PROD.TRSF.INDEX) -
CISZ(512))
//*----------------------------------------------------------------*
//* STEP 2 : INITIALISATION DU FICHIER CUSTTRSF *
//*----------------------------------------------------------------*
//STEP2 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//INIT DD DATA,DLM=AA
$$$$IWS.WORKREC.INW$TEMP
AA
/*
//BASE DD DSN=SP000.VIRT452.SAMP.TRSF,DISP=SHR
//TRSF DD DSN=SP000.SPVIR1A.SCENARIO.PROD.TRSF,DISP=SHR
//SYSIN DD *
REPRO INFILE(BASE) OUTFILE(TRSF)
/*
The DDNAME SCENPROD will be added to the VIRTEL started procedure to point to this VSAM TRSF file. We also need to a VSAM definition to the VIRTEL TCT and reassemble. Two definitions are required in the TCT. Once this has been done VIRTEL can be restarted and the VSAM file can be allocated to the pre-production directory.
....
PASSTCK=YES, *
UFILE1=(SAMPTRSF,ACBH1,0,10,01), *
UFILE2=(HTMLTRSF,ACBH2,0,10,01), *
UFILE3=(SCENTRSF,ACBH3,0,10,01), *
UFILE4=(SCENPROD,ACBH4,0,10,01), *
GATE=GENERAL, *
....
ACBH1 ACB AM=VSAM,DDNAME=SAMPTRSF,MACRF=(SEQ,DIR,OUT,LSR), *
STRNO=3
ACBH2 ACB AM=VSAM,DDNAME=HTMLTRSF,MACRF=(SEQ,DIR,OUT,LSR), *
STRNO=3
ACBH3 ACB AM=VSAM,DDNAME=SCENTRSF,MACRF=(SEQ,DIR,OUT,LSR), *
STRNO=3
ACBH4 ACB AM=VSAM,DDNAME=SCENPROD,MACRF=(SEQ,DIR,OUT,LSR), *
STRNO=3
Defining the pre-production directory
Once we have defined the VSAM file and have it allocated to VIRTEL we can associate the VSAM file with a direct using the directory management tool. In our example we are going to add this pre-production directory to SPVIR1A as directory SCNP-DIR. It will have a related development directory of SCN-DIR. When we add the directory name SCNP-DIR we associated it with the VSAM file DDNAME SCENPROD which we have just created and added to the VIRTEL started procedure.
We have set the download attribute only as we do not want developers inadvertently uploading to this TRSF pre-production file. We only want to be able to copy elements from the related SCN-DIR to SCNP-DIR.
Copy members
The next stage is to select members from our development directory and to copy them to our pre-production TRSF file. Again we use the directory management tool. In the example below we copy the .390 file #DOPSSO.390 from SCN-DIR to SCNP-DIR using the COPY sub command of the LIST function. We provide the target directory, in this case SCNP-DIR and give it a new name.
We press the Copy tab to invoke the copy function and copy the selected member.
After the copy is complete we can list the direct to ensure that the element has indeed been copied and with the correct name.
Developers will continue to populate their pre-production TRSF files set in preparation for migration to the production environment. The jobs required to perform the migration to production are discussed in the next section – Operations View.
Batch Copies
You can also copy a group of members using the batch copy facility of the directory management tool. This I selected by pressing the batch icon on the directory list page.
With the Batch copy facility I can select a list of files and copy them to another TRSF file. I select the files to be copied then click the copy all selected icon. The first icon on the right hands
After the copy has been completed I get a confirmation message.
This bulk copy facility could allow developers to copy-en-masse the pre-production TRSF file. Unlike the individual copy facility it doesn’t’ provided any rename capability.
Exporting directory list
The Directory list function also has an export facility whereby you can export the list of members in the selected directory to an EXCEL spreadsheet. This function is invoked by pressing the EXCEL export icon in the top right corner of the list window.
Operations View
The operations department have overall control in moving the developer’s pre-production TRSF file sets and promoting them into the production environment. The developers do not have direct access to any of the production datasets. Another requirement is that both development and production have an independent means of reverting back to previous TRSF level sets.
We use standard IBM JCL, software utilities and REXX to implement this process which uses GDG file structures of unloaded TRSF files sets at ZAMVS1 and ZAMVS2. The unloaded TRSF file sets at ZAMVS1 sets are ftp’d to ZAMVS2 where they are restored and promoted to the production environment under operation control. The development VIRTEL is known as SPVIR1A in ZAMVS1 and a production VIRTEL is known as SPVIR2A in ZAMVS2.
Initial Setup
The following jobs will set up the GDG structures required at both ZAMVS1 and ZAMVS2. For ZAMVS1 we run the following JCL:-
//*******************************************************************/
//* */
//* DEFINE DEVELOPMENT TRSF FILESET GDG. USED BY DEVELOPERS TO SAVE */
//* TRSF UPDATES TO TRSF FILES - SCENARIO, HTML AND SAMP */
//* */
//* SPVIR1A IS THE DEVELOPMENT ENVIRONMENT. */
//* */
//*******************************************************************/
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
DELETE SP000.SPVIR1A.TRSF.GDG GDG
DELETE SP000.SPVIR1A.TRSF.GDG.DSCB NVSAM
SET MAXCC=0
//ALLOC1 EXEC PGM=IEFBR14
//FILE DD DSN=SP000.SPVIR1A.TRSF.GDG.DSCB,
// UNIT=3390,DISP=(NEW,CATLG),
// SPACE=(TRK,(0,0)),VOL=SER=SPT301,
// DCB=BLKSIZE=6144
//*
//ALLOC2 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEF GDG(NAME(SP000.SPVIR1A.TRSF.GDG) LIMIT(5) SCRATCH NOEMPTY)
/*
For ZAMVS2 we run the following JCL:-
//*******************************************************************/
//* */
//* DEFINE PRODUCTION TRSF FILESETS. USED BY OPERATIONS TO SAVE TRSF*/
//* UPDATES FROM DEVELOPMENT INFRASTRUCTURE. */
//* */
//* SPVIR2A IS THE PRODUCTION VIRTEL. */
//* */
//*******************************************************************/
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
DELETE SP000.SPVIR2A.TRSF.GDG GDG
DELETE SP000.SPVIR2A.TRSF.GDG.DSCB NVSAM
DELETE SP000.SPVIR2A.PEND.GDG NVSAM
SET MAXCC=0
//ALLOC1 EXEC PGM=IEFBR14
//FILE DD DSN=SP000.SPVIR2A.TRSF.GDG.DSCB,
// UNIT=3390,DISP=(NEW,CATLG),
// SPACE=(TRK,(0,0)),VOL=SER=SPT301,
// DCB=BLKSIZE=6144
//PEND DD DSN=SP000.SPVIR2A.PEND.GDG,
// UNIT=3390,DISP=(NEW,CATLG),
// SPACE=(CYL,(40,10)),VOL=SER=SPT301,
// DCB=BLKSIZE=6144
//*
//ALLOC2 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEF GDG(NAME(SP000.SPVIR2A.TRSF.GDG) LIMIT(5) SCRATCH NOEMPTY)
/*
These GDG files will hold the unloaded TRSF file sets. We have establish a GDG cycle of 5 for recovery purposes. Also note that the setup job that runs at the production centre ZAMVS2 includes a holding file. This is used by the restore process to provide independence between development and production centre GDGs.
Process overview
A daily job is run in ZAMVS1 to dump the current SPVIR1A TRSF files to a GDG. This is run after the development VIRTEL SPVIR1A has been shut down. This avoids shipping OPEN VSAM datasets.
After successfully dumping the relevant TRSF file set a job is submitted, via FTP, to the production centre ZAMVS2. This job initiates a batch FTP GET from ZAMVS2 to copy across the latest GDG TRSF file set to a holding file – SP000.SPVIR2A.PEND.GDG. At any one time this holding file always contains they latest TRSF file set produced at the development centre - GDG 0. It may or may not be promoted to production. We GET from ZAMVS2 rather than PUT from ZAMVS1 as developers have no access to production datasets whereas production can access development datasets.
As and when required, the holding file is promoted to the VIRTEL production TRSF file set. Again a GDG structure is used to provide a back out capability should it be required.
Before starting VIRTEL a control step is run. This prepares the TRSF file set that the production VIRTEL will use. Depending on the parameters passed to step control step in the VIRTEL JCL will dictate which TRSF file set will be used and restore a file set if necessary.
In our example, the batch FTP job to get the latest development TRSF file set is run every day but the production TRSF file set is only promoted one a week, on a Friday. If that fails the previous production GDG (-1) is used to restore the TRSF file set. Effectively our production environment provides 5 weekly restore points and are development GDG 5 daily restore points.
The Development Process
TRSFMNT2
Job TRSFMNT2 – Daily job to back up TRSF file sets at development site. The job consists of three steps:-
TRSFDUMP Dump the relevant TRSF files.
TRSFTRS Terse the dump file and create a new GDG version.
FTP Send job TRSFMNT3, via FTP, to run at the production centre.
This is the only job that is run at the development centre. It provides the production centre with the latest TRSF file set and also enables the developers to keep a history TRSF development - in our example 5 days. A restore from an older version is possible should it be required. This GDG is independent of the production TRSF GDG file set.
TRSFMNT2 Sample JCL
//*******************************************************************/
//* */
//* DUMP DEVELOPMENT TRSF FILESET AND INSTRUCT PRODUCTION TO PULL */
//* LATEST TRSF UPDATES OVER. SHOULD BE CONTROLLED THROUGH A */
//* BATCH SCHEDULER AND RUN WHEN DEVELOPMENT VIRTEL IS DOWN! */
//* */
//* RUN ON DEVELOPMENT TO INITIATE A GET FROM PRODUCTION. MOVE ONLY */
//* PRE-PRODUCTION FILES FROM DEVELOPERS ENVIRONMENT */
//* */
//*******************************************************************/
//TRSFDUMP EXEC PGM=ADRDSSU,REGION=0M
//SYSPRINT DD SYSOUT=*
//DUMPFILE DD DSN=SP000.TEMP,DISP=(,CATLG),
// UNIT=SYSDA,
// SPACE=(CYL,(40,10),RLSE)
DUMP DS(INCLUDE( -
SP000.SPVIR1A.**.PROD.TRSF -
) ) -
ADMIN TOL(ENQF) -
OUTDD(DUMPFILE) COMPRESS
//TRSFTRS EXEC PGM=TRSMAIN,PARM=PACK
//SYSPRINT DD SYSOUT=*
//INFILE DD DSN=SP000.TEMP,DISP=(OLD,DELETE)
//OUTFILE DD DSN=SP000.SPVIR1A.TRSF.GDG(+1),DISP=(,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(40,10),RLSE)
//FTP EXEC PGM=FTP,REGION=8M,PARM='ZAMVS2 (EXIT'
//OUTPUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//NETRC DD *
MACHINE ZAMVS2 LOGIN USERID PASSWORD PASSWORD
//INPUT DD *
TYPE E
MODE B
SITE LRECL=80 BLOCKSIZE=3120 RECFM=FB
SITE FILETYPE=JES
PUT 'SP000.VIRTEL1A.CNTL(TRSFMNT3)'
QUIT
/*
TRSFMNT3
Job TRSFMNT3 – Job submitted to production JES2 via FTP from development centre.
This job consists of one step:-
‘FTP Get’ the latest pre-production TRSF file set GDG from ZAMVS1 and copy to the pending file.
//*******************************************************************/
//* */
//* INVOKE BY DEVELOPMENT FTP JOB TRSFMNT2. THIS JOB RUNS ON THE */
//* PRODUCTION SYSTEM TO PULL OVER THE LATEST TRSF FILESET UPDATES. */
//* */
//* RUN ON PRODUCTION TO OBTAIN TRSF UPDATES FROM DEVELOPMENT */
//* */
//*******************************************************************/
//FTP EXEC PGM=FTP,REGION=8M,PARM='ZAMVS1 (EXIT'
//OUTPUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//NETRC DD *
MACHINE ZAMVS1 LOGIN USERID PASSWORD PASSWORD
//INPUT DD *
TYPE E
MODE B
GET 'SP000.SPVIR1A.TRSF.GDG(0)' 'SP000.SPVIR2A.PEND.GDG' (REPLACE
QUIT
/*
TRSFREXX
Step TRSFREXX – A control setup run before the startup of VIRTEL at the production site. This job can be a part of the VIRTEL started task or can be a separate job run prior to starting VIRTEL. It prepares the TRSF file set that VIRTEL will use using conditional JCL.
The control step consists of a REXX EXEC which generates a return code from a parameter passed. This return code controls whether a TRSF file set is to be restored and if so which TRSF file set.
REXXCC REXX EXEC to set a return code based upon the symbolic GDG=
UPDATE Only executed if return code is 99. Calls TRSFUPDT procedure to restore latest TRSF file set from production.
BACKOUT Only executed if return code is greater than 1. Calls TRSFBACK procedure to restore a particular GDG historic TRSF file set. The GDG number indicates which GDD level is to be restored.
TRSFREXX Example JCL
//******************************************************************/
//* THIS CONTROL JOB PREPARES THE TRSF FILES FOR VIRTEL. IT IS */
//* CONTROLLED BY A RC SET VIA A JCL SYMBOLIC. */
//* */
//* THE SYMBOLIC GDG IS SET AS FOLLOWS:- */
//* */
//* GDG = 0. DO NOTHING. VIRTEL WILL USE THE CURRENT TRSF FILE SET */
//* THIS WOULD NORMALLY BE THE DEFAUL. */
//* */
//* GDG = 99 UPDATE THE TRSF FILE SET WITH THE LATEST PRE-PROD */
//* VERSION. THIS WOULD HAVE BEEN OBTAINED VIA AN FTP GET */
//* TO THE DEVELOPMENT INFRASTRUCTURE. */
//* */
//* GDG = N RESTORE THE TRSF FILE SET THE THE N'TH GDG VERSION. */
//* WHERE 1 = GDG(-1), 2 = (GDG(-2) ETC. */
//* */
//* */
//******************************************************************/
//LIBS JCLLIB ORDER=SP000.VIRTEL2A.CNTL
//TRSFREXX PROC GDG=0
//REXXCC EXEC PGM=IRXJCL,PARM=('REXXCC &GDG')
//SYSEXEC DD DISP=SHR,DSN=SP000.VIRTEL2A.CNTL
//RC99 IF (RC = 99) THEN
//UPDATE EXEC TRSFUPDT /* UPDATE WITH LATEST */
//NOTRC99 ELSE
//RC1 IF (RC > 0) THEN
//BACKOUT EXEC TRSFBACK,GDG=&GDG /* BACK OUT TO LEVEL */
// ENDIF
// ENDIF
//VIRTEL EXEC VIRTEL
// PEND
//S01 EXEC TRSFREXX,GDG=0 /* NO TRSF CHANGE */
Procedure TRSFUPDT
This procedure consists of three steps. It promotes that latest development TRSF file set into the production environment.
TRSCOPY Unterse the holding file and create a new production GDG TRSF file set level. This contains the latest copy of the development TRSF file set.
DELETE Delete the current VSAM TRSF files
TRSFDUMP Restore from the new GDG the TRSF file set renaming the VSAM files to production dataset names.
//TRSFUPDT PROC
//********************************************************************/
//* */
//* PROCEDURE TO RESTORE THE LATEST PENDING GDG FILE AS OBTAINED BY */
//* FTP JOB. */
//* */
//********************************************************************/
//TRSFCOPY EXEC PGM=TRSMAIN,PARM=UNPACK
//SYSPRINT DD SYSOUT=*
//INFILE DD DSN=SP000.SPVIR2A.PEND.GDG,DISP=SHR
//OUTFILE DD DSN=SP000.SPVIR2A.TRSF.GDG(+1),DISP=(,CATLG,DELETE),
// UNIT=SYSDA,VOL=SER=SPT309,
// SPACE=(CYL,(40,10),RLSE)
//DELETE EXEC PGM=IDCAMS,COND=(0,NE,TRSFCOPY)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DSN=SP000.VIRTEL2A.CNTL(TRSFDEL),DISP=SHR
//TRSFREST EXEC PGM=ADRDSSU,REGION=0M,
// COND=((0,NE,TRSFCOPY),(0,NE,DELETE))
//SYSPRINT DD SYSOUT=*
//TRSFFILE DD DSN=SP000.SPVIR2A.TRSF.GDG(+1),DISP=(SHR,KEEP,DELETE)
//DISK DD UNIT=SYSDA,VOL=SER=SPT309,DISP=SHR
//SYSIN DD DSN=SP000.VIRTEL2A.CNTL(TRSFREST),DISP=SHR
// PEND
Procedure TRSFBACK
This procedure restores the TRSF file set from an older production GDG backup.
DELBACK Delete the current VSAM TRSF files
TRSFREST Restore from an older GDG the TRSF file set, renaming the VSAM files to production dataset names.
//TRSFBACK PROC GDG=
//***************************************************************
//* *
//* PROCEDURE TO RESTORE TRSF FILES FROM A PREVIOUS GDG LEVEL *
//* *
//***************************************************************
//DELBACK EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD DSN=SP000.VIRTEL2A.CNTL(TRSFDEL),DISP=SHR
//TRSFREST EXEC PGM=ADRDSSU,REGION=0M,
// COND=(0,NE,DELBACK)
//SYSPRINT DD SYSOUT=*
//TRSFFILE DD DSN=SP000.SPVIR2A.TRSF.GDG(-&GDG),DISP=SHR
//DISK DD UNIT=SYSDA,VOL=SER=SPT309,DISP=SHR
//SYSIN DD DSN=SP000.VIRTEL2A.CNTL(TRSFREST),DISP=SHR
// PEND
Rexx Procedure REXXCC
/* REXX
*****************************************************************/
/* A little REXX EXEC to return a RC as set by a PARM value. */
/****************************************************************/
arg rc
if rc = '' then return 0
return rc
Control Statements TRSFREST
RESTORE DS(INCLUDE( -
SP000.SPVIR1A.**.PROD.TRSF -
) ) -
RENAMEU(SP000.SPVIR1A.**,SP000.SPVIR2A.**) -
ADMIN TOL(ENQF) -
OUTDD(DISK) INDD(TRSFFILE)
Control Statements TRSFDEL
DELETE SP000.SPVIR2A.SCENARIO.PROD.TRSF
SET MAXCC=0