Compile and Link Transp codes

17-Mar-2015
jec

Overview of scripts required to build Transp codes

See also - List of Known Problems

Directories

A library or executable <elem> has the following directories (under codesys); mpi elements have in addition the (mpi_)<elem> directory

source/<elem>

Source files ( both serial & mpi) for the element, + .inf files to control processing;
<elem>_link.exe files for executables ( lists library requirement)

make/(mpi_)<elem>

makefiles, generated by cmsmms (libraries), or makelink (executables)

compsc/(mpi_)<elem>

(cpp) preprocessed source, ready for compilation

obj/(mpi_)<elem>

compiled code and (private) .mod files

(marker)

No longer used

lib/(mpi_)<elem>.a

library files

exe/(mpi_)<elem>

executables

Target

The shell script $SC/Target aids traversing these directories;   >source $SC/Target  defines aliases Target, Source, Obj, mpiObj, Make, mpiMake. Target <elem> then sets $CSTGT to <elem>, after which the aliases Source, mpiObj etc provide the commands 'cd $CODESYSDIR/source/<elem>', 'cd $CODESYSDIR/obj/mpi_<elem>',

LLjob

LLjob will submit any script from codesys/csh as a loadleveller ( == grid engine) job.
Arguments -

1. Name of script to be submitted
2. identifier - log file is $LOGDIR/<script>_<id>
3. remaining args are passed to the selected script

	eg -
            >LLjob linkcheck.csh 00 clean,noexe

Libraries

Library makefiles are constructed by cmsmms, using the contents of source/<elem>;

cmsmms

cmsmms creates makefiles ( in make/(mpi_)<elem> ):

<elem>_a

to construct lib/<elem>.a

<src>_o

to construct the linrary member lib/<elem>.a(<src>.o)

The file source/<elem>/<elem>_mpi.inf flags the requirement to build an MPI variant.

Sample makefiles -

$(LOCAL)/lib/trmpi.a: \
$(LOCAL)/lib/trmpi.a(trmpi_env_update.o)\
$(LOCAL)/lib/trmpi.a(trmpi_init.o)\
$(LOCAL)/lib/trmpi.a(trmpi_listener.o)\
$(LOCAL)/lib/trmpi.a(trmpi_t4_lunset.o)\
$(LOCAL)/lib/trmpi.a(trmpi_timer.o)
csh -f csh/finishlib $(LOCAL)/lib/trmpi.a

include make/trmpi/trmpi_env_update_o
include make/trmpi/trmpi_init_o
include make/trmpi/trmpi_listener_o
include make/trmpi/trmpi_t4_lunset_o
include make/trmpi/trmpi_timer_o

ifndef trmpi-trmpi_init-make
trmpi-trmpi_init-make = TRUE

$(LOCAL)/lib/trmpi.a(trmpi_init.o): source/trmpi/trmpi_init.f90\
$(LOCAL)/obj/trmpi_helper/trmpi.module_tag\
$(LOCAL)/obj/portlib/logmod.module_tag\
$(LOCAL)/obj/portlib/execsystem.module_tag
csh -f csh/tcomp source/trmpi/trmpi_init.f90 $(LOCAL)/obj/trmpi/trmpi_init.o
csh -f csh/ckreplace $(LOCAL)/lib/trmpi.a $(LOCAL)/obj/trmpi/trmpi_init.o

endif

ifndef trmpi-trmpi_init-modincl
trmpi-trmpi_init-modincl = TRUE

include make/trmpi_helper/trmpi_o

include make/portlib/logmod_o

include make/portlib/execsystem_o

endif

uplib

>uplib <elem>  runs make, with input file make/<elem>/<elem>_a, and argument -k ( keep going) by default. The flags may be overwritten by setting $MAKE_FLAGS. If  "new" is provided as the 2nd argument, then the library and object files will be deleted, to force a complete rebuild.

If $UPLIB_LOG is defined, then the return code (from make) is written to the file $UPLIB_LOG (JET extension, see linkcheck.csh ). A status of 1 is returned if <elem>_a is not found. Status 2 is returned if any file fails to compile.

Executables

A file source/<elem>/<elem>_exe.link indicates that source/<elem> contains the source for a  Transp executable

makelink

 This script reads a file $CODESYSDIR/source/<name>/<name>_exe.link  It writes files in $CODESYSDIR/make/<name> :-

 <name>.mk

csh script to update libraries ( uplib ) & calls make with files <name>.o, <name>_x

 <name>.o

make commands to complile source ( using $SC/tcomp )

 <name>_x

make commands, defines dependencies & make command for build

 <name>.sh

csh script, calls linkload to build executable

<name>_exe.link may contain 3 different types of input -

<filename>.o

object file definitions; object files are located by $SC/scfind

<libname>.a

library file definitions. Default location in $CODESYS/lib. If first character is ., then a path from $CODESYSDIR is assumed

$<variable>

environmental variable references. These lines are stored as given. Variable expansion occurs when <name>.sh calls $SC/linkload [ so rerunning uplink issufficient if the variable definitions are changed ]

 anything else is ignored.

tcomp_test

tcomp_test and tcomp_testlib were originally written to debug the tcomp script, but provide a useful quick check on a new Transp version
 

tcomp_test.o
tcomp_testlib.a
mdstransp.a
portlib.a
$L_NETCDF
$L_HDF5

#! /bin/csh -f
# build code: check each library + the executable: tcomp_test
cd $CODESYSDIR
if ( $1 != "nolib" ) then
csh -f csh/uplib tcomp_testlib
csh -f csh/uplib mdstransp
csh -f csh/uplib portlib
endif
make -f make/tcomp_test/tcomp_test_o
cd $LOCAL
make -f $CODESYSDIR/make/tcomp_test/tcomp_test_x

ifndef tcomp_test-tcomp_test-make
tcomp_test-tcomp_test-make = TRUE

$(LOCAL)/obj/tcomp_test/tcomp_test.o: source/tcomp_test/tcomp_test.f\
$(LOCAL)/obj/tcomp_testlib/mtest_mod.module_tag
csh -f csh/tcomp source/tcomp_test/tcomp_test.f $(LOCAL)/obj/tcomp_test/tcomp_test.o

endif

ifndef tcomp_test-tcomp_test-modincl
tcomp_test-tcomp_test-modincl = TRUE

include make/tcomp_testlib/mtest_mod_o

endif

External libraries and Symbols

Libraries required for external packages ( netCDF, MPI etc ) are referenced by environment variables, conventionally $L_<libname>; for example,

L_NETCDF=/common/transp_shared/Code/transp/JET_61/external/lib/libnetcdff.a /common/transp_shared/Code/transp/JET_61/external/lib/libnetcdf.a \
	/usr/lib/libhdf5.so /usr/lib/libhdf5_hl.so /usr/lib/libhdf5_fortran.so

Since the variable references are in the .sh file, the values are resolved when the code is built, not when the makefiles are constructed. This allows libraries to be changed, without rebuilding the make files. The symbols are defined in configdir/libdef.csh;

At JET, the system libraries are compiled with pgf, while Transp uses Lahey. Lahey versions have been built where necessary (see /home/pshare/lib ) and are linked from $TRANSPROOT/external

uplink

uplink <elem>  {nolib}  will rebuild exe/<elem>; if the 2nd argument nolib is provided, then the library dependencies will not be rebuilt (which saves time if only the executable source has been modified). uplink exeutes the (csh) script make/<elem>/<elem>.mk

If $UPLINK_LOG is defined, then the return code (from <elem>.mk) is written to the file $UPLINK_LOG (JET extension, see linkcheck.csh )

Compilation & Linking

tcomp

tcomp is a script to compile all Transp source files, on all supported platforms.

Optional Arguments ( in any order)

-v

Display preprocessor & compiler options

debug

Use debug compiler switches

inb_debug

(??)

expert

Compile the expert file for transp

must precede the required arguments

name

of the file to be processed

name

of the object file produced

 Files used

$SC/Options

Link to the $WORKSTATION_TYPE_options file

$SC/System_options

General codesys options

Optional -

marker/$libname/options

library specific overrides

$cdir/User_options

user overrides

$SC/User_options

user overrides

The logic by which the compiler options are defined is inpenetrable, but the utility optest may be used to display the results for any library or executable. If all else fails, then setting $CODESYS_DEBUG = 1 will echo the commands, and also prevent deletion of the intermediate files in compsc/<elem> . Unfortunately, the amount of output produced makes this less useful than might be hoped.

module handling

    public module files are stored in codesys/mod or codesys/mpi_mod, while private module files are stored in obj/<elem>/. tcomp used codesys as a working directory, and the script $SC/move_mod would then move the .mod files as needed. Unfortunately this was poorly implemented, and if simultaneous builds were in progress, .mod files could be copied to the wrong directory.  tcomp now sets $TCOMP_SRC to the name of the file being processed, and $SC/Get_All_Options adds a line "-M<directory> to the compiler options, to direct the .mod file to the correct location, so that move_mod is no longer required

linkload

optest

optest displays the compiler options used for any element, and optionally for a constituent file :-

>optest tcomp_testlib mtest_mod
optest: executing Get_All_Options ...
mtest_mod.for
tcomp% -I- used: /common/transp_shared/Code/transp/JET_64/config/User_options
options finished 

------> optest output for LINUX ( LaheyFujitsu ), tcomp_testlib(mtest_mod)


f90opt = lf95 -c -O --trap --trace -Wa,--32 \
	-I/common/transp_shared/Code/transp/JET_64/codesys/compsc/tcomp_testlib \
	-I/common/transp_shared/Code/transp/JET_64/codesys/obj/tcomp_testlib \
	-M/common/transp_shared/Code/transp/JET_64/codesys/mod \
	-I/common/transp_shared/Code/transp/JET_64/codesys/mod \
	-I/common/transp_shared/Code/transp/JET_64/external/mod \
	-I/common/transp_shared/Code/transp/JET_64/external/inc


f90debug = lf95 -c -g --trap --trace --chk a,e,s,u \
	-I/common/transp_shared/Code/transp/JET_64/codesys/compsc/tcomp_testlib \
	-I/common/transp_shared/Code/transp/JET_64/codesys/obj/tcomp_testlib \
	-M/common/transp_shared/Code/transp/JET_64/codesys/mod \
	-I/common/transp_shared/Code/transp/JET_64/codesys/mod \
	-I/common/transp_shared/Code/transp/JET_64/external/mod \
	-I/common/transp_shared/Code/transp/JET_64/external/inc

In this example, mtest_mod is the source for a public module, so -M flags the codesys/mod directory.

tcomp_test

System Building

linkcheck

linkcheck is a script to recreate all Transp binaries from the source files, using the make files. It does not recreate generated code or make files - see code_daemon for this. The script linkcheck.csh is also available (at JET) to perform the same function. It can run more than one build operation (uplink, uplib) at the same time, and has additional options.

 Arguments

'everything'	to check sglib
'nolib'	to suppress library rebuild
'nolink'	to suppress exe/* rebuild
'files'	run check_cmsmms
'notest'	dont rebuild exe/*test*
'pro'	build production executables ( only ) - those required by JETtransp
'noshare'	to suppress rebuild of shared libraries
'nolinkmake'	to suppress make_eqtoks or anything which is shared among architectures
'clean'	only attempt link if libs OK
'vclean'	halt on 1st library build error ( for debugging.. )
'dbg'	sets $CODESYS_DEBUG  (beware excessive output)
'svnupd'	Runs subversion update in each directory, before uplib/uplink. NB this will only work if changes are minor, so the existing makefiles will work
'continue'	Dont update directories a second time (on reruns...) See $SC/svn_cmd

linkcheck accepts a single argument from the above list. linkcheck.csh will parse a string containing more than one argument, separated by  commas or other punctuation characters - eg
                >linkcheck.csh clean,pro

It also halts if an unrecognised argument is encountered, and can provide a list of recognised arguments, if invoked with none.

 References

$NO_XTRANSPIN	suppress xtranspin rebuild if defined
$MPI_ENABLE	enable MPI builds if defined
$CODESYSDIR	$TRANSPROOT/codesys
$WORKSTATION_TYPE
$FORTRAN_VARIANT	select compiler used on Linux

 Calls

lockcheck	lock manager (Disabled - more trouble than its worth)
check_cmsmms	regenerate missing or suspect library makefiles
check_sglib	( if opt = 'everything' / if not nolink ) Rebuilds graphics package
uplib	Builds a library
uplink	Builds an executable

multiple builds

linkcheck.csh first builds libraries in lib.public ( ie those which provide modules required in other libraries) sequentially, then the remainder, submitting up to 5 simultaneous instances of uplib ( or uplink ). Linkcheck.csh creates a working directory (under $LOGDIR) and each uplib or uplink process writes a separate logfile to this directory. The status returns are written to uplib.log or uplink.log. This has two advantages -

• It is easy to find which builds have failed
• It is much easier to locate the associated output for failed builds

When used in conjunction with the svnupd option, considerable time savings may result.

Module handling  See tcomp for details of changes to the .mod file handling, which are required to allow multiple simultaneous builds.

check_cmsmms

Looks for library directories, for which no file libincl/<elem>.dat exists. This file is created by ckmodules, (called by cmsmms). [ This will only check that the makefiles exist - they may not be up to date ?? ]

code_daemon

code_daemon is the main script used to build a new version of Transp, after an update. It runs the code generators, rebuilds the make system, then calls linkcheck.csh to build the executables. This may take many hours.

 Arguments

Since code_daemon does not parse an argument list, only one of these options can be given

 Calls :-

In most cases, problems can be fixed by updating the source code and rerunning linkcheck. Sometimes it is necessary to rerun cmsmms or makelink, if missing source elements (or directories) have been located; it is rarely necessary to rerun code_daemon.

Desperate measures

recompile_all

recompile_all deletes all binaries, al intermediate source files (in compsc) and all .mod files ( codesysdir/mod, /mpi_mod ) If the argument nocomp is given , it exits. Otherwise, it runs make_begin and check_sglib. If argument is "nolinkcheck", it then returns, otherwise it will run linkcheck.csh with the argument string provided..

rebuild_lib, rebuild_prog

rebuild_lib deletes binaries & .mod files associated with a library, then rebuilds it (with uplib). rebuild_prog does the same for an executable (with uplink).