Category:Post-Processing
Special Considerations for EVLA Data Calibration and Imaging in AIPS
Overview
The old VLA with its once state of the art, but now dated, correlator and electronic has been turned off. The new electronics and correlator of the EVLA has been turned on and made available to users. For the time being, this availability will be “Shared Risk Observing” of two forms: “Open” (OSRO) with limited capabilities and “Resident” (RSRO) with potentially unlimited capabilities. OSRO data are expected to be similar to those of the VLA initially although with more spectral channels and bandwidth. OSRO data for continuum science will have, in each of 2 tunable spectral windows, 64 spectral channels in each of 4 polarization products covering ≤ 128 MHz. For line work, OSRO data will have one spectral window with 256 spectral channels in both RR and LL polarizations covering ≤ 128 MHz total bandwidth. These capabilities will increase gradually with time. RSRO data may have many thousands of spectral channels and, in time, up to 8 GHz of bandwidth per polarization. AIPS software will be important to both programs, although RSRO data are expected, by management, to be processed primarily in CASA.
At this writing, the EVLA has already produced amazing scientific results, but with considerable difficulties which are expected to be corrected over time. Delays are currently not set accurately, causing the data analysis to begin with the VLBI task FRING needed to correct large slopes in phase across the bandpass. The flagging information known to the on-line system (telescope off source and the like) is now transferred to CASA and so may reach AIPS in the form of already flagged data. That on-line flagging information is not transferred to AIPS in the form of a flag table and neither software package yet receives useful flagging data, such as the antenna has no receiver, from the correlator. Substantial flagging effort is therefore still required. Also, system temperatures and gains are not transferred as yet. This means that the data weights are not meaningful and should not be adjusted by the amplitude calibration. That calibration will not be as good as it will be since corrections for differences in gains and sensitivity in the directions of the primary and secondary calibrators and the target sources cannot be made. The weather table is now available with the data so that reasonable opacities may be determined. However, the “over-the-top”, table which is used in determining antenna positions, the frequency offset table, used in managing Doppler tracking, and the CQ table, used to correct amplitudes for spectral averaging in the presence of non-zero delays, are not available.
Although these issues should be corrected, quite possibly early in 2010, the following guide will not assume that they have been completed. Steps that can be omitted or simplified when they are will be described. This appendix is written with the assumption that the reader is moderately familiar with AIPS as described in the preceding chapters. It is also written with the assumption that you are using the 31DEC10 or later releases of the software.
Getting Your Data into AIPS
Your EVLA data are stored as an “ALMA Science Data Model” (ASDM) format file in “SDMBDF” (Science Data Model Binary Data Format) in the NRAO archive. They may be read out of the archive in that format, a CASA measurement set format, or in an AIPS-friendly uvfits format. This last is produced by the CASA uvfits writing software. Go the the web page:
and select the Advanced Query Tool. Fill out enough of the form to describe your data and submit the query. If the data are not yet public, you will need the Locked Project Access Key which may be obtained from the NRAO data analysts. To avoid the need for this key, you may log in to my.nrao.edu after which it will know if you are entitled to access to locked projects. The query will return a list of the data sets which meet your specifications. On this form, enter your e-mail address, choose AIPS Friendly names (almost certainly does not work), AIPS FITS under the EVLA-WIDAR section and choose the desired spectral and time averaging. If the delays are not accurately known, spectral averaging can be damaging to the data amplitudes. However, the data are recorded at one-second intervals which is rather short, making the data voluminous. Judicious averaging can help with data set size and processing times without compromising the science. Choose the data set(s) you wish to receive and submit the request. You will be told an estimate of the output data set sizes and the amount of time you will need to wait for the format translation to occur. A 19 Gbyte SDMBDF file run as a test with no averaging was estimated to produce a UVFITS file of 30.26 Gbytes and to take 103 minutes to prepare for download. That time assumes that your download job is the only one being performed. If your download fails, you will probably be told erroneously by e-mail that it worked. The output file will however be missing or incomplete. Try again before contacting NRAO for help. Unlocked files will be downloaded to the NRAO public ftp site:
ftp://ftp.aoc.nrao.edu/e2earchive/
and you may then use ftp to copy the file to your computer. Locked files will go to a protected ftp site and you must use ftp to download those, even within NRAO. The instructions for downloading will be e-mailed to you. Be sure to specify binary for the copy. If you are located in the AOC in Socorro, you may set an environment variable to the archive location, e.g.,
export E2E=/home/acorn2/ftp/pub/e2earchive CR for bash shells
setenv E2E /home/acorn2/ftp/pub/e2earchive CR for C shells such as tcsh
and simply unlocked data files directly from the public download area. Note that the file will be deleted automatically after 48 hours in both public and protected data areas. The data file may be read from disk into AIPS using UVLOD or FITLD, using:
> DEFAULT ’UVLOD’ ; INP CR to initialize and review the inputs needed.
> DATAIN ’E2E:filename’ CR where filename is the disk file name in logical area E2E; (see § 3.10.3).
> DOUVCOMP FALSE CR to write visibilities in uncompressed format. There are no
weights at present, so there is no loss of information in
compressed format, but the conversion from compressed format
costs more than reading the larger data files.
> OUTNA ’myname’ CR to set the AIPS name.
> OUTCL ’ ’ CR to take default (UVDATA) class.
> OUTSEQ 0 CR to take next higher sequence #.
> OUTDI 3 CR to write the data to disk 3 (one with enough space).
> INP CR to review the inputs.
> GO CR to run the program when you’re satisfied with inputs.
Watch the messages from UVLOD to see where your data set goes and whether the task ran properly. When it is finished, check the output header:
> INDI n; GETN m CR to select the data set on disk n and catalog number m.
> IMHEAD CR to examine the header.
Note that the header does not show the usual complement of AIPS extension files. CASA translates the on-line data into its internal format and then writes the uvfits file read by AIPS. Since CASA does not have files comparable to AIPS index and CL tables, it does not provide them. To build index and calibration tables, use;
> TASK ’INDXR’ ; INP CR to select the task and review its inputs.
> INFILE ’ ’;PRTLEV=0 CR to be sure not to use an input text file and to avoid excess messages.
> CPARM=0,0,1/2 CR to make a CL table 1 with a 30-second interval.
> BPARM τ , 0 CR to take default VLA gains and a zenith opacity of τ . Set τ = −1
for no opacity correction. You may set τ = 0, which is now
recommended, to get new default opacities. These are based on
a detailed model predicting the opacity at any frequency from
that at 22 GHz. The combination of weather and seasonal
model long used by FILLM and INDXR is now used solely to
estimate the 22 GHz opacity.
> GO CR to run the task after checking the inputs.
It is a good idea to list the structure of your data set and your antenna locations on the printer and to keep those listings next to your work station for reference:
> DEFAULT LISTR ; INP CR to initialize the LISTR inputs and review them.
> INDI n; GETN m CR to select the data set on disk n and catalog number m.
> OPTYPE ’SCAN’ ; DOCRT -1 CR to choose a scan listing on the printer.
> GO ; GO PRTAN CR to print the scan listing and the antenna file contents.
Read these with care. There have sometimes been problems with antenna identifications, with the order of the IF frequencies, and even with identification of sources by scan. Task SUFIX may be used to correct the last problem and, if desired, FLOPM may be used to reverse the frequency order. You may have to use SETJY to change the CALCODE of some sources if your calibration sources have a blank calibrator code or your target sources have a non-blank calibrator code.
Initial Calibration FRING
We have had difficulty setting all of the delays in the EVLA to values which are sufficiently accurate. If the delay is not set correctly, the interferometer phase will vary linearly with frequency, potentially wrapping through several turns of phase within a single spectral window (“IF band”). We hope that bad delays will not arise in future, allowing you to skip this section. But this is a problem familiar to VLBI users and AIPS has a well-tested method to correct the problem. Using your LISTR output, select a time range of about one minute toward the end of a scan on a strong point-source calibrator, usually your bandpass calibrator. Then
> DEFAULT FRING ; INP CR to initialize the FRING inputs and review them.
> INDI n; GETN m CR to select the data set on disk n and catalog number m.
> TIMERANG db,hb,mb,sb,de,he,me,se CR to specify the beginning day, hour, minute, and second and
ending day, hour, minute, and second (wrt REFDATE) of the
data to be included.
> SOLINT 1.05 CR to set the averaging interval in minutes slightly longer than the
data interval selected.
> DPARM(9) = 1 CR to fit only delay, not rate.
> INP CR to check the voluminous inputs.
> GO to run the task, writing SN table 1 with delays for each antenna,
IF, and polarization.
to specify the beginning day, hour, minute, and second and ending day, hour, minute, and second (wrt REFDATE) of the data to be included. Too much data will cause trouble. to set the averaging interval in minutes slightly longer than the data interval selected. to fit only delay, not rate. to check the voluminous inputs. to run the task, writing SN table 1 with delays for each antenna, IF, and polarization. The different IFs in current EVLA data sets may come from different basebands and therefore have different residual delays. The option APARM(5)=1 to force all IFs to have the same delay solution is therefore no longer appropriate. This SN table will need to be applied to the main CL table created by INDXR.
> TASK ’CLCAL’ ; INP CR to look at the necessary inputs.
> TIMERANG 0 CR to reset the time range.
> GAINUSE 0 ; GAINVER 0 CR to select the highest CL table as input and write one higher as
output (version 1 and 2, resp. in this case).
> SNVER 1 ; INVER 1 CR to use only the SN table just created.
> INP CR to review the inputs.
> GO CR to make an update calibration table.
Be sure to apply this (or higher) CL table with DOCALIB 1 in all later steps.
Initial Editing
There will be data validity information prepared both by the on-line control software and by the WIDAR correlator and this information will in time be available as an initial flag table. The tasks above will have applied this table for you by default since FLAGVER 0. On-line flags may already have caused data to be flagged within your data set (but not deleted) by CASA. Unfortunately, at this writing, no flag table is present and, even when it does appear, it may not be fully reliable initially. Thus, we need to look at the data to flag out whatever remains of the time off source not flagged in CASA using on-line flagging information. There have also been drop outs in which the visibility is pure zero, typically for all channels and IFs and a single integration. The drop outs should now be handled by UVLOD and FITLD. Note, however, that CASA and FITLD pass along all data samples, including those that are fully flagged. This makes the data set rather larger than one might wish. Use UVCOP to remove all fully flagged data samples. Before doing this, use TVFLG to look for any more data samples that might need to be flagged fully. Check especially samples at the beginnings and ends of scans. Try
Basic Calibration
Target Source Data -- Edit and SPLIT
Spectral-line Imaging Hints
Continuum Imaging Hints
Concluding Remakrs, References, Pictures
AIPS itself, and particularly this appendix, do not begin to cover all of the issues that will arise with EVLA data. The increased bandwidth will probably cause the polarization calibration to change from one complex number per antenna per IF to a complete complex spectrum much like a bandpass. This will force major revisions to the AIPS code which deals with this area. See § 4.6 for information about current AIPS methods of polarization calibration, which average all spectral channels within an IF except for those which are flagged.
The increased sensitivity of the EVLA will means that imaging will no longer be able to ignore effects that are difficult to correct such as pointing errors, beam squint, variable antenna polarization across the field, leakage of polarized signal into the parallel-hand visibilities, etc., etc. These are research topics which may have solutions in other software packages such as OBIT and CASA.
