Category:Post-Processing: Difference between revisions

Overview

The old VLA with its once state of the art, but now dated, correlator and electronic has been turned off. The new electronics and correlator of the EVLA has been turned on and made available to users. For the time being, this availability will be “Shared Risk Observing” of two forms: “Open” (OSRO) with limited capabilities and “Resident” (RSRO) with potentially unlimited capabilities. OSRO data are expected to be similar to those of the VLA initially although with more spectral channels and bandwidth. OSRO data for continuum science will have, in each of 2 tunable spectral windows, 64 spectral channels in each of 4 polarization products covering ≤ 128 MHz. For line work, OSRO data will have one spectral window with 256 spectral channels in both RR and LL polarizations covering ≤ 128 MHz total bandwidth. These capabilities will increase gradually with time. RSRO data may have many thousands of spectral channels and, in time, up to 8 GHz of bandwidth per polarization. AIPS software will be important to both programs, although RSRO data are expected, by management, to be processed primarily in CASA.

At this writing, the EVLA has already produced amazing scientific results, but with considerable difficulties which are expected to be corrected over time. Delays are currently not set accurately, causing the data analysis to begin with the VLBI task FRING needed to correct large slopes in phase across the bandpass. The flagging information known to the on-line system (telescope off source and the like) is now transferred to CASA and so may reach AIPS in the form of already flagged data. That on-line flagging information is not transferred to AIPS in the form of a flag table and neither software package yet receives useful flagging data, such as the antenna has no receiver, from the correlator. Substantial flagging effort is therefore still required. Also, system temperatures and gains are not transferred as yet. This means that the data weights are not meaningful and should not be adjusted by the amplitude calibration. That calibration will not be as good as it will be since corrections for differences in gains and sensitivity in the directions of the primary and secondary calibrators and the target sources cannot be made. The weather table is now available with the data so that reasonable opacities may be determined. However, the “over-the-top”, table which is used in determining antenna positions, the frequency offset table, used in managing Doppler tracking, and the CQ table, used to correct amplitudes for spectral averaging in the presence of non-zero delays, are not available.

Although these issues should be corrected, quite possibly early in 2010, the following guide will not assume that they have been completed. Steps that can be omitted or simplified when they are will be described. This appendix is written with the assumption that the reader is moderately familiar with AIPS as described in the preceding chapters. It is also written with the assumption that you are using the 31DEC10 or later releases of the software.

Getting Your Data into AIPS

Your EVLA data are stored as an “ALMA Science Data Model” (ASDM) format file in “SDMBDF” (Science Data Model Binary Data Format) in the NRAO archive. They may be read out of the archive in that format, a CASA measurement set format, or in an AIPS-friendly uvfits format. This last is produced by the CASA uvfits writing software. Go the the web page:

http://archive.cv.nrao.edu/

and select the Advanced Query Tool. Fill out enough of the form to describe your data and submit the query. If the data are not yet public, you will need the Locked Project Access Key which may be obtained from the NRAO data analysts. To avoid the need for this key, you may log in to my.nrao.edu after which it will know if you are entitled to access to locked projects. The query will return a list of the data sets which meet your specifications. On this form, enter your e-mail address, choose AIPS Friendly names (almost certainly does not work), AIPS FITS under the EVLA-WIDAR section and choose the desired spectral and time averaging. If the delays are not accurately known, spectral averaging can be damaging to the data amplitudes. However, the data are recorded at one-second intervals which is rather short, making the data voluminous. Judicious averaging can help with data set size and processing times without compromising the science. Choose the data set(s) you wish to receive and submit the request. You will be told an estimate of the output data set sizes and the amount of time you will need to wait for the format translation to occur. A 19 Gbyte SDMBDF file run as a test with no averaging was estimated to produce a UVFITS file of 30.26 Gbytes and to take 103 minutes to prepare for download. That time assumes that your download job is the only one being performed. If your download fails, you will probably be told erroneously by e-mail that it worked. The output file will however be missing or incomplete. Try again before contacting NRAO for help. Unlocked files will be downloaded to the NRAO public ftp site:

ftp://ftp.aoc.nrao.edu/e2earchive/

and you may then use ftp to copy the file to your computer. Locked files will go to a protected ftp site and you must use ftp to download those, even within NRAO. The instructions for downloading will be e-mailed to you. Be sure to specify binary for the copy. If you are located in the AOC in Socorro, you may set an environment variable to the archive location, e.g.,

export E2E=/home/acorn2/ftp/pub/e2earchive CR	for bash shells 
setenv E2E /home/acorn2/ftp/pub/e2earchive CR	for C shells such as tcsh

and simply unlocked data files directly from the public download area. Note that the file will be deleted automatically after 48 hours in both public and protected data areas. The data file may be read from disk into AIPS using UVLOD or FITLD, using:

> DEFAULT ’UVLOD’ ; INP CR       to initialize and review the inputs needed.
> DATAIN ’E2E:filename’ CR       where filename is the disk file name in logical area E2E; (see § 3.10.3).
> DOUVCOMP FALSE CR              to write visibilities in uncompressed format. There are no 
                                 weights at present, so there is no loss of information in 
                                 compressed format, but the conversion from compressed format 
                                 costs more than reading the larger data files.
> OUTNA ’myname’ CR              to set the AIPS name.
> OUTCL ’ ’ CR                   to take default (UVDATA) class.
> OUTSEQ 0 CR                    to take next higher sequence #.
> OUTDI 3 CR                     to write the data to disk 3 (one with enough space).
> INP CR                         to review the inputs.
> GO CR                          to run the program when you’re satisfied with inputs.