DEEP2 spec2d Cookbook

Now that you have downloaded all of the UCB/DEEP2 and Princeton/SDSS code and calibration files, you are prepared to run the spec2d pipeline. The below text should provide the necessary information for reducing a DEIMOS mask using the pipeline. First, you will need to create a plan file. A plan file consists of a set of keywords followed by values for them, one per line. For instance, the line

          SCIENCENAME: d0911_0041.fits

would specify that the name of one of the science frames is d0911_0041.fits. The code looks for 3 letter keywords with no regard to capitalization: FLAt would work instead of FLATNAME, arc instead of ARCNAME, RAWdirectory for RAWDATADIR, etc. The keyword and value should be separated by one or more spaces or a tab; values for any keyword but CHIPS or SLITS can be followed by comments, so long as the comments are separated from the value by one or more tabs/spaces. For a more complete list of the plan file keywords, continue reading here.

A plan file can be created in 2 ways: (1) generating the plan file using the routine deimos_planfile.pro; (2) generating the plan file by hand. If all of your raw DEIMOS frames (flats, arcs, and science frames) are located in a single directory, then running deimos_planfile is a viable option. To do this, go to the directory in which your raw files reside. Launch IDL from there. Once IDL has started-up, type the following command:

    IDL> deimos_planfile

This routine will scan the headers of all of the FITS files in the directory, attempting to match files according to the mask name specified in the FITS headers. If the required files are found, then the routine will create a plan file, xxxx.plan, where xxxx is the mask name as specified in the FITS headers. The xxxx.plan file will be written to the current working directory, that is, to the raw data directory. If the deimos_planfile routine fails or your raw frames are located in multiple directories, then you can always write/edit the plan file by hand. Here is an example plan file, xxxx.plan. The first two lines are not required; these 2 lines are comment lines (preceeded by a #). It is necessary to specify the mask name. For the sake of consistency, it is best to use the same mask name as given in the FITS headers of the raw DEIMOS frames. To check the mask name given in the header, use the following lines of code:

    IDL> file = 'd0911_0041.fits'
    IDL> hdr = headfits(file)
    IDL> mask = sxpar(hdr, 'SLMSKNAM')
    IDL> print, mask

This will print the name of the mask. Use this as the mask name in your plan file.

On the next line of your plan file, specify the wavelength fitting scheme that you wish to employ (either "poly" or "trace"). See the spec2d primer for more details regarding these different wavelength fits. This line is optional since the "poly" scheme is the default method and in general provides comparable or superior results to that of the "trace" method. The CHIPS field in the plan file allows the user to restrict the number of chips over which the pipeline will run. For testing the pipeline, it is best to simply specify two chips (e.g. 2,6). If this field is omitted, then the pipeline will automatically run on all of the chips. Note that the chips are numbered as follows (where the wavelength direction runs vertically as denoted by the BLUE and RED labels):
    ---------- ---------- ---------- ---------- 
    |        | |        | |        | |        |  
    | Chip 1 | | Chip 2 | | Chip 3 | | Chip 4 | BLUE
    |        | |        | |        | |        |  
    ---------- ---------- ---------- ---------- 
    ---------- ---------- ---------- ---------- 
    |        | |        | |        | |        | 
    | Chip 5 | | Chip 6 | | Chip 7 | | Chip 8 | RED
    |        | |        | |        | |        | 
    ---------- ---------- ---------- ---------- 

The RAWDATADIR field, in conjunction with the DEIMOS_DATA variable (in your .idlenv file), specifies the location of the raw DEIMOS data. The pipeline is able to handle raw frames located in different directories. In the example xxxx.plan file, the calibration files (arcs and flats) are located in the directory "$DEIMOS_DATA/2002sep10" while the science frames are located in the "$DEIMOS_DATA/2002sep11" directory. In this manner, the user is able to switch the RAWDATADIR in the plan file.

Lastly, you must specify the names of your raw frames. Using the fields FLATNAME, ARCNAME, and SCIENCENAME, list the raw DEIMOS FITS files. Presently, the raw data files must be uncompressed in order to pass through the pipeline, but we are working on changes to handle compressed FITS files. You can specify multiple flats, arcs, and science frames. However, in the case of the flats, only the last 3 flats listed will be employed in the reduction. That is, for example, if you list 4 flats in your plan file, then the first one listed goes entirely disregarded.

It is important to note that a user may specify more than one arc frame. Unlike for flats, if multiple arcs are listed then the frames will be combined to make a single arc which is then used by the pipeline to do wavelength fitting. Additional freedom is allowed by using the fields BLUEARCFRAME and REDARCFRAME to specify different arcs for the blue and red portions of the data. Recall that each 2-d slit spectrum covers 2 chips on the DEIMOS detector (one BLUE and one RED). Since certain arc species are better suited for use at the bluer wavelengths, the arc frames denoted as BLUEARCFRAMES will only be used in performing the wavelength solution on the blue side and vice versa for the red. As an example, let's examine the following lines taken from a plan file:

    BLUEARCFRAME: d0814_0001.fits
    BLUEARCFRAME: d0814_0002.fits
    REDARCFRAME: d0814_0003.fits

In this case, the frames d0814_0001.fits and d0814_0002.fits will be combined and used for wavelength fitting on the blue side and d0814_0003.fits will be employed in the red. This feature is very helpful since some arc species may have many distinct, well-spaced lines (good for lambda fitting) in the blue wavelength range, but these same species may also have features at red wavelengths which are saturated or horribly blended.

The final field in a plan file is the LAMPLIST field which allows the user to specify a file containing good arc lines and their corresponding wavelengths for use in wavelength fitting. Since at different grating settings the available lines and ability to resolve those lines varies, the pipeline allows for different lamplists. Presently, 2 lamplists exist: lamp_NIST.dat and lamp_NIST_blue.dat. These ascii-format files can be found in the ~/cvs/spec2d/etc/ directory. For DEEP2-like settings (1200 line/mm grating centered at 7800Å), the lamp_NIST.dat list is recommended. But for observations centered blueward or using a lower-dispersion grating, the lamp_NIST_blue.dat list will likely provide better wavelength fits.

In case more example are needed, refer to these additional example plan files: yyyy.plan and zzzz.plan.


Now, you are ready to run the pipeline. This also can be done in two manners: (1) interactively; (2) via a shell script. To run the pipeline, interactively, simply go to the directory into which you wish to write the output files. From there, start-up IDL in 32-bit mode. Due to pieces of code used in tracing out the edges of the slitlets, the spec2d pipeline must be run in 32-bit mode otherwise it will crash. At the IDL prompt, execute the following command:

    IDL> domask, nlsky=0

Now just sit back and wait for IDL and the spec2d pipeline to do its thing! Note that you must set nlsky=0; the default setting for the pipeline is to do non-local sky subtraction using sky-only slits as employed in the DEEP2 observations.

Alternatively, you can write a shell script to run the pipeline. An example script can be seen, here. Simply, substitute your result directory (where you wish to write the output files) in lieu of result_dir. The lines in red text, are the only lines needed in a simple script. To execute this script, you do not need to start IDL and you do not need to be in the result directory. Just type the following at the unix prompt:

    > source xxxx.sh

where xxxx.sh is the filename of your script. In this case, all of the IDL messages normally printed to the display are written to the file out.log. Note that IDL must be run in 32-bit mode; you can launch IDL in 32-bit mode using the command 'idl-32'.

Running on a machine with a 1.2 GHz processor and 2 GB of RAM, it takes roughly 6 hours to reduce a single DEIMOS mask with 3 science frames and ~125 slitlets.