CCD Data reduction (ccdproc)

Introduction

Note

ccdproc works only with astropy version 1.0 or later.

The ccdproc package provides:

  • An image class, CCDData, that includes an uncertainty for the data, units and methods for performing arithmetic with images including the propagation of uncertainties.
  • A set of functions performing common CCD data reduction steps (e.g. dark subtraction, flat field correction) with a flexible mechanism for logging reduction steps in the image metadata.
  • A function for reprojecting an image onto another WCS, useful for stacking science images. The actual reprojection is done by the reproject package.
  • A class for combining and/or clipping images, Combiner, and associated functions.
  • A class, ImageFileCollection, for working with a directory of images.

Getting Started

A CCDData object can be created from a numpy array (masked or not) or from a FITS file:

>>> import numpy as np
>>> from astropy import units as u
>>> import ccdproc
>>> image_1 = ccdproc.CCDData(np.ones((10, 10)), unit="adu")

An example of reading from a FITS file is image_2 = astropy.nddata.CCDData.read('my_image.fits', unit="electron") (the electron unit is defined as part of ccdproc).

The metadata of a CCDData object may be any dictionary-like object, including a FITS header. When a CCDData object is initialized from FITS file its metadata is a FITS header.

The data is accessible either by indexing directly or through the data attribute:

>>> sub_image = image_1[:, 1:-3]  # a CCDData object
>>> sub_data =  image_1.data[:, 1:-3]  # a numpy array

See the documentation for CCDData for a complete list of attributes.

Most operations are performed by functions in ccdproc:

>>> dark = ccdproc.CCDData(np.random.normal(size=(10, 10)), unit="adu")
>>> dark_sub = ccdproc.subtract_dark(image_1, dark,
...                                  dark_exposure=30*u.second,
...                                  data_exposure=15*u.second,
...                                  scale=True)

See the documentation for subtract_dark for more compact ways of providing exposure times.

Every function returns a copy of the data with the operation performed.

Every function in ccdproc supports logging through the addition of information to the image metadata.

Logging can be simple – add a string to the metadata:

>>> dark_sub_gained = ccdproc.gain_correct(dark_sub, 1.5 * u.photon/u.adu, add_keyword='gain_corrected')

Logging can be more complicated – add several keyword/value pairs by passing a dictionary to add_keyword:

>>> my_log = {'gain_correct': 'Gain value was 1.5',
...           'calstat': 'G'}
>>> dark_sub_gained = ccdproc.gain_correct(dark_sub,
...                                        1.5 * u.photon/u.adu,
...                                        add_keyword=my_log)

You might wonder why there is a gain_correct at all, since the implemented gain correction simple multiplies by a constant. There are two things you get with gain_correct that you do not get with multiplication:

  • Appropriate scaling of uncertainties.
  • Units

The same advantages apply to operations that are more complex, like flat correction, in which one image is divided by another:

>>> flat = ccdproc.CCDData(np.random.normal(1.0, scale=0.1, size=(10, 10)),
...                        unit='adu')
>>> image_1_flat = ccdproc.flat_correct(image_1, flat)

In addition to doing the necessary division, flat_correct propagates uncertainties (if they are set).

The function wcs_project allows you to reproject an image onto a different WCS.

To make applying the same operations to a set of files in a directory easier, use an ImageFileCollection. It constructs, given a directory, a Table containing the values of user-selected keywords in the directory. It also provides methods for iterating over the files. The example below was used to find an image in which the sky background was high for use in a talk:

>>> from __future__ import division, print_function
>>> from ccdproc import ImageFileCollection
>>> import numpy as np
>>> from glob import glob
>>> dirs = glob('/Users/mcraig/Documents/Data/feder-images/fixed_headers/20*-??-??')
>>> for d in dirs:
...     print(d)
...     ic = ImageFileCollection(d, keywords='*')
...     for data, fname in ic.data(imagetyp='LIGHT', return_fname=True):
...         if data.mean() > 4000.:
...             print(fname)

ccdproc Package

The ccdproc package is a collection of code that will be helpful in basic CCD processing. These steps will allow reduction of basic CCD data as either a stand-alone processing or as part of a pipeline.

Functions

background_deviation_box(data, bbox) Determine the background deviation with a box size of bbox.
background_deviation_filter(data, bbox) Determine the background deviation for each pixel from a box with size of bbox.
bitfield_to_boolean_mask(bitfield[, ...]) Convert an integer bit field to a boolean mask.
block_average(ccd, block_size) Like block_reduce but with predefined func=np.mean.
block_reduce(ccd, block_size[, func]) Thin wrapper around astropy.nddata.block_reduce.
block_replicate(ccd, block_size[, conserve_sum]) Thin wrapper around astropy.nddata.block_replicate.
ccd_process(ccd[, oscan, trim, error, ...]) Perform basic processing on ccd data.
ccdmask(ratio[, findbadcolumns, byblocks, ...]) Uses method based on the IRAF ccdmask task to generate a mask based on the given input.
combine(img_list[, output_file, method, ...]) Convenience function for combining multiple images.
cosmicray_lacosmic(ccd[, sigclip, sigfrac, ...]) Identify cosmic rays through the lacosmic technique.
cosmicray_median(ccd[, error_image, thresh, ...]) Identify cosmic rays through median technique.
create_deviation(ccd_data[, gain, ...]) Create a uncertainty frame.
flat_correct(ccd, flat[, min_value, ...]) Correct the image for flat fielding.
gain_correct(ccd, gain[, gain_unit, add_keyword]) Correct the gain in the image.
median_filter(data, *args, **kwargs) See scipy.ndimage.median_filter for arguments.
rebin(*args, **kwargs)

Deprecated since version 1.1.

sigma_func(arr[, axis]) Robust method for calculating the deviation of an array.
subtract_bias(ccd, master[, add_keyword]) Subtract master bias from image.
subtract_dark(ccd, master[, dark_exposure, ...]) Subtract dark current from an image.
subtract_overscan(ccd[, overscan, ...]) Subtract the overscan region from an image.
test([package, test_path, args, plugins, ...]) Run the tests using py.test.
transform_image(ccd, transform_func[, ...]) Transform the image.
trim_image(ccd[, fits_section, add_keyword]) Trim the image to the dimensions indicated.
wcs_project(ccd, target_wcs[, target_shape, ...]) Given a CCDData image with WCS, project it onto a target WCS and return the reprojected data as a new CCDData image.

Classes

Combiner(ccd_list[, dtype]) A class for combining CCDData objects.
ImageFileCollection([location, keywords, ...]) Representation of a collection of image files.
Keyword(name[, unit, value])

Class Inheritance Diagram

Inheritance diagram of ccdproc.combiner.Combiner, ccdproc.image_collection.ImageFileCollection, ccdproc.core.Keyword

ccdproc.utils.slices Module

Define utility functions and classes for ccdproc

Functions

slice_from_string(string[, fits_convention]) Convert a string to a tuple of slices.