ImageFileCollection¶

class ccdproc.ImageFileCollection(location=None, keywords=None, find_fits_by_reading=False, filenames=None, glob_include=None, glob_exclude=None, ext=0)[source]¶

Bases: object

Representation of a collection of image files.

The class offers a table summarizing values of keywords in the FITS headers of the files in the collection and offers convenient methods for iterating over the files in the collection. The generator methods use simple filtering syntax and can automate storage of any FITS files modified in the loop using the generator.

Parameters:

location : str or None, optional: Path to directory containing FITS files. Default is None.
keywords : list of str, ‘*’ or None, optional: Keywords that should be used as column headings in the summary table. If the value is or includes ‘*’ then all keywords that appear in any of the FITS headers of the files in the collection become table columns. Default value is ‘*’ unless info_file is specified. Default is None.
find_fits_by_reading: bool, optional: If True, read each file in location to check whether the file is a FITS file and include it in the collection based on that, rather than by file name. Compressed files, e.g. image.fits.gz, will NOT be properly detected. Will be ignored if `filenames` is not ``None``.
filenames: str, list of str, or None, optional: List of the names of FITS files which will be added to the collection. The filenames may either be in location or the name can be a relative or absolute path to the file. Default is None.
glob_include: str or None, optional: Unix-style filename pattern to select filenames to include in the file collection. Can be used in conjunction with glob_exclude to easily select subsets of files in the target directory. Default is None.
glob_exclude: str or None, optional: Unix-style filename pattern to select filenames to exclude from the file collection. Can be used in conjunction with glob_include to easily select subsets of files in the target directory. Default is None.
ext: str or int, optional: The extension from which the header and data will be read in all files.Default is 0.

Raises:

ValueError: Raised if keywords are set to a combination of ‘*’ and any other value.

Attributes Summary

`ext`	str or int, The extension from which the header and data will be read in all files.
`files`	list of str, Unfiltered list of FITS files in location.
`glob_exclude`	str or None, Unix-style filename pattern to select filenames to exclude in the file collection.
`glob_include`	str or None, Unix-style filename pattern to select filenames to include in the file collection.
`keywords`	list of str, Keywords currently in the summary table.
`location`	str, Path name to directory containing FITS files.
`summary`	`Table` of values of FITS keywords for files in the collection.

Methods Summary

`ccds`(self[, ccd_kwargs])	Generator that yields each CCDData in the collection.
`data`(self[, do_not_scale_image_data])	Generator that yields each image in the collection.
`files_filtered`(self, \\kwd)	Determine files whose keywords have listed values.
`filter`(self, \\kwd)	Create a new collection by filtering the current collection.
`hdus`(self[, do_not_scale_image_data])	Generator that yields each HDUList in the collection.
`headers`(self[, do_not_scale_image_data])	Generator that yields each header in the collection.
`refresh`(self)	Refresh the collection by re-reading headers.
`sort`(self, keys)	Sort the list of files to determine the order of iteration.
`values`(self, keyword[, unique])	List of values for a keyword.

Attributes Documentation

ext¶: str or int, The extension from which the header and data will be read in all files.

files¶: list of str, Unfiltered list of FITS files in location.

glob_exclude¶: str or None, Unix-style filename pattern to select filenames to exclude in the file collection.

glob_include¶: str or None, Unix-style filename pattern to select filenames to include in the file collection.

keywords¶

list of str, Keywords currently in the summary table.

Setting the keywords causes the summary table to be regenerated unless the new keywords are a subset of the old.

Changed in version 1.3: Added deleter for keywords property.

location¶: str, Path name to directory containing FITS files.

summary¶

Table of values of FITS keywords for files in the collection.

Each keyword is a column heading. In addition, there is a column called file that contains the name of the FITS file. The directory is not included as part of that name.

The first column is always named file.

The order of the remaining columns depends on how the summary was constructed.

If a wildcard, * was used then the order is the order in which the keywords appear in the FITS files from which the summary is constructed.

If an explicit list of keywords was supplied in setting up the collection then the order of the columns is the order of the keywords.

Methods Documentation

ccds(self, ccd_kwargs=None, **kwd)[source]¶

Generator that yields each CCDData in the collection.

If any of the parameters save_with_name, save_location or overwrite evaluates to True the generator will write a copy of each FITS file it is iterating over. In other words, if save_with_name and/or save_location is a string with non-zero length, and/or overwrite is True, a copy of each FITS file will be made.

Parameters:

save_with_name : str, optional: string added to end of file name (before extension) if FITS file should be saved after iteration. Unless save_location is set, files will be saved to location of the source files self.location. Default is ''.
save_location : str, optional: Directory in which to save FITS files; implies that FITS files will be saved. Note this provides an easy way to copy a directory of files–loop over the CCDData with save_location set. Default is ''.
overwrite : bool, optional: If True, overwrite input FITS files. Default is False.
clobber : bool, optional: Alias for overwrite. Default is False.
do_not_scale_image_data : bool, optional: If True, prevents fits from scaling images. Default is True. Default is True.
return_fname : bool, optional: If True, return the tuple (header, file_name) instead of just header. The file name returned is the name of the file only, not the full path to the file. Default is False.
ccd_kwargs : dict, optional: Dict with parameters for fits_ccddata_reader. For instance, the key 'unit' can be used to specify the unit of the data. If 'unit' is not given then 'adu' is used as the default unit. See fits_ccddata_reader for a complete list of parameters that can be passed through ccd_kwargs.
regex_match : bool, keyword-only: If True, then string values in the **kwd dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive.
**kwd :: Any additional keywords are used to filter the items returned; see files_filtered examples for details.

Returns:

astropy.nddata.CCDData: If return_fname is False, yield the next CCDData in the collection.
(astropy.nddata.CCDData, str): If return_fname is True, yield a tuple of (CCDData, file name) for the next item in the collection.

data(self, do_not_scale_image_data=False, **kwd)[source]¶

Generator that yields each image in the collection.

If any of the parameters save_with_name, save_location or overwrite evaluates to True the generator will write a copy of each FITS file it is iterating over. In other words, if save_with_name and/or save_location is a string with non-zero length, and/or overwrite is True, a copy of each FITS file will be made.

Parameters:

save_with_name : str, optional: string added to end of file name (before extension) if FITS file should be saved after iteration. Unless save_location is set, files will be saved to location of the source files self.location. Default is ''.
save_location : str, optional: Directory in which to save FITS files; implies that FITS files will be saved. Note this provides an easy way to copy a directory of files–loop over the image with save_location set. Default is ''.
overwrite : bool, optional: If True, overwrite input FITS files. Default is False.
clobber : bool, optional: Alias for overwrite. Default is False.
do_not_scale_image_data : bool, optional: If True, prevents fits from scaling images. Default is False. Default is True.
return_fname : bool, optional: If True, return the tuple (header, file_name) instead of just header. The file name returned is the name of the file only, not the full path to the file. Default is False.
ccd_kwargs : dict, optional: Dict with parameters for fits_ccddata_reader. For instance, the key 'unit' can be used to specify the unit of the data. If 'unit' is not given then 'adu' is used as the default unit. See fits_ccddata_reader for a complete list of parameters that can be passed through ccd_kwargs.
regex_match : bool, keyword-only: If True, then string values in the **kwd dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive.
**kwd :: Any additional keywords are used to filter the items returned; see files_filtered examples for details.

Returns:

numpy.ndarray: If return_fname is False, yield the next image in the collection.
(numpy.ndarray, str): If return_fname is True, yield a tuple of (image, file name) for the next item in the collection.

files_filtered(self, **kwd)[source]¶

Determine files whose keywords have listed values.

Parameters:

include_path : bool, keyword-only: If the keyword include_path=True is set, the returned list contains not just the filename, but the full path to each file. Default is False.
regex_match : bool, keyword-only: If True, then string values in the **kwd dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive.
**kwd :: **kwd is dict of keywords and values the files must have. The value ‘*’ represents any value. A missing keyword is indicated by value ‘’.

Returns:

filenames : list: The files that satisfy the keyword-value restrictions specified by the **kwd.

Notes

Value comparison is case insensitive for strings, whether matching exactly or matching with regular expressions.

Examples

Some examples for filtering:

>>> keys = ['imagetyp','filter']
>>> collection = ImageFileCollection('test/data', keywords=keys)
>>> collection.files_filtered(imagetyp='LIGHT', filter='R')
>>> collection.files_filtered(imagetyp='*', filter='')

In case you want to filter with keyword names that cannot be used as keyword argument name, you have to unpack them using a dictionary. For example if a keyword name contains a space or a -:

>>> add_filters = {'exp-time': 20, 'ESO TPL ID': 1050}
>>> collection.files_filtered(imagetyp='LIGHT', **add_filters)

filter(self, **kwd)[source]¶

Create a new collection by filtering the current collection.

Parameters:	regex_match : bool, keyword-only If `True`, then string values in the `kwd` dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive. kwd : `*kwd` is dict of keywords and values the files must have. The value ‘’ represents any value. A missing keyword is indicated by value ‘’.
Returns:	`ImageFileCollection` A new collection with the files matched by the arguments to filter.

hdus(self, do_not_scale_image_data=False, **kwd)[source]¶

Generator that yields each HDUList in the collection.

If any of the parameters save_with_name, save_location or overwrite evaluates to True the generator will write a copy of each FITS file it is iterating over. In other words, if save_with_name and/or save_location is a string with non-zero length, and/or overwrite is True, a copy of each FITS file will be made.

Parameters:

save_with_name : str, optional: string added to end of file name (before extension) if FITS file should be saved after iteration. Unless save_location is set, files will be saved to location of the source files self.location. Default is ''.
save_location : str, optional: Directory in which to save FITS files; implies that FITS files will be saved. Note this provides an easy way to copy a directory of files–loop over the HDUList with save_location set. Default is ''.
overwrite : bool, optional: If True, overwrite input FITS files. Default is False.
clobber : bool, optional: Alias for overwrite. Default is False.
do_not_scale_image_data : bool, optional: If True, prevents fits from scaling images. Default is False. Default is True.
return_fname : bool, optional: If True, return the tuple (header, file_name) instead of just header. The file name returned is the name of the file only, not the full path to the file. Default is False.
ccd_kwargs : dict, optional: Dict with parameters for fits_ccddata_reader. For instance, the key 'unit' can be used to specify the unit of the data. If 'unit' is not given then 'adu' is used as the default unit. See fits_ccddata_reader for a complete list of parameters that can be passed through ccd_kwargs.
regex_match : bool, keyword-only: If True, then string values in the **kwd dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive.
**kwd :: Any additional keywords are used to filter the items returned; see files_filtered examples for details.

Returns:

astropy.io.fits.HDUList: If return_fname is False, yield the next HDUList in the collection.
(astropy.io.fits.HDUList, str): If return_fname is True, yield a tuple of (HDUList, file name) for the next item in the collection.

headers(self, do_not_scale_image_data=True, **kwd)[source]¶

Generator that yields each header in the collection.

If any of the parameters save_with_name, save_location or overwrite evaluates to True the generator will write a copy of each FITS file it is iterating over. In other words, if save_with_name and/or save_location is a string with non-zero length, and/or overwrite is True, a copy of each FITS file will be made.

Parameters:

save_with_name : str, optional: string added to end of file name (before extension) if FITS file should be saved after iteration. Unless save_location is set, files will be saved to location of the source files self.location. Default is ''.
save_location : str, optional: Directory in which to save FITS files; implies that FITS files will be saved. Note this provides an easy way to copy a directory of files–loop over the header with save_location set. Default is ''.
overwrite : bool, optional: If True, overwrite input FITS files. Default is False.
clobber : bool, optional: Alias for overwrite. Default is False.
do_not_scale_image_data : bool, optional: If True, prevents fits from scaling images. Default is True. Default is True.
return_fname : bool, optional: If True, return the tuple (header, file_name) instead of just header. The file name returned is the name of the file only, not the full path to the file. Default is False.
ccd_kwargs : dict, optional: Dict with parameters for fits_ccddata_reader. For instance, the key 'unit' can be used to specify the unit of the data. If 'unit' is not given then 'adu' is used as the default unit. See fits_ccddata_reader for a complete list of parameters that can be passed through ccd_kwargs.
regex_match : bool, keyword-only: If True, then string values in the **kwd dictionary are treated as regular expression patterns and matching is done by regular expression search. The search is always case insensitive.
**kwd :: Any additional keywords are used to filter the items returned; see files_filtered examples for details.

Returns:

astropy.io.fits.Header: If return_fname is False, yield the next header in the collection.
(astropy.io.fits.Header, str): If return_fname is True, yield a tuple of (header, file name) for the next item in the collection.

refresh(self)[source]¶: Refresh the collection by re-reading headers.

sort(self, keys)[source]¶

Sort the list of files to determine the order of iteration.

Sort the table of files according to one or more keys. This does not create a new object, instead is sorts in place.

Parameters:	keys : str, list of str The key(s) to order the table by.

values(self, keyword, unique=False)[source]¶

List of values for a keyword.

Parameters:	keyword : str Keyword (i.e. table column) for which values are desired. unique : bool, optional If True, return only the unique values for the keyword. Default is `False`.
Returns:	list Values as a list.

Navigation

ImageFileCollection¶