Processing Raster and Vector files with hubdc.applier

Basic tools for setting up a function to be applied over a raster processing chain. The Applier class is the main point of entry in this module.

See Applier Examples for more information.

class hubdc.applier.Applier(controls=None)

Bases: object

apply(operator=None, description=None, overwrite=True, *ufuncArgs, **ufuncKwargs)

Applies the operator blockwise over a raster processing chain and returns a list of results, one for each block.

The operator must be a subclass of ApplierOperator and needs to implement the ufunc() method to specify the image processing.

For example:

class MyOperator(ApplierOperator):
    def ufunc(self):
        # process the data

applier.apply(operator=MyOperator)

or:

def my_ufunc(operator):
    # process the data

applier.apply(operator=my_ufunc)

Parameters:

• operator (ApplierOperator or function) – applier operator
• description – short description that is displayed on the progress bar
• ufuncArgs – additional arguments that will be passed to the operators ufunc() method.
• ufuncKwargs – additional keyword arguments that will be passed to the operators ufunc() method.

Returns:

list of results, one for each processed block

mainGrid

class hubdc.applier.ApplierControls

Bases: object

setAutoFootprint(footprintType='union')

Derive extent of the reference pixel grid from input files. Possible options are ‘union’ or ‘intersect’.

setAutoResolution(resolutionType='minimun')

Derive resolution of the reference pixel grid from input files. Possible options are ‘minimum’, ‘maximum’ or ‘average’.

setCreateEnviHeader(createEnviHeader=True)

Set to True to create additional ENVI header files for all output rasters. The header files store all metadata items from the GDAL PAM ENVI domain, so that the images can be correctly interpreted by the ENVI software. Currently only the native ENVI format and the GTiff format is supported.

setDisableWindowTrimToFootprint(disable=False)

setFootprint(xMin=None, xMax=None, yMin=None, yMax=None)

Set spatial footprint of the reference pixel grid.

setGDALCacheMax(bytes=104857600)

For details see the GDAL_CACHEMAX Configuration Option.

setGDALDisableReadDirOnOpen(disable=True)

For details see the GDAL_DISABLE_READDIR_ON_OPEN Configuration Option.

setGDALMaxDatasetPoolSize(nfiles=100)

For details see the GDAL_MAX_DATASET_POOL_SIZE Configuration Option.

setGDALSwathSize(bytes=104857600)

For details see the GDAL_SWATH_SIZE Configuration Option.

setGDALWarpErrorThreshold(errorThreshold=0.0)

setGDALWarpMemoryLimit(memoryLimit=104857600)

setGDALWarpMultithread(multithread=False)

setNumThreads(nworker=None)

Set the number of pool worker for multiprocessing. Set to None to disable multiprocessing (recommended for debugging).

setNumWriter(nwriter=None)

Set the number of writer processes. Set to None to disable multiwriting (recommended for debugging).

setProgressBar(progressBar=None)

Set the progress display object. Default is an CUIProgress object. For suppressing outputs use an SilentProgress object

setProjection(projection=None)

Set projection of the reference pixel grid.

setReferenceGrid(grid=None)

Set the reference pixel grid. Pass an instance of the PixelGrid class.

setReferenceGridByImage(filename=None)

Set an image defining the reference pixel grid.

setReferenceGridByVector(filename, xRes, yRes, layerNameOrIndex=0)

Set a vector layer defining the reference pixel grid footprint and projection.

setResolution(xRes=None, yRes=None)

Set resolution of the reference pixel grid.

setWindowFullSize()

Set the block size to full extent.

setWindowXSize(windowxsize=256)

Set the X size of the blocks used. Images are processed in blocks (windows) of ‘windowxsize’ columns, and ‘windowysize’ rows.

setWindowYSize(windowysize=256)

Set the Y size of the blocks used. Images are processed in blocks (windows) of ‘windowxsize’ columns, and ‘windowysize’ rows.

DEFAULT_CREATEENVIHEADER = True

DEFAULT_FOOTPRINTTYPE = 'union'

DEFAULT_GDALCACHEMAX = 104857600

DEFAULT_GDALDISABLEREADDIRONOPEN = True

DEFAULT_GDALMAXDATASETPOOLSIZE = 100

DEFAULT_GDALSWATHSIZE = 104857600

DEFAULT_NWORKER = None

DEFAULT_NWRITER = None

DEFAULT_RESOLUTIONTYPE = 'minimun'

DEFAULT_WINDOWXSIZE = 256

DEFAULT_WINDOWYSIZE = 256

class hubdc.applier.ApplierIOGroup

Bases: object

getGroup(key)

setGroup(key, value)

operator

class hubdc.applier.ApplierInputRaster(filename)

classmethod fromDataset(dataset)

getBandArray(indicies, overlap=0, resampleAlg=<Mock name='mock.gdal.GRA_NearestNeighbour' id='139778651126096'>, noData=None, errorThreshold=0.0, warpMemoryLimit=104857600, multithread=False)

getFractionArray(categories, overlap=0, noData=None, index=None)

getImageArray(overlap=0, resampleAlg=<Mock name='mock.gdal.GRA_NearestNeighbour' id='139778651126096'>, noData=None, errorThreshold=0.0, warpMemoryLimit=104857600, multithread=False)

getMetadataDict()

Return metadata dictionary.

getMetadataItem(key, domain)

Return a metadata item.

getNoDataValue(default=None)

Return image no data value.

getNoDataValues(default=None)

Return band no data values

dataset

operator

class hubdc.applier.ApplierInputRasterArchive(folder, extensions, filter=None, overwrite=False)

Bases: object

getIntersection()

operator

class hubdc.applier.ApplierInputRasterArchiveGroup

Bases: hubdc.applier.ApplierIOGroup

getArchive(key)

setArchive(key, value)

class hubdc.applier.ApplierInputRasterGroup

Bases: hubdc.applier.ApplierIOGroup

findRaster(endswith=None)

getFlatRasterKeys()

getFlatRasters()

getGroupKeys()

getGroups()

getRaster(key)

getRasterKeys()

getRasters()

setRaster(key, value)

class hubdc.applier.ApplierInputVector(filename, layerNameOrIndex=0)

Bases: object

getFractionArray(categories, categoryAttribute=None, oversampling=10, overlap=0)

Returns aggregated category fractions of the current block in form of a 3d numpy array.

Parameters:

• categories – list of categories (numbers or names)
• categoryAttribute – attribute field on the features holding the categories
• oversampling – factor defining the relative degree of rasterization detail compared to the target resolution. If for example the target resolution is 30m and the oversampling factor is 10, then the categories are first rasterized at 3m, and finally aggregated to the target resolution.
• overlap – the amount of margin (number of pixels) added to the image data block in each direction, so that the blocks will overlap; this is important for spatial operators like filters.

getImageArray(initValue=0, burnValue=1, burnAttribute=None, allTouched=False, filterSQL=None, overlap=0, dtype=<Mock name='mock.float32' id='139778651126544'>)

Returns the vector rasterization of the current block in form of a 3-d numpy array.

Parameters:

• initValue – value to pre-initialize the output array
• burnValue – value to burn into the output array for all objects; exclusive with burnAttribute
• burnAttribute – identifies an attribute field on the features to be used for a burn-in value; exclusive with burnValue
• allTouched – whether to enable that all pixels touched by lines or polygons will be updated, not just those on the line render path, or whose center point is within the polygon
• filterSQL – set an SQL WHERE clause which will be used to filter vector features
• overlap – the amount of margin (number of pixels) added to the image data block in each direction, so that the blocks will overlap; this is important for spatial operators like filters.

layer

Return the Layer object

operator

Return the parent ApplierOperator object

class hubdc.applier.ApplierInputVectorGroup

Bases: hubdc.applier.ApplierIOGroup

getFlatVectorKeys()

getFlatVectors()

getVector(key)

setVector(key, value)

class hubdc.applier.ApplierOperator(mainGrid, inputRaster, inputRasterArchive, inputVector, outputRaster, controls, ufuncArgs, ufuncKwargs, ufuncFunction=None)

Bases: object

This is the baseclass for an user defined applier operator. For details on user defined operators see hubdc.applier.Applier.apply()

getFull(value, bands=1, dtype=None, overlap=0)

isFirstBlock()

Returns wether or not the current block is the first one.

isLastBlock()

Returns wether or not the current block is the last one.

isLastXBlock()

Returns wether or not the current block is the last block in x direction.

isLastYBlock()

Returns wether or not the current block is the last block in y direction.

ufunc(*args, **kwargs)

Overwrite this method to specify the image processing.

See Applier Examples for more information.

mainGrid

Returns the PixelGrid.

progressBar

Returns the progress bar.

workingGrid

Returns the PixelGrid of the currently processed block.

class hubdc.applier.ApplierOutputRaster(filename, format=None, creationOptions=None)

Bases: object

getBand(index)

getBandIterator()

getFlatList()

initialize(bands)

Specify the number of output bands. This is only required if the output is written band-wise.

setImageArray(array, overlap=0)

Write data to an output raster image. The name identifier must match the identifier used with hubdc.applier.Applier.setOutput().

Parameters:

• name – output raster name
• array – 3-d or 2-d numpy array to be written
• overlap – the amount of margin (number of pixels) to be removed from the image data block in each direction; this is useful when the overlap keyword was also used with getArray()

setMetadataDict(metadataDict)

Set metadata dictionary.

setMetadataItem(key, value, domain)

Set image metadata item.

setNoDataValue(value)

Set no data value to all bands.

filename

operator

class hubdc.applier.ApplierOutputRasterBand(parent, bandNumber)

Bases: object

setArray(array, overlap=0)

setDescription(value)

setMetadataItem(key, value, domain='')

Set metadata item.

setNoDataValue(value)

Set no data value.

class hubdc.applier.ApplierOutputRasterGroup

Bases: hubdc.applier.ApplierIOGroup

getFlatRasterKeys()

getFlatRasters()

getRaster(key)

setRaster(key, value)

hubdc.applier.applierInputRasterGroupFromFolder(folder, extensions)

hubdc.applier.is3darray(array)

hubdc.applier.is3darrayDict(arrayDict)

hubdc.applier.is3darrayDictOfDicts(arrayDictOfDicts)

hubdc.applier.is3darrayDictOfLists(arrayDictOfLists)

hubdc.applier.is3darrayList(arrayList)

hubdc.applier.is3darrayListOfDicts(arrayListOfLists)

hubdc.applier.is3darrayListOfLists(arrayListOfLists)