New, Version 4, SIMAP is posted.Major improvements impact the way simap handles
the internal geolocations of all MODIS Level 1B products, including the
institutional Direct Broadcast products.Another big upgrade is the inclusion
of patches that allow certain AIRS Level 2 and 1B products to be mapped.
In better details, the changes introduced with SIMAP.4 are:
1. Introduced special inter- and extrapolation of geolocations
for MODIS L1B, including direct broadcast. Thus, the
georeferencing (mapping) is substantially improved,
especially in the case of 1-km L1B and internal geolocation arrays.
2. Introduced dynamic offsets,ia,ja, to better fill holes in the
3.Made changes allowing certain AIRS data to be mapped as well,
including the visible L1B, AIRVBRAD.
SIMAP can now stitch that AIRS data.
4. The CONVERT_COORD is vectorized to speed up the process.If the
geolocation arrays are too big, the code will switch to a FOR-cycle
to avoid possible memory problems.
Added a function SCALEGEO to scale integer geolocations
to real values, if local attributes like "Slope" and "Intercept"
to the geolocation arrays are present.
Such is the case with MODOCQC/MYDOCQC, and the
joint atmosphere product, MODATML2/MYDATML2.
simap also has a slight change to the common block, that
makes more variables be seen from the batch file.This was
done to allow more sophisticated map generations where more
than three layers can be superimposed, e.g. to combine
true color visible image with some science parameter.This, however,
requires advanced knowledge of MODIS quality flags, science
parameters valid ranges, and logical operators in IDL.
This job can be quite different from parameter to parameters. Thus, we can
provide sample batch file on request which may not match you needs though,
and regret to not being able to support it further.
More and more users have newer versions of IDL where the GIF
license was not acquired by RSI.IDL5.3 had both, GIF and PNG,
but needed the image flipped over before saving to the PNG file.
This flipping is not neccesary in IDL5.5.All these annoying
differences made us introduce a check of the IDL version, before
writing a one-layered image to a GIF or PNG file.If the IDL
version is 5.3 or older, then the program will use GIF interface.
versions (IDL5.5) the program will use
the PNG instead.Image examples here,
however, are gif files produced by IDL 5.3,
and are intentionally
left for users with older browsers.
The tool described here is an IDL based script that
is designed to read and map MODIS level 1B (L1B) and level 2 (L2
) Ocean and Atmosphere products. The main intention is to have a
non-interactive, command-line-executed tool, that can generate various maps
of the above products. The resulting maps are scaled to physical
units (radiances, concentrations, brightness temperatures, etc.) and are
saved in binary files. At the same time image files are also saved,
and their type depends on the task performed. A main feature is composition
of global maps, or stitching of multiple L1B or L2 granules on a single
world map. This can be useful in e.g. momentary display of Direct
Broadcast data, or producing high resolution continental maps of
US. Another useful feature is producing of large galleries of images
from a large amount of granules. This can be used to quickly and
automatically produce browse images from arbitrary amount of granules and
parameters, instead of doing images manually one by one. The feature
can be used to produce movies from time series of MODIS L1B and L2 granules,
or for quality control of big volumes of data. A feature is also
a module in the main code that detects invalid geolocations, and patches
them by means of an interpolation that uses neighboring correct
To repeat, the design of the tool is intended for
automatic production systems (like S4P, for example), where mapped remote
sensing products are desired (GIS for example).
2. SYSTEM REQUIREMENTS
IDL 5.3 or higher version is required to be installed
on the machine where the script will run. When working on MODIS 250m
resolution data (MOD02QKM) a good memory resource must be provided, in
general more than 500 MB. The script by its
self does not need installation.
There are only two text files that experienced IDL users can even mold
to their needs. Thus, it is platform independent, as long as the
platform has the IDL 5.3 or higher. However, the batch mode design
of the script is intended to take the full benefit ofa UNIX system (inclusion
in shell or Perl scripts, running in cron jobs, etc.)
3. HOW TO RUN
The tool consists of just two files.
One is an IDL batch file, and the other is the
main code, simap.pro. The batch
file contains configuration switches and all needed information to run
the main code. The batch file consists of simple commands that you
would normally type in the IDL command line. In the batch file, after
a setup section, the main code is executed by a simple single line instruction.
In general, the main code does not need any intervention. Yet, there
are things that user may want to change, and these will be explained.
The batch file format,
can be repeated arbitrary number
of times in a single batch file thus giving the user a lot of choices of
batch jobs. E.g. once the main setup has been made, a gallery
of [mapped] 40 Ocean L2 parameters can be generated, switching only
linear/logarithmic scales, color scales, or image intensities, for example.
These 40 images can be single granules, or 40 continental maps made from
tens of granules.
You may want to place the main code in one
place, and then to keep different batch files for different batch jobs
in separate directories. All of them can use
just that one main code, simap.pro.
However, it is your responsibility to ensure that IDL 'sees' needed directories.
Setting $PATH in UNIX, e.g., won't help! Here is what you can
First, regarding the main code, it is compiled from the batch file, and
you have to ensure that the
main code,simap.pro , is visible to the IDL compiler.
There are several ways of doing that, choose one:
i) The simplest way is to hard code
the path to the main code in you batch files in the .compile line (see
example batch files below):
ii) Add a line before the .compile line,
in the batch files (see example batch files below) that says and uses the
right separator for your operating system (e.g. ':' for UNIX, ';' for
iii) Add a line before the .compile
line in the batch files (see example batch file below) that says (don't
worry about the right separator)
iv) Keep batch files and the main
code in one directory, make this directory current, and run IDL from the
Second, regarding the batch file.IT IS THE BATCH FILE that executes the
program.Thus, if running from an IDL command line, then type and execute (idl>
is the IDL prompt):
where "path" is the name of directory where you keep your IDL batch
file, batchfile.bat (see example batch files below).
If running from a UNIX box, type and execute
(% is the UNIX shell prompt):
% idl /path/batchfile.bat
4. BATCH FILE DESCRIPTION
You don't need to be a programmer to put together
your own batch file, given the examples we provide here. The
switches and parameters of the batchfile will define in what mode and how
the main code will execute. Be very careful when choosing your
setup, as it will determine how the program will run, if run at
all. Once you set all switches and parameters in the batch file for
a given job, you don't need to reset them if you do same job over many
granules. The main code, as we pointed out, does not need
The following is an explanation of all switches and parameters
in the example simapL1B_1km.bat
we provide. More batch examples for many particular cases are provided
afterwards. simapL1B_1km.bat is a batch file setup that makes a gallery
of gray scale mapped images from MODIS land bands (1,4,3) at 1-km resolution.
The result is automatically saved in png and in binary "dat" files.
In the png files, all components of a scientific graphics are included:
axis box, labels, gray scale legend, name of the parameter and units (taken
from the HDF attributes). In the binaries, only what you see inside
the map is saved in physical units and is available for further processing.
So, the mapped data is available in image and binary files. Let's
go through this batch file line by line. We will remind again, once
you set all switches and parameters in the batch file, you don't need to
reset them if you do same job over many granules.
The first line makes a full reset of the IDL state, and the second
is compiling the main code.
These two lines do not need to be repeated any longer in the batch
file. However, they need to be before any other commands and be in
that order. Note, that IDL must see where the main code, simap,
resides. We mentioned how to do it in the section "HOW TO RUN" above.
The simplest way is just to hard code the path: .compile /path/simap.pro
These two lines initialize a data structure that will keep the names
of Scientific Data Sets (SDS) and channels, given SDS are three-dimesional,
you want to retrieve. In the rare case you want to make more than
20 selections, change this number. Otherwise, no changes needed here.
Note, it is actually an array of structures. The elements are sel,
sel and so forth. Each element has (or points) three thingies:
"name", "chans" and "numch". You don't need to worry about "numch".
How to set "name" and "chans" will be described shortly.
These are the path to your input HDF files, and the path to output
the mapped data. I.e. image and binary files will go to the directory
stored in outpath. These may be changed any time before the main
code is executed. Thus for every run of the main code you can take
data from, and distribute to, different locations.
This is the name of your input HDF file. It can be a single HDF
file name, or a text file containing the list of granules (HDF files) you
want to process. In the latter case, the name must be literally "filelist",
and so hdfname="filelist". The file "filelist" must contain an HDF
name per line. It may contain just one name. Paths can be included,
but then you must set inpath to a null string: inpath="".
This is NOT a good practice, though, and we discourage from that.
The reason is, if you ask for external geolocations, then the code will
search in the "inpath" directory and will fail, because it expects external
geolocations and the data in the same directory.
These are the SDS selections, and channels names for the corresponding
SDS, you make for the following run of the main code. All of these
need to be quoted and comma separated within the brackets! The reason
is the MODIS L1B 1km data: it has band names '13lo', '13hi', '14lo', '14hi'
that can only be entered as strings. The SDS EV_1km_RefSB has band
names starting from '8', going through '13lo', '13hi', '14lo', '14hi',
and the last two are '19' and '26'. This is why the code expects
band names, and not consecutive indexes of the third dimension,
for L1B data! All other data when needed, e.g. Atmosphere L2 profiles,
can be selected as indexes of the third dimension. You will see detailed
If set to 1, this switch will activate an interpolation that will attempt
to fill up artifacts of mapping procedure. Such artifacts, seen as
'holes', will appear when the requested image size is comparable to or
larger than the SDS array size. Almost certainly you will encounter this
case when working with MODIS Atmosphere L2 products. Set to 0 to
turn off this interpolation.When stitching granules (global maps), it turns
Set this switch to 1 to use EXTERNAL geolocations from MOD03 product.
Set to 0 to use INTERNAL geolocations. When mapping MODIS Ocean products,
you must supply corresponding MOD03 file and set extgeo=1. You may
choose to use external geolocation when mapping L1B and Atmosphere L2 as
well, but for these cases this is not necessary.
Set to 1 to extrapolate geolocations to the size of the SDS.
It must be 1 when using internal geolocations and L1B data. It is
a good idea to keep it always 1. You may set igeo=0 when using external
geolocations with L1B 1km or Ocean L2. When doing large jobs in these
cases it will be beneficial to set igeo=0, because geolocation arrays size
is the same SDS and there is no need to extrapolate. However, setting
igeo=1 will save you from thinking all the time of relative array sizes.
This is a switch to list SDS and print them on the current output device.
If you ever find use of it, here are the states:
1=short list, SDS names only, no attributes
2=long list, SDS names, attributes and their values
This is a flag for the type of image to be generated after all procedures
Set to 3 to show image composite from the first 3 channels selected.
Set to 1 to show b/w first layer
Set to 0 not to show image on the monitor
Note: when doing global maps (see gallery=0), the resulting image will
be displayed only if you run in debug mode (see debug=-1) and after all
granules are placed on the global map. When doing galleries of images
this flag will automatically turn to 0 to save time. You may run
a job for a large gallery unattended, and then review it with some png-
or jpg- viewer.
This is a switch to fill continents with a uniform gray color.
It is useful for Ocean L2 products.
Set to 1 to fill continents
Set to 0 not to fill continents
If 1, this flag will cause map limits (boundaries) to be automatically
Set to 0 to use the default map limits that you will set. See
Set to 1 to automatically set image brightness.
Set to 0 to use default image brightness ranges that you will set. See
This flag causes the image to be presented in log scale.
Set to 1 to show image in log scale. This is useful for certain
Ocean L2 parameters. This will not affect the data stored in
the binary files, only the images.
This flag regulates whether a global map will be produced from a list
of granules, (stitching mode), or a gallery of image files. When
this flag is set to 1, the code will generate galleries of image files.
0=no gallery, will do global map (stitching mode)
1=png or jpeg gallery of gray/color scale, of true color, RGB, image
files. If True_col=1 then grey/color scale. If True_col=3 then
the gallery will consist of RGB images composed from the first three selections.
See below for the ctab parameter
to understand how to set color table.
This flag regulates the debug mode. When debug=-1 all processing
messages, including errors, will print to the monitor. Set to 4 to
print to a log file. The name of the log file will be log.txt
and currently is set to be in the directory set by 'outpath'
xstart="";leave as NULL string for full size
xend="";or give numerical value
This section allows for a spatial subset by taking a
row/column subset.The quotes are an empty string that
signals that no subset is needed.To initiate a subset,
replace any one (or all) of the empty strings with a numerical value.
xstart/xend correspond to the along scan dimension ("frame
numbers"), while the ystart/yend correspond to the along track
dimension ("line numbers").
;---------Default map settings------------
These are default, hard coded, map boundaries. They will be active
if automap=0. If automap=1, they are ignored and the map boundaries
are automatically determined from geolocation arrays. In case automap=1 and
polar regions (DAYNIGHTFLAG is "Both"), a special procedure is used to form an 8-element
vector instead. The valid range for
latitudes is +-90 deg, and for longitudes is +-180 deg.
The latitude of the point on the Earth's surface to be mapped to the
center of the projection plane. Setting it to 0 will give an easy
to handle rectangular grid. However, it results in large distortions
at higher latitudes. If such distortions are an issue, try something
The longitude of the point on the Earth's surface to be mapped to the
center of the projection plane.
This defines map position and size relative to the graphic window.
This will define the type of projection. Two are available so
far, but more can be added easily.
1 = cylindrical
2 = stereo
These are default image birightness ranges. They will be active
For 3D arrays, this is the dimension to fix. In IDL, arrays dimensions
are e.g. array[x,y,z]. Set slice=1 or 2 or 3 to fix
x or y or z dimension, correspondingly. For most SDS's slice=3,
i.e. they are stacked over the z-dimension. However, for example MODIS
Atmosphere L2 Quality_Assurance needs slice=1 !!! I.e. it is stacked
over the x-dimension.
This is the size in pixels of the map to be produced. NXM and
NYM are the horizontal and vertical sizes, respectively. Note: the
size of the graphic window, that accommodates the map, is calculated from
this input and is larger. Thus, the array sizes stored in the binary
files are NXM x NYM. The binary files are always 4B (32 bit) FLOATS,
because the data there are scaled to physical units.
In the case you make a multichannel selection and a global map, all channels
are stacked in one binary file. In the case you make a gallery,
all channels are saved in a separate binary files. Each file has
the channel/parameter name embedded in the file name. The file names
are formed automatically, you don't need to worry about that. Now,
the size of the image file is the size of the graphic window. The
size of the graphic window is NX x NY and is calculated below, you don't
need to change or set it. Do not worry if the map is larger than
your monitor. E.g. if you monitor is 640x480 and the map is 2000x2000,
you still will be able to see the map (pending on the mode) in the IDL
environment because we use a sliding window. Recall that in "gallery"
mode graphics are not send to monitor but are saved in image files to speed
up the operation.
This parameter holds the IDL color table number. 0 is linear
black and white and is the default. ctab=33 will call for a rainbow.
This parameter does not influence the binary files, only the image files
that a single layer. That is, ctab does not influence the composite
RGB jpg files either. See for tricks and tips below how to use it
when choosing the right color table.
This is a background inside the map and outside the granule(s) in terms
of color index. In general, IDL has 256 color indexes in a given
color table (this, however, depending on your monitor and terminal is not
granted). Thus, when the color table is ctab=33 (rainbow), then backgnd=128
will give the green background.
;this will be the size of the graphic window to
;hold the map:
Finally, this will execute the code with the settings from above.