Skip to content. | Skip to navigation

Personal tools



3. Data Organization


The TOVS Pathfinder level 3 output consist of binary files containing 2- and 3-dimensional geophysical parameters mapped to a rectangular latitude-longitude grid. For each parameter, each cell in the grid is characterized by a mean value of that parameter, the associated standard deviation, and the number of observations used in computing the first two quantities. These statistics are stored as Scientific Data Sets (SDSs) in the Hierarchical Data Format (HDF). Means and standard deviations are stored as 32-bit floating point words, while the number of observations or "counts" are represented as 16 bit integers. Version 3.2 release 4 of HDF (HDF3.2r4) was used to create all level 3 files. The schematic below shows the overall structure of a TOVS HDF file:

                        |     FILE IDENTIFIER    |
                        |     FILE DESCRIPTION   |
                        |      DAAC METADATA     |
                        |     SCIENCE METADATA   |
                       /             |            \
                      /              |             \
                     /               |              \
                    /                |               \
                   /                 |                \
        -----------             -----------            -----------
        |Parameter|             |Parameter|            |Parameter|
        |  Means  |             | Std Dev |            |  Counts |
        -----------             -----------            -----------
           /   \                   /   \                   /   \
          / ... \                 / ... \                 / ... \
   ---------   ---------   ---------   ----------  ---------   ----------
   | SDS 1 |...|SDS 17 |   |SDS 18 |...| SDS 34 |  |SDS 35 |...| SDS 51 |
   | TEMP  |   | PSURF |   |TEMP_SD|   |PSURF_SD|  |TEMP_CT|   |PSURF_CT|
   ---------   ---------   ---------   ----------  ---------   ----------
        |  .....  |             |  .....  |             |  .....  |
        |         |             |         |             |         |
    --------   --------     --------   --------     --------   --------
    |Labels|   |Labels|     |Labels|   |Labels|     |Labels|   |Labels|
    --------   --------     --------   --------     --------   --------
    |Scales|   |Scales|     |Scales|   |Scales|     |Scales|   |Scales|
    --------   --------     --------   --------     --------   --------
    |Type  |   |Type  |     |Type  |   |Type  |     |Type  |   |Type  |
    --------   --------     --------   --------     --------   --------

Each file includes the following parameter SDSs (written out in that order):

    SDS Name   Units     Description
    --------   -----     -----------
    (1) TEMP     K       Temperatures at the surface and at 1000, 850, 700,
                         500, 400, 300, 200, 100, 70, 50, and 30 mb
    (2) CLTEMP   K       Coarse layer mean temperature for the surface-500,
                         500-300, 300-100, and 100-30 mb layers
    (3) PRWAT    cm      Total precipitable water values above surface,
                         850, 700, 500, and 300 mb
    (4) TSURF    K       Surface skin temperature
    (5) FCLD     0-1     Total cloud fraction *
    (6) FCLDP    0-1     Cloud fraction in 7 ISSCP layers *
                            180 mb to top of atmosphere
                            180 mb to 310 mb
                            310 mb to 440 mb
                            440 mb to 560 mb
                            560 mb to 680 mb
                            680 mb to 800 mb
                            800 mb to surface
    (7) PCLD     mb      Cloud top pressure *
    (8) TCLD     K       Cloud top temperature *
    (9) ZANGLE   deg     Effective satellite zenith angle defined as the
                         arccosine of the average value of the cosines of
                         the individual satellite zenith angles
   (10) TIME     hrs     Time of observation in Z hours (HH.hh)
   (11) QFLAG            Coarse indicator of agreement between observed and
                         computed brightness temperatues for the sounding -
                         0 (best) to 3 (accepted but not as good)
   (12) TOZ      D.U.    Total ozone index (1 Dobson Unit=1 matm-cm) *
   (13) OLR      W/m^2   Outgoing longwave radiation
   (14) LCRF     W/m^2   Longwave cloud radiative forcing defined as the
                         difference between the cloudy and clear sky OLR
   (15) PRECIP   mm/day  Precipitation estimate *
   (16) SPHUM    g/kg    Specific humidity values at 1000, 850, 700, 500,
                         and 300 mb
   (17) PSURF    mb      Ancillary field -- forecast surface pressure

The '*' indicates that these derived parameters were designated as experimental by the TOVS Implementation Team, pending further validation of the results. Items (1), (2), (3), (6), and (16) are stored as 3-dimensional SDSs where x=longitude, y=latitudeand z=pressure altitude. All other items are stored as 2-dimensional SDSs in latitude and longitude. There is a total of 51 SDSs in each TOVS HDF file. The first group of 17 SDSs contain the parameter means, the second group of 17 the parameter standard deviations, and the final group of 17 the associated counts.

Associated with each SDS is a set of ASCII labels containing the name, units, display format, and a brief description of the data in that array. The axes scales for each dimension of the SDS are also stored in the same representation as that used for the data, i.e., if the data are represented as 16-bit integers, then the scales are also stored as 16-bit integers. This number type, as well as the other information described above, is accessible via the HDF command line utilities (see section 4.2 and the example in Appendix A). For the TOVS data, the x-axis scales refer to longitude, the y-axis scales to latitude, and (for 3-dimensional SDSs) the z-axis scales to pressure altitude. It should be noted that in the case of layer quantities such as CLTEMP and FCLDP, the values for the z-axis scales denote the midpoints of the layers rather than the upper or lower boundaries. It should also be noted that the z-axis scale value associated with temperature at the surface (see TEMP above) is given a dummy value of "8888", since the surface pressure as defined in SDS #17 (PSURF) is itself variable over the entire grid.

There are 6 level 3 data product files, each representing a different averaging time period. The spatial and temporal resolutions of the data are listed below:

        Averaging periods   :  daily, 5-day and monthly AM (descending nodes)
                               daily, 5-day and monthly PM (ascending nodes)
        Gridbox dimensions  :  1 degree latitude by 1 degree longitude
        Center of gridbox 1 :  (-179.5,-89.5), with negative longitudes
                               west of the Greenwich meridian
        Gridbox progression :  proceeds from (-179.5,-89.5) to (179.5,-89.5),
                               continuing northward and ending at the gridbox
                               with central coordinates (179.5,89.5)

An HDF file identifier has been included with each of the 6 files which indicates the type of data set, the satellite, the time period and the date pertaining to the data. This label takes the following form (using March 1988 AM as an example) :

          TOVS_NOAA10_PATHA_GLOBAL_GRIDDED_DAILY_AM_880320           (daily)
          TOVS_NOAA10_PATHA_GLOBAL_GRIDDED_5DAYS_AM_B880317.E880321  (5day)
          TOVS_NOAA10_PATHA_GLOBAL_GRIDDED_MONTHLY_AM_8803          (monthly)

The HDF file identifiers for the PM files are named in an analagous manner. There are also three HDF ASCII annotations in each file. The first contains a general file description containing useful information related to the contents of the file and the Path A methodology. The second contains a short list of ASCII keywords (metadata) extracted by the Goddard DAAC as part of the archive process and used to populate a database for inventory purposes. The third HDF annotation contains auxiliary scientific data and statistics (science metadata) associated with the generation of the data contained in the SDSs. This is of interest only to expert users of the data requiring indepth knowledge of the special adjustments and corrections made by the algorithm during the retrieval process.

Local equator crossing time for even numbered satellites such as NOAA-10 is nominally 7:30 PM for the ascending node and 7:30 AM for the descending node. For gridding purposes, the global AM (PM) map is not created by including data from all descending (ascending) portions of the orbits between 00Z and 24Z. This would result in data from 2 consecutive days being included in either map since the satellite crosses the date line during this 24-hour period. Rather, the gridding is based upon the local date and time of the orbits, e.g., only those ascending orbits with the same local day and time (which is always 7:30 PM) are used in the construction of the PM map. A similar procedure is used in the construction of the AM map.

Each level 2 spot retrieval is assigned to a single gridbox according to which gridbox the spatial coordinates of the center of the level 2 FOV falls within. For a gridbox with multiple observations, a simple unweighted average of all points within the box is used for determining mean values. No interpolation or data filling is used. For all mean and standard deviation arrays, a fill of -999.99 is used to indicate that data for that gridbox is either missing, invalid or suspect. For the count arrays, a corresponding fill value of 0 is used under these circumstances.

The quality flag QFLAG is assigned a value according to the following procedure:

All retrievals are objectively validated according to the agreement between observed (cloud corrected) brightness temperatures for the temperature sounding channels and those computed from the solution. Two measures are used in this validation: the absolute difference between observed and computed brightness temperatures for MSU channel 2 and the root mean square difference between observed and computed brightness temperatures for the remainder of the temperature sounding channels. If either value is greater than 1 degree, the retrieval is rejected and a fill value is written out for all parameters except those related to cloud properties, which are always computed and written out. If each difference is less than 1 degree, a quality flag indicator is then computed based on the sum of the two measures as follows :

                     QFLAG =   ( | resid | +  | rms | ) * 2.

A quality flag value less than 1.0 indicates that the overall disagreement was rather small and the retrieval is assumed to be of very good quality. Quality flag values between 1.0 and 3.0 indicate an increasing difference between the observed and computed MSU channel 2 brightness temperatures and/or a higher RMS error for the remainder of the temperature sounding channels.


Document Actions
NASA Logo -
NASA Privacy Policy and Important Notices
Last updated: Nov 23, 2009 02:26 PM ET