Content-Based Data Services

A number of content-based data services are currently being explored at the GES DISC. These are illustrated in the image below and discussed in the following paragraphs.

Content-Based Subscriptions

Currently, the best a user can hope for is to qualify subscriptions based on characteristics of the whole scene. However, a scene that is 75% clear overall may still have the user's area of interest completely covered in clouds (top right). Content-based subscriptions will allow a user to specify that a certain area within the scene must be clear in order for it to fill the subscription.

Content-Based Subsetting

One technique is to deliver to the user only the pixels meeting certain content criteria, such as clear-sky. In this case, we use the content assessment to "zero out" cloudy pixels, and then apply a compression algorithm to the data (thus preserving the data structure). As a result, data files can be reduced by one-half to two-thirds. This is currently implemented for MODIS real-time data at the GES DAAC using on-the-fly subsetting. In order to invoke it, one replaces the ".hdf" extension with the ".clr" extension when doing an ftp "get".

Content-Based Cache Management

Online disk space, while much more available today than previously, is nevertheless limited. In order to make maximum use of this area, we are developing a mechanism to set the expiration time on disk based on how "interesting" we expect the data to be to the user community at large. Thus, relatively clear scenes will stay on the disk longer than mostly cloudy scenes, and scence where particular areas of interest are clear (e.g. validation sites) will stay on disk even longer.