Skip to content. | Skip to navigation

Personal tools
You are here: GES DISC Home Additional Features Technology Lab S4PM 5.12.0 RELEASE NOTES

S4PM 5.12.0 RELEASE NOTES

Release Date

May 30, 2006

Functional Changes

 

  1. Production Summary Bug Fixed - A bug when using the Production Summary feature caused Stringmaker to fail complaining that the Production Summary data type was not set in the Stringmaker data types configuration file. This should have been done automatically without the user having to explicitly set it. The bug has been fixed.

  2. Fill Hole Tool Fixes - A fix made in 5.11.0 caused the Fill Hole tool to be filled out with output data types; these data types should not be listed there. In this release, the code responsible for this has been fixed. If there is no implicit algorithm chaining in the string, the Fill Hole tool lists the same data types as the Compose Data Request tool, but with smaller time increments.

  3. AIRS Version 5 Production Rule Enhancements - Added support for START_OF_AIRS_PENTAD. Made changes to allow fully-hands-off S4PM processing of monthly and pentad DPRs. Prior releases required operator modification of the algorithm select_data config files to adjust the coverage to represent the actual number of days in the period being processed -- now done automatically by S4PM.

  4. Leap Second Boundary Crossing Bug Fixed - A bug occurring when a DPR straddles a leap second transition caused the DPR times to have the same leap second correction applied. This has been fixed.

  5. Enhancement to s4p_mk_pdr.pl Script - Added -n nodename argument to override the default from gethostbyname.

  6. Enhancements in Find Data - The Find Data station code has been reworked to improve performance. In particular, if the file name pattern configured in a string is such that file names are fully predictable (i.e. no glob patterns), the Find Data code is now smart enough to simply test for the file's existence rather than running an expensive glob operation. Note that the default file name pattern (the $s4pm_filename_pattern parameter in the Stringmaker Data Types configuration file) does not allow for this performance enhancement since it includes non-predictable elements in the file names (i.e. production date/time). Thus, to take advantage of this enhancement, this parameter must be overridden with one that leaves out these elements. Note that in low load strings, the performance increase may not be noticeable.

  7. Enhancement to the Compose Data Request Tool - Added ability to "query" Level 0 S4PA archives.

  8. Function Added to S4P::MetFile - Added a convenience function, get_datetime_range(), to retrieve BeginningDateTime and EndingDateTime from a metadata file.

  9. Fix to S4PM.xs - Added flush (stdout) to see if we could eliminate problem of occasional non-output on some Linux machines

  10. S4PM::GdDbi Enhancement - Changed recognition of Sybase to include other DAACs and g3 machines.

  11. Terminate Job Fix in S4P - Fixed terminate_job() to kill the "real" children, not the "sh -c" process.

  12. Time Calculation Fix to Select Data - Fixed bug in Select Data for the end time calculation with START_OF_MONTH.

  13. Leap Second Bug Fix – Fixed a leap second bug in S4PM.pm and s4pm_select_data.pl.

  14. Bug Fix in s4p_mk_pdr.pl - Fixed bug that did not recognize -e days and -o orig_system.

Detailed File Changes

S4P.pm

  • Fixed terminate_job to kill the "real" children, not the "sh -c" process.

S4P::FileGroup.pm

  • Modified subroutines ftp_get() and ssh_get() to accomodate a new environment variable: FTP_FIREWALL. This is used, in turn, to pass firewall information to the Net::FTP module. If FTP_FIREWALL is undefined, however, no firewall options are passed to Net::FTP. The purpose of implementing this as an environment variable is to allow the API to ftp_get() and ssh_get() to remain unchanged.

S4P::MetFile.pm

  • Add subroutine get_datetime_range() to obtain BeginningDateTime and EndingDateTime from metadata file in one fell swoop.

s4p_mk_pdr.pl

  • Added -n nodename argument to override default from gethostbyname.
  • Added getopts() call to pick up all command line arguments (formerly ignored).
  • Fixed bug that did not recognize -e days and -o orig_system.

S4P::PAN.pm

  • Modified subroutine write() to accomodate a new environment variable: FTP_FIREWALL. As with the changes in S4P::FileGroup, this is used to pass firewall information to the Net::FTP module. If FTP_FIREWALL is undefined, however, no firewall options are passed to Net::FTP. The purpose of implementing this as an environment variable is to allow the API to write() to remain unchanged.

S4PM.pm

  • Changed order of application of AIRS instrument offset and leapsecond correction in leapsec_correction().

S4PM::Algorithm.pm

  • Added method update_info() that updates the algorithm processing_period, product_coverage, and trigger_coverage in the invoking object. Updates are made only for objects with processing_start = START_OF_MONTH or START_OF_AIRS_PENTAD. The start time of the processing interval is passed as an argument to the method. The updated values reflect the number of days in the processing period, which varies based on the start time.

S4PM::DataRequest.pm

  • Added ability to handle multi-granule files in find_urls_by_datatime(), such as those available in a Level 0 S4PA archive.

s4pm_find_data.pl

  • Modified subroutine get_data_pathname() rather significantly so that what Find Data looks for are specifically the .ur files. This should lead the way to not globbing when the file name patterns are adjusted to not contain production date/time fields. For now, it is just cleaner, more efficient code.
  • Another modification to get_data_pathname() is that if the file name pattern is fully qualified (that is, contains to * glob characters), it runs a simple file existence test rather than executing an expensive glob.

s4pm_prepare_run.pl

  • Added invocation of $algorithm->update_info(), using the start time from the input workorder as the argument.

s4pm_preselect_data.pl

  • Added invocation of $algorithm->update_info(), modified one program variable to incorporate the updated info.

s4pm_regular_block.pl

  • Added support for START_OF_AIRS_PENTAD boundary.

s4pm_run_test.pl

  • Modified so that if it can't find the S4PM binaries, it fails rather than trying to recover. This was necessary because the s4pstart.ksh script, which it calls, assumes that the PATH environment variable includes the location of stationmaster.pl.

s4pm_select_data.pl

  • get_process_end_time() was subtracting 1 second from the end time for START_OF_MONTH processing_start, but for no others. The inconsistency was fixed by not doing that subtraction.
  • Added support for START_OF_AIRS_PENTAD boundary.
  • Added invocation of $algorithm->update_info()
  • Modified get_process_end_time() so that the leapsecond correction is applied independently to both start and end, rather than just to start.

s4pm_stringmaker.pl

  • Fixed some code in sub read_algorithm_cfg() that was supposed to automatically set the data type max size of production summary data types. It was trying to do this via the %all_datatype_max_sizes hash when it should have been using the %datatype_max_sizes hash instead. This was fixed.
  • Modified a line where it checks whether $max_children has been set or not. It was doing a simple: if ($max_children), but this fails if you purposely set $max_children to zero. To allow it to accept a setting of zero, the test was changed to: if ( defined $max_children ).

s4pm_stringmaker_datatypes.cfg

  • Added description of the new @non_lgid_datatypes parameter.
  • Updated documentation on the $s4pm_filename_pattern parameter to explain the impact on performance of having the machine date/time included in the file names. Also emphasized the need to set the S4PM_CONFIGDIR environment variable if this option is used.
  • Added description and example of the new %s4pa_data_map hash that defines where data are located in S4PA systems. This parameter is ignored when S4PM is only interoperating with ECS.

s4pm_stringmaker_derived.cfg

  • Added code to add the @non_lgid_datatypes parameter to the s4pm_export.cfg file if present, non-empty, and the external archive system is S4PA.
  • A number of fixes to the Request Data and Acquire Data sections to address better handling of the Fill Hole tool.
  • Modified the Request Data station setup so that if inputs in the INPUT disk pool are symbolic links, the $cfg_max_children is set to the default of 5 rather than 1 (single threaded). We can do this since no allocations are made when this is the case.
    For DME, set the polling interval to 20 seconds from the default of 120 seconds to improve DME peformance.
  • Added support for properly populating the s4pm_tk_compose_request.cfg file with the new %cfg_location hash when S4PM is interoperating with S4PA. The changes affect the Acquire Data and Auto Request stations.
  • Modified stuff for the Auto Request station which now has functionality to support S4PA interoperability.

 

Document Actions
NASA Logo - nasa.gov
NASA Privacy Policy and Important Notices
Last updated: Dec 08, 2010 11:06 AM ET
Top