MAPSERVER AND OGC WEB SERVICES

INTRODUCTION

MAPSERVER

Introduction and Compilation
MapFiles
MapScript

Creating Surface Velocity Vector Fields for MapServer Display
Creating Raster Files for Display on MapServer
Static vs Dynamic MAP Files

OGC TECHNOLOGIES AND MAPSERVER

Web Map Service (WMS)

Introduction to WMS

WMS Implementations

Web Coverage Service (WCS)

Introduction to WCS

WCS Implementations

Introduction to the Web Coverage Processing Service

Web Feature Service (WFS)

Introduction to WFS

WFS Implementations

Web Processing Service (WPS)

Introduction to WPS

WPS Implementations

Sensor Web Enablement (SWE)

Introduction to SWE

Introduction to Sensor Observations Service (SOS)
      Core Operations
      Transactional Operations
      Enhanced Operations

Configuring a MapServer for SOS

Web Services, SOAP and SOS
      SOAP Services
      SOS-Compatible SOAP Services

TGLO/TABS INTEROPERABILITY WITH GNOME MODEL

The Motivation and the Crisis
NetCDF
Metadata Issues
      NetCDF and XML
      Geoscience and XML

Horizontal Grid Issues
Vertical Coordinate Issues
Data Serving Issues
      Static
      Dynamic
      OPeNDAP/DODS

INTRODUCTION

The requirements of this project were to:

The unifying concept that underlies most of these requirements is the application of the concept of web services to the goal of providing geophysical data more openly and easily. A simple definition of web services is that of applications that interact with each other using web standards. A useful analogy is that of electrical services. Anyone connected to the electrical grid can immediately use the service, i.e. the electricity, by simply throwing a switch or plugging something into an electrical socket. This is possible because all of the components needed for using this service were standardized long ago, from the decision to use AC instead of DC, to the choice of cycles per second for the AC, to the transformer interface at the house, to the physical wiring of the house, and even to the design of the standard AC outlet. This is not yet true of the Internet, although the application of the web services concept is gradually making it a reality.

Some things are already standardized, for instance the HTML, HTTP and TCP/IP standards that basically created the Internet. Respectively, these were standards created to (1) provide a means to describe the structure of text-based information in a document, (2) provide a means to publish and retrieve the HTML pages, and (3) provide a means for networked machines to exchange such data in packet form. All of this is hidden from the average user who simply uses a web browser to, for example, look at the headlines on cnn.com, because the web browser and the operating system on which it is being used both smoothly implement a surprisingly large number of standards and protocols. For example, the TCP/IP protocol is really a set of seven protocols, and beneath the HTML protocol lies a range of text encoding standards from simple ASCII for encoding the English alphabet to Unicode for allowing all the text and symbols from all of the writing systems of the world to be consistently represented and manipulated by computers.

Although these three standards jump-started the web, they are fundamentally limited to exchanging just text among applications or machines. There are, for example, no provisions for exchanging pictures, binary data or money. This led to the genesis of the web services concept as well as further standards such as XML and SOAP for the implementation thereof. The characteristics of web services include:

The basic components of the web service architecture are the Service, the Service Requestor and the Service Container. A typical chain of events within this architecture is:

SERVICE CREATION

A required Service is created using the available and appropriate tools and languages. For example, herein we create a Web Map Service (WMS) to make our predicted surface current vectors available to clients, and we use the MapServer software and the Perl language to do so.

SERVICE PUBLICATION

After the service is created, it is published at a registry called a Universal Description, Discovery and Integration (UDDI) registry in the form of a Service Container, i.e. a Web Service Description Language (WSDL) file written in XML. The Service Container contains complete information about the service and whoever created it. In our example, the MapServer software creates a WSDL from the information provided in the WMS-enabled MapFile. The creation of a UDDI for such services is currently a research project of several interested groups.

SEARCH

The Service Requestor searches the UDDI registry for a specific service via an interface provided by UDDI registry provider. In our example, there is as yet no UDDI to be searched. Presently, those with services contact possible clients and provide the needed web address information. Eventually, we hope to have an appropriate UDDI that can be searched for "surface currents" and "Gulf of Mexico" and automatically provide the information.

REFERENCE

After finding the requested Service, the Service Requestor gets a reference to the Service and inspects the WSDL Service specification. In our example, the MapServer on the client side obtains the MapFile produced by the MapServer on the server side.

BINDING

The Service Requestor uses the information available in the WSDL reference to bind itself to the Service. In our example, he client-side MapServer uses the MapFile information to bind to the server-side MapServer.

INVOCATION

The Service is invoked using the WSDL via technologies such as SOAP or XML-RPC, and the Service Requestor gets the required result. In our example, when a WMS layer is requested on the client-side MapServer, it is automatically obtained from the server-side MapServer.

Now for a brief explanation of the protocols/standards mentioned:

XML

The eXtensible Markup Language (XML) is an extension and generalization of HTML, with the significant difference being that while HTML tags are for presenting data, XML tags are for describing data. Example HTML tags are BR, TABLE, STRONG, etc., while example XML tags are NAME, AGE, DATE, etc. The XML tags are useful for creating well-formed documents that can be easily exchanged and understood between applications. Possible ambiguity is avoided in XML by the use of namespaces, wherein an identifier is added to a tag to convert a local name into a qualified name, e.g. adding OCEAN to TEMPERATURE to obtain OCEAN:TEMPERATURE to distinguish it from, say, ATMOSPHERE:TEMPERATURE.

SOAP and XML-RPC

These are both standards for calling procedures on remote machines. That is, they provide the equivalent of local commands for non-local machines. Both of these are lightweight, XML-based protocols that enable the exchange of structured and typed information on the web. SOAP defines elements for specifying method names, parameters and return types, and also defines various data types for exchanging parameters and return values.

WSDL

WSDL is an XML format for describing the web service interface. It defines the set of operations supported by the server and the format that a client must use when requesting the service.

UDDI

UDDI can be seen as the Yellow Pages of web services. It allows service providers to publish information about their services for service requestors.

MAPSERVER

MapServer Introduction and Compilation

MapServer is an open-source environment for creating spatially-enabled web applications. It renders spatial data - maps. images and vector data - for the web. A basic MapServer set-up consists of:

A MapServer-enabled web page can be as simple as a single specified object, e.g. state boundaries, that is displayed when the page is loaded, to an arbitrarily complex page containing JavaScript, AJAX, etc. commands allowing the interactive display, animation, zooming, panning, etc. of thousands of available objects. The capabilities are limited only by the imagination of the developer, the available examples (quickly increasing), and the documentation (scattered but quickly improving).

While the MapServer package itself creates images from the available objects, much of the underlying work such as translating among GIS formats and enabling remote access of objects is performed by libraries that are compiled into the MapServer binary. Translation tasks are handled by the Geospatial Data Abstraction Library (GDAL), a translator library for raster geospatial data formats, and the OGR translator library for vector formats contained within the GDAL package. The remote object access essential for the implementation of OGC standards such as WMS, WFS and WCS is provided by libcurl. Translation among the numerous available map projects is provided by libproj.

Binary installation packages are available for some Linux distributions as well as for Windows and Mac platforms. We chose to compile our initial MapServer installation, largely because none of the available binary distributions contained all of the capabilities we desired. For example, none contained the NetCDF capabilities for GDAL. The initial compilation was performed in early 2005, at which point the compilation process was lengthy and error-prone, especially with GDAL, the largest component of the MapServer compilation process. Problems included the need for specific versions of included libraries (which seemingly changed with each new version release), PHP compatibility issues (the MapScript scripting language is written in PHP), and the same library being compiled by two different packages (at one point both MapServer and GDAL compiled a version of the GIF imaging library, although eventually this was confined to GDAL). In retrospect, most if not all of the problems were due to the rapid and significant developments made throughout the MapServer code hierarchy during 2005. The MapServer code appears to have reached a reasonably stable plateau during early 2006, as is evidenced by the trouble-free compilation of the MapServer version available in late May 2006.

MapFiles

In MapServer, the MapFile sets the configuration of how the map looks, what size it will be, and which layers to show. In form, a MapFile is a text file containing hierarchical object structures that define map settings. Each object or section of the MapFile starts with a keyword and ends with END, with one or more parameters or sub-objects between them. The primary documentation for this is the:

MapFile Reference - http://mapserver.gis.umn.edu/docs/reference/mapfile

A frighteningly skeletal MapFile follows: 

MAP
	...
	WEB
		...
	END
	
	LAYER
               ...
	END
	
END
A MAP object is the uber-object of the MapFile, and it defines map-wide parameters. The primary documentaton for this is the:

MAP Reference - http://mapserver.gis.umn.edu/docs/reference/mapfile/mapObj

The sub-objects available for defining below the MAP object are:

ANGLE [double] - The angle in degrees to rotate the map, with a default of 0.
CONFIG [key] [value] - Defines the location of the EPSG files from the PROJ.4 library.
DATAPATTERN [regular expression] - Defines a regular expression to be applied to requests to change DATA parameters via URL requests.
DEBUG [on/off] - Enables debugging of the MAP object.
EXTENT [minx] [miny] [maxx] [maxy] - The spatial extent of the map to be created.
FONTSET [filename] - Absolute filename of fontset file to use.
IMAGECOLOR [r] [g] [b] - The background color for the map.
IMAGETYPE [gif|png|jpeg|wbmp|gtiff|swf|userdefined] - The output image format to generate.
LAYER - The start of a LAYER object.
LEGEND - The start of a LEGEND object.
MAXSIZE [integer] - The maximum size of the image, e.g. 2048 sets a maximum of 2048 pixels in both dimensions.
NAME - [name] - The prefix attached to map, scalebar and legend GIF filenames created.
PROJECTION - The start of a PROJECTION object.
QUERYMAP - The start of a QUERYMAP object.
REFERENCE - The start of a REFERENCE object.
RESOLUTION [int] - Sets the pixels per inch for output, but only affects scale computations. The default is 72.
SCALE [double] - The computed scale of the map.
SCALEBAR - The start of a SCALEBAR object.
SHAPEPATH [pathname] - The path to the directory holding the shapefiles or tiles.
SIZE [x][y] - The size in pixels of the output image.
STATUS [on/off] - Sets the activity of the map.
SYMBOLSET [filename] - Absolute filename of the symbolset to use.
SYMBOL - The start of a SYMBOL object.
TEMPLATEPATTERN [regular expression] - Defines a regular expression to be applied to requests to change TEMPLATE parameters via URL requests.
UNITS [feet|inches|kilometers|meters|miles|dd] - Units of the map coordinates as used for scalebar and scale computations.
WEB - The start of a WEB object.^

The sub-objects in the skeletal example - with which we are immediately concerned - are LAYER:

CLASS - The start of a CLASS object.
CLASSITEM [attribute] - An item name in an attribute table to use for class lookups.
CONNECTION [string] - A database connection string to retrieve remote data.
CONNECTIONTYPE [local|sde|ogr|postgis|oraclespatial|wms] - The type of connection, with a default of local.
DATA [filename]|[sde parameters][postgis table/column][oracle table/column] - The full filename of the spatial data to process, with no file extension is necessary for shapefiles. This can be specified relative to the SHAPEPATH option from the MAP object.
DEBUG [on/off] - Enables debugging of this object.
DUMP [true/false] - A switch to allow Mapserver to return data in GML format.
FEATURE - The start of a FEATURE object.
FILTER [string] - This allows for data specific attribute filtering that is done at the same time spatial filtering is done.
FILTERITEM [attribute] - An item to use with simple FILTER expressions ( for OGR and shapefiles only).
FOOTER [filename] - A template to use after a layer's set of results have been sent (for multiresult query modes only).
GRID - The start of a GRID object.
GROUP [name] - The name of a group to which this layer belongs.
HEADER [filename] - A template to use before a layer's set of results have been sent (multiresult query modes only).
LABELANGLEITEM [attribute] - An item name in attribute table to use for class annotation angles (in degrees).
LABELCACHE [on/off] - Specifies whether labels should be drawn as the features for this layer are drawn, or whether they should be cached and drawn after all layers have been drawn.
LABELITEM [attribute] - An item name in an attribute table to use for class annotation (i.e. labeling).
LABELMAXSCALE [double] - The maximum scale at which the layer is labeled.
LABELMINSCALE [double] - The minimum scale at which the layer is labeled.
LABELREQUIRES [expression] - Sets the context for labeling this layer.
LABELSIZEITEM [attribute] - An item name in an attribute table to use for class annotation sizes (in pixels).
MAXFEATURES [integer] - Specifies the number of features that should be drawn for this layer in the current window.
MAXSCALE [double] - The maximum scale at which this layer is drawn.
METADATA Allows for arbitrary data to be stored as name value pairs.
MINSCALE [double] - The minimum scale at which this layer is drawn.
NAME [string] - A short name for this layer.
OFFSITE [r][g][b] - Sets the color index to treat as transparent for raster layers.
POSTLABELCACHE [true/false] - Whether or not to render this layer after all labels in the cache have been drawn.
PROCESSING [string] - Passes a processing directive to be used with this layer.
PROJECTION - The start of a PROJECTION object.
REQUIRES [expression] - Sets the context for displaying this layer
SIZEUNITS [pixels|feet|inches|kilometers|meters|miles| - Sets the unit of CLASS object SIZE values (default is pixels).
STATUS [on|off|default] - Sets the current status of the layer.
STYLEITEM [attribute] - Item to use for feature specific styling.
SYMBOLSCALE [double] - The scale at which symbols and/or text appear full size.
TEMPLATE [file|url] - A global alternative to CLASS TEMPLATE.
TILEINDEX [filename] - Full filename for the index or tile definition for this layer.
TILEITEM [attribute] - An item that contains the location of an individual tile, with the default "location".
TOLERANCE [double] - Sensitivity for point based queries (i.e. via mouse and/or map coordinates).
TOLERANCEUNITS [pixels|feet|inches|kilometers|meters|miles|dd] - Units of the TOLERANCE value.
TRANSPARENCY [integer|alpha] - Sets the transparency level of all classed pixels for a given layer.
TRANSFORM [true/false] - Tells MapServer whether or not a particular layer needs to be transformed from some coordinate system to image coordinates. Default is true.
TYPE [point|line|polygon|circle|annotation|raster|query] - Specifies how the data should be drawn.

and WEB:

EMPTY [url] - URL to forward users to if a query fails. If not defined the value for ERROR is used.
ERROR [url] - URL to forward users to if an error occurs. Ugly old MapServer error messages will appear if this is not defined
FOOTER [filename] - Template to use AFTER anything else is sent. Multiresult query modes only.
HEADER [filename] - Template to use BEFORE everything else has been sent. Multiresult query modes only.
IMAGEPATH [path] - Path to the temporary directory fro writing temporary files and images. Must be writable by the user the web server is running as. Must end with a / or depending on your platform.
IMAGEURL [path] - Base URL for IMAGEPATH. This is the URL that will take the web browser to IMAGEPATH to get the images.
LOG [filename] - File to log MapServer activity in. Must be writable by the user the web server is running as.
MAXSCALE [double] - Maximum scale at which this interface is valid. When a user requests a map at a bigger scale, MapServer automatically returns the map at this scale. This effectively prevents user from zooming too far out.
MAXTEMPLATE [file|url] - Template to be used if above the maximum scale for the app, useful for nesting apps.
METADATA - This keyword allows for arbitrary data to be stored as name value pairs. This is used with OGC WMS to define things such as layer title. It can also allow more flexibility in creating templates, as anything you put in here will be accessible via template tags.
MINSCALE [double] - Minimum scale at which this interface is valid. When a user reuqests a map at a smaller scale, MapServer automatically returns the map at this scale. This effectively prevents the user from zooming in too far.
MINTEMPLATE - Template to be used if above the minimum scale for the app, useful for nesting apps.
OUTPUTFORMAT [mime-type] - Format of the query output. Default is "text/html". This is experimental, the use of the OUTPUTFORMAT object is recommended instead.
TEMPLATE [filename|url] - Template file or URL tto use in presenting the results to the user in an interactive mode (i.e. map generates map and so on ... )^

MapScript

All of the functionality available within the MapServer binary program is also available as an API in several scripting languages, including Perl, Python. PHP and Java. MapScript is a loadable module that embeds MapServer capabilities in each of these languages, which in turn allows the full capabilities of each language to be used in service of MapServer.

MapScript API Reference - http://mapserver.gis.umn.edu/docs/reference/mapscript/index_html#introduction

Perl MapScript

Some useful links:

Perl MapScript Installation - http://mapserver.gis.umn.edu/cgi-bin/wiki.pl?PerlMapScript
Perl MapScript Examples - http://mapserver.gis.umn.edu/docs/howto/perl_examples
Perl MapScript, CGI and RDF Store Example - http://mappinghacks.com/projects/mapscript/01_cgi-perl-pan.html

A basic example (from http://proquestcombo.safaribooksonline.com/0596008651/webmapping-CHP-14-SECT-6) is: 
use mapscript;

# Set the map file to use
$mapfile = "global.map";

# Create a mapObj, initialized with the mapfile above
$mapobject = new mapscript::mapObj($mapfile);

# Create an imgObj that has an image of the map
$mapimage = $mapobject->draw(  );

# Save the mapimage to a file
$mapimage->save("worldmap.png");

A slightly less basic example is: 
#!/usr/bin/perl
use strict;
use mapscript;
use CGI ":cgi";
my $resp = new CGI;
# Create a unique image name every time through
#
my $image_name = sprintf("ms-hello%0.6d",rand(1000000)).".png";
# Create a new instance of a map object
#
my $map = new mapscript::mapObj("/home/mapdata/hello.map");
# Create an image of the map and save it to disk
#
my $img = $map->draw();
$img->save("/var/www/htdocs/tmp/".$image_name);
# Output the HTML form and map image
#
print $resp->header();
print $resp->start_html(-title=>'MapScript Hello World ');
print <<END_OF_HTML;
<form name="pointmap" action="perlms_hello.pl" method="POST">
<input type="image" name="img" src="/tmp/$image_name">
</form>
END_OF_HTML
print $resp->end_html();

where the MapFile hello.map is: 

# This is our "Hello World" mapfile
NAME "Hello World"
SIZE 400 300
IMAGECOLOR 249 245 186
IMAGETYPE png
EXTENT -1.0 -1.0 1.0 1.0
WEB
     TEMPLATE "/var/www/htdocs/hello.html"
     IMAGEPATH "/var/www/htdocs/tmp/"
     IMAGEURL "/tmp/"
END
LAYER
     STATUS default
     TYPE point
     FEATURE
          POINTS  0.00 0.00 END
          TEXT "Hello World"
     END # end feature
     CLASS
          STYLE
               COLOR 255 0 0
          END
          LABEL
               TYPE bitmap
          END
     END
END
END

Python MapScript

Python MapScript Installation - http://mapserver.gis.umn.edu/cgi-bin/wiki.pl?PythonMapScript
Python MapScript Examples - http://mapserver.gis.umn.edu/docs/howto/mapscript_python

A basic example is: 
import mapscript

# Set the map file to use
mapfile = "global.map"

# Create a mapObj, initialized with the mapfile above
mapobject = mapscript.mapObj(mapfile)

# Create an imgObj that has an image of the map
mapimage = mapobject.draw(  )

# Save the mapimage to a file
mapimage.save("worldmap.png")

PHP MapScript

Some useful links:

PHP MapScript Installation - http://mapserver.gis.umn.edu/docs/howto/phpmapscript-install
PHP MapScript Class Reference - http://mapserver.gis.umn.edu/docs/reference/phpmapscript-class
PHP MapScript by Example - http://mapserver.gis.umn.edu/docs/howto/phpmapscript-byexample

A basic example is: 
<?PHP
// map1.php
// PHP MapScript Example 1

// MapScript extension could also be loaded in php.ini
if (!extension_loaded("MapScript"))
   dl('php_mapscript.'.PHP_SHLIB_SUFFIX);


// Set the map file to use
$mapfile = "global.map";

// Create a mapObj, initialized with the mapfile above
$mapobject = ms_newMapObj($mapfile);

// Create an imgObj that has an image of the map
$mapimage = $mapobject->draw(  );

// Save the mapimage to a file
$mapimage->saveImage("worldmap.png");
?>

Creating Surface Velocity Vector Fields for MapServer Display

Once we had compiled a working MapServer configuration, our first desire was to make our current main web products available via MapServer. This led to some interesting issues as our TGLO web display featured surface current vector fields for both our ROMS and POM simulation model results. Our problem, most generally stated, was how to convert our output NetCDF files containing u and v components of the surface velocity vectors on a curvilinear grid into velocity vectors that could be displayed via MapServer.

MapServer can display both raster- and vector-based geospatial data. Raster data fields contain arrays of pixels that represent spatially continuous fields, while vector data consists of lines connecting points to represent boundaries or other non-continuous data. While the field represented by our displayed vectors is continuous, the vectors themselves are discrete, so we needed to develop a method to represent our surface current velocity vectors as vector-based geospatial data. ( And yes, the two nearly contiguous and disparate uses of the word "vector" did create a bit of confusion along the way.)

First, we searched for previous solutions to the problem. MapServer will render symbols as well as raster- and vector-based files, which provided the basis for the previous solutions we found. Current and wind vectors were plotted as symbols, with the symbols part of a font containing a separate symbol for each direction. For instance, a font containing 180 symbols for the angles 0 through 180 (and, also, for 180 through 360) is used, with if-then statements used to decide which symbol will be used for each vector, i.e. into which directional "bin" and corresponding symbol each vector will fall. Differing vector magnitudes were handled by placing different numbers of barbs on the vector tail, that is, the stronger the vector, the more barbs. Discussion and examples of this method can be found at:

http://nautilus.baruch.sc.edu/bb/viewtopic.php?t=169&start=0&postdays=0&postorder=asc&highlight=
http://lists.gis.umn.edu/pipermail/mapserver-users/2002-December/000041.html

We considered this solution inflexible and inelegant, and searched for another.

After significant trial and error, and working from both endpoints towards the middle, we created the following method for converting our NetCDF output files into surface vectors ready for displaying via MapServer.

  1. Extract the surface layer values of U and V from the ROMS output history file, and also the field A containing the angle between the curvilinear grid and a lat/lon grid at each ROMS grid point.
  2. Interpolate the U, V and A fields from their separate staggered grid locations to a single grid point location for all.
  3. Calculate the angle and magnitude of the vector at each grid point.
  4. Rotate the angle from the curvilinear axes to lon/lat axes using the A angle field.
  5. Convert the lon/lat values to cartesian coordinates using the program proj from the PROJ.4 package.
  6. Convert a file containing lon/lat/angle/magnitude quartets at each grid point into an ASCII ArcInfo generate format file containing information on how to construct a vector arrow in vector format at each grid point. This is done by looping over all the llines in the first file and:
    • writing a consecutive number for each feature, i.e. vector arrow, for each quartet line in the first file;
    • specifying the first of four coordinate pairs as the gridpoint location, i.e. the foot of our vector;
    • calculating the second coordinate pair, i.e. the head of the vector, from a scaling factor and the magnitude and direction of the vector at that grid point;
    • calculating the third coordinate pair as extending 45 degrees to the left of the vector head and a specified (and adjustable) distance;
    • calculating the fourth coordinate pair as extending 45 degrees to the right of the vector head and the same specified distance as the third pair;
    • writing "end" to specify the end of this vector/feature;
    • continuing until the last grid point, and then ending the ArcInfo file with another "end".
  7. Use the gen2shp program to convert the ArcInfo generate file into an ESRI ShapeFile, which can be rendered by MapServer.

This process creates an ESRI Shapefile containing a surface velocity vector field in vector format from a NetCDF history file containing the U and V components of the surface velocity field.

Creating Raster Files for Display on MapServer

In addition to the discontinuous surface velocity vectors, we also wanted to develop the capability to create and display continuous fields such as the sea surface height. MapServer can display raster-based images, but it requires that such files contain georeferencing information. We've been routinely creating raster-based images of various fields from the ROMS simulation output using the GMT graphics display package, but they do not contain georeferencing information. The solution to our problem was in the discovery of MB-System, a package built on top of GMT for displaying swatch sonar data.

The general procedure for creating georeferenced raster files from NetCDF history files is:

  1. Extract the desired field from the NetCDF history file and interpolate it from the curvilinear coordinate grid to a regular lat/lon grid.
  2. Create an ASCII file containing lon/lat/ssh triples.
  3. Use the MB-System program mbgrid to convert the lon/lat/ssh triples file into a GMT-compatible NetCDF grd file.
  4. Use the MB-System program mbgrdtiff to generate a georeferenced GeoTIFF file from the GMT-compatible NetCDF grd file.

The resulting GeoTIFF file can be rendered and display by MapServer.

Static vs Dynamic MAP Files

The development and use of MapServer and other GIS systems has until relatively recently been predicated on the use and display of fields that do not vary rapidly in time. Typical fields used for display include continental outlines, national and local boundaries, bathymetry, voting trends, city and other feature locations, etc. These types of fields typically vary on the order of years or longer, with monthly changes being about the most rapid variation encountered. If it is desired to add another field such as an updated or new bathymetry file to an operational MapServer site, then it is typically added either by editing the MAP file and web page text files with a text editor, or via the use of one the GUI interfaces that have been created for MapServer development, e.g. Chameleon. While this method is practical for adding the occasional new or changed field, it is not realistically applicable to our situation. We run 48-hour prediction simulations of the Gulf of Mexico four times per day as part of the TGLO/TABS system, and wish to have our latest fields, e.g. surface velocity vectors, sea surface height, available for MapServer display as soon as possible after they are created. Since hand editing the MAP and web page files 4 times per day at 6 hour intervals to create 48 new LAYER objects and a new pulldown menu is a largely impractical endeavor, we have created a system that dynamically and programmatically creates these files. The basic steps in the system are:

The primary goal of this MapServer configuration is to make the fields from the latest prediction simulation available as soon as possible, and to keep them available until the next simulation results can be processed.

OGC TECHNOLOGIES AND MAPSERVER

The Open Geospatial Consortium (OGC) is an international consortium of over 300 companies, government agencies and universities who participate in a consensus process to develop publicly available interface specifications. The core mission is to develop standards that enable the interoperability and seamless integration of spatial information, processing software, and spatial services. Spatial information and processing encompass geographic information systems (GIS), remote sensing, surveying and mapping, navigation, location-based services, access to spatial databases, sensor webs and any other technologies and information sources related to spatial information. Interoperability is defined as software components working with each other to overcome tedious batch conversion tasks, import/export obstacles, and distributed resource access barriers imposed by heterogeneous processing environments and heterogeneous data.

Online map services have been provided for years by many companies and organizations, albeit in a tangled jungle of data formats, programming languages and operating systems. The OGC intends to tame this jungle by providing specifications for various services, e.g. obtaining maps, obtaining data about maps, obtaining sensor data, obtaining information about the sensors, etc., that provide a common interface such that these services are interoperable across all platforms.

Towards this goal the OGC has implemented a Specification Program, which works in a format concensus process to develop approved or adopted OpenGIS Specifications; an Interoperability Program, which is a series of hands-on engineering initiatives designed to accelerate the development and acceptance of the specifications; and an Outreach and Adoption Program, wherein OGC and its members offer resources to help technology developers and users take advantage of the open standards.

The Specification Program produces a series of documents, the most important of which are the OpenGIS Specifications that detail the interface structure between software components. A Specification is considered to contain enough detail for implementing an interface if, when implemented by two independent agents, the resulting components will plug and play with each other. Abstract Specifications are first developed and provide the conceptual foundation or reference model for the development of the OpenGIS Specifications. For example, while the former might specify that a map image should be made available in a requested geographic range, the latter will specify in detail the syntax of how that request should be made. Other document series are Discussion Papers, which document a wide range of technology issues currently under discussion for possible adoption as standards, and Best Practices Documents, which discuss recommended procedures for using or implementing an adopted specification.

The most significant of the specifications produced thus far are:

Introduction to Web Map Services (WMS)

The Web Map Service (WMS) is one of the OGC technologies for employment in this project. The WMS specification provides for obtaining data from several different locations to create a single digital map. If the software clients and servers conform to this specification, a user can - using an ordinary web browser - overlay into a single image map images obtained from multiple map servers regardless of the map scale, projection, earth coordinate system or digital format at the server end. The maps produced by WMS are usually rendered in a raster-based format such as PNG, GIF or JPEG, but can be rendered in vector-based formats such as SVG or WebCGM. The clients can specify requested layers, layer styles, the geographic area of interest or bounding box, the projected or geographic coordinate reference systems, and the image file format as well as the size of the image.

The WMS standard defines three mandatory operations:

Useful documents about WMS include:

OGC WMS Standards Documents - http://www.opengeospatial.org/standards/wms
Guide to Distributing Your Data Products Via WMS - http://oceanesip.jpl.nasa.gov/esipde/guide.html
WMS Cookbook - http://www.intl-interfaces.com/cookbook/WMS/ Three web log entries were created in the process of learning about WMS and installing WMS capabilities in our local MapServer configuration. These provide further specific details about the procedures and methods described below. WMS Specification Overview - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=107
WMS Server Implementation with MapServer - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=108
WMS Client Implementation with MapServer - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=109

Configuring a MapServer for WMS

The MapServer software includes the capabilities of acting as both a WMS server and client, providing for the handling of all three required requests: GetCapabilities, GetMap, and GetFeatureInfo (although the features available for the last request are currently limited). In the context of the way a MapServer is installed and configured, all three requests are handled via extensions to the items required for the MAP and LAYER objects in the MAP file. Useful guides for configuring MapServer as a WMS server and client are:

Required MapFile Extensions for WMS

At the MAP object level, the following items are required or recommended:

NAME - Sets the root layer's name in the GetCapabilities request output.
PROJECTION - Sets the available advertised projections in which data can be served, at present a small subset of the available EPSG codes.
METADATA - The following are required or recommended in the WEB object:
wms_title - Sets the root layer's title in the GetCapabilities XML output.
wms_onlineresource - Specifies the URL for accessing data from the server.
wms_srs - Specifies the EPSG projections in which the data can be served if an EPSG code matching the projection of the data is not available. This is required unless PROJECTION is defined using an EPSG code.

At the LAYER level, the following items are required/recommended:

NAME - Sets a unique LAYER name in the GetCapabilities request output.
PROJECTION - See the same entry in the MAP level listing.
STATUS - If set to DEFAULT, the layer will always be sent to the client; if set to ON or OFF, it can be requested.
METADATA - The following items are required in the LAYER object:
wms_title - Sets a unique LAYER title.
wms_srs - See the wms_srs item in the MAP level listing.

WMS MapFile Example

An example of a MapFile properly configured for WMS support shown below. The bold items are those that are required or strongly recommended. Note that wms_srs is not used since a supported EPSG code is used for PROJECTION. Note also that the wms_time* items are explained below.

MAP
  NAME TGLO_ROMS_SURFACE_CURRENT_VECTORS
  IMAGETYPE PNG
  EXTENT -10891452.18 2000842.589 -8985428.89 3602117.913
  SIZE 600 450
  SHAPEPATH "data"
  IMAGECOLOR 255 255 255

  WEB
    TEMPLATE '/var/www/html/MAP/tutorial/tglo_perl.html'
     IMAGEPATH '/var/www/html/tmp/'
    IMAGEURL '/tmp/'
    METADATA
      "wms_title" "WMS Server for TGLO ROMS Surface Current Vectors"
      "wms_onlineresource" "http://csanady.tamu.edu/cgi-bin/mapserv?wms_tglo_sfc_vec.map"
    END
  END

  PROJECTION
    "init=epsg:4326"
  END
  ...
  LAYER
    NAME sfcvec-06-06-23-01
    METADATA
      "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-23T01:00:00Z"
      "wms_timeextent" "2006-06-23T01:00:00Z/2006-06-23T01:00:00Z"
      "wms_timeitem" "TIME"
      "wms_timedefault" "2006-06-23T01:00:00Z"
    END
    DATA sfcvec-06-06-23-01
    STATUS OFF
    TYPE LINE
    PROJECTION
      "init=epsg:4326"
    END
  END
END

Optional WEB and LAYER Object Metadata

A few items are strongly encouraged for the LAYER object level, but most can be inherited from the MAP object level. Many other non-required items are available for the WEB and LAYER METADATA objects. They are useful for the GetCapabilities request, that is, the more METADATA objects that are specified in the MapFile, the more information will be available to those perusing your data via a GetCapabilities request.

Available optional WEB object metadata that can be specified are:

ows_schemas_location - The root of the web tree where the family of OGC WMS XMLSchema files are located. This must be a valid URL where the actual .xsd files are located if you want your WMS output to validate in a validating XML parser. The efault is http://schemas.opengeospatial.net.
wms_abstract - A text description of the WMS server.
wms_accesscontraints - Information about access constraints. Use none if there are no constraints.
wms_addresstype, wms_address, wms_city, wms_stateorprovince, wms_postcode, wms_country - Optional contact address information. All six metadata items are required if any are given.
wms_attribution_logourl_format - The MIME type of the logo image. (e.g. "image/png"). Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_height - Height of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_href - URL of the logo image. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_width - Width of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_onlineresource - The data provider's URL.
wms_attribution_title - Human-readable string naming the data provider.
wms_contactelectronicmailaddress - Contact email address.
wms_contactfacsimiletelephone - Contact facsimile telephone number.
wms_contactperson, wms_contactorganization, wms_contactposition - Contact information. If any are provided then all three are required.
wms_contactvoicetelephone - Contact voice telephone number.
wms_encoding - XML capabilities encoding type. The default is ISO-8859-1.
wms_feature_info_mime_type - Used to specify an additional MIME type that can be used when responding to the GetFeature request.
wms_fees - Fees information. Use the reserved word "none" if there are no fees.
wms_keywordlist - A comma-separated list of keywords or keyword phrases to help catalog searching.

Available optional LAYER object metadata are:

gml_exclude_items - A comma-delimited list of items to exclude. This applies only to GetFeatures GML requests.
gml_groups - A comma-delimited list of group names for the layer. This applies only to GetFeatures GML requests.
gml_[group name]_group - A comma-delimited list of attributes in the group. Here is an example: 
      "gml_include_items" "all"
      "gml_groups"        "display"
      "gml_display_group" "Name_e,Name_f"
gml_include_items - A comma delimited list of items to include, or keyword "all". The default behavior is to include none. This applies only to GetFeatureInfo GML requests.
gml_[item name]_alias - An alias for an attribute's name. The served GML will refer to this attribute by the alias. Here is an example: 
      "gml_province_alias" "prov"
gml_[item name]_type - Specifies the type of the attribute. Valid values are Integer|Real|Character|Date|Boolean.
gml_xml_items - A comma delimited list of items that should not be XML-encoded.
wms_abstract - The same as wms_abstract in the Web Object.
wms_attribution_logourl_format - The MIME type of the logo image. (e.g. "image/png"). Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_height - Height of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_href - URL of the logo image. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_logourl_width - Width of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified.
wms_attribution_onlineresource - The data provider's URL.
wms_attribution_title - Human-readable string naming the data provider.
wms_dataurl_format- Non-standardized file format of the metadata. The layer metadata wms_dataurl_href must also be specified.
wms_dataurl_href - The URL to the layer's metadata. The layer metadata wms_dataurl_format must also be specified.
wms_extent - Used for the layer's BoundingBox tag for cases where it is impossible (or very inefficient) for MapServer to probe the data source to figure its extents. The value for this metadata is "minx miny maxx maxy" separated by spaces, with the values in the layer's projection units. If wms_extent is provided then it has priority and MapServer will NOT try to read the source file's extents.
wms_group_abstract - A blurb of text providing more information about the group. Only one layer for the group needs to contain wms_group_abstract, MapServer will find and use the value. The value found for the first layer in the group is used. So if multiple layers have wms_group_abstract set then only the first value is used.
wms_group_title - A human-readable name for the group that this layer belongs to. Only one layer for the group needs to contain wms_group_title, MapServer will find and use the value. The value found for the first layer in the group is used. So if multiple layers have wms_group_title set then only the first value is used.
wms_keywordlist - Same as wms_keywordlist in the Web Object .
wms_layer_group - Can be used to assign a layer to a number of hierarchically nested groups. This grouped hierarchy will be expressed in the capabilities. All group names should be preceded by a forward slash (/). It is not allowed to use both the WMS_LAYER_GROUP setting and the GROUP keyword for a single layer. An example is: 
      LAYER
        NAME "mylayer"
        DATA "mylayer"
        TYPE LINE
        CLASS
          STYLE
            COLOR 100 100 255
          END
        END
        METADATA
         WMS_LAYER_GROUP "/rootgroup/subgroup"
        END
      END
wms_metadataurl_format - The file format MIME type of the metadata record (e.g. "text/plain"). The layer metadata wms_metadataurl_type and wms_metadataurl_href must also be specified.
wms_metadataurl_href - The URL to the layer's metadata. The layer metadata wms_metadataurl_format and wms_metadataurl_type must also be specified.
wms_metadataurl_type - The standard to which the metadata complies. Currently only two types are valid: "TC211" which refers to [ISO 19115], and "FGDC" which refers to [FGDC-STD-001-1988]. The layer metadata wms_metadataurl_format and wms_metadataurl_href must also be specified.
wms_opaque - Set this metadata to 1 to indicate that the layer represents an area-filling coverage of space (e.g. a bathymetry and elevation layer). This should be taken by the client as a hint that this layer should be placed at the bottom of the stack of layers.
wms_srs - Same as wms_srs in the Web Object .
wms_style - The LegendURL style name. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style_[style's_name]_legendurl_format, wms_style_[style's_name]_legendurl_href
wms_style_[style's_name]_legendurl_height - The height of the legend image in pixels. Requires the following metadata: wms_style_[style's_name]_width, wms_style, wms_style_[style's_name]_legendurl_format, wms_style_[style's_name]_legendurl_href.
wms_style_[style's_name]_legendurl_href - The URL to the layer's legend. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style_[style's_name]_legendurl_format, wms_style.
wms_style_[style's_name]_legendurl_format - The file format MIME type of the legend image. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style, wms_style_[style's_name]_legendurl_href.
wms_style_[style's_name]_legendurl_width- The width of the legend image in pixels. Requires the following metadata: wms_style_[style's_name]_format, wms_style_[style's_name]_legendurl_height, wms_style, wms_style_[style's_name]_legendurl_href.
wms_timedefault - This value is used if it is defined and the Time value is missing in the request. Please see the WMS Time Support howto for more information.
wms_timeextent - (Mandatory for Time Support) This is used in the capabilities to return the valid time values for the layer. The value defined here should be a valid time range. Please see the WMS Time Support howto for more information.
wms_timeitem - (Mandatory for Time Support) This is the name of the field in the DB that contains the time values. Please see the WMS Time Support howto for more information.
wms_title - Same as wms_title in the Web Object.^

WMS Time Support

Although not a critical matter in traditional GIS applications that employ temporally static layers, the definition of time becomes much more important when one is dealing with layers that change on an hourly basis. The manner in which MapServer handles time is described in:

WMS Time Support - http://mapserver.gis.umn.edu/docs/howto/wms_time_support/view

A valid WMS layer with time support must have the following METADATA at the LAYER level:

wms_timeextent - The single time value represented by each of the 48 LAYER objects is not directly supported in MapServer, although there is an easy workaround. One simply specifies the single time value for both the minimum and maximum in the supported min/max format (required).
wms_timeitem - The name of the field in the database that contains the time values (required).
wms_timedefault - This value is used if it is defined and the TIME value is missing in the request (optional).^

In regards to the time format, WMS specifies that the basic format used for TIME requests is based on the ISO 8601:1988(E) format. MapServer supports a limited set of patterns that are defined in the ISO 8601 specifications, as well as few other patterns that are useful but not compliant to ISO 8601. A table of supported formats can be found in the [http://mapserver.gis.umn.edu/docs/howto/wms_time_support/ | WMS Time Support] document listed above and is repeated here:

Time PatternsExamples
YYYYMMDD20041012
YYYY-MM-DDTHH:MM:SSZ2004-10-12T13:55:20Z
YYYY-MM-DDTHH:MM:SS2004-10-12T13:55:20
YYYY-MM-DD HH:MM:SS2004-10-12 13:55:20
YYYY-MM-DDTHH:MM2004-10-12T13:55
YYYY-MM-DD HH:MM2004-10-12 13:55
YYYY-MM-DDTHH2004-10-12T13
YYYY-MM-DD HH2004-10-12 13
YYYY-MM-DD2004-10-12
YYYY-MM2004-10
YYYY2004
THH:MM:SSZT13:55:20Z
THH:MM:SST13:55:20

Creating the MapFiles for TGLO/TABS

A Perl script was created to programmatically create the WMS server MapFile four times per day for the Shapefiles containing the surface velocity vectors derived from the TGLO/TABS ROMS simulation results. 

#!/usr/bin/perl

use Template::Magic;
use CGI::Pretty qw(:standard);
use Date::Calc qw(Day_of_Week Week_Number Day_of_Year);
use Date::Calc qw(Add_Delta_Days);
use Date::Calc qw(Add_Delta_YMDHMS);
use Time::Local;
use Getopt::Declare;
use mapscript;

$zed = "0";

# ----------------------------------------------------------------------
#  Locate the most recent shapefiles and process them.
# ----------------------------------------------------------------------

# Directory containing the tarred and gzipped 48-hour shapefile packages.
$ROMS_CUR_DIR = "/bfd6000/TGLO/GRAPH/ROMS-sfcvec-shp";

# Directory from which WMS is served.
$WMS_CUR_DIR = "/bfd6000/TGLO/WMS";

# Sample shapefile name:  ROMS-sfcvec-shp-48-05-10-05-00.tar.gz

#    Contains:  vecshp-05-10-05-00-01.dbf,shx,shp,prj
#               vecshp-05-10-05-00-02.dbf,shx,shp,prj
#                               ...
#               vecshp-05-10-05-00-48.dbf,shx,shp,prj

$roms_cur_prefix = "vecshp";
$roms_cur_tar_prefix = "ROMS-sfcvec-shp-48";

#  Find all the archived ROMS surface current files available.

@roms_cur_files = glob "$ROMS_CUR_DIR/$roms_cur_tar_prefix-*.tar*";

#  Find most recent file, i.e. the last one in the array.
$roms_cur_current = $roms_cur_files[$#roms_cur_files];
($name,$tar,$gz) = split ('\.',$roms_cur_current);
$roms_cur_current_tar = "$name.$tar";
print "  Most recent current file:  $roms_cur_current_tar\n";

# Example:
#  /bfd6000/TGLO/GRAPH/ROMS-sfcvec-shp/ROMS-sfcvec-shp-48-06-06-23-00.tar

#  Split the directories from the file name.
@cur_string = split("/",$roms_cur_current);

#  Set the name of the tar file containing the shapefiles.
$shptarfile = $cur_string[$#cur_string];
print " shptarfile = $shptarfile\n";

#  Split the numbers from the prefix.
($pre1,$pre2,$pre3,$hrs,$year,$month,$mday,$fhtgz) = split("-",$cur_string[$#cur_string]);

#  Split the fh from the tar.gz.
($fh,$tar,$gz) = split ('\.',$fhtgz);
$fhfour = $fh."00";
$yearfour = "20".$year;

#  Uncompress and untar the files within.
system "cp $ROMS_CUR_DIR/$shptarfile $WMS_CUR_DIR/data";
system "cd $WMS_CUR_DIR/data; tar xzf $shptarfile";
@vecfiles = `cd $WMS_CUR_DIR/data; ls $roms_cur_prefix-$year-$month-$mday-$fh*.shp`;
system "cd $WMS_CUR_DIR/data; rm $shptarfile";

$nromfields = 48;
$npomfields = 24;

# -------------------------------------------------------------------
#  Set the non-volatile parameters, directories, etc.
# -------------------------------------------------------------------
$mapname = "TGLO_ROMS_SURFACE_CURRENT_VECTORS";
$imagetype = "PNG";
$minx = -10891452.18;
$miny = 2000842.589;
$maxx = -8985428.89;
$maxy = 3602117.913;
$xsize = 600;
$ysize = 450;
$shapepath = "/bfd6000/TGLO/WMS/data";
$imagecolor = "255 255 255";
$html_template = "tglo-test.html";
$imagepath = "/var/www/html/tmp/";
$imageurl = "/tmp/";
$map_proj = "merc";
$current_proj = "latlong";
$ellps = "WGS84";
$lon_0 = "0";
$lat_0 = "0";
$color = "50 50 50";
$outline_color = "32 32 32";

$wms_title = "WMS Server for TGLO ROMS Surface Current Vectors";

$boundary_layer = "gulf";
$boundary_layer_data = "ESRI/Country_Boundaries_3";

# ----------------------------------------------------------------------
#  Create the MAP file
# ----------------------------------------------------------------------

#$mapfilename = "wms_tglo_sfv_vec.map";
$mapfilename = "wms.map";
$map = "/bfd6000/TGLO/WMS/$mapfilename";

open (MAP, ">$mapfilename") or die "Cannot open $mapfilename";

print MAP "# MAP file for $year-$month-$mday-$fh\n";

#  The MAP object

print MAP "MAP\n";
print MAP "  NAME $mapname\n";
print MAP "  IMAGETYPE $imagetype\n";
print MAP "  EXTENT $minx $miny $maxx $maxy\n";
print MAP "  SIZE $xsize $ysize\n";
print MAP "  SHAPEPATH \"$shapepath\"\n";
print MAP "  IMAGECOLOR $imagecolor\n";
print MAP "\n";

#  The WEB object.

print MAP "WEB\n";
print MAP "  TEMPLATE \'$html_template\'\n";
print MAP "  IMAGEPATH \'$imagepath\'\n";
print MAP "  IMAGEURL \'$imageurl\'\n";
print MAP "  METADATA\n";
print MAP "    \"wms_title\" \"$wms_title\"\n";
print MAP "    \"wms_onlineresource\" \"http://csanady.tamu.edu/cgi-bin/mapserv?$map\"\n";
print MAP "    \"wms_abstract\" \"A WMS server for the TGLO/TABS ROMS model Gulf of Mexico prediction
	 simulation results.  A 48-hour prediction simulation is performed
	 four times per day, i.e. at 0000, 0600, 1200 and 1800 UCT.  The surface
	 current vectors are provided in ESRI Shapefile format.\"\n";
print MAP "    \"wms_accessconstraints\" \"none\"\n";
print MAP "    \"wms_contactelectronicmailaddress\" \"baum\@stommel.tamu.edu\"\n";
print MAP "    \"wms_fees\" \"none\"\n";
print MAP "  END\n";
print MAP "END\n";
print MAP "\n";

#  The PROJECTION object.

print MAP "PROJECTION\n";
print MAP "  \"init=epsg:4326\"\n";
print MAP "END\n";
print MAP "\n";

#  Creating LAYER objects for each of the 48 ROMS current vector time slice fields.

for ($n=0; $n <= $nromfields - 1; $n++) {
$np = $n + 1;

#  Creating WMS-conformant time information.

($e_yr,$e_mo,$e_dy,$e_hr,$e_mn,$e_sc) =
  Add_Delta_YMDHMS($year,$month,$mday,$fh,0,0, 0,0,0,$np,0,0);
if ($e_hr < 10) {$e_hr = "0"."$e_hr";};
if ($e_dy < 10) {$e_dy = "0"."$e_dy";};
if ($e_mo < 10) {$e_mo = "0"."$e_mo";};
if ($e_yr < 10) {$e_yr = "0"."$e_yr";};
$timeend = "20$e_yr-$e_mo-$e_dy".T."$e_hr:00:00Z";
($shpfile_prefix,$suffix) = split ('\.',$vecfiles[$n]);

#  To create the LAYER NAME and DATA objects - and the corresponding files in the
#  /data subdirectory - create copies of the shapefiles based on
#  relative times, e.g.
#
#           vecshp-06-06-23-00-01.*
#           ...
#           vecshp-06-06-23-00-48.*
#
#  that are based on absolute times:
#
#           sfcvec-shp-06-06-23-01.*
#           ...
#           sfcvec-shp-06-06-25-00.*
#
#  and then delete the original shapefiles with relative times in their names.

($e_yr,$e_mo,$e_dy,$e_hr,$e_mn,$e_sc) =
  Add_Delta_YMDHMS($year,$month,$mday,$fh,0,0, 0,0,0,$np,0,0);
if ($e_hr < 10) {$e_hr = "0"."$e_hr";};
if ($e_dy < 10) {$e_dy = "0"."$e_dy";};
if ($e_mo < 10) {$e_mo = "0"."$e_mo";};
if ($e_yr < 10) {$e_yr = "0"."$e_yr";};
$timeend = "20$e_yr-$e_mo-$e_dy".T."$e_hr:00:00Z";
($shpfile_prefix,$suffix) = split ('\.',$vecfiles[$n]);

#  To create the LAYER NAME and DATA objects - and the corresponding files in the
#  /data subdirectory - create copies of the shapefiles based on
#  relative times, e.g.
#
#           vecshp-06-06-23-00-01.*
#           ...
#           vecshp-06-06-23-00-48.*
#
#  that are based on absolute times:
#
#           sfcvec-shp-06-06-23-01.*
#           ...
#           sfcvec-shp-06-06-25-00.*
#
#  and then delete the original shapefiles with relative times in their names.

$layername = "sfcvec-$e_yr-$e_mo-$e_dy-$e_hr";
$filename = "$layername";
system "cp $WMS_CUR_DIR/data/$shpfile_prefix.shp $WMS_CUR_DIR/data/$layername.shp";
system "cp $WMS_CUR_DIR/data/$shpfile_prefix.shx $WMS_CUR_DIR/data/$layername.shx";
system "cp $WMS_CUR_DIR/data/$shpfile_prefix.dbf $WMS_CUR_DIR/data/$layername.dbf";
system "cp $WMS_CUR_DIR/data/$shpfile_prefix.prj $WMS_CUR_DIR/data/$layername.prj";
system "rm $WMS_CUR_DIR/data/$shpfile_prefix.*";


$romfields[$n] = $filename;
print MAP "LAYER  # surface current vector layer at $np hr\n";
print MAP "  NAME $layername\n";
print MAP "  METADATA\n";
print MAP "    \"wms_title\" \"TGLO ROMS Gulf of Mexico surface current vectors for $timeend\"\n";
print MAP "    \"wms_timeextent\" \"$timeend/$timeend\"\n";
print MAP "    \"wms_timeitem\" \"TIME\"\n";
print MAP "    \"wms_timedefault\" \"$timeend\"\n";
print MAP "  END\n";
print MAP "  DATA $filename\n";
print MAP "  STATUS OFF\n";
print MAP "  TYPE LINE\n";
print MAP "  PROJECTION\n";
print MAP "    \"init=epsg:4326\"\n";
print MAP "  END\n";
print MAP "  CLASS\n";
print MAP "    COLOR $color\n";
print MAP "    OUTLINECOLOR $outline_color\n";
print MAP "  END\n";
print MAP "END\n";
print MAP "\n";
}
print MAP "END\n";
close MAP;

Here is an example of the MapFiles created by the Perl script: 

# MAP file for 06-06-23-00
MAP
  NAME TGLO_ROMS_SURFACE_CURRENT_VECTORS
  IMAGETYPE PNG
  EXTENT -10891452.18 2000842.589 -8985428.89 3602117.913
  SIZE 600 450
  SHAPEPATH "/bfd6000/TGLO/WMS/data"
  IMAGECOLOR 255 255 255

WEB
  TEMPLATE 'tglo-test.html'
  IMAGEPATH '/var/www/html/tmp/'
  IMAGEURL '/tmp/'
  METADATA
    "wms_title" "WMS Server for TGLO ROMS Surface Current Vectors"
    "wms_onlineresource" "http://csanady.tamu.edu/cgi-bin/mapserv?/bfd6000/TGLO/WMS/wms.map"
    "wms_abstract" "A WMS server for the TGLO/TABS ROMS model Gulf of Mexico prediction
                                  simulation results.  A 48-hour prediction simulation is performed
                                  four times per day, i.e. at 0000, 0600, 1200 and 1800 UCT.  The surface
                                  current vectors are provided in ESRI Shapefile format."
    "wms_accessconstraints" "none"
    "wms_contactelectronicmailaddress" "baum@stommel.tamu.edu"
    "wms_fees" "none"
  END
END

PROJECTION
  "init=epsg:4326"
END

LAYER  # surface current vector layer at 1 hr
  NAME sfcvec-06-06-23-01
  METADATA
    "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-23T01:00:00Z"
    "wms_timeextent" "2006-06-23T01:00:00Z/2006-06-23T01:00:00Z"
    "wms_timeitem" "TIME"
    "wms_timedefault" "2006-06-23T01:00:00Z"
  END
  DATA sfcvec-06-06-23-01
  STATUS OFF
  TYPE LINE
  PROJECTION
    "init=epsg:4326"
  END
  CLASS
    COLOR 50 50 50
    OUTLINECOLOR 32 32 32
  END
END

...

LAYER  # surface current vector layer at 48 hr
  NAME sfcvec-06-06-25-00
  METADATA
    "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-25T00:00:00Z"
    "wms_timeextent" "2006-06-25T00:00:00Z/2006-06-25T00:00:00Z"
    "wms_timeitem" "TIME"
    "wms_timedefault" "2006-06-25T00:00:00Z"
  END
  DATA sfcvec-06-06-25-00
  STATUS OFF
  TYPE LINE
  PROJECTION
    "init=epsg:4326"
  END
  CLASS
    COLOR 50 50 50
    OUTLINECOLOR 32 32 32
  END
END

END

Testing the Server

Once the server has been successfully configured, it can be tested and used. A typical GetCapabilities request looks like :

http://csanady.tamu.edu/cgi-bin/mapserv?map=/bfd6000/TGLO/WMS/wms_tglo_sfv_vec.map&SERVICE=WMS&REQUEST=GetCapabilities

which will retrieve an XML file detailing what is available at that particular WMS server (with the amount of detail, as discussed above, dependent on the number METADATA items included in the MAP file).

A typical GetMap request looks like:

http://csanady.tamu.edu/cgi-bin/mapserv?map=/bfd6000/TGLO/WMS/wms.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&LAYERS=sfcvec-06-08-18-01

Configuring MapServer as a WMS Client

As with other OGC services, there are specific requirements that must be met by the WMS-client MapFile to implement a MapServer application that reads remote WMS data. First of all, IMAGEPATH parameter must be specified in the WEB object since this directory is used to store the temporary files downloaded from the remote WFS server. Next, it it necessary to include a specific set of parameters in the LAYER objects in the MapFile. The required LAYER objects and metadata are:

CONNECTIONTYPE - This must be specified as WMS.
CONNECTION - The URL of the remote WMS server. is the remote server's online resource URL.
METADATA - Each LAYER must contain this object with the following parameters:
wms_format - The image format to use in GetMap requests. If wms_formatlist is provided, then this is optional and MapServer will pick the first supported format in wms_formatlist for use in GetMap requests.
wms_name - A comma-separated list of layers to be fetched from the remote WMS server.
wms_server_version - The version of the WMS protocol specification supported by the remote WMS server.
wms_srs - A space-delimited list of EPSG projection codes supported by the remote server, as found from a GetCapabilities request.

The optional LAYER objects and metadata are:

MINSCALE, MAXSCALE - These can be set if the remote server's capabilities contains a ScaleHint value for this LAYER. This will enable MapServer to request the layer only at scales where it makes sense.
PROJECTION - MapServer will create this internally if needed.
METADATA - Each LAYER can contain the following parameters:
wms_connectiontimeout - The maximum time in seconds to wait for a remote WMS layer to load, set in seconds, with a default set to 30. (default is 30 seconds). This can also be set at the MAP level to be inherited by the LAYER objects.
wms_exceptions_format - Sets the format for exceptions. This defaults to application/vnd.ogc.se_inimage (the exception will be in a picture format).
wms_force_separate_request If this is set to 1, it will force this WMS layer to be requested using its own separate GetMap request. By default MapServer will try to merge multiple adjacent WMS layers from the same server into a single multi-layer GetMap request to reduce the load on remote servers and improve response time.
wms_formatlist - A comma-separated list of image formats supported by the remote WMS server. This is not needed if wms_formatlist, in which case it will use the first item in that list.
wms_latlonboundingbox - The bounding box of this layer in geographic coordinates in the format lon_min lat_min lon_max lat_max. If it is set then MapServer will request the layer only when the map view overlaps that bounding box.
wms_style - The name of the style to use for the STYLES parameter in GetMap requests for this layer.
wms_[stylename]_sld - The URL of a SLD to use in GetMap requests. Replace [stylename] in the metadata name with the name of the style to which the SLD applies. An example is: 
        "wms_style"        "mystyle"
        "wms_mystyle_sld"  "http://my.host.com/mysld.xml"

wms_time - The value to use for the TIME parameter in GetMap requests for this layer.^

An example of a correctly configured layer for WMS client capabilities - with the required portions in bold - is:

LAYER
  NAME "prov_bound"
  TYPE RASTER
  STATUS ON
  CONNECTION "http://www2.dmsolutions.ca/cgi-bin/mswms_gmap?"
  CONNECTIONTYPE WMS
  METADATA
    "wms_srs" "EPSG:42304"
    "wms_name" "prov_bound"
    "wms_server_version" "1.1.1"
    "wms_format" "image/gif"
  END
END

deegree WMS

The

deegree WMS - http://www.deegree.org/docs/wms/deegree_wms_configuration_2006-07-31.html

is written in Java and is deployed as a Tomcat application.

OpenLayers WMS

The

OpenLayers JavaScript Library - http://www.openlayers.org/

is a JavaScript mapping application containing an implementation of WMS.

msCross WMS

The

msCross Javascript GIS Client - http://datacrossing.crs4.it/en_Documentation_mscross.html

is a JavaScript mapping application containing a client implementation of WMS.

BeeldBlad WMS

The

BeeldBlad WMS Client - http://nlr-gis.nlr.nl/beeldblad/

is a Flash implementation of a WMS client.

Javascript WMS

The

WMS JavaScript Library - http://wms-map.sourceforge.net/

is a JavaScript implementation of WMS.

Introduction to WCS

The Web Coverage Service (WCS) extends the Web Map Server (WMS) interface to allow access to geospatial coverages (raster data sets) that represent values or properties of geographic locations, rather than WMS generated maps. That is, it allows you to obtain the data used to render the maps as well as the maps. The OGC WCS standard provides for three operations:

Useful documents about WCS include: The first of these documents is the official OGC WMS specification, while the latter three provide details on a WCS implementation within the very interesting THREDDS project, i.e.

http://www.unidata.ucar.edu/projects/THREDDS/

which has provided and continues to provide information and guidelines about implementing the XML/metadata and OGC technologies of interest to this project. It also provides useful implementations of these technologies. The THREDDS implementation is particularly valuable in that it deals directly with the NetCDF/NCML files that we and many other geoscientists use to store our data.

WCS Implementations

  • MapServer WCS

    Configuring MapServer for WCS

    Useful resources for MapServer WCS configuration are:

    Within MapServer, the WCS capabilities are implemented in a manner similar to the WMS capabilities - i.e. extra information is supplied via the object attributes in the MapFile - although the support for WCS is at an earlier and less complete stage than for WMS.

    MapServer will serve only the LAYER objects that meet the following conditions:

    The data is of raster type as supported by GDAL.
    NAME - A unique name must be supplied.
    TYPE - This must be set to RASTER.
    DUMP - This must be set to TRUE.

    Available optional WEB object METADATA parameters are listed below. These can be pre-pended with wcs_, ows_ or wms_.

    metadatalink -
    wcs_encoding -
    description -
    name -
    label -
    keywordlist -
    fees -
    accessconstraints -

    Available optional LAYER object METADATA parameters are listed below. These can be pre-pended with wcs_, ows_ or wms_.

    metadatalink -
    description -
    name -
    label -
    keywordlist -
    timeposition -
    timeperiod -
    timeitem - The attribute in the spatio/temporal index that contains time values.
    extent - Must be provided with tiled data.
    srs -
    nativeformat -
    formats -

    Axis descriptions are also available as optional LAYER object parameters. MapServer allows you to define a number of these for a layer. Individual axes are identified by name when defining specific metadata. All defines axes must be listed in the rangeset_axes metadata tag so MapServer knows in advance what to expect. A special rangeset for multiband data is automatically generated by adding the name bands to the rangeset_axes list. If found, MapServer will automatically generate metadata for the image bands. You may of course extend that basic support using the naming conventions below.

    rangeset_axes -
    [name]_semantic -
    [name]_refsys -
    [name]_refsyslabel -
    [name]_description -
    [name]_label -
    [name]_values -
    [name]_values_semantic -
    [name]_values_type -
    [name]_interval - Only a single interval is supported per axis.

    WCS and NetCDF

    The use of WCS for specific cases is described in:

    MapServer WCS Use Cases - http://mapserver.gis.umn.edu/docs/howto/WCSServerFormatHowTo

    with examples given for Modis, Landsat, DEM and NetCDF data. The last case interests us most given our extensive use of NetCDF. However, some issues involving GDAL - the library used by MapServer to handle raster-based files - must be addressed. NetCDF is one of many

    GDAL-Supported Raster Formats - http://www.remotesensing.org/gdal/formats_list.html

    that the library can read and/or write, and translate from one format to another. A major issue that must be addressed, however, is that while the NetCDF format fundamentally supports 3-D fields, the underlying data structures of GDAL were designed to handle 2-D fields. Fortunately, a workaround is available in the use of bands by GDAL. The band data structure arose because of remote sensing devices that can scan using different bands or wavelengths, and thus make available raster fields of the same 2-D area in several different bands. Given a bit of programming prestidigitation, the GDAL bands can be used as proxies for levels in the vertical dimension in 3-D NetCDF fields.

    Installing and Testing NetCDF with GDAL/MapServer

    To use NetCDF with GDAL, support for the former must be compiled into the latter (which is then compiled into MapServer). The default compilation procedure for GDAL does not include NetCDF support, so it must be specifically added. The procedure follows:

    Compiling GDAL/MapServer with NetCDF Support
    Obtain the latest NetCDF V3 distribution from:

    NetCDF - http://www.unidata.ucar.edu/software/netcdf/.

    Enter the following commands: 

    tar xzvf netcdf.tar.gz
cd netcdf-3.6.1/src
./configure --prefix=/usr/local >& conf.log &
make >& make.log &
make test
su
make install >& inst.log &
    This procedure (under the bash shell) will create files containing diagnostic information about the configuration (configure), compilation (make) and installation (make install) phases, which is useful if something goes wrong in any of the stages. You can follow the progress of any of the phases via the command tail -f *.log.

    Having successfully installed NetCDF, now obtain the latest GDAL distribution from:

    GDAL - http://www.remotesensing.org/gdal/download.html

    You can either get the official source release from:

    http://www.gdal.org/dl/

    or the CVS version (which contains the most recent functionality at perhaps the risk of a bit more instability) via:

    export CVSROOT=:pserver:cvsanon@cvs.maptools.org:/cvs/maptools/cvsroot
    cvs login
    Hit ENTER key.
    cvs checkout gdal

    Going the CVS route will create a gdal subdirectory in the directory in which you run the last command. Now enter the following commands: 

    cd gdal
./configure --prefix=/usr/local --with-netcdf=/usr/local >& conf.log &
make >& make.log &
su
make install >& inst.log &
    to install the GDAL library.

    After installing GDAL, MapServer should be compiled with the GDAL library, although we won't go into that here.

    To discover whether NetCDF has been successfully compiled into GDAL, we will use the gdalinfo command, more about which can be found at:

    gdalinfo Web Page - http://www.remotesensing.org/gdal/gdalinfo.html

    to interrogate a NetCDF file that is CF-1.0 compliant. First, we'll look at the file using the ncdump command which dumps an ASCII version of the entire file to the screen or a file. The result of this for GROM-fore-reg-06-08-24-12-48.nc, a CF-1.0 compliant file, is:

    Interrogating a NetCDF File with ncdump
    netcdf GROM-fore-reg-06-08-24-12-48 {
dimensions:
        lon = 171 ;
        lat = 131 ;
        time = UNLIMITED ; // (49 currently)
variables:
        double lon(lon) ;
                lon:long_name = "Longitude" ;
                lon:standard_name = "longitude" ;
                lon:units = "degrees_east" ;
                lon:point_spacing = "even" ;
        double lat(lat) ;
                lat:long_name = "Latitude" ;
                lat:standard_name = "latitude" ;
                lat:units = "degrees_north" ;
                lat:point_spacing = "even" ;
        double mask(lat, lon) ;
                mask:long_name = "Land Mask" ;
                mask:units = "nondimensional" ;
                mask:standard_name = "land_binary_mask" ;
        double water_u(time, lat, lon) ;
                water_u:long_name = "Eastward Water Velocity" ;
                water_u:standard_name = "surface_eastward_sea_water_velocity" ;
                water_u:units = "m/s" ;
                water_u:axis = "X" ;
                water_u:_FillValue = -9.9999e+32 ;
                water_u:scale_factor = 1. ;
                water_u:add_offset = 0. ;
        double water_v(time, lat, lon) ;
                water_v:long_name = "Northward Water Velocity" ;
                water_v:standard_name = "surface_northward_sea_water_velocity" ;
                water_v:units = "m/s" ;
                water_v:axis = "Y" ;
                water_v:_FillValue = -9.9999e+32 ;
                water_v:scale_factor = 1. ;
                water_v:add_offset = 0. ;
        double time(time) ;
                time:long_name = "Valid Time" ;
                time:standard_name = "time" ;
                time:units = "seconds since 2003-10-28 0:00:00 0:00" ;
                time:axis = "T" ;

// global attributes:
                :title = "ROMS output 48-hour regular grid surface currents" ;
                :institution = "Texas A&M University, Dept.of Oceanography" ;
                :source = "ROMS 2.1" ;
                :history = "none" ;
                :references = "none" ;
                :comment = "none" ;
                :Conventions = "CF-1.0" ;
                :grid_type = "REGULAR" ;
    The result of running gdalinfo on the same file is:
    Interrogating a NetCDF File with gdalinfo
    Driver: netCDF/Network Common Data Format
Size is 512, 512
Coordinate System is `'
Metadata:
  NC_GLOBAL#title=ROMS output 48-hour regular grid surface currents
  NC_GLOBAL#institution=Texas A&M University, Dept.of Oceanography
  NC_GLOBAL#source=ROMS 2.1
  NC_GLOBAL#history=none
  NC_GLOBAL#references=none
  NC_GLOBAL#comment=none
  NC_GLOBAL#Conventions=CF-1.0
  NC_GLOBAL#grid_type=REGULAR
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":mask
  SUBDATASET_1_DESC=[131x171] land_binary_mask (64-bit floating-point)
  SUBDATASET_2_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":water_u
  SUBDATASET_2_DESC=[49x131x171] surface_eastward_sea_water_velocity (64-bit floating-point)
  SUBDATASET_3_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":water_v
  SUBDATASET_3_DESC=[49x131x171] surface_northward_sea_water_velocity (64-bit floating-point)
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  512.0,    0.0)
Lower Right (  512.0,  512.0)
Center      (  256.0,  256.0)

    Serving NetCDF Files with WCS on MapServer

    Now that NetCDF has been successfully compiled into GDAL (and MapServer), we take advantage of the features discussed in these list messages:

    GDAL NetCDF Support I - http://lists.maptools.org/pipermail/gdal-dev/2005-August/006393.html
    GDAL NetCDF Support II - http://www.unidata.ucar.edu/support/help/MailArchives/galeon/msg00068.html

    which tell us that GDAL now supports the NetCDF format using the CF metadata convention. Basically, GDAL/MapServer looks for the presence of the variables lat/latitude and lon/longitude in the NetCDF file and, if it finds them, uses them to specify the georeferencing information needed for displaying information on MapServer. To put it another way, MapServer can now read NetCDF/CF files directly from disk and render them, and also transform them into other formats if needed.

    Script to Convert NetCDF Levels Into GDAL/MapServer Bands
    #!/usr/bin/perl
use XBase;
opendir(DIR, '.'); # open the current directory
foreach $file (readdir(DIR)) {
  next if !($file =~ /\.dbf$/); # read the dbf file in this directory created by gdaltindex
  print "Working on $file...\n";
  $tfile = 'temporary.dbf';
  system("mv $file $tfile");
  $oldtable = new XBase $tfile or die XBase->errstr;
  print join("\t", $oldtable->field_names) ."\n";
  print join("\t", $oldtable->field_types) ."\n";
  print join("\t", $oldtable->field_lengths) ."\n";
  print join("\t", $oldtable->field_decimals) ."\n";
  $newtable = XBase->create("name" => $file,
                "field_names" => [$oldtable->field_names, "IMGDATE"],  # this is the FILTERITEM in the corresponding tile index layer within your mapfile
                "field_types" => [$oldtable->field_types, "C"],  # character column type
                "field_lengths" => [$oldtable->field_lengths, 13], # length of the date string
                "field_decimals" => [$oldtable->field_decimals, undef]) or die "Error creating new table.";
  foreach (0 .. $oldtable->last_record) {
  ($deleted, @data) = $oldtable->get_record($_);
  print "  ...record $data[0]\n";
      # extract the date
      $year = substr $data[0], 8, 4;  # year  is at position 8 in the filename string
      $month = substr $data[0], 12, 2;  # month is at position 12 in the filename string
      $day = substr $data[0], 14, 2; # day is at position 14 in the filename string
      $hour = substr $data[0], 16, 2;  # hour is at position 16 in the filename string
      $date =  "$year-$month-$day" . "T" . "$hour\n"; # format is YYYY-MM-DDTHH, or any ISO format
   print "$date";
      push @data, "$date";
  $newtable->set_record($_, @data);
  }
  $newtable->close();
  $oldtable->close();
  unlink($tfile);
}

    Useful Perl libraries:

    GDAL-Perl - http://map.hut.fi/
    Geo::Proj4 - http://perl.overmeer.net/
    Geo::Point - http://perl.overmeer.net/
    Geo-perl Archives - https://list.hut.fi/pipermail/geo-perl/

    GDAL OPenDAP - http://gdal.maptools.org/frmt_dods.html | http://home.gdal.org/~warmerda/gdal_opendap_design.html

    Accessing MapServer Configured for WCS

    To be completed.

    WCS, THREDDS and GALEON

    In the realm of open-source solutions to serving data via an OGC WMS server, MapServer is not the only option. The

    THREDDS Project - http://www.unidata.ucar.edu/projects/THREDDS/

    server implementation currently includes a working WCS server, although it is considered experimental. It implements the WCS 1.0.0 specification and serves gridded data in GeoTIFF or NetCDF format. A working THREDDS/WCS server with examples can be found at NCAR:

    THREDDS/WCS Server - http://motherlode.ucar.edu:8080/thredds/catalog.html
    Server Configuration and Overview - http://motherlode.ucar.edu:8080/thredds/docs/WCS/index.html
    WCS-THREDDS Gateway - http://www.unidata.ucar.edu/projects/THREDDS/OGC/WCS-THREDDS%20Gateway.htm

    The THREDDS WCS implementation has become part of a larger WCS initiative called GALEON (Geo-interface to Atmosphere, Land, Earth, Ocean and NetCDF) which started out as an OGC WCS Interoperability Experiment (IE). The:

    GALEON Network - http://www.ogcnetwork.net/galeon

    site provides a central clearinghouse for all information about the project.

    The original GALEON IE implemented a geo-interface to NetCDF files via WCS 1.0.0 via a wrapper or layer around existing clients, servers and catalog protocols. Specifically, it leveraged the OPeNDAP and THREDDS projects to attempt to provide 5D grids from forecast model output, where 5D grids consist of 3 spatial dimensions and 2 time dimensions: model run time and forecast time. The objectives of the original project included determining whether:

    A dozen or so organizations participated in various parts of the original project, developing server/client implementations and exploring the viability of the various technologies and standards. One of the significant results of this was a set of requests for changing the WCS 1.1 draft specification to extend the GetCoverage request. It currently does not specify the contents of the coverage returned as a WCS GetCoverage response. The proposed GetCoverage evaluation model contains the following sequential operations:

    deegree WCS

    The

    deegree WCS - http://www.deegree.org/docs/wcs/deegree_wcs_configuration_2006-07-31.html

    implementation is written in Java and installed as a Tomcat application.

    Introduction to the Web Coverage Processing Service (WCPS)

    Also originating from the original project was an OGC discussion paper about a proposed service called the Web Coverage Processing Service (WCPS) for the retrieval and processing of geospatial coverage data. WCPS includes the functionality of WCS and extends it with an expression language for forming requests of arbitrary complexity. WCPS arose from the realization that extending WCS with further specific functions was less feasible and elegant than defining a new service with a coverage processing language that provides arbitrary functionality.

    The WCPS proposal extends the coverage concept of WCS by subdividing it into four layers:

    WCPS Coverage Layers
    Layer 0 - The grid data, the values of which are of all of the same data type inside a single grid.

    Layer 1 - The technical metadata, i.e. the number of axes, spatial extent per axis, cell type, interpolation methods, null or missing values, etc. This is the sort of information typically found in NetCDF 3 header files.

    Layer 2 - The spatio-temporal metadata, i.e. the coordinate reference system, the geo/time refrences and axis semantics. This level allows the semantics of space and time to be associated with an axis in Level 1, e.g. if any axis represents horizontal spatial coordinates, a coordinate system describes the axis.

    Layer 3 - General metadata, i.e. everything not included in the above including ontologies.

    although the proposed standard is only concerned with the first three layers.

    A WCPS coverage consists of the following constituents:

    WCPS Coverage Constituents
    Coverage values - The grid data of Level 0.

    Cell domain - The cell domain or coordinate extent, represented by the lower and upper bound coordinates of each axis.

    Range list - The cell type, consisting of an ordered list of (ComponentName, TypeName) pairs. ComponentName identifies the component (e.g. temperature, density), and TypeName is the type of integer or floating point number of each component.

    Null value - The number to be interpreted as a null value.

    Interpolation methods - The available interpolation methods, which are nearest neighbor (default), bilinear, bicubic, lost area and barycentric.

    Name - An identifier for the coverage.

    Domain - The available coverage locations and space and/or time, as expressed in geographic, elevation, or time coordinates. This is a list of (AxisName, DomainExtent, AxisType) triples where AxisName identifies the axis, DomainExtent indicates the extent on the axis, and AxisType indicates the type of axis, i.e. one of x, y, temporal, elevation or other.

    CRS - The identifier of the coverage's coordinate reference system.

    A set of accessor functions is available for extracting the coverage constituents.

    WCPS Coverage Constituent Accessor Functions
    Coverage Characteristic Accessor Function for Coverage C Comment
    Cell values value(C,p) Level 0 and 1 function
    Cell domain cdom(C) Level 0 and 1 function
    Range list celltype(C) Level 0 and 1 function
    Null value null(C) Level 0 and 1 function
    Interpolation method list im(C) Level 0 and 1 function
    Name name(C) Stored coverages only
    Domain dom(C) or dom(C,a) Level 2 function
    CRS crs(C) Level 2 function
    Dimension dim(C) Redundant level 2 convenience function
    Axis resolution res(C,a) Redundant level 2 convenience function

    Two operations are defined in WCPS: GetCapabilities and ProcessCoverage. The GetCapabilities operation is identical to that of WCS, except for extending coverage names to names of coverages and coverage sets, of axis types, and with supported format being a server rather than individual coverage property.

    The ProcessCoverage operation extends the WCS GetCoverage operation via more powerful processing capabilities including further coverage processing primitives, and the nesting of functions (which allows for arbitrarily complex requests). The WCPS primitives plus the nesting capabilities form an expression languaged called the WCPS expression language.

    The WCPS expression language contains the following elements:

    ProcessCoverage Request Elements
    coverageListExpr - Processes a list of coverages in turn. Each coverage request consists of exactly one of these. 
    for c in (A, B, C)
return tiff(c)
    processingExpr - Either an encodedCoverageExpr (which evaluates to an encoded coverage), or a scalarExpr (which avalues to coverage description or summary data).
    encodedCoverageExpr - Specifies a result with a data format encoding specified by the format name, e.g. 
    encode( C, f, extraParams)
encode( C, "hdf-eos" )
    booleanExpr - A scalarExpr whose result type is Boolean.
    scalarExpr - Either a getMetaDataExpr or a condenseExpr that returns a non-coverage result.
    getMetaDataExpr - Extracts a coverage description element, with the following extraction functions defined:
    • cdom - cell domain
    • celltype - cell type
    • null - null value
    • crs - coordinate reference system
    • xDomain - east-west extent
    • yDomain - north-south extent
    • tDomain - temporal coordinate extent
    • dim - dimension
    • res - resolution
    coverageExpr - A coverageName, setMetaDataExpr, inducedExpr, subsetExpr, scaleExpr, crsTransformExpr, coverageConstructorExpr or condenseExpr which always evaluates to a single coverage.
    coverageName - The name of a single coverage offered by the server.
    setMetaDataExpr - Specifies a change in a coverage's metadata, with the following defined:
    • setNull
    • setInterpolation
    • setCrs
    • setAxis
    inducedExpr - Either a unaryInducedExpr or a binaryInducedExpr. These allow the simultaneous application of a function on a single cell to all cells of a coverage.
    unaryInducedExpr - Specifies a unary induced operation.
    arithmeticExpr - Specifies a unary induced arithmetic operation, with the following defined:
    • +
    • -
    • sqrt
    • abs
    and the example: 
    sqrt ( C + D )
    trigonometricExpr - A unary induced trigonometric operation, with the following defined:
    • sin
    • cos
    • tan
    • sinh
    • cosh
    • arcsin
    • arccos
    • arctan
    and the example: 
    sin( C )
    exponentialExpr - A unary induced exponential operation, with the following defined:
    • exp
    • log
    • ln
    and the following example: 
    ln( C )
    boolExpr - A unary induced Boolean operation, with the following defined:
    • not
    • bit
    and the example: 
    not C
    castExpr - A unary induced cast operation wherein the cell type of the coverage is changed while leaving all other constituents unchanged, i.e. data type conversion.
    selectExpr - A unary induced record selection operation.
    binaryInducedExpr - A binary induced operation, i.e. one involving two coverage-valued arguments. The following are defined:
    • +
    • -
    • *
    • /
    • and
    • or
    • xor
    • = =
    • <
    • >
    • <=
    • >=
    • !=
    • overlay
    subsetExpr - A spatial and temporal domain subsetting, encompassing spatial and temporal trimming as well as sectioning.
    trimExpr - Extracts a subset from a given coverage expression along a specified axis, as specified by a lower and upper bound.
    sectionExpr - Extracts a spatial slice from a given coverage expression along one of its axes, as specified by section axis and position.
    crsTransformExpr - Transforms a coverage into the specified coordinate reference system. 
    ctransform( C, "EPSG:12345", "nearest-neighbor" )
    scaleExpr - Performs scaling in one axis of the source coverage using the specified interpolation method. 
    scale( scale( C, x, s, bicubic), y, s, bicubic )
    coverageConstructorExpr - Creates an n-dimensional coverage with content defined by a general expression.
    condenseExpr - Either a reduceExpr or a generalCondenseExpr. It takes a coverage and summarizes its values using a summarization function.
    generalCondenseExpr - Consolidates cell values or a coverage to a scalar value based on a specified condensing operation.
    reduceExpr - Derives a summary value from a coverage.
    coordinateTransformExpr - Specifies translation from a spatial or temporal reference system into cell (grid) coordinates.
    spatialTransformExpr - Specifies the transformation of a geographic point coordinate in a given reference system into a coverage's cell (grid) coordinates.
    temporalTransformExpr - Specifies the transformation of a time point coordinate into a cell (grid) coordinate.

    Introduction to Web Feature Service (WFS)

    The Web Feature Service (WFS) is an OGC specification that allows a client to retrieve and update geospatial data encoded in GML from multiple WFS implementations/servers.

    Useful links include:

    WFS Simple - http://www.ogcnetwork.net/wfssimple - A proposed WFS extension supporting BBOX and TIME parameters in queries.

    The operations defined in the OGC WFS specification are:

    GetCapabilities - Describes the capabilities of a WFS server by returning service metadata in response to a request.
    DescribeFeatureType -
    GetFeature -
    GetGmlObject - Allows the request of features and elements by ID from a WFS, which returns an XML document fragment containing the result set. This provides an interface through which a WFS can be asked to traverse and resolve XLinks to the features and elements it serves.
    Transaction - Used to describe data transformation operations that are to be applied to web accessible feature instances. Upon transaction completion, an XML response is generated.
    LockFeature - Implements a long-term feature locking mechanism that allows only one transaction at a time to modify the same data item.

    WFS Implementations

    MapServer WFS

    Configuring a MapServer for WFS

    The procedure described in:

    WFS Servers with MapServer - http://mapserver.gis.umn.edu/docs/howto/wfs_server

    is summarized below.

    A MapServer can be configured to act as a WFS server by suitably modifying the entries in a MapFile. The following conditions must be met for MapServer to serve a layer or include it in the WFS capabilities document.

    The data source must be an OGR-supported vector file.
    Required LAYER parameters:
    NAME - Must be set and start with a letter.
    TYPE - Must be LINE, POINT or POLYGON.
    DUMP - Must be set to TRUE.
    Required WEB METADATA:
    wfs_onlineresource - The URL used to access the WFS server must be specified.

    An example of a MapFile suitably configured for WFS is: 

    NAME WFS_server
STATUS ON
SIZE 400 300
SYMBOLSET ../etc/symbols.sym
EXTENT -2200000 -712631 3072800 3840000
UNITS METERS
SHAPEPATH "../data"
IMAGECOLOR 255 255 255
FONTSET ../etc/fonts.txt

WEB
  IMAGEPATH "/ms4w/tmp/ms_tmp/" 
  IMAGEURL "/ms_tmp/"
  METADATA
    "wfs_title"            "GMap WFS Demo Server"  ## REQUIRED
    "wfs_onlineresource"   "http://127.0.0.1/cgi-bin/mapserv.exe?"  ## Recommended
    "wfs_srs"               "EPSG:42304 EPSG:42101 EPSG:4269 EPSG:4326"  ## Recommended
    "ows_schemas_location" "http://ogc.dmsolutions.ca"  ## Optional
  END
END

PROJECTION
  "init=epsg:42304"
END

LAYER
  NAME province
  METADATA
    "wfs_title"    "Provinces" ## REQUIRED
    "gml_include_items" "all"  ## Optional (serves all attributes for layer)
  END
  TYPE POLYGON
  STATUS ON
  DATA province
  PROJECTION
    "init=epsg:42304"
  END
  DUMP TRUE           ## REQUIRED
  CLASS
    NAME "Canada"
    STYLE
      COLOR 200 255 0
      OUTLINECOLOR 120 120 120
    END
    TEMPLATE "ttt_query.html"
  END
END # Layer

END # Map File

    In addition to these requirements, the following mostly optional WEB object parameters are available.

    ows_schema_location - The root of the web tree where a the family of OGC WFS XML schema files are located. This must be a valid URL where the .xsd files are located if the WFS output is to be validated via a validating XML parser. The default value is http://schemas.opengeospatial.net.
    wfs_abstract - A description providing more information about the server.
    wfs_accessconstraints - Any access constraints imposed by the service provided on the WFS or data retrieved.
    wfs_encoding - An XML encoding for all XML documents returned by the server, with a default of ISO-8859-1.
    wfs_fees - Any fees imposed by the service provider for using the WFS or for data retrieved from it.
    wfs_keywordlist - A list of words to assist catalog searching.
    wfs_maxfeatures - The maximum number of elements the server will return.
    wfs_namespace_prefix - A user-defined namespace prefix to be used in the response to a WFS GetFeature request.
    wfs_namespace_uri - A user-defined namespace URI to be used in the response to a WFS GetFeature request.
    wfs_onlineresource - The URL prefix for HTTP GET requests (recommended).
    wfs_service_onlineresource - A top-level onlineresource URL. MapServer uses the onlineresource metadata (if provided) in the following order:
    1. wfs_service_onlineresource (optional)
    2. ows_service_onlineresource (optional)
    3. wfs_onlineresource (required)

    wfs_title - A human-readable title to identify the server (required).
    wfs_srs - A SRS to use for all layers in this server (recommended).

    The following LAYER object metadata parameters are available, with all optional unless otherwise noted.

    gml_exclude_items, gml_include_items - A comma-delimited list of items to exclude or include. To expose any attributes gml_include_items must first be invoked, followed by a list (if any) of specific items for exclusion. For example, if all attributes but one are included: 
         "gml_include_items"  "all"
     "gml_exclude_items"  "Phonenum"
    gml_groups - A comma-delimited list of group names for the layer, allowing attributes to be collected into related groups.
    gml_[group_name]_group - A comma-delimited list of attributes in the group. For example, for a group display containing attributes Name_e and Name_: 
         "gml_include_items"  "all"
     "gml_groups"  "display"
     "gml_display_group"  "Name_e,Name_f"
    gml_[item_name]_alias - An alias for the name of an attribute.
    gml_[item_name]_type [Integer|Real|Character|Date|Boolean] - The type of the attribute.
    gml_xml_items - A comma-delimited list of items that should not be XML-encoded.
    wfs_abstract - Same as wfs_abstract in the WEB object.
    wfs_extent - Used for the layer's BoundingBox tage for cases where it is impossible (or inefficient) for MapServer to probe the data source to figure its extents. The value for this is minx miny maxx maxy separated by spaces, with the values set in the layer's projection units.
    wfs_keywordlist - Same as wfs_keywordlist in the WEB object.
    wfs_metadataurl_format - The file format of the metadata record, e.g. XML, SGML or HTML. If this or any of the next two are specified, all three are required.
    wfs_metadataurl_href - The URL for the layer's metadata.
    wfs_metadataurl_type - The standard to which the metadata complies.
    wfs_srs - A SRS used to advertise a LAYER in the capabilities if no SRS is defined at the top level.
    wfs_title - The same as wfs_title in the WEB object.

    Configuring a MapServer as a WFS Client

    The procedure described in:

    WFS Clients with MapServer - http://mapserver.gis.umn.edu/docs/howto/wfs_client

    is summarized below.

    A WFS server publishes feature-level geospatial data to the web, and this data can be used as a source to render a map. It is similar to accessing a Shapefile over the web, except instead of the binary Shapefile you access an ASCII GML file containing the geospatial data, including both geometry and attribute information.

    As with other OGC services, there are specific requirements that must be met by the WFS client MapFile to read remote WFS data. There are:


    Required WEB object parameters:
    IMAGEPATH - Must be specified since this directory is used to store the temporary files downloaded from a WFS server.
    Required LAYER object parameters:
    CONNECTIONTYPE - Must be LINE, POINT or POLYGON.
    CONNECTION - Must be set to TRUE.
    Required LAYER METADATA parameters:
    wfs_typename - The NAME of the layer as obtained via GetCapabilities (required).
    wfs_version - The version of the WFS specification (required).
    wfs_connectiontimeout - The number of seconds to wait for a remote WFS layer to load, with a default of 30 seconds (optional). This can also be added at the MAP level and inherited here.
    wfs_filter - For including a filter encoding parameter in the getFeature request (optional).
    wfs_latlongboundingbox - Sets a bounding box in geographic coordinates of the form lon_min lat_min lon_max lat_max (optional). If this is set, the layer will be requested only when the map view overlaps the bounding box.
    wfs_maxfeatures - The maximum number of GML features to return (optional).
    wfs_request_method - Set to GET for WFS servers that do not support POST requests (optional).

    An example of a LAYER properly configured to act as a WFS client is: 

    LAYER
  NAME park
  TYPE POLYGON
  STATUS ON
  CONNECTIONTYPE WFS
  CONNECTION "http://www2.dmsolutions.ca/cgi-bin/mswfs_gmap?"
  METADATA
    "wfs_typename"          "park"
    "wfs_version"           "1.0.0"
    "wfs_request_method"    "GET"
    "wfs_connectiontimeout" "60"
    "wfs_maxfeatures"       "1"
  END
  PROJECTION
    "init=epsg:42304"
  END
  CLASS
    NAME "Parks"
    STYLE
      COLOR 200 255 0
      OUTLINECOLOR 120 120 120
    END
  END
END # Layer

    deegree WFS

    The

    deegree WFS - http://www.deegree.org/docs/wfs/deegree_wfs_configuration_2006-10-26.html

    implementation is written in Java and installed as a Tomcat application.

    Introduction to Web Processing Service (WPS)

    The Web Processing Service (WPS) provides access to pre-programmed calculations and/or computation models for operating on spatially referenced data. Useful links include:

    OGC WPS Page - http://www.opengeospatial.org/standards/requests/28
    52north WPS Page - https://incubator.52north.org/twiki/bin/view/Processing/52nWebProcessingService
    From GrassLinks to WPS with GRASS GIS - http://wgrass.media.osaka-cu.ac.jp/gisideas06/viewabstract.php?id=174
    Computing with GML using the WPS - http://www.gmldays.com/gml2005/presentations.html

    The operations specified by the OGC as mandatory for all WPS server implementations are:

    GetCapabilities - Allows a client to request and receive service metadata (or Capabilities) documents describing the capabilities of the specific server implementation.
    DescribeProcess - Allows a client to request and receive detailed information about one or more processes that can be executed by an Execute operation, including the input parameters and formats, and the outputs.
    Execute - Allows a client to run a specified process implemented by the WPS, using the provided input parameters and returning the output.

    The Execute operation allows WPS clients to run one of the processes provided by the server using the available input parameter values, and obtain the output. The output can be returned as a direct response to the request, or the results can be stored as web accessible resources. In the latter case, the response will be an XML document including a URL for each of the output responses.

    WPS Implementations

    PyWPS

    The package

    PyPWS - http://pywps.wald.intevation.org/

    is an implementation of WPS using Python as a scripting language and a combination of GRASS, R, GDAL and PROJ.4 for writing and supplying processes.

    PyWPS Installation

    The package is obtained from:

    PyPWS Download - http://pywps.wald.intevation.org/download/

    and installed in the local CGI directory, usually the cgi-bin subdirectory located below the root of the local Apache installation. The file wps.py is the CGI script, and both settings.py and grass.py are configuration files that need modification.

    As an example, the most recent version of PyWPS is contained in the file pypws-1.0.0.tar.gz and, when downloaded into the local server document root, e.g. /var/www/html, creates the subdirectory:

    /var/www/html/pywps-1.0.0

    Assuming that the local CGI script directory is at /var/www/cgi-bin, the CGI script wps.py is moved to the proper place (starting from the directory /var/www/html) via:

    cd pywps-1.0.0
    cp wps.py ../../cgi-bin

    52nWPS

    The

    52nWPS - https://incubator.52north.org/twiki/bin/view/Processing/52nWebProcessingService

    package is a WPS implementation written in Java, and is installed as an Apache Tomcat application.

    Introduction to Sensor Web Enablement (SWE)

    The Sensor Observation Service (SOS) is part of the ongoing OGC Sensor Web Enablement (SWE) project, which works on implementing interoperability interfaces and metadata encoding to enable the real-time integration of heterogeneous sensor webs into the information infrastructure. A series of specifications are being developed for creating appliations, platforms and products involving any web-connected device that can serve as a sensor. This category includes ocean data buoys, flood gauges, air pollution monitors, stress gauges on bridges, webcams, robits, space and airborne earth imaging devices, etc. Basically, any autonomous or semi-autonomous device that can record observations that can also be connected to the web qualifies for SWE.

    The currently available SWE projects with accompanying candidate specifications are:

    Introduction to Sensor Observations Service (SOS)

    The Sensor Observations Service (SOS) provides an API for managing deployed sensors and retrieving sensor data, specifically observation data. When used with the other OGC specifications, SOS provides a range of interoperable capability for discovering, binding to, and interrogating individual sensors, sensor platforms, or networked groups of sensors in real-time, archived or simulated environments.

    An SOS organizes collections of related sensor system observations into Observation Offerings (OO), which are analogous to WMS layers in that each OO is intended to be a non-overlapping group of related observations. When grouping observations in an SOS into OO, the goal should be to minimize the likelihood that a valid query request will get an empty response. That is, reasonable queries should usually produce results.

    A critical interoperability issue is to define a standard way to refer to phenomena measures by sensors, and the units of measure for those phenomena. Since the range of phenomena and units of measure is large, unknown a-priori, and possibly even unknowable, the solution is to use dictionaries in GML format. Servers and clients can then use URIs to refer to specific entries in a particular dictionary.

    The concepts of units of measure and phenomena are defined separately within the SWE initiative. Units of measure are defined based on the schema developed for GML, while a GML-conformant scheme for defining base and derived phenomena has been developed as part of the initiative.

    The operations defined in the SOS specification are divided into four profiels: core, enhanced, transactional and entire.

    The SOS Core Operations

    The three mandatory core operations are GetObservation, DescribeSensor and GetCapabilities.

    The GetObservation operation provides access to sensor observations and measurement data through a spatio-temporal query that can be filtered by time, space, sensor and phenomena. The request and response requirements are:

    GetObservation Request Requirements and Options
    Offering - The offering ID advertised in the GetCapabilities response document (required).
    ObservedProperty - The observable(s) for which observations are requested (required).
    eventTime - The time period(s) for which observations are requested (optional).
    procedure - The sensor system(s) for which observations are requested (optional).
    featureOfInterest - The feature for which observations are requested (optional). This is either a feature ID advertised in the GetCapabilities document, or a generic GML spatial constraint.
    Result - A parameter for providing OGC filter expressions based on property values (optional).
    resultFormat - The desired MIME type for results (required).
    responseModel - The desired format for results, with the choices being generic Observation or Common Observation format (optional).
    responseMode - The requested mode of response, with the choices being in-line, out-of-band, as an attachment, or as an observation template to be used for a subsequent call to GetResult
    service - The service type identifier, i.e. SOS.
    version - The version of the SOS specification.

    GetObservation Response Requirements and Options
    The response is encapsulated within Observation Features, Observation Collections and Observation Arrays, and within Discrete Observation Coverages. These are described in the Observations & Measurements (O&M) discussion paper (OGC 05-087).

    The DescribeSensor operation provides detailed information about the sensors making the measurements, and the platforms that carry the sensors. The request and response requirements are:

    DescribeSensor Request Requirements and Options
    SensorId - The ID of the sensor for which the description is desired (required).
    service - The service type identification, i.e. SOS (required).
    version - The version of the specification (required).
    outputFormat - A specification of the desired output format (optional).

    DescribeSensor Response Requirements and Options
    A SensorML or TML document describing the sensor system.

    The GetCapabilities provides SOS service metadata. The request and response requirements are:

    GetCapabilities Request Requirements and Options
    The allowed set of service metadata XML document section names and meanings are specified in Tables 3 and 7 of the Web Service Common Implementation Specification (OGC 05-008). The section name values and meanings are:

    ServiceIdentification - Returns the ServiceIdentification element of the service metadata document. This contains metadata about the specific server.
    ServiceProvider - Returns the ServiceProvider element. This contains metadata about the organization operating the server.
    OperationsMetadata - Returns the OperationsMetadata element. This contains metadata about the operations specified by this service and implemented by this server, including the URL for operation requests.
    Contents - Returns the Contents metadata element. This contains metadata about the data provided by this server.
    All - Returns the complete metadata service document.

    In addition to these, the following are available:

    service - The service type identification, i.e. SOS (required).
    version - The version of the specification (required).
    Contents - Returns the contents section of the service metadata document (optional).
    Filter_Capabilities - Returns the filter capabilities section of the service metadata document (optional).

    GetCapabilities Response Requirements and Options
    A SensorML or TML document describing the sensor system.

    The SOS Transactional Operations

    The optional transactional operations are RegisterSensor and InsertObservation.

    The RegisterSensor operation allows the client to register a new sensor system with the SOS as part of the transactional profile. Sensor observations can only be inserted via InsertObservation for sensors that have first been registered via this operation.

    RegisterSensor Request Requirements and Options
    SensorML_Process or TML_Process - The head of a substitution group that can include a Sensor or System element from either SensorML or TML, i.e. a detailed description of this sensor or system (required).
    O&M:Observation - A template for observation instances that will be inserted (required).
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    RegisterSensor Response Requirements and Options
    An InsertId used by SOS to identify the new sensor.

    The InsertObservation operation allows a client to insert new observations into a sensor system.

    InsertObservation Request Requirements and Options
    InsertID - The identifier for the sensor that made this observation.
    O&M:Observation - A parameter that provides the new observation for insertion.
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    InsertObservation Response Requirements and Options
    An empty InsertObservationResponse tag.

    The SOS Enhanced Operations

    The optional enhanced operations are GetResult, GetFeatureOfInterest, GetFeatureOfInterestTime, DescribeFeatureOfInterest, DescribeObservationType and DescribeResultModel.

    The GetResult operation allows a client to repeatedly obtain data from the same set of sensors without having to send and receive requests and responses that are redundant except for the timestamp. This operation relies on the creation of an O&M template from a previous call to GetObservation, with the ID of the template used for subsequent GetResult calls rather than a duplicate GetObservation XML document. The response contains only the result portion of the observation, with the other components included by reference to the template.

    GetResult Request Requirements and Options
    ObservationId - The Observation template whose result is desired.
    TimeInstant - Supports the common use case where data is desired for the period of time since the last call (optional).
    gml:GeometricPrimitive - The abstract head of the substitution group for all geometric primitives (optional). It can be used to subset the response by geographic area.
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    GetResult Response Requirements and Options
    A GetResultResponse document.

    The GetFeatureOfInterest operation obtains information about features of interest.

    GetFeatureOfInterest Request Requirements and Options
    FeatureOfInterestId - Requests a feature based on ID (this or FeatureOfInterestLocation required).
    EventTime - Specifies the time for which the target feature is of interest.
    FeatureOfInterestLocation - Requests a feature based on location rather than ID (this or FeatureOfInterestId required).
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    GetFeatureOfInterest Response Requirements and Options
    A GML feature.

    The GetFeatureOfInterestTime operation returns the time periods for which the SOS will return data.

    GetFeatureOfInterestTime Request Requirements and Options
    Ogc:spatialOps - A request for selecting a feature by spatial extent (this or ObjectID required).
    ObjectID - A request for selecting a feature by ID (this or Ogc:spatialOps required).
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    GetFeatureOfInterestTime Response Requirements and Options
    A GML time primitive listing one or more time periods for which observations are available.

    The DescribeFeatureOfInterest operation returns the XML schema for the specified GML feature of interest advertised in the GetCapabilities document.

    DescribeFeatureOfInterest Request Requirements and Options
    FeatureOfInterestId - The gml:id of the feature of interest (required).
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    DescribeFeatureOfInterest Response Requirements and Options
    A something.

    The DescribeObservationType operation returns the XML schema that describes the Observation type returned for a particular phenomenon.

    DescribeObservationType Request Requirements and Options
    Phenomenon - The name of the phenomenon (required).
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).

    DescribeObservationType Response Requirements and Options
    The XML schema that defines the specialized observation type returned for the given phenomenon.

    The DescribeResultModel operation returns the schema for the result element returned when the client asks for the result model by the ResultName.

    DescribeResultModel Request Requirements and Options
    ResultName - Specifies the qualified name of the result element in an Observation for which this operation is performed.
    service - The service type identifier, i.e. SOS (required).
    version - The version of the specification (required).
    -

    DescribeResultModel Response Requirements and Options
    The XML schema that defines the specialized result element type returned for the given qualified name.

    Configuring MapServer as an SOS Server

    The configuration procedure described in:

    Sensor Observation Server with MapServer - http://mapserver.gis.umn.edu/docs/howto/sos_server

    is summarized in the following.

    A MapServer can be configured to act as an SOS server, with each instance of an SOS server requiring a separate MapFile. The MapFile is configured for SOS duty by adding some parameters and metadata entries. The required or strongly recommended parameters and metadata at the MAP level are:

    NAME -
    PROJECTION -

    The required WEB object METADATA are:

    sos_title -
    sos_onlineresource - This specifies the URL used to access the SOS server.
    sos_srs - EPSG projection code denoting the projection of the offering. Required unless PROJECTION object is defined using "init=epsg:...

    The required or strongly recommended parameters and metadata at the LAYER level are:

    NAME -
    PROJECTION -

    The required LAYER object METADATA are:

    sos_offering_id - The ID of the offering.
    sos_observedproperty_id - The ID of the observed property.
    sos_describesensor_url - A temporary measure until the describe sensor is generated from MapServer. For DescribeSensor requests, it will redirect to the URL defined by:

    "sos_describesensor_url" "http://some.edu/url/NS01EE0014.xml"

    sos_procedure - A unique sensor ID, with one per layer.

    An example of an SOS server MapFile that meets the bare minimum requirements is: 

    NAME SOS_DEMO
STATUS ON
SIZE 300 300
EXTENT -66 44 -62 45 
UNITS METERS
SHAPEPATH "./data/"
IMAGECOLOR 255 255 0
SYMBOLSET "./etc/symbols.sym"

IMAGETYPE png

WEB
 IMAGEPATH "/ms4w/tmp/ms_tmp/"
 IMAGEURL "/ms_tmp/"  

 METADATA
   "sos_onlineresource" "http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map" ## REQUIRED
   "sos_title"          "My SOS Demo Server" ## Recommended
   "sos_srs"            "epsg:4326" ## Recommended
 END
END

PROJECTION
  "init=epsg:4326"
END

LAYER
  NAME test_sos_layer
  METADATA
    "sos_procedure"  "NS01EE0014" ## REQUIRED
    "sos_offering_id" "WQ1289" ## REQUIRED
    "sos_observedproperty_id" "Water Quality" ## REQUIRED  
    "sos_describesensor_url" "http://some/url/NS01EE0014.xml" ## REQUIRED 
  END  
  TYPE POINT
  STATUS ON
  DATA sos_test

  PROJECTION
    "init=epsg:4326"
  END

  CLASS
    NAME "water quality"
    STYLE
      COLOR 255 0 0
      SYMBOL "circle"
      SIZE 8
    END
  END
END    

END #map

    Additional optional metadata are available for the WEB and LAYER objects. The full list of available WEB object SOS metadata is:

    ows_schemas_location - The root of the web tree when the family of OGC SOS XML schema files are located. This must be a valid URL where the .xsd files are located if the SOS output is to be validated with a validating XML parser.
    sos_abstract - A description of the SOS server.
    sos_accessconstraints - Access constraints imposed on the SOS or data provided.
    sos_addresstype, sos_address, sos_city, sos_postcode, sos_country - If any of these optional contact address items are provided, then all are required.
    sos_contactelectronicmailaddress - The email address of a contact.
    sos_contactperson, sos_contactposition -
    sos_contactvoicetelephone - A voice telephone number for contact.
    sos_fees - Information about fees if any, with a default value of none.

    A list of optional MAP object SOS metadata is:

    sos_[item_name]_alias - An alias for an attribute's name that will be returned with GetObservation requests.
    sos_[item_name]_componenturl - An associated URL for a component that will be returned with GetObservation requests.
    sos_observedproperty_name - The name of the observed property.
    sos_offering_description - A description of the offering.
    sos_offering_extent - The spatial extent of the offering in minx, miny, maxx, maxy format. An example is:

    "sos offering_extent" "-66, 43, -62, 45"

    sos_offering_name - The name of the offering.
    sos_offering_timeextent - The time extent of the offering, in a "begin/end" format. If the end portion is not specified, it will be set to now. An example is:

    "sos offering_timeextent" "1990/2003"

    sos_timeitem - The name of the time field. This is layer specific and should be set in all layers.
    sos_srs - EPSG projection code denoting the projection of the offering.

    Web Services, SOAP and SOS

    SOAP Services

    One of the goals of the

    OOS Tech 2005 Follow-Up Plan - http://twiki.sura.org/twiki/bin/view/Main/OosTechFollowUp

    was to promote using and serving data via generic web services as implemented with various open source software packages designed for such tasks. A few definitions are in order at this point:

    Web Service - A software system designed to support interoperable machine-to-machine interaction over a network. This commonly refers to services that use SOAP-formatted XML envelopes and have their interfaces described by WSDL.

    SOAP - An XML-based message envelope format with bindings to underlying protocols such as HTTP.

    WSDL (Web Services Definition Language) - An XML format for describing web services and their bindings to specific protocols.

    UDDI (Universal Description Discovery and Integration) - A protocol for publishing and discovering metadata about web services, to enable applications to find them.

    Basically, WSDL is used to describe what sort of web service you are offering, e.g. salinity or temperature values, SOAP is used to obtain what is being offered, and UDDI is used to discover if any salinity or temperature values are being offered via SOAP.

    The initial OOSTech implementation of web services employed the

    Perl SOAP::Lite - http://www.soaplite.com/

    implementation of the SOAP protocol, and the work done to implement the

    OOS Tech Service Definition - http://twiki.sura.org/twiki/bin/view/Main/OosTechServiceDefinition

    used for the

    OpenIOOS Interoperability Test Bed - http://twiki.sura.org/twiki/bin/view/Main/InteroperabilityDemo

    is documented in the

    OpenIOOS SOAP::Lite Cookbook - http://twiki.sura.org/twiki/bin/view/Main/SoapliteCookBook

    The following web services were planned to be the initial OpenIOOS offerings:

    Initial OpenIOOS Web Services
    getLatest(platform,parameter)

    where:

    • platformid - A description of the geographical extent of the area of interest.
    • parameter - A parameter such as salinity or temperature.

    getPlatforms(parameter, westLimit, southLimit, eastLimit, northLimit)

    where:

    • parameter - A parameter such as salinity or temperature.
    • *Limit - The geographic limits of the search for the specific parameter in a given area.

    getPlatformMetadata

    TABS SOAP Services

    Here we detail the steps used to create a simple SOAP service for obtaining temperature values from the TABS buoys. While the eventual goal of such technology is to request such things directory from the sensors or platforms, the TGLO/TABS buoys are not yet ready for that task. We obtain the processed temperatures and velocity components from a GERG-based server on a daily basis, from which flat database files are created for each of the TABS buoys. The SOAP server detailed below makes available the most recent parameter available in a flat database file for a given buoy.

    After the installation of the SOAP::Lite package on our server, we created the following server script:

    SOAP Server Script tabs-server.pl
    #!/usr/bin/perl

#----------------------------------------------------------------------------------

#  These lines set up a SOAP wrapper around a class.

use SOAP::Transport::HTTP;

SOAP::Transport::HTTP::CGI

#  The next line refers to the class package TABSData below.  Dispatching
#  is the process of determining what class should handle a request, and
#  then loading that class.  The subroutine TABSGetW contains the only
#  class available in this simple example.

#  More than one class package can be specified.  If another module, e.g.
#  TABSModel.pm, were available, the next line would be:
#
#        ->dispatch_to('TABSData','TABSModel')
#
#  A dynamic dispatch method is also available wherein at line such as:
#
#        ->dispatch_to('/usr/soaplite/modules')
#
#  could be specified, with the given directory automatically searched
#  for the requested modules.

-> dispatch_to('TABSData')
-> handle;

#-----------------------------------------------------------------------------------

#  These lines are the package TABSData containing the class TABSGetW.
#  These could also exist in a separate file TABSData.pm in the same
#  or a different directory than TABS-server.pl.  It is only required
#  that the module TABSData.pm be in a location specified within @INC,
#  the Perl variable containing the list of directories to search for
#  library files.

package TABSData;

sub TABSGetW {

my ($class, $parameter) = @_;
my $result = -9999;

#  These lines define the name and location of the archive file containing
#  the buoy data; in this case buoy W.

my $buoy = "W";
my $soap_pre = "soap_archive_";
my $DAT_DIR = "/var/www/html/TABS-DAT";
my $datfile = "$DAT_DIR/$soap_pre$buoy.d";

my %data;

#  Here we open the W archive file and obtain the last line in a
#  hash containing all the individual column values.  For this example
# our data file looks like this:
#                                             Current
#    Date       Time        Lon.       Lat.   Speed  Dir.  Temp.
#2006-04-03T23:00:00   -96.0221   28.33476   0.2299   209   19.66
#2006-04-03T23:30:00   -96.0221   28.33476   0.2043   220   19.62

open(DATAFILE, "<$datfile");
while (<DATAFILE>) {
  @data{'datetime','lon','lat','currentspd','currentdir','temp'} = split(" ",$_);
}
close(DATAFILE);

$result = $data{$parameter};

return ($result);
}
1;

    along with a matching client program:

    SOAP Client Script tabs-client.pl
    #!/usr/bin/perl -w

use SOAP::Lite;

#  The package name TABSData from the server script is called as
#  part of the following URI to identify the class package
#  desired by the client.

my $uri   = "http://csanady.tamu.edu/TABSData";

#  This identifies the CGI script that supplies access to the
#  desired module, i.e. it is the server script placed in a
#  suitable CGI directory.

my $proxy = "http://csanady.tamu.edu/cgi-bin/TABS-server.pl";

$tglo_response = SOAP::Lite
  -> uri("$uri")
   -> proxy("$proxy")

#  This is where the specific subroutine/class within the module
#  is called.  If multiple values are available - e.g. the six
#  values stored in @data in the server script - then the desired
#  value is requested as a parameter of TABSGetW.

   -> TABSGetW('currentspd');

$result = $tglo_response->result;

print " Result is $result\n";

    SOS-Compatible SOAP Services

    Eventually, it was decided to align the existing web service activities with the OGC Sensor Web Enablement (SWE) initiative. Therefore, the existing operational SOAP-based XML service definition and demonstration project were to adopt the standards and interoperability demonstrations being developed by the OGC, while attempting to adhere to the CF-1.0 conventions. These activities are documented at:

    Moving Toward SWE - http://twiki.sura.org/twiki/bin/view/Main/OOSTechSWE

    TGLO/TABS INTEROPERABILITY WITH GNOME MODEL

    The Motivation and the Crisis

    GNOME (General NOAA Oil Modeling Environment) is the oil spill trajectory model used by HAZMAT responders during an oil spill. When there is a spill, the GNOME model predicts how wind, water currents, and other processes might move and spread the oil. Traditionally, climatological average values of wind and water currents have been used to force the GNOME model, but recently the capability to ingest and use spatially and temporally varying fields of wind and water currents has been incorporated into the model. The required formats for this are detailed in the:

    GNOME Data Formats Report - http://response.restoration.noaa.gov/type_topic_entry.php?RECORD_KEY%28entry_topic_type%29=entry_id,topic_id,type_id&entry_id(entry_topic_type)=298&topic_id(entry_topic_type)=1&type_id(entry_topic_type)=3

    Part of the long-term goals of the TGLO/TABS program is to enable those in charge of the TGLO oil spill remediation program to force the GNOME model with the TGLO/TABS surface current prediction system model output. The development of this part of the program was accelerated by Hurricane Katrina. The NOAA facilities at Bay St. Louis that had been operationally providing predicted surface current fields for the Gulf of Mexico were unavailable for several weeks after the passage of the hurricane. This created a situation wherein NOAA only had climatological wind and surface currents to force force the GNOME model. We received a request from NOAA to provide wind and surface current fields while the Bay St. Louis facilities were down. We used the information available in the GNOME Data Formats Report to produce GNOME-ready surface current and wind fields within a week of the request. Here we describe the process of and the problems encountered in producing these fields.

    http://www.unidata.ucar.edu/software/netcdf/

    Both the data and metadata are stored in a machine-independent binary format that conserves storage space. In the case of GNOME fields, the required ASCII format would require huge files for storing and transmitting the needed 2-D (and even larger 3-D) current fields, so we chose to go with the NetCDF format. While there is some "spin-up" time required for a novice to become familiar with the use of NetCDF, we have extensive experience with it. Also, NetCDF has become a de-facto standard in the geoscience community for storing and transmitting array-oriented data, with many data analysis programs and circulation models capable of NetCDF input and/or output. Additionally, our choice was motivated by the fact that the ROMS circulation model, i.e.

    ROMS - http://marine.rutgers.edu/po/index.php?model=roms

    we employ to produce our Gulf of Mexico prediction simulations uses NetCDF as the default standard for both input and output files.

    Metadata Issues

    NetCDF and XML

    Although our experience with and the use of NetCDF by both the front-end (ROMS) and back-end (GNOME) models would seem to make our task straightforward, a few curveballs were encountered along the way. First of all, while NetCDF is a widely accepted and used standard data encoding format wherein the files that are created are machine independent, the data model consists only of abstract variables, dimensions and attributes. A convention is required to transform the NetCDF format from the abstract to the realm of the concrete. For example, there has been until relatively recently no widely accepted method for specifying the name used for the horizontal velocity components created by circulation models. Examples of terminology that has been used for this variable include "u", "uvel", "u-velocity" and "eastward sea water velocity."

    There have been several projects that have attempted to standardize the NetCDF metadata conventions, e.g.

    COARDS - http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
    CF - http://www.cgd.ucar.edu/cms/eaton/cf-metadata/
    GDT - http://www-pcmdi.llnl.gov/drach/GDT_convention.html
    SEACOOS - http://marinemetadata.org/content/refs/scncdfstd/

    with discussion and information about previous and ongoing work at:

    Unidata NetCDF Conventions - http://www.unidata.ucar.edu/software/netcdf/conventions.html

    providing an overview of the state-of-the-art. The CF convention seems to be becoming the de facto community standard NetCDF profile.

    A further complicating factor is the collision of the XML bulldozer (rollercoaster?) with this work. Not only must work continue on standardizing the names of the variables and attributes that will populate the metadata portion of the file, but also on an XML schema into which these will fit. The most advanced project in this regard - and the one that seems to be gaining wider acceptance - is the CF project, although there are still unresolved issues that need to be addressed, e.g. staggered and unstructured grids, GIS/spatial information content, ontologies, discovery information and others.

    The primary document explaining the current plans for extending NetCDF and incorporating XML is

    Encoding netCDF Semantics and/or Data into XML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/NcMLAndGML.htm

    which explains the various projects underway. First, an XML version of the original NetCDF Common data form Description Language (CDL) called NcML was created. Then, a series of extensions to NcML were made to address the various needs.

    ncML - NcML is an XML description of the data stored in a NetCDF file. The schema is based on an abstract object model for expressing metadata associated with generic NetCDF data, and is closely related to the existing NetCDF data model. The NcML Core object model contains four types of objects:

    The object model was encoded in XML, thereby introducing an

    ncML XML Schema - http://zeus.pin.unifi.it/schemas/ncml/netcdf-coverage.xsd

    ncML Dataset - The NcML Dataset extension defines a new type of NetCDF file that allows:

    A NetCDF object as defined by an NcML Dataset can be either real or virtual, i.e. it may or may not exist as an actual file. It can also be used to redefine, aggregate and subset existing NetCDF files.

    ncML-CS - This extends the NcML Core model to add explicit support for general and geographic coordinate systems. It consists of the following extensions to the NcML Core model:

    ncML-G - The NcML Geography (NcML-G) extension was created to provide access to legacy atmospheric and ocean sciences datasets through GIS tools. The NcML-G model addresses the problem of interconnecting the traditional atmospheric science data model with the traditional GIS data model to achieve interoperability. As such, it extends the NcML-CS model to add explicit support for information characteristic of GIS data.

    GIS systems use two fundamental geographic data types: features and coverages. Features represental geometric entities on the surface of the Earth such as rivers, streets and lakes. Coverages - actually a special case of features - are used to map composite data such as is found in satellite images or radar observations. With these definitions in mind, four steps were taken to extend the model to prepare NetCDF data for GIS-based applications:

    The NcML-G model extends the NcML-CS model via five object packages: The encoding of the NcML-G model in GML is called NcML-GML.

    An appreciation of the similarities and differences among these representations can be obtained by perusing the same header file represented in four different ways:

    NetCDF - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncdump.txt - The text output of the ncdump utility. 

    netcdf C:/WebDocs/THREDDS/GALEON/SST.NC.test/sst-again.nc {
 dimensions:
   lon = 180;   // (has coord.var)
   lat = 170;   // (has coord.var)
   time = UNLIMITED;   // (24 currently)   // (has coord.var)
   bnds = 2;
 variables:
   double lon(lon);
     :standard_name = "longitude";
     :long_name = "longitude";
     :units = "degrees_east";
     :axis = "X";
     :bounds = "lon_bnds";
     :original_units = "degrees_east";
   double lon_bnds(lon, bnds);
   double lat(lat);
     :standard_name = "latitude";
     :long_name = "latitude";
     :units = "degrees_north";
     :axis = "Y";
     :bounds = "lat_bnds";
     :original_units = "degrees_north";
   double lat_bnds(lat, bnds);
   double time(time);
     :standard_name = "time";
     :long_name = "time";
     :units = "days since 2001-1-1";
     :axis = "T";
     :calendar = "360_day";
     :bounds = "time_bnds";
     :original_units = "seconds since 2001-1-1";
   double time_bnds(time, bnds);
   float tos(time, lat, lon);
     :standard_name = "sea_surface_temperature";
     :long_name = "Sea Surface Temperature";
     :units = "K";
     :cell_methods = "time: mean (interval: 30 minutes)";
     :_FillValue = 1.0E20; // float
     :missing_value = 1.0E20; // float
     :original_name = "sosstsst";
     :original_units = "degC";
     :history = " At   16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units;  Cyclical dimension was output starting at a different lon;";

 :title = "IPSL  model output prepared for IPCC Fourth Assessment SRES A2 experiment";
 :institution = "IPSL (Institut Pierre Simon Laplace, Paris, France)";
 :source = "IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v";
 :contact = "Sebastien Denvil, sebastien.denvil@ipsl.jussieu.fr";
 :project_id = "IPCC Fourth Assessment";
 :table_id = "Table O1 (13 November 2004)";
 :experiment_id = "SRES A2 experiment";
 :realization = 1; // int
 :cmor_version = 0.96; // float
 :Conventions = "CF-1.0";
 :history = "YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed  At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements";
 :references = "Dufresne et al, Journal of Climate, 2015, vol XX, p 136";
 :comment = "Test drive";

    ncML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml.xml - The attribute information in the netCDF file encoded into core ncML. 

    <?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location="C:/WebDocs/THREDDS/GALEON/SST.NC.test/sst-again.nc">
  <dimension name="lon" length="180"/>
  <dimension name="lat" length="170"/>
  <dimension name="time" length="24" isUnlimited="true"/>
  <dimension name="bnds" length="2"/>
  <attribute name="title" type="String" value="IPSL  model output prepared for IPCC Fourth Assessment SRES A2 experiment"/>
  <attribute name="institution" type="String" value="IPSL (Institut Pierre Simon Laplace, Paris, France)"/>
  <attribute name="source" type="String" value="IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v"/>
  <attribute name="contact" type="String" value="Sebastien Denvil, sebastien.denvil@ipsl.jussieu.fr"/>
  <attribute name="project_id" type="String" value="IPCC Fourth Assessment"/>
  <attribute name="table_id" type="String" value="Table O1 (13 November 2004)"/>
  <attribute name="experiment_id" type="String" value="SRES A2 experiment"/>
  <attribute name="realization" type="int" value="1"/>
  <attribute name="cmor_version" type="float" value="0.96"/>
  <attribute name="Conventions" type="String" value="CF-1.0"/>
  <attribute name="history" type="String" value="YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed  At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements"/>
  <attribute name="references" type="String" value="Dufresne et al, Journal of Climate, 2015, vol XX, p 136"/>
  <attribute name="comment" type="String" value="Test drive"/>
  <variable name="lon" shape="lon" type="double">
    <attribute name="standard_name" type="String" value="longitude"/>
    <attribute name="long_name" type="String" value="longitude"/>
    <attribute name="units" type="String" value="degrees_east"/>
    <attribute name="axis" type="String" value="X"/>
    <attribute name="bounds" type="String" value="lon_bnds"/>
    <attribute name="original_units" type="String" value="degrees_east"/>
  </variable>
  <variable name="lon_bnds" shape="lon bnds" type="double"/>
  <variable name="lat" shape="lat" type="double">
    <attribute name="standard_name" type="String" value="latitude"/>
    <attribute name="long_name" type="String" value="latitude"/>
    <attribute name="units" type="String" value="degrees_north"/>
    <attribute name="axis" type="String" value="Y"/>
    <attribute name="bounds" type="String" value="lat_bnds"/>
    <attribute name="original_units" type="String" value="degrees_north"/>
  </variable>
  <variable name="lat_bnds" shape="lat bnds" type="double"/>
  <variable name="time" shape="time" type="double">
    <attribute name="standard_name" type="String" value="time"/>
    <attribute name="long_name" type="String" value="time"/>
    <attribute name="units" type="String" value="days since 2001-1-1"/>
    <attribute name="axis" type="String" value="T"/>
    <attribute name="calendar" type="String" value="360_day"/>
    <attribute name="bounds" type="String" value="time_bnds"/>
    <attribute name="original_units" type="String" value="seconds since 2001-1-1"/>
  </variable>
  <variable name="time_bnds" shape="time bnds" type="double"/>
  <variable name="tos" shape="time lat lon" type="float">
    <attribute name="standard_name" type="String" value="sea_surface_temperature"/>
    <attribute name="long_name" type="String" value="Sea Surface Temperature"/>
    <attribute name="units" type="String" value="K"/>
    <attribute name="cell_methods" type="String" value="time: mean (interval: 30 minutes)"/>
    <attribute name="_FillValue" type="float" value="1.0E20"/>
    <attribute name="missing_value" type="float" value="1.0E20"/>
    <attribute name="original_name" type="String" value="sosstsst"/>
    <attribute name="original_units" type="String" value="degC"/>
    <attribute name="history" type="String" value=" At   16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units;  Cyclical dimension was output starting at a different lon;"/>
  </variable>
</netcdf>

    ncML-CS - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml-cs.xml - Core ncML plus coordinate system extensions. 

    <?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.ucar.edu/schemas/netcdf" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ucar.edu/schemas/netcdf http://www.unidata.ucar.edu/schemas/netcdf-cs.xsd" uri="C:/WebDocs/THREDDS/GALEON/TestDatasets/sst.nc">
  <dimension name="lon" length="180"/>
  <dimension name="lat" length="170"/>
  <dimension name="time" length="24" isUnlimited="true"/>
  <dimension name="bnds" length="2"/>
  <attribute name="title" type="String" value="IPSL  model output prepared for IPCC Fourth Assessment SRES A2 experiment"/>
  <attribute name="institution" type="String" value="IPSL (Institut Pierre Simon Laplace, Paris, France)"/>
  <attribute name="source" type="String" value="IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v"/>
  <attribute name="contact" type="String" value="Sebastien Denvil, sebastien.denvil@ipsl.jussieu.fr"/>
  <attribute name="project_id" type="String" value="IPCC Fourth Assessment"/>
  <attribute name="table_id" type="String" value="Table O1 (13 November 2004)"/>
  <attribute name="experiment_id" type="String" value="SRES A2 experiment"/>
  <attribute name="realization" type="int" value="1"/>
  <attribute name="cmor_version" type="float" value="0.96"/>
  <attribute name="Conventions" type="String" value="CF-1.0"/>
  <attribute name="history" type="String" value="YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed  At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements"/>
  <attribute name="references" type="String" value="Dufresne et al, Journal of Climate, 2015, vol XX, p 136"/>
  <attribute name="comment" type="String" value="Test drive"/>
  <coordinateAxis name="lon" shape="lon" type="double" units="degrees_east" axisType="Lon">
    <attribute name="standard_name" type="String" value="longitude"/>
    <attribute name="long_name" type="String" value="longitude"/>
    <attribute name="units" type="String" value="degrees_east"/>
    <attribute name="axis" type="String" value="X"/>
    <attribute name="bounds" type="String" value="lon_bnds"/>
    <attribute name="original_units" type="String" value="degrees_east"/>
    <attribute name="_CoordinateAxisType" type="String" value="Lon"/>
  </coordinateAxis>
  <coordinateAxis name="lat" shape="lat" type="double" units="degrees_north" axisType="Lat">
    <attribute name="standard_name" type="String" value="latitude"/>
    <attribute name="long_name" type="String" value="latitude"/>
    <attribute name="units" type="String" value="degrees_north"/>
    <attribute name="axis" type="String" value="Y"/>
    <attribute name="bounds" type="String" value="lat_bnds"/>
    <attribute name="original_units" type="String" value="degrees_north"/>
    <attribute name="_CoordinateAxisType" type="String" value="Lat"/>
  </coordinateAxis>
  <coordinateAxis name="time" shape="time" type="double" units="days since 2001-1-1" axisType="Time">
    <attribute name="standard_name" type="String" value="time"/>
    <attribute name="long_name" type="String" value="time"/>
    <attribute name="units" type="String" value="days since 2001-1-1"/>
    <attribute name="axis" type="String" value="T"/>
    <attribute name="calendar" type="String" value="360_day"/>
    <attribute name="bounds" type="String" value="time_bnds"/>
    <attribute name="original_units" type="String" value="seconds since 2001-1-1"/>
    <attribute name="_CoordinateAxisType" type="String" value="Time"/>
  </coordinateAxis>
  <variable name="lon_bnds" shape="lon bnds" type="double"/>
  <variable name="lat_bnds" shape="lat bnds" type="double"/>
  <variable name="time_bnds" shape="time bnds" type="double"/>
  <variable name="tos" shape="time lat lon" type="float" coordinateSystems="time-lat-lon">
    <attribute name="standard_name" type="String" value="sea_surface_temperature"/>
    <attribute name="long_name" type="String" value="Sea Surface Temperature"/>
    <attribute name="units" type="String" value="K"/>
    <attribute name="cell_methods" type="String" value="time: mean (interval: 30 minutes)"/>
    <attribute name="_FillValue" type="float" value="1.0E20"/>
    <attribute name="missing_value" type="float" value="1.0E20"/>
    <attribute name="original_name" type="String" value="sosstsst"/>
    <attribute name="original_units" type="String" value="degC"/>
    <attribute name="history" type="String" value=" At   16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units;  Cyclical dimension was output starting at a different lon;"/>
  </variable>
  <coordinateSystem name="time-lat-lon">
    <coordinateAxisRef ref="time"/>
    <coordinateAxisRef ref="lat"/>
    <coordinateAxisRef ref="lon"/>
  </coordinateSystem>
</netcdf>

    ncML-GML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml-gml.xml - ncML-GML semantic information and data in ASCII XML. 

    <?xml version="1.0" encoding="UTF-8"?>
<ncco:netcdfCoverageSet uri="test/sampleNetCDF-CF/sst.nc" xmlns="http://www.ucar.edu/schemas/netcdf"
 xmlns:gml="http://www.opengis.net/gml" xmlns:nc="http://www.ucar.edu/schemas/netcdf" 
 xmlns:ncco="http://www.ucar.edu/schemas/netcdf/coverage" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.ucar.edu/schemas/netcdf/coverage
 http://apollo.pin.unifi.it:8080/galeon/schemas/netcdf-coverage.xsd">
  <dimension length="180" name="lon"/>
  <dimension length="170" name="lat"/>
  <dimension isUnlimited="true" length="24" name="time"/>
  <dimension length="2" name="bnds"/>
  <attribute name="title" type="string" value="IPSL  model output prepared for IPCC Fourth Assessment SRES A2 experiment"/>
  <attribute name="institution" type="string" value="IPSL (Institut Pierre Simon Laplace, Paris, France)"/>
  <attribute name="source" type="string" value="IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v"/>
  <attribute name="contact" type="string" value="Sebastien Denvil, sebastien.denvil@ipsl.jussieu.fr"/>

  <attribute name="project_id" type="string" value="IPCC Fourth Assessment"/>
  <attribute name="table_id" type="string" value="Table O1 (13 November 2004)"/>
  <attribute name="experiment_id" type="string" value="SRES A2 experiment"/>
  <attribute name="realization" type="int" value="1"/>
  <attribute name="cmor_version" type="float" value="0.96"/>
  <attribute name="Conventions" type="string" value="CF-1.0"/>
  <attribute name="history" type="string" value="YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed  At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements"/>
  <attribute name="references" type="string" value="Dufresne et al, Journal of Climate, 2015, vol XX, p 136"/>
  <attribute name="comment" type="string" value="Test drive"/>

  <coordinateAxis axisType="Lon" name="lon" shape="lon" type="double" units="degrees_east">
    <attribute name="standard_name" type="string" value="longitude"/>
    <attribute name="long_name" type="string" value="longitude"/>
    <attribute name="units" type="string" value="degrees_east"/>
    <attribute name="axis" type="string" value="X"/>
    <attribute name="bounds" type="string" value="lon_bnds"/>
    <attribute name="original_units" type="string" value="degrees_east"/>
    <attribute name="_CoordinateAxisType" type="string" value="Lon"/>
    <values increment="2.0" npts="180" start="1.0"/>

  </coordinateAxis>
  <coordinateAxis axisType="Lat" name="lat" shape="lat" type="double" units="degrees_north">
    <attribute name="standard_name" type="string" value="latitude"/>
    <attribute name="long_name" type="string" value="latitude"/>
    <attribute name="units" type="string" value="degrees_north"/>
    <attribute name="axis" type="string" value="Y"/>
    <attribute name="bounds" type="string" value="lat_bnds"/>
    <attribute name="original_units" type="string" value="degrees_north"/>
    <attribute name="_CoordinateAxisType" type="string" value="Lat"/>

    <values increment="1.0" npts="170" start="-79.5"/>
</coordinateAxis>
<coordinateAxis axisType="Time" name="time" shape="time" type="double" units="days since 2001-1-1">
    <attribute name="standard_name" type="string" value="time"/>
    <attribute name="long_name" type="string" value="time"/>
    <attribute name="units" type="string" value="days since 2001-1-1"/>
    <attribute name="axis" type="string" value="T"/>
    <attribute name="calendar" type="string" value="360_day"/>
    <attribute name="bounds" type="string" value="time_bnds"/>
    <attribute name="original_units" type="string" value="seconds since 2001-1-1"/>
    <attribute name="_CoordinateAxisType" type="string" value="Time"/>
    <values increment="30.0" npts="24" start="15.0"/>
</coordinateAxis>
<variable name="lon_bnds" shape="lon bnds" type="double"/>
<variable name="lat_bnds" shape="lat bnds" type="double"/>
<variable name="time_bnds" shape="time bnds" type="double"/>
<variable coordinateSystem="time-lat-lon" name="tos" shape="time lat lon" type="float">
    <attribute name="standard_name" type="string" value="sea_surface_temperature"/>
    <attribute name="long_name" type="string" value="Sea Surface Temperature"/>
    <attribute name="units" type="string" value="K"/>
    <attribute name="cell_methods" type="string" value="time: mean (interval: 30 minutes)"/>
    <attribute name="_FillValue" type="float" value="1.0E20"/>
    <attribute name="missing_value" type="float" value="1.0E20"/>
    <attribute name="original_name" type="string" value="sosstsst"/>
    <attribute name="original_units" type="string" value="degC"/>
    <attribute name="history" type="string" value=" At   16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units;  Cyclical dimension was output starting at a different lon;"/>
</variable>
    <coordinateSystem name="time-lat-lon">
    <coordinateAxisRef ref="time"/>
    <coordinateAxisRef ref="lat"/>
    <coordinateAxisRef ref="lon"/>
</coordinateSystem>
<ncco:netcdfGridCoverage>
    <gml:rectifiedGridDomain>
        <gml:RectifiedGrid dimension="3">
            <gml:limits>
                <gml:GridEnvelope>
                    <gml:low>0 0 0</gml:low>
                    <gml:high>179 169 23</gml:high>
                </gml:GridEnvelope>
            </gml:limits>
            <gml:axisName>lon</gml:axisName>
            <gml:axisName>lat</gml:axisName>
            <gml:axisName>time</gml:axisName>
	    <gml:origin>
	        <gml:Point>
                    <gml:coordinates>1.0,-79.5,15.0</gml:coordinates>
                </gml:Point>
            </gml:origin>
            <gml:offsetVector>2.0 0.0 0.0</gml:offsetVector>
            <gml:offsetVector>0.0 1.0 0.0</gml:offsetVector>
            <gml:offsetVector>0.0 0.0 30.0</gml:offsetVector>
        </gml:RectifiedGrid>
    </gml:rectifiedGridDomain>
    <ncco:scalarRangeSet>
        <ncco:netcdfVariableRef referenceName="tos"/>
        <ncco:asciiData>
	    <gml:QuantityList uom="http://my.unidata.ucar.edu/content/software/udunits/udunits.txt#deg_C">
               1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 1.0E20 ...
               ...
               271.46524 271.46527 271.4653 271.46533 271.46536 271.4654 271.46542 ...
            </gml:QuantityList>
        </ncco:asciiData>
    </ncco:scalarRangeSet>
    <ncco:coordinateSystemRef referenceName="time-lat-lon"/>
    <ncco:coordinateReferenceSystem>
        <ncco:spatialCRS>
            <gml:GeographicCRS gml:id="EPSG4326">
                <gml:srsName codeSpace="EPSG">WGS 84</gml:srsName>
                <gml:srsID>
                    <gml:name codeSpace="EPSG">4326</gml:name>
                    <gml:version>6.7</gml:version>
                </gml:srsID>
                <gml:remarks>CRS kind: geographic 2D</gml:remarks>
                <gml:validArea>
                    <gml:description>World</gml:description>
                </gml:validArea>
                <gml:scope>
                    GPS satellite navigation and NATO military surveying
                </gml:scope>
                <gml:usesEllipsoidalCS>
                    <gml:EllipsoidalCS gml:id="EPSG6422">
                        <gml:csName>Ellipsoidal 2D CS</gml:csName>
                        <gml:remarks>Axis order is by element order</gml:remarks>
                        <gml:usesAxis>
                            <gml:CoordinateSystemAxis gml:id="EPSG9901" gml:uom="urn:x-epsg:v0.1:uom:degree">
                                <gml:name>Geodetic latitude</gml:name>
                                <gml:axisID>
                                    <gml:name codeSpace="NcML-CS">lat</gml:name>
                                </gml:axisID>
                                <gml:axisAbbrev>Lat</gml:axisAbbrev>
                                <gml:axisDirection>north</gml:axisDirection>
                            </gml:CoordinateSystemAxis>
                        </gml:usesAxis>
	                <gml:usesAxis>
                            <gml:CoordinateSystemAxis gml:id="EPSG9902" gml:uom="urn:x-epsg:v0.1:uom:degree">
                                <gml:name>Geodetic longitude</gml:name>
                                <gml:axisID>
                                    <gml:name codeSpace="NcML-CS">lon</gml:name>
                                </gml:axisID>
                                <gml:axisAbbrev>Long</gml:axisAbbrev>
                                <gml:axisDirection>east</gml:axisDirection>
                            </gml:CoordinateSystemAxis>
                        </gml:usesAxis>
                    </gml:EllipsoidalCS>
                </gml:usesEllipsoidalCS>
                <gml:usesGeodeticDatum>
	            <gml:GeodeticDatum gml:id="EPSG6326">
                        <gml:datumName>World Geodetic System 1984</gml:datumName>
	                <gml:usesPrimeMeridian>
                            <gml:PrimeMeridian gml:id="EPSG8901">
                                <gml:meridianName>Greenwich</gml:meridianName>
                                <gml:greenwichLongitude>
                                    <gml:angle uom="urn:x-epsg:v0.1:uom:degree">0</gml:angle>
                                </gml:greenwichLongitude>
                            </gml:PrimeMeridian>
                        </gml:usesPrimeMeridian>
                        <gml:usesEllipsoid>
                            <gml:Ellipsoid gml:id="EPSG7030">
                                <gml:ellipsoidName>WGS 84</gml:ellipsoidName>
                                <gml:semiMajorAxis uom="urn:x-si:v1999:uom:metre">6378137</gml:semiMajorAxis>
	                        <gml:secondDefiningParameter>
                                    <gml:inverseFlattening uom="urn:x-bagug:v0.1:dictionary:ifu">298.257223563</gml:inverseFlattening>
                                </gml:secondDefiningParameter>
                            </gml:Ellipsoid>
                        </gml:usesEllipsoid>
                    </gml:GeodeticDatum>
                </gml:usesGeodeticDatum>
            </gml:GeographicCRS>
        </ncco:spatialCRS>
	<gml:TemporalCRS gml:id="customTemporalCRS">
            <gml:srsName>days since 2001-1-1</gml:srsName>
            <gml:usesTemporalCS>
                <gml:TemporalCS gml:id="NetCDF-CF_standard">
                    <gml:csName codeSpace="NetCDF-CF_v1.0">standard/gregorian calendar</gml:csName>
	            <gml:remarks>
                        Mixed Gregorian/Julian calendar as defined by Udunits: Gregorian since 15 oct. 1582, Julian before
                    </gml:remarks>
                    <gml:usesAxis>
                        <gml:CoordinateSystemAxis gml:id="time" gml:uom="http://my.unidata.ucar.edu/content/software/udunits/udunits.txt#days">
                            <gml:name>Time</gml:name>
                            <gml:axisID>
                                <gml:name codeSpace="NcML-CS">time</gml:name>
                            </gml:axisID>
                            <gml:axisAbbrev>Time</gml:axisAbbrev>
                            <gml:axisDirection>up</gml:axisDirection>
                        </gml:CoordinateSystemAxis>
                    </gml:usesAxis>
                </gml:TemporalCS>
            </gml:usesTemporalCS>
            <gml:usesTemporalDatum>
	        <gml:TemporalDatum gml:id="customTemporalOrigin">
                    <gml:datumName>January 1, 2001</gml:datumName>
                    <gml:origin>2001-01-01T00:00:00Z</gml:origin>
               </gml:TemporalDatum>
            </gml:usesTemporalDatum>
        </gml:TemporalCRS>
    </ncco:coordinateReferenceSystem>
</ncco:netcdfGridCoverage>
</ncco:netcdfCoverageSet>

    Geoscience and XML

    There is also much work being done external to NetCDF with XML in the geoscience community. XML-related projects concerning data interoperability in the geosciences include:

    Geography Markup Language (GML) - http://opengis.net/gml/ - An XML grammar defined to express geographic features that serves as a modeling language for geographic systems as well as an open interchange format.
    Earth Science Markup Language (ESML) - http://esml.itsc.uah.edu/index.jsp - An interchange XML technology that enables data (both structural and semantic) interoperability with applications without enforcing a standard format within the Earth science community. Users write external files using ESML schema to describe the structure of the data file, and applications employ the ESML library to parse this description file and decode the data format.
    Climate Science Modeling Language (CSML) - http://ndg.nerc.ac.uk/csml/ - A set of feature types and GML application schema for the standards-based integration of climate-related data from both observations and models.
    MarineXML - http://www.iode.org/marinexml/categories.php?category_no=2 - An XML grammar developed to encapsulate marine data, and to provide an efficient means to store, transfer and display marine data.

    with the:

    Marine Metadata Interoperability - http://marinemetadata.org/

    project providing clearinghouse and guidance roles for XML initiatives in the geosciences.

    While there are many issues that have to be dealt with before a widely accepted and useful standard is available for facilitating the interoperability of data in the geoscience community, the time limits imposed on our project of converting our ROMS model output into input files for GNOME meant converting the ROMS output "standard" metadata directly into GNOME input "standard" metadata.

    Horizontal Grid Issues

    The output fields created by the TGLO/TABS ROMS simulations are on staggered, curvilinear grids. A curvilinear grid is an orthogonal grid mesh created over a non-regular area (such as the Gulf of Mexico) that is mathematically transformed into a regular grid for numerical computations. Geometrical simplicity is gained at the cost of extra terms in the difference equations and therefore extra computational time. A staggered grid is one on which the u and v horizontal velocity components and the density variable are placed on three different overlapping grids. This allows the difference equations and computational algorithms to possess various useful mathematical properties, and also provides some computational advantages.

    When the work to transform ROMS output files into GNOME input files began, our goal was to transform the staggered, curvilinear ROMS fields in NetCDF files into nonstaggered, regular GNOME fields in NetCDF files. After we had completed this task, we obtained a newer version of the GNOME Data Formats document wherein a standard for NetCDF files containing curvilinear grids for input into GNOME was detailed. Not knowing at the time whether the curvilinear grid support was beta or operational, we decided to maintain our production of regular grids in additional to also providing curvilinear grids. Also, since the use of curvilinear grids is not yet as widespread as regular grids - along with the tools with which to handle them - it was decided that it was simply a good idea to maintain the production of regular grid files.

    We initially developed scripts and programs to provide the following GNOME-ready fields:

    These fields were provided in three temporal formats: The nowcast fields are the first 6 hours of each 48-hour forecast field, and the combination of all four of these for a given day creates the hindcast fields.

    An accident involving the sinking of a tanker with unusually dense cargo that spread deeper than the surface layer prompted us to offer additional GNOME-ready fields:

    Both additional fields were made available in the same temporal formats as the original fields.

    The general procedure for tranforming curvilinear grid ROMS output files into regular grid GNOME input files is:

    Only the first and last steps of this procedure are used when transforming curvilinear ROMS output files into curvilinear GNOME input files.

    Vertical Coordinate Issues

    Providing the 3-D current fields exposed another interoperability problem. In additional to using curvilinear coordinates in the horizontal plane, the ROMS model employs what is known as an s-coordinate in the vertical, the details about which can be seen on a page about:

    S-Coordinate Design Philosophy - http://www.ocean-modeling.org/docs_main.php?page=s-coordinate

    Basically, the s-coordinate is a generalization or extension of the more well known terrain-following sigma coordinates. Just as with the curvilinear coordinates in the horizontal, s- and sigma-coordinates are used to transform the vertical plane from irregular topography to a regular rectangle for computational convenience. While sigma-coordinates employ a linear stretching of the vertical coordinate to increase resolution near the surface, s-coordinates apply a nonlinear stretching that depends on the local water depth. This allows a more uniform vertical resolution near the surface and therefore a better representation of the mixed layer and thermocline.

    Data Serving Issues

    Static Web Page

    After the fields have been appropriately transformed into the required format(s), the issue of how to get them from the producer to the consumer must be addressed. The ideal is to make all needed data available in a timely and seamless manner. The fields for a 48-hour prediction simulation must be made available immediately after the simulation has been finished and the post-processing has been performed, and they must be both easily found and obtained. Our initial approach to the issue of making our GNOME-ready input fields available was to place all of the files in a single directory on one of our net-accessible machines. We then configured the Apache web server to create a web page consisting of a list of all of the files in that directory. A page containing a brief explanation of the contents was then created, with a link to the page containing the file list. The results of this approach can be seen at:

    dods.tamu.edu GNOME Files - http://dods.tamu.edu/

    where one can see the less than satisfactory nature of this solution in terms of making the files easy to find and obtain.

    Dynamic Web Page

    Our second attempt at making our GNOME-ready fields available involving creating a custom web page that provides the files in a much more easily understandable format. The second page can be found at:

    csanady.tamu.edu GNOME Files - http://csanady.tamu.edu/GNOME/gnome.html

    and illustrates the improvements made for easier access. The key changes were:

    This web page is created after each TGLO/TABS ROMS simulation, i.e. four times per day, via a Perl script and an HTML template file. The Perl script is:

    Perl Script for Creating GNOME File Web Page
    #!/usr/bin/perl

use Template::Magic;
use Date::Calc qw(Day_of_Week Week_Number Day_of_Year);
use Date::Calc qw(Add_Delta_Days);
use Date::Calc qw(Add_Delta_DHMS);
use Time::Local;
use Getopt::Declare;

#  Set parameters.

#  The URL pointing to where the files will be served from.
$gnomeurlbase = "http://csanady.tamu.edu/GNOME";
$gnomeurldods = "http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/GNOME";

#  The directory containing the script and template file(s).
$SCRIPTDIR = "/home/baum/TGLO/GNOME";
$GNOMEDIR = "/var/www/html/GNOME";

#  The directory in which the calculations are performed.
$WORKDIR = "/home/baum/TGLO/GNOME";

#  Directory in which the GIF, PS and PDF files are archived.
#$ARCHIVEDIR = "/usr/local/apache2/htdocs/TGLO";
#$ARCHIVEDIR = "/usr/local/apache2/htdocs/TGLO";
$GNOMEARCHIVEDIR = "/bfd6000/TGLO/GNOME";
#$ARCHIVEDIR = "/home/baum/TGLO/TSPLOTS/PS";

#  The prefixes used for the daily and weekly time-series comparison files.

$zed = "0";

#  Calculate today.

$year = (localtime)[5] - 100;
$month = (localtime)[4] + 1;
$mday = (localtime)[3];

#$year = 6;
#$month = 5;
#$mday = 22;

#  Calculate yesterday.

$diff_day = -1;
($y_year, $y_month, $y_mday) = Add_Delta_Days($year, $month, $mday, $diff_day);

$diff_week = -7;
($w_year, $w_month, $w_mday) = Add_Delta_Days($year, $month, $mday, $diff_week);

#  Prepend a zero to date strings (if necessary) for labeling purposes.

if ($y_year < 10) {$y_year = $zed.$y_year;};
if ($y_month < 10) {$y_month = $zed.$y_month;};
if ($y_mday < 10) {$y_mday = $zed.$y_mday;};

if ($w_year < 10) {$w_year = $zed.$w_year;};
if ($w_month < 10) {$w_month = $zed.$w_month;};
if ($w_mday < 10) {$w_mday = $zed.$w_mday;};

$date_yest = "$y_year-$y_month-$y_mday";
$date_last = "$w_year-$w_month-$w_mday";

$yest_human = "$y_month/$y_mday/20$y_year";
$last_human = "$w_month/$w_mday/20$w_year";

#  Now to creating the archived file list.  The general strategy:
#
#  1.  Find and create lists of all of the available archived files.
#  2.  Loop through the daily and weekly files and
#     a.  Create a human readable date string from each file name
#     b.  Create an array of hashes containing the linked date strings
#           and file names
#  3.  Use the {hasharray} {datestring}{filename} {\hasharray} construct
#         from Template::Magic to print these in the template file.

#  Find all the archived daily and weekly graphs in PDF format.

@GNAM_fore_reg_48 = glob "$GNOMEARCHIVEDIR/GNAM-fore-reg-48/*";
@GNAM_hind_reg_24 = glob "$GNOMEARCHIVEDIR/GNAM-hind-reg-24/*";
@GROM_fore_crv_all_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-all-48/*";
@GROM_fore_crv_bot_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-bot-48/*";
@GROM_fore_crv_sfc_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-sfc-48/*";
@GROM_fore_reg_sfc_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-reg-sfc-48/*";
@GROM_hind_crv_all_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-all-24/*";
@GROM_hind_crv_bot_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-bot-24/*";
@GROM_hind_crv_sfc_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-sfc-24/*";
@GROM_hind_reg_sfc_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-reg-sfc-24/*";


#  Create an array of hashes containing date strings and PDF file names.
#    ...for the daily files

# split("\Q - \E",$file);
# GNAM-fore-reg-06-05-16-06-48.nc

$no = 0;
$size = @GROM_fore_reg_sfc_48 -1;
foreach $file (@GROM_fore_reg_sfc_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-reg-sfc-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_reg_sfc_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_reg_sfc_48_latest = "$filelist[$size]";
$GROM_fore_reg_sfc_48_url = "$gnomeurlbase/GROM-fore-reg-sfc-48/";
$tmp = "$gnomeurldods/GROM-fore-reg-sfc-48/$GROM_fore_reg_sfc_48_latest";
$GROM_fore_reg_sfc_48_dods_html = "$tmp".".html";
$GROM_fore_reg_sfc_48_dods_dds = "$tmp".".dds";
$GROM_fore_reg_sfc_48_dods_das = "$tmp".".das";
$GROM_fore_reg_sfc_48_date = "$dates[$size]";

#goto FIN;

$no = 0;
$size = @GROM_fore_crv_sfc_48 -1;
foreach $file (@GROM_fore_crv_sfc_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-sfc-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_sfc_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_sfc_48_latest = "$filelist[$size]";
$GROM_fore_crv_sfc_48_url = "$gnomeurlbase/GROM-fore-crv-sfc-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-sfc-48/$GROM_fore_crv_sfc_48_latest";
$GROM_fore_crv_sfc_48_dods_html = "$tmp".".html";
$GROM_fore_crv_sfc_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_sfc_48_dods_das = "$tmp".".das";
$GROM_fore_crv_sfc_48_date = "$dates[$size]";

$no = 0;
$size = @GROM_fore_crv_bot_48 -1;
foreach $file (@GROM_fore_crv_bot_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-bot-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$bot,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_bot_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_bot_48_latest = "$filelist[$size]";
$GROM_fore_crv_bot_48_url = "$gnomeurlbase/GROM-fore-crv-bot-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-bot-48/$GROM_fore_crv_bot_48_latest";
$GROM_fore_crv_bot_48_dods_html = "$tmp".".html";
$GROM_fore_crv_bot_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_bot_48_dods_das = "$tmp".".das";
$GROM_fore_crv_bot_48_date = "$dates[$size]";

$no = 0;
$size = @GROM_fore_crv_all_48 -1;
foreach $file (@GROM_fore_crv_all_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-all-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$all,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_all_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_all_48_latest = "$filelist[$size]";
$GROM_fore_crv_all_48_url = "$gnomeurlbase/GROM-fore-crv-all-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-all-48/$GROM_fore_crv_all_48_latest";
$GROM_fore_crv_all_48_dods_html = "$tmp".".html";
$GROM_fore_crv_all_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_all_48_dods_das = "$tmp".".das";
$GROM_fore_crv_all_48_date = "$dates[$size]";

$no = 0;
$size = @GNAM_fore_reg_48 -1;
foreach $file (@GNAM_fore_reg_48) {
#  Strip off file path.
	($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GNAM-fore-reg-48/\E",$file);
	$filelist[$no] = $filename;
#	print " no = $no  filename = $filename\n";
#	print " file = $file  filename = $filename\n";
#  Split on the dash to get the date and time information.
	($GNAM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
	$date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#	print " daystring = $daystring   dayfilename = $dayfilename\n";
	$GNAM_fore_reg_48_hash[$no] = {datestring => $date,  filename => "$filename"};
	$no = $no + 1;
}
$GNAM_fore_reg_48_latest = "$filelist[$size]";
$GNAM_fore_reg_48_url = "$gnomeurlbase/GNAM-fore-reg-48/";
$tmp = "$gnomeurldods/GNAM-fore-reg-48/$GNAM_fore_reg_48_latest";
$GNAM_fore_reg_48_dods_html = "$tmp".".html";
$GNAM_fore_reg_48_dods_dds = "$tmp".".dds";
$GNAM_fore_reg_48_dods_das = "$tmp".".das";
$GNAM_fore_reg_48_date = "$dates[$size]";


$no = 0;
$size = @GNAM_hind_reg_24 - 1;
foreach $file (@GNAM_hind_reg_24) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GNAM-hind-reg-24/\E",$file);
 	$filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GNAM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
	$GNAM_hind_reg_24_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GNAM_hind_reg_24_latest = "$filelist[$size]";
$GNAM_hind_reg_24_url = "$gnomeurlbase/GNAM-hind-reg-24/";
$tmp = "$gnomeurldods/GNAM-hind-reg-24/$GNAM_hind_reg_24_latest";
$GNAM_hind_reg_24_dods_html = "$tmp".".html";
$GNAM_hind_reg_24_dods_dds = "$tmp".".dds";
$GNAM_hind_reg_24_dods_das = "$tmp".".das";
$GNAM_hind_reg_24_date = "$dates[$size]";

Template::Magic->new->print("$SCRIPTDIR/gnome_template");

system "cd $WORKDIR; cp gnome.html /bfd6000/TGLO/GNOME";

FIN:

    and the HTML template file is:

    HTML Template for Creating GNOME File Web Page
    <HTML>

<HEAD>
<TITLE>GNOME-Ready Input Files from TGLO/TABS ROMS Prediction Simulations</TITLE>
</HEAD>

<FONT SIZE=6>
GNOME-Ready Input Files from TGLO/TABS ROMS Prediction Simulations
</FONT>

<BODY>

<P><HR><P>
<FONT SIZE=5>
Latest Files
</FONT>
<P>
<FONT SIZE=4>
Forecasts
</FONT>
<P>
<FONT SIZE=3>
<TABLE BORDER="4" CELLSPACING="15">
<TR>
<TH>File Description</TH>
<TH>Date and Time</TH>
<TH>Filename</TH>
<TH COLSPAN="3">OPeNDAP</TH>
<TH>Archives</TH>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - surface - regular grid</TD>
<TD>{GROM_fore_reg_sfc_48_date}</TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_url}{GROM_fore_reg_sfc_48_latest}">{GROM_fore_reg_sfc_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_reg_sfc_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - surface - curvilinear grid</TD>
<TD>{GROM_fore_crv_sfc_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_url}{GROM_fore_crv_sfc_48_latest}">{GROM_fore_crv_sfc_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_sfc_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - bottom - curvilinear grid</TD>
<TD>{GROM_fore_crv_bot_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_bot_48_url}{GROM_fore_crv_bot_48_latest}">{GROM_fore_crv_bot_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_bot_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - 3-D - curvilinear grid</TD>
<TD>{GROM_fore_crv_all_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_all_48_url}{GROM_fore_crv_all_48_latest}">{GROM_fore_crv_all_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_all_48">Archive</A></TD>
</TR>
<TR>
<TD>ETA Winds - 48 hour forecast - regular grid</TD>
<TD>{GNAM_fore_reg_48_date}</TD>
<TD><A HREF="{GNAM_fore_reg_48_url}{GNAM_fore_reg_48_latest}">{GNAM_fore_reg_48_latest}</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_das}">das</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_html}">html</A></TD>
<TD><A HREF="#GNAM_fore_reg_48">Archive</A></TD>
</TR>
</TABLE>
<P>
<FONT SIZE=4>
Hindcasts
</FONT>
<P>
<FONT SIZE=3>
<TABLE BORDER="4" CELLSPACING="15">
<TR>
<TH>File Description</TH>
<TH>Date and Time</TH>
<TH>Filename</TH>
<TH COLSPAN="3">OPeNDAP</TH>
<TH>Archives</TH>
</TR>
<TR>
<TD>ETA Winds - 24 hour hindcast - regular grid</TD>
<TD>{GNAM_hind_reg_24_date}</TD>
<TD><A HREF="{GNAM_hind_reg_24_url}{GNAM_hind_reg_24_latest}">{GNAM_hind_reg_24_latest}</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_dds}">dds</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_das}">das</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_html}">html</A></TD>
<TD><A HREF="#GNAM_hind_reg_24">Archive</A></TD>
</TR>
</TABLE>

<P><HR><P>

<FONT SIZE=5>
Archives
</FONT>

<P><HR><P>
<A NAME="GROM_fore_reg_sfc_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - surface - regular grid
</FONT>
<P>
{GROM_fore_reg_sfc_48_hash}
{datestring}: <A HREF="{GROM_fore_reg_sfc_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_reg_sfc_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_sfc_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - surface - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_sfc_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_sfc_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_sfc_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_bot_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - bottom - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_bot_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_bot_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_bot_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_all_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - 3-D - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_all_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_all_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_all_48_hash}

<P><HR><P>
<A NAME="GNAM_fore_reg_48"></A>
<FONT SIZE=4>
ETA Surface Winds - 48 hour forecast - regular grid
</FONT>
<P>
{GNAM_fore_reg_48_hash}
{datestring}: <A HREF="{GNAM_fore_reg_48_url}{filename}">{filename}</A><BR>
{/GNAM_fore_reg_48_hash}

<P><HR><P>
<A NAME="GNAM_hind_reg_24"></A>
<FONT SIZE=4>
ETA Surface Winds - 24 hour hindcast - regular grid
</FONT>
<P>
{GNAM_hind_reg_24_hash}
{datestring}: <A HREF="{GNAM_hind_reg_24_url}{filename}">{filename}</A><BR>
{/GNAM_hind_reg_24_hash}

<P><HR><P>

</BODY>

</HTML>

    When the Perl script is executed, the variables and arrays created therein are substituted into their corresponding locations (within curly brackets) in the HTML template file via the use of a Perl module called Template::Magic.

    OPeNDAP/DODS

    The ultimate goal in serving the GNOME-ready NetCDF files involves the use of software designed for serving these types of files. The

    OPeNDAP/DODS - http://www.opendap.org/

    project is creating software to make array-based data produced anywhere available to those who wish to use it anywhere, as long as web connections are available at both ends. OPeNDAP (Open-source Project for a Network Data Access Protocol) (the name DODS is now a historical artifact) is built on the client/server model wherein servers provide data and clients allow access to that data. OPeNDAP servers can provide data in many formats including NetCDF, and the available clients - over a dozen to date - can download data in all of those formats.

    Particularly interesting and useful for our purposes is the NetCDF client library component, a drop-in replacement for the standard NetCDF library. The standard library is typically used by being compiled into a program or application, which enables them to read and write files in NetCDF format. The ROMS circulation model we use for TGLO/TABS is an example of this. Basically, a NetCDF-enabled program can read a local file in NetCDF format, and access the data in that file natively, e.g. a Fortran program compiled with the library is enabled to read the binary NetCDF-format data in an external file into Fortran arrays, and can write data in Fortran arrays out to a NetCDF file. The OPeNDAP version of the NetCDF library does exactly the same thing, but adds the capability of reading and writing non-local files. If a NetCDF file can be accessed via a web address, then it can be read directly into a program from its location on the web. This is done simply by replacing the file path with a fully qualified URL in the appropriate location in either the program or a data file read by the program.

    Although OPeNDAP is already useful in the capacity just described, it has additional capabilities that greatly extend its utility. In addition to being able to download entire NetCDF files from anywhere on the web, OPeNDAP can be used to:

    The OPeNDAP server is implemented as a CGI script, and files are accessed via a URL resembling:

    http://csanady.tamu.edu/cgi-bin/nph-dods/

    where nph-dods is simply appended to the URL of your CGI bin root address. A quick and simple test to see if the server is working is:

    http://csanady.tamu.edu/cgi-bin/nph-dods/version

    For demonstration purposes, we will now assume a NetCDF file TGLO-his-06-05-24-00-06.nc located in the subdirectory /TGLO/HIS under the base URL.

    The metadata or information about the file can be accessed via appending .das the the URL. The will return the Dataset Attribute Structure, which contains information about the data such as units and variable names. For example,

    http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.das

    will return something looking like this:

    Result of OPeNDAP .das Query
    Attributes {
    ntimes {
        String long_name "number of long time-steps";
    }
    ndtfast {
        String long_name "number of short time-steps";
    }
    dt {
        String long_name "size of long time-steps";
        String units "second";
    }
    dtfast {
        String long_name "size of short time-steps";
        String units "second";
...

    Appending .asc or .ascii to the URL will return the contents of the entire file to your screen in ASCII format. This can be quite huge for our example file, so we'll use an example of Constraint Expression Syntax to return a limited portion of the file. If we append ?dt to our URL (after .asc) we will obtain only the specified variable. Thus, for:

    http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.asc?dt

    we would get:

    Result of OPeNDAP .asc?dt Query
    Dataset: TGLO-his-06-05-24-00-06.nc
dt, 200

    The Constraint Expression Syntax provides the capabilities to access temporal and spatial subsets of the data contained within NetCDF files. The full capabilities of this syntax are explained in the:

    OPeNDAP User Guide - http://www.opendap.org/user/guide-html/

    Appending .html to the URL will obtain a Server Dataset Access Form. For example, for:

    http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.html

    you will receive a web page resembling:

    Result of OPeNDAP .html Query
    OPeNDAP Form

    This HTML form allows you to sample the data at a URL and obtain either an ASCII or a NetCDF file containing the subset you have selected.

    These examples serve to show that OPeNDAP is capable of much more than merely downloading a remote NetCDF file. And since OPeNDAP is an ongoing project, further capabilities are frequently being added.