The unifying concept that underlies most of these requirements is the application of the concept of web services to the goal of providing geophysical data more openly and easily. A simple definition of web services is that of applications that interact with each other using web standards. A useful analogy is that of electrical services. Anyone connected to the electrical grid can immediately use the service, i.e. the electricity, by simply throwing a switch or plugging something into an electrical socket. This is possible because all of the components needed for using this service were standardized long ago, from the decision to use AC instead of DC, to the choice of cycles per second for the AC, to the transformer interface at the house, to the physical wiring of the house, and even to the design of the standard AC outlet. This is not yet true of the Internet, although the application of the web services concept is gradually making it a reality.

Some things are already standardized, for instance the HTML, HTTP and TCP/IP standards that basically created the Internet. Respectively, these were standards created to (1) provide a means to describe the structure of text-based information in a document, (2) provide a means to publish and retrieve the HTML pages, and (3) provide a means for networked machines to exchange such data in packet form. All of this is hidden from the average user who simply uses a web browser to, for example, look at the headlines on cnn.com, because the web browser and the operating system on which it is being used both smoothly implement a surprisingly large number of standards and protocols. For example, the TCP/IP protocol is really a set of seven protocols, and beneath the HTML protocol lies a range of text encoding standards from simple ASCII for encoding the English alphabet to Unicode for allowing all the text and symbols from all of the writing systems of the world to be consistently represented and manipulated by computers.

Although these three standards jump-started the web, they are fundamentally limited to exchanging just text among applications or machines. There are, for example, no provisions for exchanging pictures, binary data or money. This led to the genesis of the web services concept as well as further standards such as XML and SOAP for the implementation thereof. The characteristics of web services include:

• Optional registration with a lookup service.
• A published public interface that enables the client to invoke the service.
• The use of standard web protocols for communication.
• Accessibility via the web.
• Loose coupling among uncoupled, distributed systems during the web services transaction.

SERVICE CREATION A required Service is created using the available and appropriate tools and languages. For example, herein we create a Web Map Service (WMS) to make our predicted surface current vectors available to clients, and we use the MapServer software and the Perl language to do so. SERVICE PUBLICATION After the service is created, it is published at a registry called a Universal Description, Discovery and Integration (UDDI) registry in the form of a Service Container, i.e. a Web Service Description Language (WSDL) file written in XML. The Service Container contains complete information about the service and whoever created it. In our example, the MapServer software creates a WSDL from the information provided in the WMS-enabled MapFile. The creation of a UDDI for such services is currently a research project of several interested groups. SEARCH The Service Requestor searches the UDDI registry for a specific service via an interface provided by UDDI registry provider. In our example, there is as yet no UDDI to be searched. Presently, those with services contact possible clients and provide the needed web address information. Eventually, we hope to have an appropriate UDDI that can be searched for "surface currents" and "Gulf of Mexico" and automatically provide the information. REFERENCE After finding the requested Service, the Service Requestor gets a reference to the Service and inspects the WSDL Service specification. In our example, the MapServer on the client side obtains the MapFile produced by the MapServer on the server side. BINDING The Service Requestor uses the information available in the WSDL reference to bind itself to the Service. In our example, he client-side MapServer uses the MapFile information to bind to the server-side MapServer. INVOCATION The Service is invoked using the WSDL via technologies such as SOAP or XML-RPC, and the Service Requestor gets the required result. In our example, when a WMS layer is requested on the client-side MapServer, it is automatically obtained from the server-side MapServer.

Now for a brief explanation of the protocols/standards mentioned:

XML The eXtensible Markup Language (XML) is an extension and generalization of HTML, with the significant difference being that while HTML tags are for presenting data, XML tags are for describing data. Example HTML tags are BR, TABLE, STRONG, etc., while example XML tags are NAME, AGE, DATE, etc. The XML tags are useful for creating well-formed documents that can be easily exchanged and understood between applications. Possible ambiguity is avoided in XML by the use of namespaces, wherein an identifier is added to a tag to convert a local name into a qualified name, e.g. adding OCEAN to TEMPERATURE to obtain OCEAN:TEMPERATURE to distinguish it from, say, ATMOSPHERE:TEMPERATURE. SOAP and XML-RPC These are both standards for calling procedures on remote machines. That is, they provide the equivalent of local commands for non-local machines. Both of these are lightweight, XML-based protocols that enable the exchange of structured and typed information on the web. SOAP defines elements for specifying method names, parameters and return types, and also defines various data types for exchanging parameters and return values. WSDL WSDL is an XML format for describing the web service interface. It defines the set of operations supported by the server and the format that a client must use when requesting the service. UDDI UDDI can be seen as the Yellow Pages of web services. It allows service providers to publish information about their services for service requestors.

MAPSERVER

MapServer Introduction and Compilation

MapServer is an open-source environment for creating spatially-enabled web applications. It renders spatial data - maps. images and vector data - for the web. A basic MapServer set-up consists of:

• a file containing the web page that both displays the rendered spatial data and enables users to interactively change the image being displayed;
• a MAP file containing information about the objects, i.e. images, vector data, etc., available to be rendered on that web page; and
• a MapServer program that creates a rendered image from the directions contained within and the MAP file specified on the web page.

While the MapServer package itself creates images from the available objects, much of the underlying work such as translating among GIS formats and enabling remote access of objects is performed by libraries that are compiled into the MapServer binary. Translation tasks are handled by the Geospatial Data Abstraction Library (GDAL), a translator library for raster geospatial data formats, and the OGR translator library for vector formats contained within the GDAL package. The remote object access essential for the implementation of OGC standards such as WMS, WFS and WCS is provided by libcurl. Translation among the numerous available map projects is provided by libproj.

Binary installation packages are available for some Linux distributions as well as for Windows and Mac platforms. We chose to compile our initial MapServer installation, largely because none of the available binary distributions contained all of the capabilities we desired. For example, none contained the NetCDF capabilities for GDAL. The initial compilation was performed in early 2005, at which point the compilation process was lengthy and error-prone, especially with GDAL, the largest component of the MapServer compilation process. Problems included the need for specific versions of included libraries (which seemingly changed with each new version release), PHP compatibility issues (the MapScript scripting language is written in PHP), and the same library being compiled by two different packages (at one point both MapServer and GDAL compiled a version of the GIF imaging library, although eventually this was confined to GDAL). In retrospect, most if not all of the problems were due to the rapid and significant developments made throughout the MapServer code hierarchy during 2005. The MapServer code appears to have reached a reasonably stable plateau during early 2006, as is evidenced by the trouble-free compilation of the MapServer version available in late May 2006.

MapFiles

In MapServer, the MapFile sets the configuration of how the map looks, what size it will be, and which layers to show. In form, a MapFile is a text file containing hierarchical object structures that define map settings. Each object or section of the MapFile starts with a keyword and ends with END, with one or more parameters or sub-objects between them. The primary documentation for this is the:

MapFile Reference - http://mapserver.gis.umn.edu/docs/reference/mapfile

A frighteningly skeletal MapFile follows:

MAP
	...
	WEB
		...
	END
	
	LAYER
               ...
	END
	
END

MAP Reference - http://mapserver.gis.umn.edu/docs/reference/mapfile/mapObj

The sub-objects available for defining below the MAP object are:

ANGLE [double] - The angle in degrees to rotate the map, with a default of 0. CONFIG [key] [value] - Defines the location of the EPSG files from the PROJ.4 library. DATAPATTERN [regular expression] - Defines a regular expression to be applied to requests to change DATA parameters via URL requests. DEBUG [on/off] - Enables debugging of the MAP object. EXTENT [minx] [miny] [maxx] [maxy] - The spatial extent of the map to be created. FONTSET [filename] - Absolute filename of fontset file to use. IMAGECOLOR [r] [g] [b] - The background color for the map. IMAGETYPE [gif|png|jpeg|wbmp|gtiff|swf|userdefined] - The output image format to generate. LAYER - The start of a LAYER object. LEGEND - The start of a LEGEND object. MAXSIZE [integer] - The maximum size of the image, e.g. 2048 sets a maximum of 2048 pixels in both dimensions. NAME - [name] - The prefix attached to map, scalebar and legend GIF filenames created. PROJECTION - The start of a PROJECTION object. QUERYMAP - The start of a QUERYMAP object. REFERENCE - The start of a REFERENCE object. RESOLUTION [int] - Sets the pixels per inch for output, but only affects scale computations. The default is 72. SCALE [double] - The computed scale of the map. SCALEBAR - The start of a SCALEBAR object. SHAPEPATH [pathname] - The path to the directory holding the shapefiles or tiles. SIZE [x][y] - The size in pixels of the output image. STATUS [on/off] - Sets the activity of the map. SYMBOLSET [filename] - Absolute filename of the symbolset to use. SYMBOL - The start of a SYMBOL object. TEMPLATEPATTERN [regular expression] - Defines a regular expression to be applied to requests to change TEMPLATE parameters via URL requests. UNITS [feet|inches|kilometers|meters|miles|dd] - Units of the map coordinates as used for scalebar and scale computations. WEB - The start of a WEB object.^

The sub-objects in the skeletal example - with which we are immediately concerned - are LAYER:

CLASS - The start of a CLASS object. CLASSITEM [attribute] - An item name in an attribute table to use for class lookups. CONNECTION [string] - A database connection string to retrieve remote data. CONNECTIONTYPE [local|sde|ogr|postgis|oraclespatial|wms] - The type of connection, with a default of local. DATA [filename]|[sde parameters][postgis table/column][oracle table/column] - The full filename of the spatial data to process, with no file extension is necessary for shapefiles. This can be specified relative to the SHAPEPATH option from the MAP object. DEBUG [on/off] - Enables debugging of this object. DUMP [true/false] - A switch to allow Mapserver to return data in GML format. FEATURE - The start of a FEATURE object. FILTER [string] - This allows for data specific attribute filtering that is done at the same time spatial filtering is done. FILTERITEM [attribute] - An item to use with simple FILTER expressions ( for OGR and shapefiles only). FOOTER [filename] - A template to use after a layer's set of results have been sent (for multiresult query modes only). GRID - The start of a GRID object. GROUP [name] - The name of a group to which this layer belongs. HEADER [filename] - A template to use before a layer's set of results have been sent (multiresult query modes only). LABELANGLEITEM [attribute] - An item name in attribute table to use for class annotation angles (in degrees). LABELCACHE [on/off] - Specifies whether labels should be drawn as the features for this layer are drawn, or whether they should be cached and drawn after all layers have been drawn. LABELITEM [attribute] - An item name in an attribute table to use for class annotation (i.e. labeling). LABELMAXSCALE [double] - The maximum scale at which the layer is labeled. LABELMINSCALE [double] - The minimum scale at which the layer is labeled. LABELREQUIRES [expression] - Sets the context for labeling this layer. LABELSIZEITEM [attribute] - An item name in an attribute table to use for class annotation sizes (in pixels). MAXFEATURES [integer] - Specifies the number of features that should be drawn for this layer in the current window. MAXSCALE [double] - The maximum scale at which this layer is drawn. METADATA Allows for arbitrary data to be stored as name value pairs. MINSCALE [double] - The minimum scale at which this layer is drawn. NAME [string] - A short name for this layer. OFFSITE [r][g][b] - Sets the color index to treat as transparent for raster layers. POSTLABELCACHE [true/false] - Whether or not to render this layer after all labels in the cache have been drawn. PROCESSING [string] - Passes a processing directive to be used with this layer. PROJECTION - The start of a PROJECTION object. REQUIRES [expression] - Sets the context for displaying this layer SIZEUNITS [pixels|feet|inches|kilometers|meters|miles| - Sets the unit of CLASS object SIZE values (default is pixels). STATUS [on|off|default] - Sets the current status of the layer. STYLEITEM [attribute] - Item to use for feature specific styling. SYMBOLSCALE [double] - The scale at which symbols and/or text appear full size. TEMPLATE [file|url] - A global alternative to CLASS TEMPLATE. TILEINDEX [filename] - Full filename for the index or tile definition for this layer. TILEITEM [attribute] - An item that contains the location of an individual tile, with the default "location". TOLERANCE [double] - Sensitivity for point based queries (i.e. via mouse and/or map coordinates). TOLERANCEUNITS [pixels|feet|inches|kilometers|meters|miles|dd] - Units of the TOLERANCE value. TRANSPARENCY [integer|alpha] - Sets the transparency level of all classed pixels for a given layer. TRANSFORM [true/false] - Tells MapServer whether or not a particular layer needs to be transformed from some coordinate system to image coordinates. Default is true. TYPE [point|line|polygon|circle|annotation|raster|query] - Specifies how the data should be drawn.

and WEB:

EMPTY [url] - URL to forward users to if a query fails. If not defined the value for ERROR is used. ERROR [url] - URL to forward users to if an error occurs. Ugly old MapServer error messages will appear if this is not defined FOOTER [filename] - Template to use AFTER anything else is sent. Multiresult query modes only. HEADER [filename] - Template to use BEFORE everything else has been sent. Multiresult query modes only. IMAGEPATH [path] - Path to the temporary directory fro writing temporary files and images. Must be writable by the user the web server is running as. Must end with a / or depending on your platform. IMAGEURL [path] - Base URL for IMAGEPATH. This is the URL that will take the web browser to IMAGEPATH to get the images. LOG [filename] - File to log MapServer activity in. Must be writable by the user the web server is running as. MAXSCALE [double] - Maximum scale at which this interface is valid. When a user requests a map at a bigger scale, MapServer automatically returns the map at this scale. This effectively prevents user from zooming too far out. MAXTEMPLATE [file|url] - Template to be used if above the maximum scale for the app, useful for nesting apps. METADATA - This keyword allows for arbitrary data to be stored as name value pairs. This is used with OGC WMS to define things such as layer title. It can also allow more flexibility in creating templates, as anything you put in here will be accessible via template tags. MINSCALE [double] - Minimum scale at which this interface is valid. When a user reuqests a map at a smaller scale, MapServer automatically returns the map at this scale. This effectively prevents the user from zooming in too far. MINTEMPLATE - Template to be used if above the minimum scale for the app, useful for nesting apps. OUTPUTFORMAT [mime-type] - Format of the query output. Default is "text/html". This is experimental, the use of the OUTPUTFORMAT object is recommended instead. TEMPLATE [filename|url] - Template file or URL tto use in presenting the results to the user in an interactive mode (i.e. map generates map and so on ... )^

Creating Surface Velocity Vector Fields for MapServer Display

Once we had compiled a working MapServer configuration, our first desire was to make our current main web products available via MapServer. This led to some interesting issues as our TGLO web display featured surface current vector fields for both our ROMS and POM simulation model results. Our problem, most generally stated, was how to convert our output NetCDF files containing u and v components of the surface velocity vectors on a curvilinear grid into velocity vectors that could be displayed via MapServer.

MapServer can display both raster- and vector-based geospatial data. Raster data fields contain arrays of pixels that represent spatially continuous fields, while vector data consists of lines connecting points to represent boundaries or other non-continuous data. While the field represented by our displayed vectors is continuous, the vectors themselves are discrete, so we needed to develop a method to represent our surface current velocity vectors as vector-based geospatial data. ( And yes, the two nearly contiguous and disparate uses of the word "vector" did create a bit of confusion along the way.)

First, we searched for previous solutions to the problem. MapServer will render symbols as well as raster- and vector-based files, which provided the basis for the previous solutions we found. Current and wind vectors were plotted as symbols, with the symbols part of a font containing a separate symbol for each direction. For instance, a font containing 180 symbols for the angles 0 through 180 (and, also, for 180 through 360) is used, with if-then statements used to decide which symbol will be used for each vector, i.e. into which directional "bin" and corresponding symbol each vector will fall. Differing vector magnitudes were handled by placing different numbers of barbs on the vector tail, that is, the stronger the vector, the more barbs. Discussion and examples of this method can be found at:

http://nautilus.baruch.sc.edu/bb/viewtopic.php?t=169&start=0&postdays=0&postorder=asc&highlight=
http://lists.gis.umn.edu/pipermail/mapserver-users/2002-December/000041.html

We considered this solution inflexible and inelegant, and searched for another.

After significant trial and error, and working from both endpoints towards the middle, we created the following method for converting our NetCDF output files into surface vectors ready for displaying via MapServer.

1. Extract the surface layer values of U and V from the ROMS output history file, and also the field A containing the angle between the curvilinear grid and a lat/lon grid at each ROMS grid point.
2. Interpolate the U, V and A fields from their separate staggered grid locations to a single grid point location for all.
3. Calculate the angle and magnitude of the vector at each grid point.
4. Rotate the angle from the curvilinear axes to lon/lat axes using the A angle field.
5. Convert the lon/lat values to cartesian coordinates using the program proj from the PROJ.4 package.
6. Convert a file containing lon/lat/angle/magnitude quartets at each grid point into an ASCII ArcInfo generate format file containing information on how to construct a vector arrow in vector format at each grid point. This is done by looping over all the llines in the first file and:
   • writing a consecutive number for each feature, i.e. vector arrow, for each quartet line in the first file;
   • specifying the first of four coordinate pairs as the gridpoint location, i.e. the foot of our vector;
   • calculating the second coordinate pair, i.e. the head of the vector, from a scaling factor and the magnitude and direction of the vector at that grid point;
   • calculating the third coordinate pair as extending 45 degrees to the left of the vector head and a specified (and adjustable) distance;
   • calculating the fourth coordinate pair as extending 45 degrees to the right of the vector head and the same specified distance as the third pair;
   • writing "end" to specify the end of this vector/feature;
   • continuing until the last grid point, and then ending the ArcInfo file with another "end".
7. Use the gen2shp program to convert the ArcInfo generate file into an ESRI ShapeFile, which can be rendered by MapServer.

This process creates an ESRI Shapefile containing a surface velocity vector field in vector format from a NetCDF history file containing the U and V components of the surface velocity field.

Creating Raster Files for Display on MapServer

In addition to the discontinuous surface velocity vectors, we also wanted to develop the capability to create and display continuous fields such as the sea surface height. MapServer can display raster-based images, but it requires that such files contain georeferencing information. We've been routinely creating raster-based images of various fields from the ROMS simulation output using the GMT graphics display package, but they do not contain georeferencing information. The solution to our problem was in the discovery of MB-System, a package built on top of GMT for displaying swatch sonar data.

The general procedure for creating georeferenced raster files from NetCDF history files is:

1. Extract the desired field from the NetCDF history file and interpolate it from the curvilinear coordinate grid to a regular lat/lon grid.
2. Create an ASCII file containing lon/lat/ssh triples.
3. Use the MB-System program mbgrid to convert the lon/lat/ssh triples file into a GMT-compatible NetCDF grd file.
4. Use the MB-System program mbgrdtiff to generate a georeferenced GeoTIFF file from the GMT-compatible NetCDF grd file.

The resulting GeoTIFF file can be rendered and display by MapServer.

Static vs Dynamic MAP Files

The development and use of MapServer and other GIS systems has until relatively recently been predicated on the use and display of fields that do not vary rapidly in time. Typical fields used for display include continental outlines, national and local boundaries, bathymetry, voting trends, city and other feature locations, etc. These types of fields typically vary on the order of years or longer, with monthly changes being about the most rapid variation encountered. If it is desired to add another field such as an updated or new bathymetry file to an operational MapServer site, then it is typically added either by editing the MAP file and web page text files with a text editor, or via the use of one the GUI interfaces that have been created for MapServer development, e.g. Chameleon. While this method is practical for adding the occasional new or changed field, it is not realistically applicable to our situation. We run 48-hour prediction simulations of the Gulf of Mexico four times per day as part of the TGLO/TABS system, and wish to have our latest fields, e.g. surface velocity vectors, sea surface height, available for MapServer display as soon as possible after they are created. Since hand editing the MAP and web page files 4 times per day at 6 hour intervals to create 48 new LAYER objects and a new pulldown menu is a largely impractical endeavor, we have created a system that dynamically and programmatically creates these files. The basic steps in the system are:

• The vector- and raster-based fields desired for display via MapServer, e.g. surface velocity vector Shapefiles and sea surface height GeoTIFF files created as described above, are created as part of the post-processing phase of the TGLO/TABS prediction system. These fields are then placed in a quasi-permanent archive and retains and preserves them. In our case this is a large RAID array that ensures the availibility of both the most current fields and a goodly number of archived fields. Each type of field is placed in a different directory within the archive, and the 48 hourly fields for each simulation are themselves placed into a single tape archive or tar file that significantly reduces the number of files in each directory.
• For each type of field, the appropriate archive directory is searched for the most recently added file. The 48 hourly files contained therein are extracted.
• A MAP file is programmatically created by looping over each individual hourly filename and creating a LAYER object for that file. Basically, NAME and DATA attributes are created for each LAYER object based on the year, month, day and hour contained within the filename. All of the other attributes required for each LAYER object are usually remain unchanged for each 48-hour group of files. The 48 new LAYER objects are combined with standard pre- and post-pended items such as the MAP and PROJECTION objects, and other LAYER objects such as continental outlines and boundaries, to create a fully functional MAP file for this set of simulation results. The MAP file created is given a name that codifies the time frame represented by its contents, e.g. tglo-06-07-23-06.map for a MAP file containing the 48 hourly results for the simulation that started at 0600 on July 23, 2006. A copy of this file is also made with the name tglo-latest.map so the latest available MAP file can be accessed generically.
• A web page file is programmatiically created for the same group of 48 hourly files. Since the individual files are usually accessed via a menu, usually of the drop-down variety, this pragmatically entails creating a new item list for the menu. A few titles may also need to be changed. In the same manner as with the MAP file, both time-specific and generic copies of this file are created.
• Since only one set of 48 hourly files are made available at a time in this MapServer configuration, only those 48 individual Shapefile- and GeoTIFF-format files are kept available in unarchived form for MapServer access. As is discussed below, a larger range of unarchived files may be kept on a sliding basis depending on trade-offs between available machine cycles and storage space.

OGC TECHNOLOGIES AND MAPSERVER

The Open Geospatial Consortium (OGC) is an international consortium of over 300 companies, government agencies and universities who participate in a consensus process to develop publicly available interface specifications. The core mission is to develop standards that enable the interoperability and seamless integration of spatial information, processing software, and spatial services. Spatial information and processing encompass geographic information systems (GIS), remote sensing, surveying and mapping, navigation, location-based services, access to spatial databases, sensor webs and any other technologies and information sources related to spatial information. Interoperability is defined as software components working with each other to overcome tedious batch conversion tasks, import/export obstacles, and distributed resource access barriers imposed by heterogeneous processing environments and heterogeneous data.

Online map services have been provided for years by many companies and organizations, albeit in a tangled jungle of data formats, programming languages and operating systems. The OGC intends to tame this jungle by providing specifications for various services, e.g. obtaining maps, obtaining data about maps, obtaining sensor data, obtaining information about the sensors, etc., that provide a common interface such that these services are interoperable across all platforms.

Towards this goal the OGC has implemented a Specification Program, which works in a format concensus process to develop approved or adopted OpenGIS Specifications; an Interoperability Program, which is a series of hands-on engineering initiatives designed to accelerate the development and acceptance of the specifications; and an Outreach and Adoption Program, wherein OGC and its members offer resources to help technology developers and users take advantage of the open standards.

The Specification Program produces a series of documents, the most important of which are the OpenGIS Specifications that detail the interface structure between software components. A Specification is considered to contain enough detail for implementing an interface if, when implemented by two independent agents, the resulting components will plug and play with each other. Abstract Specifications are first developed and provide the conceptual foundation or reference model for the development of the OpenGIS Specifications. For example, while the former might specify that a map image should be made available in a requested geographic range, the latter will specify in detail the syntax of how that request should be made. Other document series are Discussion Papers, which document a wide range of technology issues currently under discussion for possible adoption as standards, and Best Practices Documents, which discuss recommended procedures for using or implementing an adopted specification.

The most significant of the specifications produced thus far are:

• Geography Markup Language (GML) - The XML standard or grammar defined to express geographical features. This serves as both a modeling language for geographic systems, and an open interchange format for geographic transactions on the Internet.
• Web Map Service (WMS) - A standard for producing and providing digital raster-based maps of spatially referenced data from geographic information.
• Web Coverage Service (WCS) - A standard for providing the raster data sets themselves rather than maps produced from them.
• Web Feature Service (WFS) - A standard for retrieving and updating geospatial data encoded in GML.

Introduction to Web Map Services (WMS)

The Web Map Service (WMS) is one of the OGC technologies for employment in this project. The WMS specification provides for obtaining data from several different locations to create a single digital map. If the software clients and servers conform to this specification, a user can - using an ordinary web browser - overlay into a single image map images obtained from multiple map servers regardless of the map scale, projection, earth coordinate system or digital format at the server end. The maps produced by WMS are usually rendered in a raster-based format such as PNG, GIF or JPEG, but can be rendered in vector-based formats such as SVG or WebCGM. The clients can specify requested layers, layer styles, the geographic area of interest or bounding box, the projected or geographic coordinate reference systems, and the image file format as well as the size of the image.

The WMS standard defines three mandatory operations:

• GetCapabilities - This returns service-level metadata, i.e. a machine- and human-readable description of the content available on the server and the acceptable request parameter values that can be used to obtain the content.
• GetMap - This returns a map with well-defined geographic and dimension parameters
• GetFeatureInfo - This returns information about particular features shown on a map. It is typically used after a GetMap request to choose a point on a map about which more information is desired.

OGC WMS Standards Documents - http://www.opengeospatial.org/standards/wms
Guide to Distributing Your Data Products Via WMS - http://oceanesip.jpl.nasa.gov/esipde/guide.html
WMS Cookbook - http://www.intl-interfaces.com/cookbook/WMS/ Three web log entries were created in the process of learning about WMS and installing WMS capabilities in our local MapServer configuration. These provide further specific details about the procedures and methods described below. WMS Specification Overview - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=107
WMS Server Implementation with MapServer - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=108
WMS Client Implementation with MapServer - http://pong.tamu.edu/tiki/tiki-view_blog_post.php?blogId=8&postId=109

Configuring a MapServer for WMS

The MapServer software includes the capabilities of acting as both a WMS server and client, providing for the handling of all three required requests: GetCapabilities, GetMap, and GetFeatureInfo (although the features available for the last request are currently limited). In the context of the way a MapServer is installed and configured, all three requests are handled via extensions to the items required for the MAP and LAYER objects in the MAP file. Useful guides for configuring MapServer as a WMS server and client are:

• WMS Servers with MapServer - http://mapserver.gis.umn.edu/docs/howto/wms_server/
• WMS Clients with MapServer - http://mapserver.gis.umn.edu/docs/howto/wms_client/
• WMS Time Support with MapServer - http://mapserver.gis.umn.edu/docs/howto/wms_time_support/
• MapScript Wrappers for WxS Services - http://mapserver.gis.umn.edu/docs/howto/wxs_mapscript/

Required MapFile Extensions for WMS

At the MAP object level, the following items are required or recommended:

NAME - Sets the root layer's name in the GetCapabilities request output. PROJECTION - Sets the available advertised projections in which data can be served, at present a small subset of the available EPSG codes. METADATA - The following are required or recommended in the WEB object: wms_title - Sets the root layer's title in the GetCapabilities XML output. wms_onlineresource - Specifies the URL for accessing data from the server. wms_srs - Specifies the EPSG projections in which the data can be served if an EPSG code matching the projection of the data is not available. This is required unless PROJECTION is defined using an EPSG code.

At the LAYER level, the following items are required/recommended:

NAME - Sets a unique LAYER name in the GetCapabilities request output. PROJECTION - See the same entry in the MAP level listing. STATUS - If set to DEFAULT, the layer will always be sent to the client; if set to ON or OFF, it can be requested. METADATA - The following items are required in the LAYER object: wms_title - Sets a unique LAYER title. wms_srs - See the wms_srs item in the MAP level listing.

WMS MapFile Example

An example of a MapFile properly configured for WMS support shown below. The bold items are those that are required or strongly recommended. Note that wms_srs is not used since a supported EPSG code is used for PROJECTION. Note also that the wms_time* items are explained below.

MAP   NAME TGLO_ROMS_SURFACE_CURRENT_VECTORS   IMAGETYPE PNG   EXTENT -10891452.18 2000842.589 -8985428.89 3602117.913   SIZE 600 450   SHAPEPATH "data"   IMAGECOLOR 255 255 255   WEB     TEMPLATE '/var/www/html/MAP/tutorial/tglo_perl.html'      IMAGEPATH '/var/www/html/tmp/'     IMAGEURL '/tmp/'     METADATA       "wms_title" "WMS Server for TGLO ROMS Surface Current Vectors"       "wms_onlineresource" "http://csanady.tamu.edu/cgi-bin/mapserv?wms_tglo_sfc_vec.map"     END   END   PROJECTION     "init=epsg:4326"   END   ...   LAYER     NAME sfcvec-06-06-23-01     METADATA       "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-23T01:00:00Z"       "wms_timeextent" "2006-06-23T01:00:00Z/2006-06-23T01:00:00Z"       "wms_timeitem" "TIME"       "wms_timedefault" "2006-06-23T01:00:00Z"     END     DATA sfcvec-06-06-23-01     STATUS OFF     TYPE LINE     PROJECTION       "init=epsg:4326"     END   END END

Optional WEB and LAYER Object Metadata

A few items are strongly encouraged for the LAYER object level, but most can be inherited from the MAP object level. Many other non-required items are available for the WEB and LAYER METADATA objects. They are useful for the GetCapabilities request, that is, the more METADATA objects that are specified in the MapFile, the more information will be available to those perusing your data via a GetCapabilities request.

Available optional WEB object metadata that can be specified are:

ows_schemas_location - The root of the web tree where the family of OGC WMS XMLSchema files are located. This must be a valid URL where the actual .xsd files are located if you want your WMS output to validate in a validating XML parser. The efault is http://schemas.opengeospatial.net. wms_abstract - A text description of the WMS server. wms_accesscontraints - Information about access constraints. Use none if there are no constraints. wms_addresstype, wms_address, wms_city, wms_stateorprovince, wms_postcode, wms_country - Optional contact address information. All six metadata items are required if any are given. wms_attribution_logourl_format - The MIME type of the logo image. (e.g. "image/png"). Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_height - Height of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_href - URL of the logo image. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_width - Width of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_onlineresource - The data provider's URL. wms_attribution_title - Human-readable string naming the data provider. wms_contactelectronicmailaddress - Contact email address. wms_contactfacsimiletelephone - Contact facsimile telephone number. wms_contactperson, wms_contactorganization, wms_contactposition - Contact information. If any are provided then all three are required. wms_contactvoicetelephone - Contact voice telephone number. wms_encoding - XML capabilities encoding type. The default is ISO-8859-1. wms_feature_info_mime_type - Used to specify an additional MIME type that can be used when responding to the GetFeature request. wms_fees - Fees information. Use the reserved word "none" if there are no fees. wms_keywordlist - A comma-separated list of keywords or keyword phrases to help catalog searching.

Available optional LAYER object metadata are:

gml_exclude_items - A comma-delimited list of items to exclude. This applies only to GetFeatures GML requests. gml_groups - A comma-delimited list of group names for the layer. This applies only to GetFeatures GML requests. gml_[group name]_group - A comma-delimited list of attributes in the group. Here is an example: "gml_include_items" "all" "gml_groups" "display" "gml_display_group" "Name_e,Name_f" gml_include_items - A comma delimited list of items to include, or keyword "all". The default behavior is to include none. This applies only to GetFeatureInfo GML requests. gml_[item name]_alias - An alias for an attribute's name. The served GML will refer to this attribute by the alias. Here is an example: "gml_province_alias" "prov" gml_[item name]_type - Specifies the type of the attribute. Valid values are Integer|Real|Character|Date|Boolean. gml_xml_items - A comma delimited list of items that should not be XML-encoded. wms_abstract - The same as wms_abstract in the Web Object. wms_attribution_logourl_format - The MIME type of the logo image. (e.g. "image/png"). Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_height - Height of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_href - URL of the logo image. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_logourl_width - Width of the logo image in pixels. Note that the other wms_attribution_logourl_* metadata must also be specified. wms_attribution_onlineresource - The data provider's URL. wms_attribution_title - Human-readable string naming the data provider. wms_dataurl_format- Non-standardized file format of the metadata. The layer metadata wms_dataurl_href must also be specified. wms_dataurl_href - The URL to the layer's metadata. The layer metadata wms_dataurl_format must also be specified. wms_extent - Used for the layer's BoundingBox tag for cases where it is impossible (or very inefficient) for MapServer to probe the data source to figure its extents. The value for this metadata is "minx miny maxx maxy" separated by spaces, with the values in the layer's projection units. If wms_extent is provided then it has priority and MapServer will NOT try to read the source file's extents. wms_group_abstract - A blurb of text providing more information about the group. Only one layer for the group needs to contain wms_group_abstract, MapServer will find and use the value. The value found for the first layer in the group is used. So if multiple layers have wms_group_abstract set then only the first value is used. wms_group_title - A human-readable name for the group that this layer belongs to. Only one layer for the group needs to contain wms_group_title, MapServer will find and use the value. The value found for the first layer in the group is used. So if multiple layers have wms_group_title set then only the first value is used. wms_keywordlist - Same as wms_keywordlist in the Web Object . wms_layer_group - Can be used to assign a layer to a number of hierarchically nested groups. This grouped hierarchy will be expressed in the capabilities. All group names should be preceded by a forward slash (/). It is not allowed to use both the WMS_LAYER_GROUP setting and the GROUP keyword for a single layer. An example is: LAYER NAME "mylayer" DATA "mylayer" TYPE LINE CLASS STYLE COLOR 100 100 255 END END METADATA WMS_LAYER_GROUP "/rootgroup/subgroup" END END wms_metadataurl_format - The file format MIME type of the metadata record (e.g. "text/plain"). The layer metadata wms_metadataurl_type and wms_metadataurl_href must also be specified. wms_metadataurl_href - The URL to the layer's metadata. The layer metadata wms_metadataurl_format and wms_metadataurl_type must also be specified. wms_metadataurl_type - The standard to which the metadata complies. Currently only two types are valid: "TC211" which refers to [ISO 19115], and "FGDC" which refers to [FGDC-STD-001-1988]. The layer metadata wms_metadataurl_format and wms_metadataurl_href must also be specified. wms_opaque - Set this metadata to 1 to indicate that the layer represents an area-filling coverage of space (e.g. a bathymetry and elevation layer). This should be taken by the client as a hint that this layer should be placed at the bottom of the stack of layers. wms_srs - Same as wms_srs in the Web Object . wms_style - The LegendURL style name. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style_[style's_name]_legendurl_format, wms_style_[style's_name]_legendurl_href wms_style_[style's_name]_legendurl_height - The height of the legend image in pixels. Requires the following metadata: wms_style_[style's_name]_width, wms_style, wms_style_[style's_name]_legendurl_format, wms_style_[style's_name]_legendurl_href. wms_style_[style's_name]_legendurl_href - The URL to the layer's legend. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style_[style's_name]_legendurl_format, wms_style. wms_style_[style's_name]_legendurl_format - The file format MIME type of the legend image. Requires the following metadata: wms_style_[style's_name]_width, wms_style_[style's_name]_legendurl_height, wms_style, wms_style_[style's_name]_legendurl_href. wms_style_[style's_name]_legendurl_width- The width of the legend image in pixels. Requires the following metadata: wms_style_[style's_name]_format, wms_style_[style's_name]_legendurl_height, wms_style, wms_style_[style's_name]_legendurl_href. wms_timedefault - This value is used if it is defined and the Time value is missing in the request. Please see the WMS Time Support howto for more information. wms_timeextent - (Mandatory for Time Support) This is used in the capabilities to return the valid time values for the layer. The value defined here should be a valid time range. Please see the WMS Time Support howto for more information. wms_timeitem - (Mandatory for Time Support) This is the name of the field in the DB that contains the time values. Please see the WMS Time Support howto for more information. wms_title - Same as wms_title in the Web Object.^

Time Support

Although not a critical matter in traditional GIS applications that employ temporally static layers, the definition of time becomes much more important when one is dealing with layers that change on an hourly basis. The manner in which MapServer handles time is described in:

WMS Time Support - http://mapserver.gis.umn.edu/docs/howto/wms_time_support/view

A valid WMS layer with time support must have the following METADATA at the LAYER level:

wms_timeextent - The single time value represented by each of the 48 LAYER objects is not directly supported in MapServer, although there is an easy workaround. One simply specifies the single time value for both the minimum and maximum in the supported min/max format (required). wms_timeitem - The name of the field in the database that contains the time values (required). wms_timedefault - This value is used if it is defined and the TIME value is missing in the request (optional).^

In regards to the time format, WMS specifies that the basic format used for TIME requests is based on the ISO 8601:1988(E) format. MapServer supports a limited set of patterns that are defined in the ISO 8601 specifications, as well as few other patterns that are useful but not compliant to ISO 8601. A table of supported formats can be found in the [http://mapserver.gis.umn.edu/docs/howto/wms_time_support/ | WMS Time Support] document listed above and is repeated here:

Time Patterns	Examples
YYYYMMDD	20041012
YYYY-MM-DDTHH:MM:SSZ	2004-10-12T13:55:20Z
YYYY-MM-DDTHH:MM:SS	2004-10-12T13:55:20
YYYY-MM-DD HH:MM:SS	2004-10-12 13:55:20
YYYY-MM-DDTHH:MM	2004-10-12T13:55
YYYY-MM-DD HH:MM	2004-10-12 13:55
YYYY-MM-DDTHH	2004-10-12T13
YYYY-MM-DD HH	2004-10-12 13
YYYY-MM-DD	2004-10-12
YYYY-MM	2004-10
YYYY	2004
THH:MM:SSZ	T13:55:20Z
THH:MM:SS	T13:55:20

Creating the WMS MapFiles for TGLO/TABS

A Perl script was created to programmatically create the WMS server MapFile four times per day for the Shapefiles containing the surface velocity vectors derived from the TGLO/TABS ROMS simulation results.

#!/usr/bin/perl use Template::Magic; use CGI::Pretty qw(:standard); use Date::Calc qw(Day_of_Week Week_Number Day_of_Year); use Date::Calc qw(Add_Delta_Days); use Date::Calc qw(Add_Delta_YMDHMS); use Time::Local; use Getopt::Declare; use mapscript; $zed = "0"; # ---------------------------------------------------------------------- # Locate the most recent shapefiles and process them. # ---------------------------------------------------------------------- # Directory containing the tarred and gzipped 48-hour shapefile packages. $ROMS_CUR_DIR = "/bfd6000/TGLO/GRAPH/ROMS-sfcvec-shp"; # Directory from which WMS is served. $WMS_CUR_DIR = "/bfd6000/TGLO/WMS"; # Sample shapefile name: ROMS-sfcvec-shp-48-05-10-05-00.tar.gz # Contains: vecshp-05-10-05-00-01.dbf,shx,shp,prj # vecshp-05-10-05-00-02.dbf,shx,shp,prj # ... # vecshp-05-10-05-00-48.dbf,shx,shp,prj $roms_cur_prefix = "vecshp"; $roms_cur_tar_prefix = "ROMS-sfcvec-shp-48"; # Find all the archived ROMS surface current files available. @roms_cur_files = glob "$ROMS_CUR_DIR/$roms_cur_tar_prefix-*.tar*"; # Find most recent file, i.e. the last one in the array. $roms_cur_current = $roms_cur_files[$#roms_cur_files]; ($name,$tar,$gz) = split ('\.',$roms_cur_current); $roms_cur_current_tar = "$name.$tar"; print " Most recent current file: $roms_cur_current_tar\n"; # Example: # /bfd6000/TGLO/GRAPH/ROMS-sfcvec-shp/ROMS-sfcvec-shp-48-06-06-23-00.tar # Split the directories from the file name. @cur_string = split("/",$roms_cur_current); # Set the name of the tar file containing the shapefiles. $shptarfile = $cur_string[$#cur_string]; print " shptarfile = $shptarfile\n"; # Split the numbers from the prefix. ($pre1,$pre2,$pre3,$hrs,$year,$month,$mday,$fhtgz) = split("-",$cur_string[$#cur_string]); # Split the fh from the tar.gz. ($fh,$tar,$gz) = split ('\.',$fhtgz); $fhfour = $fh."00"; $yearfour = "20".$year; # Uncompress and untar the files within. system "cp $ROMS_CUR_DIR/$shptarfile $WMS_CUR_DIR/data"; system "cd $WMS_CUR_DIR/data; tar xzf $shptarfile"; @vecfiles = `cd $WMS_CUR_DIR/data; ls $roms_cur_prefix-$year-$month-$mday-$fh*.shp`; system "cd $WMS_CUR_DIR/data; rm $shptarfile"; $nromfields = 48; $npomfields = 24; # ------------------------------------------------------------------- # Set the non-volatile parameters, directories, etc. # ------------------------------------------------------------------- $mapname = "TGLO_ROMS_SURFACE_CURRENT_VECTORS"; $imagetype = "PNG"; $minx = -10891452.18; $miny = 2000842.589; $maxx = -8985428.89; $maxy = 3602117.913; $xsize = 600; $ysize = 450; $shapepath = "/bfd6000/TGLO/WMS/data"; $imagecolor = "255 255 255"; $html_template = "tglo-test.html"; $imagepath = "/var/www/html/tmp/"; $imageurl = "/tmp/"; $map_proj = "merc"; $current_proj = "latlong"; $ellps = "WGS84"; $lon_0 = "0"; $lat_0 = "0"; $color = "50 50 50"; $outline_color = "32 32 32"; $wms_title = "WMS Server for TGLO ROMS Surface Current Vectors"; $boundary_layer = "gulf"; $boundary_layer_data = "ESRI/Country_Boundaries_3"; # ---------------------------------------------------------------------- # Create the MAP file # ---------------------------------------------------------------------- #$mapfilename = "wms_tglo_sfv_vec.map"; $mapfilename = "wms.map"; $map = "/bfd6000/TGLO/WMS/$mapfilename"; open (MAP, ">$mapfilename") or die "Cannot open $mapfilename"; print MAP "# MAP file for $year-$month-$mday-$fh\n"; # The MAP object print MAP "MAP\n"; print MAP " NAME $mapname\n"; print MAP " IMAGETYPE $imagetype\n"; print MAP " EXTENT $minx $miny $maxx $maxy\n"; print MAP " SIZE $xsize $ysize\n"; print MAP " SHAPEPATH \"$shapepath\"\n"; print MAP " IMAGECOLOR $imagecolor\n"; print MAP "\n"; # The WEB object. print MAP "WEB\n"; print MAP " TEMPLATE \'$html_template\'\n"; print MAP " IMAGEPATH \'$imagepath\'\n"; print MAP " IMAGEURL \'$imageurl\'\n"; print MAP " METADATA\n"; print MAP " \"wms_title\" \"$wms_title\"\n"; print MAP " \"wms_onlineresource\" \"http://csanady.tamu.edu/cgi-bin/mapserv?$map\"\n"; print MAP " \"wms_abstract\" \"A WMS server for the TGLO/TABS ROMS model Gulf of Mexico prediction simulation results. A 48-hour prediction simulation is performed four times per day, i.e. at 0000, 0600, 1200 and 1800 UCT. The surface current vectors are provided in ESRI Shapefile format.\"\n"; print MAP " \"wms_accessconstraints\" \"none\"\n"; print MAP " \"wms_contactelectronicmailaddress\" \"baum\@stommel.tamu.edu\"\n"; print MAP " \"wms_fees\" \"none\"\n"; print MAP " END\n"; print MAP "END\n"; print MAP "\n"; # The PROJECTION object. print MAP "PROJECTION\n"; print MAP " \"init=epsg:4326\"\n"; print MAP "END\n"; print MAP "\n"; # Creating LAYER objects for each of the 48 ROMS current vector time slice fields. for ($n=0; $n <= $nromfields - 1; $n++) { $np = $n + 1; # Creating WMS-conformant time information. ($e_yr,$e_mo,$e_dy,$e_hr,$e_mn,$e_sc) = Add_Delta_YMDHMS($year,$month,$mday,$fh,0,0, 0,0,0,$np,0,0); if ($e_hr < 10) {$e_hr = "0"."$e_hr";}; if ($e_dy < 10) {$e_dy = "0"."$e_dy";}; if ($e_mo < 10) {$e_mo = "0"."$e_mo";}; if ($e_yr < 10) {$e_yr = "0"."$e_yr";}; $timeend = "20$e_yr-$e_mo-$e_dy".T."$e_hr:00:00Z"; ($shpfile_prefix,$suffix) = split ('\.',$vecfiles[$n]); # To create the LAYER NAME and DATA objects - and the corresponding files in the # /data subdirectory - create copies of the shapefiles based on # relative times, e.g. # # vecshp-06-06-23-00-01.* # ... # vecshp-06-06-23-00-48.* # # that are based on absolute times: # # sfcvec-shp-06-06-23-01.* # ... # sfcvec-shp-06-06-25-00.* # # and then delete the original shapefiles with relative times in their names. ($e_yr,$e_mo,$e_dy,$e_hr,$e_mn,$e_sc) = Add_Delta_YMDHMS($year,$month,$mday,$fh,0,0, 0,0,0,$np,0,0); if ($e_hr < 10) {$e_hr = "0"."$e_hr";}; if ($e_dy < 10) {$e_dy = "0"."$e_dy";}; if ($e_mo < 10) {$e_mo = "0"."$e_mo";}; if ($e_yr < 10) {$e_yr = "0"."$e_yr";}; $timeend = "20$e_yr-$e_mo-$e_dy".T."$e_hr:00:00Z"; ($shpfile_prefix,$suffix) = split ('\.',$vecfiles[$n]); # To create the LAYER NAME and DATA objects - and the corresponding files in the # /data subdirectory - create copies of the shapefiles based on # relative times, e.g. # # vecshp-06-06-23-00-01.* # ... # vecshp-06-06-23-00-48.* # # that are based on absolute times: # # sfcvec-shp-06-06-23-01.* # ... # sfcvec-shp-06-06-25-00.* # # and then delete the original shapefiles with relative times in their names. $layername = "sfcvec-$e_yr-$e_mo-$e_dy-$e_hr"; $filename = "$layername"; system "cp $WMS_CUR_DIR/data/$shpfile_prefix.shp $WMS_CUR_DIR/data/$layername.shp"; system "cp $WMS_CUR_DIR/data/$shpfile_prefix.shx $WMS_CUR_DIR/data/$layername.shx"; system "cp $WMS_CUR_DIR/data/$shpfile_prefix.dbf $WMS_CUR_DIR/data/$layername.dbf"; system "cp $WMS_CUR_DIR/data/$shpfile_prefix.prj $WMS_CUR_DIR/data/$layername.prj"; system "rm $WMS_CUR_DIR/data/$shpfile_prefix.*"; $romfields[$n] = $filename; print MAP "LAYER # surface current vector layer at $np hr\n"; print MAP " NAME $layername\n"; print MAP " METADATA\n"; print MAP " \"wms_title\" \"TGLO ROMS Gulf of Mexico surface current vectors for $timeend\"\n"; print MAP " \"wms_timeextent\" \"$timeend/$timeend\"\n"; print MAP " \"wms_timeitem\" \"TIME\"\n"; print MAP " \"wms_timedefault\" \"$timeend\"\n"; print MAP " END\n"; print MAP " DATA $filename\n"; print MAP " STATUS OFF\n"; print MAP " TYPE LINE\n"; print MAP " PROJECTION\n"; print MAP " \"init=epsg:4326\"\n"; print MAP " END\n"; print MAP " CLASS\n"; print MAP " COLOR $color\n"; print MAP " OUTLINECOLOR $outline_color\n"; print MAP " END\n"; print MAP "END\n"; print MAP "\n"; } print MAP "END\n"; close MAP;

Here is an example of the MapFiles created by the Perl script:

# MAP file for 06-06-23-00 MAP NAME TGLO_ROMS_SURFACE_CURRENT_VECTORS IMAGETYPE PNG EXTENT -10891452.18 2000842.589 -8985428.89 3602117.913 SIZE 600 450 SHAPEPATH "/bfd6000/TGLO/WMS/data" IMAGECOLOR 255 255 255 WEB TEMPLATE 'tglo-test.html' IMAGEPATH '/var/www/html/tmp/' IMAGEURL '/tmp/' METADATA "wms_title" "WMS Server for TGLO ROMS Surface Current Vectors" "wms_onlineresource" "http://csanady.tamu.edu/cgi-bin/mapserv?/bfd6000/TGLO/WMS/wms.map" "wms_abstract" "A WMS server for the TGLO/TABS ROMS model Gulf of Mexico prediction simulation results. A 48-hour prediction simulation is performed four times per day, i.e. at 0000, 0600, 1200 and 1800 UCT. The surface current vectors are provided in ESRI Shapefile format." "wms_accessconstraints" "none" "wms_contactelectronicmailaddress" "baum@stommel.tamu.edu" "wms_fees" "none" END END PROJECTION "init=epsg:4326" END LAYER # surface current vector layer at 1 hr NAME sfcvec-06-06-23-01 METADATA "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-23T01:00:00Z" "wms_timeextent" "2006-06-23T01:00:00Z/2006-06-23T01:00:00Z" "wms_timeitem" "TIME" "wms_timedefault" "2006-06-23T01:00:00Z" END DATA sfcvec-06-06-23-01 STATUS OFF TYPE LINE PROJECTION "init=epsg:4326" END CLASS COLOR 50 50 50 OUTLINECOLOR 32 32 32 END END ... LAYER # surface current vector layer at 48 hr NAME sfcvec-06-06-25-00 METADATA "wms_title" "TGLO ROMS Gulf of Mexico surface current vectors for 2006-06-25T00:00:00Z" "wms_timeextent" "2006-06-25T00:00:00Z/2006-06-25T00:00:00Z" "wms_timeitem" "TIME" "wms_timedefault" "2006-06-25T00:00:00Z" END DATA sfcvec-06-06-25-00 STATUS OFF TYPE LINE PROJECTION "init=epsg:4326" END CLASS COLOR 50 50 50 OUTLINECOLOR 32 32 32 END END END

Testing the Server

Once the server has been successfully configured, it can be tested and used. A typical GetCapabilities request looks like :

http://csanady.tamu.edu/cgi-bin/mapserv?map=/bfd6000/TGLO/WMS/wms_tglo_sfv_vec.map&SERVICE=WMS&REQUEST=GetCapabilities

which will retrieve an XML file detailing what is available at that particular WMS server (with the amount of detail, as discussed above, dependent on the number METADATA items included in the MAP file).

A typical GetMap request looks like:

http://csanady.tamu.edu/cgi-bin/mapserv?map=/bfd6000/TGLO/WMS/wms.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&LAYERS=sfcvec-06-08-18-01

Configuring MapServer as a WMS Client

As with other OGC services, there are specific requirements that must be met by the WMS-client MapFile to implement a MapServer application that reads remote WMS data. First of all, IMAGEPATH parameter must be specified in the WEB object since this directory is used to store the temporary files downloaded from the remote WFS server. Next, it it necessary to include a specific set of parameters in the LAYER objects in the MapFile. The required LAYER objects and metadata are:

CONNECTIONTYPE - This must be specified as WMS. CONNECTION - The URL of the remote WMS server. is the remote server's online resource URL. METADATA - Each LAYER must contain this object with the following parameters: wms_format - The image format to use in GetMap requests. If wms_formatlist is provided, then this is optional and MapServer will pick the first supported format in wms_formatlist for use in GetMap requests. wms_name - A comma-separated list of layers to be fetched from the remote WMS server. wms_server_version - The version of the WMS protocol specification supported by the remote WMS server. wms_srs - A space-delimited list of EPSG projection codes supported by the remote server, as found from a GetCapabilities request.

The optional LAYER objects and metadata are:

MINSCALE, MAXSCALE - These can be set if the remote server's capabilities contains a ScaleHint value for this LAYER. This will enable MapServer to request the layer only at scales where it makes sense. PROJECTION - MapServer will create this internally if needed. METADATA - Each LAYER can contain the following parameters: wms_connectiontimeout - The maximum time in seconds to wait for a remote WMS layer to load, set in seconds, with a default set to 30. (default is 30 seconds). This can also be set at the MAP level to be inherited by the LAYER objects. wms_exceptions_format - Sets the format for exceptions. This defaults to application/vnd.ogc.se_inimage (the exception will be in a picture format). wms_force_separate_request If this is set to 1, it will force this WMS layer to be requested using its own separate GetMap request. By default MapServer will try to merge multiple adjacent WMS layers from the same server into a single multi-layer GetMap request to reduce the load on remote servers and improve response time. wms_formatlist - A comma-separated list of image formats supported by the remote WMS server. This is not needed if wms_formatlist, in which case it will use the first item in that list. wms_latlonboundingbox - The bounding box of this layer in geographic coordinates in the format lon_min lat_min lon_max lat_max. If it is set then MapServer will request the layer only when the map view overlaps that bounding box. wms_style - The name of the style to use for the STYLES parameter in GetMap requests for this layer. wms_[stylename]_sld - The URL of a SLD to use in GetMap requests. Replace [stylename] in the metadata name with the name of the style to which the SLD applies. An example is: "wms_style" "mystyle" "wms_mystyle_sld" "http://my.host.com/mysld.xml" wms_time - The value to use for the TIME parameter in GetMap requests for this layer.^

An example of a correctly configured layer for WMS client capabilities - with the required portions in bold - is:

LAYER   NAME "prov_bound"   TYPE RASTER   STATUS ON   CONNECTION "http://www2.dmsolutions.ca/cgi-bin/mswms_gmap?"   CONNECTIONTYPE WMS   METADATA     "wms_srs" "EPSG:42304"     "wms_name" "prov_bound"     "wms_server_version" "1.1.1"     "wms_format" "image/gif"   END END

Introduction to Web Coverage Service (WCS)

The Web Coverage Service (WCS) extends the Web Map Server (WMS) interface to allow access to geospatial coverages (raster data sets) that represent values or properties of geographic locations, rather than WMS generated maps. That is, it allows you to obtain the data used to render the maps as well as the maps. The OGC WCS standard provides for three operations:

• GetCapabilities - Returns an XML document that describes the service and provides brief descriptions of the available data collections.
• GetCoverage - - Returns a coverage, i.e. a set of data, in a known format.
• DescribeCoverage - Returns an XML document containing a full description of one or more coverages served by a WCS server.

• OGC WCS Specification Documents - http://www.opengeospatial.org/standards/wcs/
  
• NcML/NetCDF Dataset as Form of OGC Web Coverage Service - http://www.unidata.ucar.edu/projects/THREDDS/BenStuff/Documents/WCSnetCDF.htm
  
• THREDDS Web Coverage Server - http://eclipse.ncdc.noaa.gov:9090/thredds/docs/WCS/index.html
• WCS-THREDDS Gateway - http://www.unidata.ucar.edu/projects/THREDDS/OGC/WCS-THREDDS%20Gateway.htm

http://www.unidata.ucar.edu/projects/THREDDS/

which has provided and continues to provide information and guidelines about implementing the XML/metadata and OGC technologies of interest to this project. It also provides useful implementations of these technologies. The THREDDS implementation is particularly valuable in that it deals directly with the NetCDF/NCML files that we and many other geoscientists use to store our data.

Configuring MapServer for WCS

Useful resources for MapServer WCS configuration are:

• WCS Servers with MapServer - http://mapserver.gis.umn.edu/docs/howto/wcs_server
• WCS Server Use Cases - http://mapserver.gis.umn.edu/docs/howto/WCSServerFormatHowTo - Explains how to use MapServer WCS with Modis, Landsat, DEM and NetCDF data.
• MapScript Wrappers for WxS Services - http://mapserver.gis.umn.edu/docs/howto/wxs_mapscript

Within MapServer, the WCS capabilities are implemented in a manner similar to the WMS capabilities - i.e. extra information is supplied via the object attributes in the MapFile - although the support for WCS is at an earlier and less complete stage than for WMS.

MapServer will serve only the LAYER objects that meet the following conditions:

The data is of raster type as supported by GDAL. NAME - A unique name must be supplied. TYPE - This must be set to RASTER. DUMP - This must be set to TRUE.

Available optional WEB object METADATA parameters are listed below. These can be pre-pended with wcs_, ows_ or wms_.

metadatalink - wcs_encoding - description - name - label - keywordlist - fees - accessconstraints -

Available optional LAYER object METADATA parameters are listed below. These can be pre-pended with wcs_, ows_ or wms_.

metadatalink - description - name - label - keywordlist - timeposition - timeperiod - timeitem - The attribute in the spatio/temporal index that contains time values. extent - Must be provided with tiled data. srs - nativeformat - formats -

Axis descriptions are also available as optional LAYER object parameters. MapServer allows you to define a number of these for a layer. Individual axes are identified by name when defining specific metadata. All defines axes must be listed in the rangeset_axes metadata tag so MapServer knows in advance what to expect. A special rangeset for multiband data is automatically generated by adding the name bands to the rangeset_axes list. If found, MapServer will automatically generate metadata for the image bands. You may of course extend that basic support using the naming conventions below.

rangeset_axes - [name]_semantic - [name]_refsys - [name]_refsyslabel - [name]_description - [name]_label - [name]_values - [name]_values_semantic - [name]_values_type - [name]_interval - Only a single interval is supported per axis.

WCS and NetCDF

The use of WCS for specific cases is described in:

MapServer WCS Use Cases - http://mapserver.gis.umn.edu/docs/howto/WCSServerFormatHowTo

with examples given for Modis, Landsat, DEM and NetCDF data. The last case interests us most given our extensive use of NetCDF. However, some issues involving GDAL - the library used by MapServer to handle raster-based files - must be addressed. NetCDF is one of many

GDAL-Supported Raster Formats - http://www.remotesensing.org/gdal/formats_list.html

that the library can read and/or write, and translate from one format to another. A major issue that must be addressed, however, is that while the NetCDF format fundamentally supports 3-D fields, the underlying data structures of GDAL were designed to handle 2-D fields. Fortunately, a workaround is available in the use of bands by GDAL. The band data structure arose because of remote sensing devices that can scan using different bands or wavelengths, and thus make available raster fields of the same 2-D area in several different bands. Given a bit of programming prestidigitation, the GDAL bands can be used as proxies for levels in the vertical dimension in 3-D NetCDF fields.

Installing and Testing NetCDF with GDAL/MapServer

To use NetCDF with GDAL, support for the former must be compiled into the latter (which is then compiled into MapServer). The default compilation procedure for GDAL does not include NetCDF support, so it must be specifically added. The procedure follows:

tar xzvf netcdf.tar.gz
cd netcdf-3.6.1/src
./configure --prefix=/usr/local >& conf.log &
make >& make.log &
make test
su
make install >& inst.log &

cd gdal
./configure --prefix=/usr/local --with-netcdf=/usr/local >& conf.log &
make >& make.log &
su
make install >& inst.log &

To discover whether NetCDF has been successfully compiled into GDAL, we will use the gdalinfo command, more about which can be found at:

gdalinfo Web Page - http://www.remotesensing.org/gdal/gdalinfo.html

to interrogate a NetCDF file that is CF-1.0 compliant. First, we'll look at the file using the ncdump command which dumps an ASCII version of the entire file to the screen or a file. The result of this for GROM-fore-reg-06-08-24-12-48.nc, a CF-1.0 compliant file, is:

netcdf GROM-fore-reg-06-08-24-12-48 {
dimensions:
        lon = 171 ;
        lat = 131 ;
        time = UNLIMITED ; // (49 currently)
variables:
        double lon(lon) ;
                lon:long_name = "Longitude" ;
                lon:standard_name = "longitude" ;
                lon:units = "degrees_east" ;
                lon:point_spacing = "even" ;
        double lat(lat) ;
                lat:long_name = "Latitude" ;
                lat:standard_name = "latitude" ;
                lat:units = "degrees_north" ;
                lat:point_spacing = "even" ;
        double mask(lat, lon) ;
                mask:long_name = "Land Mask" ;
                mask:units = "nondimensional" ;
                mask:standard_name = "land_binary_mask" ;
        double water_u(time, lat, lon) ;
                water_u:long_name = "Eastward Water Velocity" ;
                water_u:standard_name = "surface_eastward_sea_water_velocity" ;
                water_u:units = "m/s" ;
                water_u:axis = "X" ;
                water_u:_FillValue = -9.9999e+32 ;
                water_u:scale_factor = 1. ;
                water_u:add_offset = 0. ;
        double water_v(time, lat, lon) ;
                water_v:long_name = "Northward Water Velocity" ;
                water_v:standard_name = "surface_northward_sea_water_velocity" ;
                water_v:units = "m/s" ;
                water_v:axis = "Y" ;
                water_v:_FillValue = -9.9999e+32 ;
                water_v:scale_factor = 1. ;
                water_v:add_offset = 0. ;
        double time(time) ;
                time:long_name = "Valid Time" ;
                time:standard_name = "time" ;
                time:units = "seconds since 2003-10-28 0:00:00 0:00" ;
                time:axis = "T" ;

// global attributes:
                :title = "ROMS output 48-hour regular grid surface currents" ;
                :institution = "Texas A&M University, Dept.of Oceanography" ;
                :source = "ROMS 2.1" ;
                :history = "none" ;
                :references = "none" ;
                :comment = "none" ;
                :Conventions = "CF-1.0" ;
                :grid_type = "REGULAR" ;

Driver: netCDF/Network Common Data Format
Size is 512, 512
Coordinate System is `'
Metadata:
  NC_GLOBAL#title=ROMS output 48-hour regular grid surface currents
  NC_GLOBAL#institution=Texas A&M University, Dept.of Oceanography
  NC_GLOBAL#source=ROMS 2.1
  NC_GLOBAL#history=none
  NC_GLOBAL#references=none
  NC_GLOBAL#comment=none
  NC_GLOBAL#Conventions=CF-1.0
  NC_GLOBAL#grid_type=REGULAR
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":mask
  SUBDATASET_1_DESC=[131x171] land_binary_mask (64-bit floating-point)
  SUBDATASET_2_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":water_u
  SUBDATASET_2_DESC=[49x131x171] surface_eastward_sea_water_velocity (64-bit floating-point)
  SUBDATASET_3_NAME=NETCDF:"GROM-fore-reg-06-08-24-12-48.nc":water_v
  SUBDATASET_3_DESC=[49x131x171] surface_northward_sea_water_velocity (64-bit floating-point)
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  512.0,    0.0)
Lower Right (  512.0,  512.0)
Center      (  256.0,  256.0)

Serving NetCDF Files with WCS on MapServer

Now that NetCDF has been successfully compiled into GDAL (and MapServer), we take advantage of the features discussed in these list messages:

GDAL NetCDF Support I - http://lists.maptools.org/pipermail/gdal-dev/2005-August/006393.html
GDAL NetCDF Support II - http://www.unidata.ucar.edu/support/help/MailArchives/galeon/msg00068.html

which tell us that GDAL now supports the NetCDF format using the CF metadata convention. Basically, GDAL/MapServer looks for the presence of the variables lat/latitude and lon/longitude in the NetCDF file and, if it finds them, uses them to specify the georeferencing information needed for displaying information on MapServer. To put it another way, MapServer can now read NetCDF/CF files directly from disk and render them, and also transform them into other formats if needed.

#!/usr/bin/perl
use XBase;
opendir(DIR, '.'); # open the current directory
foreach $file (readdir(DIR)) {
  next if !($file =~ /\.dbf$/); # read the dbf file in this directory created by gdaltindex
  print "Working on $file...\n";
  $tfile = 'temporary.dbf';
  system("mv $file $tfile");
  $oldtable = new XBase $tfile or die XBase->errstr;
  print join("\t", $oldtable->field_names) ."\n";
  print join("\t", $oldtable->field_types) ."\n";
  print join("\t", $oldtable->field_lengths) ."\n";
  print join("\t", $oldtable->field_decimals) ."\n";
  $newtable = XBase->create("name" => $file,
                "field_names" => [$oldtable->field_names, "IMGDATE"],  # this is the FILTERITEM in the corresponding tile index layer within your mapfile
                "field_types" => [$oldtable->field_types, "C"],  # character column type
                "field_lengths" => [$oldtable->field_lengths, 13], # length of the date string
                "field_decimals" => [$oldtable->field_decimals, undef]) or die "Error creating new table.";
  foreach (0 .. $oldtable->last_record) {
  ($deleted, @data) = $oldtable->get_record($_);
  print "  ...record $data[0]\n";
      # extract the date
      $year = substr $data[0], 8, 4;  # year  is at position 8 in the filename string
      $month = substr $data[0], 12, 2;  # month is at position 12 in the filename string
      $day = substr $data[0], 14, 2; # day is at position 14 in the filename string
      $hour = substr $data[0], 16, 2;  # hour is at position 16 in the filename string
      $date =  "$year-$month-$day" . "T" . "$hour\n"; # format is YYYY-MM-DDTHH, or any ISO format
   print "$date";
      push @data, "$date";
  $newtable->set_record($_, @data);
  }
  $newtable->close();
  $oldtable->close();
  unlink($tfile);
}

Useful Perl libraries:

GDAL-Perl - http://map.hut.fi/
Geo::Proj4 - http://perl.overmeer.net/
Geo::Point - http://perl.overmeer.net/
Geo-perl Archives - https://list.hut.fi/pipermail/geo-perl/

GDAL OPenDAP - http://gdal.maptools.org/frmt_dods.html | http://home.gdal.org/~warmerda/gdal_opendap_design.html

Accessing MapServer Configured for WCS

To be completed.

WCS, THREDDS and GALEON

In the realm of open-source solutions to serving data via an OGC WMS server, MapServer is not the only option. The

THREDDS Project - http://www.unidata.ucar.edu/projects/THREDDS/

server implementation currently includes a working WCS server, although it is considered experimental. It implements the WCS 1.0.0 specification and serves gridded data in GeoTIFF or NetCDF format. A working THREDDS/WCS server with examples can be found at NCAR:

THREDDS/WCS Server - http://motherlode.ucar.edu:8080/thredds/catalog.html
Server Configuration and Overview - http://motherlode.ucar.edu:8080/thredds/docs/WCS/index.html
WCS-THREDDS Gateway - http://www.unidata.ucar.edu/projects/THREDDS/OGC/WCS-THREDDS%20Gateway.htm

The THREDDS WCS implementation has become part of a larger WCS initiative called GALEON (Geo-interface to Atmosphere, Land, Earth, Ocean and NetCDF) which started out as an OGC WCS Interoperability Experiment (IE). The:

GALEON Network - http://www.ogcnetwork.net/galeon

site provides a central clearinghouse for all information about the project.

The original GALEON IE implemented a geo-interface to NetCDF files via WCS 1.0.0 via a wrapper or layer around existing clients, servers and catalog protocols. Specifically, it leveraged the OPeNDAP and THREDDS projects to attempt to provide 5D grids from forecast model output, where 5D grids consist of 3 spatial dimensions and 2 time dimensions: model run time and forecast time. The objectives of the original project included determining whether:

• a viable WCS getCapabilities interface could be built on THREDDS;
• the ncML-G data model was adequate for providing describeCoverage responses for NetCDF files; and
• NetCDF is a viable WCS binary encoding format.

• select source coverage;
• perform spatial subsetting;
• perform temporal subsetting;
• perform range subsetting;
• perform spatial scaling into the requested bounding box;
• perform reprojection into the requested coordinate system
• perform data format encoding into the requested format; and
• transmit the result.

Introduction to the Web Coverage Processing Service (WCPS)

Also originating from the original project was an OGC discussion paper about a proposed service called the Web Coverage Processing Service (WCPS) for the retrieval and processing of geospatial coverage data. WCPS includes the functionality of WCS and extends it with an expression language for forming requests of arbitrary complexity. WCPS arose from the realization that extending WCS with further specific functions was less feasible and elegant than defining a new service with a coverage processing language that provides arbitrary functionality.

The WCPS proposal extends the coverage concept of WCS by subdividing it into four layers:

although the proposed standard is only concerned with the first three layers.

A WCPS coverage consists of the following constituents:

A set of accessor functions is available for extracting the coverage constituents.

Two operations are defined in WCPS: GetCapabilities and ProcessCoverage. The GetCapabilities operation is identical to that of WCS, except for extending coverage names to names of coverages and coverage sets, of axis types, and with supported format being a server rather than individual coverage property.

The ProcessCoverage operation extends the WCS GetCoverage operation via more powerful processing capabilities including further coverage processing primitives, and the nesting of functions (which allows for arbitrarily complex requests). The WCPS primitives plus the nesting capabilities form an expression languaged called the WCPS expression language.

The WCPS expression language contains the following elements:

for c in (A, B, C)
return tiff(c)

encode( C, f, extraParams)
encode( C, "hdf-eos" )

sqrt ( C + D )

sin( C )

ln( C )

not C

ctransform( C, "EPSG:12345", "nearest-neighbor" )

scale( scale( C, x, s, bicubic), y, s, bicubic )

Introduction to Web Feature Service (WFS)

The Web Feature Service (WFS) is an OGC specification that allows a client to retrieve and update geospatial data encoded in GML from multiple WFS implementations/servers.

The operations defined in the OGC WFS specification are:

GetCapabilities - Describes the capabilities of a WFS server by returning service metadata in response to a request. DescribeFeatureType - GetFeature - GetGmlObject - Allows the request of features and elements by ID from a WFS, which returns an XML document fragment containing the result set. This provides an interface through which a WFS can be asked to traverse and resolve XLinks to the features and elements it serves. Transaction - Used to describe data transformation operations that are to be applied to web accessible feature instances. Upon transaction completion, an XML response is generated. LockFeature - Implements a long-term feature locking mechanism that allows only one transaction at a time to modify the same data item.

Configuring a MapServer for WFS

The procedure described in:

WFS Servers with MapServer - http://mapserver.gis.umn.edu/docs/howto/wfs_server

is summarized below.

A MapServer can be configured to act as a WFS server by suitably modifying the entries in a MapFile. The following conditions must be met for MapServer to serve a layer or include it in the WFS capabilities document.

The data source must be an OGR-supported vector file. Required LAYER parameters: NAME - Must be set and start with a letter. TYPE - Must be LINE, POINT or POLYGON. DUMP - Must be set to TRUE. Required WEB METADATA: wfs_onlineresource - The URL used to access the WFS server must be specified.

An example of a MapFile suitably configured for WFS is:

NAME WFS_server STATUS ON SIZE 400 300 SYMBOLSET ../etc/symbols.sym EXTENT -2200000 -712631 3072800 3840000 UNITS METERS SHAPEPATH "../data" IMAGECOLOR 255 255 255 FONTSET ../etc/fonts.txt WEB IMAGEPATH "/ms4w/tmp/ms_tmp/" IMAGEURL "/ms_tmp/" METADATA "wfs_title" "GMap WFS Demo Server" ## REQUIRED "wfs_onlineresource" "http://127.0.0.1/cgi-bin/mapserv.exe?" ## Recommended "wfs_srs" "EPSG:42304 EPSG:42101 EPSG:4269 EPSG:4326" ## Recommended "ows_schemas_location" "http://ogc.dmsolutions.ca" ## Optional END END PROJECTION "init=epsg:42304" END LAYER NAME province METADATA "wfs_title" "Provinces" ## REQUIRED "gml_include_items" "all" ## Optional (serves all attributes for layer) END TYPE POLYGON STATUS ON DATA province PROJECTION "init=epsg:42304" END DUMP TRUE ## REQUIRED CLASS NAME "Canada" STYLE COLOR 200 255 0 OUTLINECOLOR 120 120 120 END TEMPLATE "ttt_query.html" END END # Layer END # Map File

In addition to these requirements, the following mostly optional WEB object parameters are available.

ows_schema_location - The root of the web tree where a the family of OGC WFS XML schema files are located. This must be a valid URL where the .xsd files are located if the WFS output is to be validated via a validating XML parser. The default value is http://schemas.opengeospatial.net. wfs_abstract - A description providing more information about the server. wfs_accessconstraints - Any access constraints imposed by the service provided on the WFS or data retrieved. wfs_encoding - An XML encoding for all XML documents returned by the server, with a default of ISO-8859-1. wfs_fees - Any fees imposed by the service provider for using the WFS or for data retrieved from it. wfs_keywordlist - A list of words to assist catalog searching. wfs_maxfeatures - The maximum number of elements the server will return. wfs_namespace_prefix - A user-defined namespace prefix to be used in the response to a WFS GetFeature request. wfs_namespace_uri - A user-defined namespace URI to be used in the response to a WFS GetFeature request. wfs_onlineresource - The URL prefix for HTTP GET requests (recommended). wfs_service_onlineresource - A top-level onlineresource URL. MapServer uses the onlineresource metadata (if provided) in the following order: wfs_service_onlineresource (optional) ows_service_onlineresource (optional) wfs_onlineresource (required) wfs_title - A human-readable title to identify the server (required). wfs_srs - A SRS to use for all layers in this server (recommended).

The following LAYER object metadata parameters are available, with all optional unless otherwise noted.

gml_exclude_items, gml_include_items - A comma-delimited list of items to exclude or include. To expose any attributes gml_include_items must first be invoked, followed by a list (if any) of specific items for exclusion. For example, if all attributes but one are included: "gml_include_items" "all" "gml_exclude_items" "Phonenum" gml_groups - A comma-delimited list of group names for the layer, allowing attributes to be collected into related groups. gml_[group_name]_group - A comma-delimited list of attributes in the group. For example, for a group display containing attributes Name_e and Name_: "gml_include_items" "all" "gml_groups" "display" "gml_display_group" "Name_e,Name_f" gml_[item_name]_alias - An alias for the name of an attribute. gml_[item_name]_type [Integer|Real|Character|Date|Boolean] - The type of the attribute. gml_xml_items - A comma-delimited list of items that should not be XML-encoded. wfs_abstract - Same as wfs_abstract in the WEB object. wfs_extent - Used for the layer's BoundingBox tage for cases where it is impossible (or inefficient) for MapServer to probe the data source to figure its extents. The value for this is minx miny maxx maxy separated by spaces, with the values set in the layer's projection units. wfs_keywordlist - Same as wfs_keywordlist in the WEB object. wfs_metadataurl_format - The file format of the metadata record, e.g. XML, SGML or HTML. If this or any of the next two are specified, all three are required. wfs_metadataurl_href - The URL for the layer's metadata. wfs_metadataurl_type - The standard to which the metadata complies. wfs_srs - A SRS used to advertise a LAYER in the capabilities if no SRS is defined at the top level. wfs_title - The same as wfs_title in the WEB object.

Configuring a MapServer as a WFS Client

The procedure described in:

WFS Clients with MapServer - http://mapserver.gis.umn.edu/docs/howto/wfs_client

is summarized below.

A WFS server publishes feature-level geospatial data to the web, and this data can be used as a source to render a map. It is similar to accessing a Shapefile over the web, except instead of the binary Shapefile you access an ASCII GML file containing the geospatial data, including both geometry and attribute information.

As with other OGC services, there are specific requirements that must be met by the WFS client MapFile to read remote WFS data. There are:

Required WEB object parameters: IMAGEPATH - Must be specified since this directory is used to store the temporary files downloaded from a WFS server. Required LAYER object parameters: CONNECTIONTYPE - Must be LINE, POINT or POLYGON. CONNECTION - Must be set to TRUE. Required LAYER METADATA parameters: wfs_typename - The NAME of the layer as obtained via GetCapabilities (required). wfs_version - The version of the WFS specification (required). wfs_connectiontimeout - The number of seconds to wait for a remote WFS layer to load, with a default of 30 seconds (optional). This can also be added at the MAP level and inherited here. wfs_filter - For including a filter encoding parameter in the getFeature request (optional). wfs_latlongboundingbox - Sets a bounding box in geographic coordinates of the form lon_min lat_min lon_max lat_max (optional). If this is set, the layer will be requested only when the map view overlaps the bounding box. wfs_maxfeatures - The maximum number of GML features to return (optional). wfs_request_method - Set to GET for WFS servers that do not support POST requests (optional).

An example of a LAYER properly configured to act as a WFS client is:

LAYER NAME park TYPE POLYGON STATUS ON CONNECTIONTYPE WFS CONNECTION "http://www2.dmsolutions.ca/cgi-bin/mswfs_gmap?" METADATA "wfs_typename" "park" "wfs_version" "1.0.0" "wfs_request_method" "GET" "wfs_connectiontimeout" "60" "wfs_maxfeatures" "1" END PROJECTION "init=epsg:42304" END CLASS NAME "Parks" STYLE COLOR 200 255 0 OUTLINECOLOR 120 120 120 END END END # Layer

Introduction to Sensor Web Enablement (SWE)

The Sensor Observation Service (SOS) is part of the ongoing OGC Sensor Web Enablement (SWE) project, which works on implementing interoperability interfaces and metadata encoding to enable the real-time integration of heterogeneous sensor webs into the information infrastructure. A series of specifications are being developed for creating appliations, platforms and products involving any web-connected device that can serve as a sensor. This category includes ocean data buoys, flood gauges, air pollution monitors, stress gauges on bridges, webcams, robits, space and airborne earth imaging devices, etc. Basically, any autonomous or semi-autonomous device that can record observations that can also be connected to the web qualifies for SWE.

The currently available SWE projects with accompanying candidate specifications are:

• Observations & Measurements (O&M) - Standard models and XML schema for encoding observations and measurements from a sensor, both archived and in real-time.
• Sensor Model Language (SML) - Standard models and XML schema for describing sensor systems and processes. This provides the information needed for the discovery of sensors, the location of sensor observations, the processing of low-level sensor observations, and the listing of taskable properties.
• Transducer Model Language (TML) - The conceptual model and XML schema for describing transducers and supporting the real-time streaming of data to and from sensor systems.
• Sensor Observations Service (SOS) - A standard web interface for requesting, filtering, and retrieving observations and sensor system information. This is the intermediary between a client and an observation repository or near real-time sensor channel.
• Sensor Planning Service (SPS) - A standard web interface for requesting user-driven acquisitions and observations. This is the intermediary between a client and a sensor collection management environment.
• Sensor Alert Service (SAS) - A standard web interface for publishing and subscribing to alerts from sensors.
• Web Notification Services (WNS) - A standard web interface for the asynchronous delivery of messages or alerts from SAS and SPS web services and other elements of service workflows.

Introduction to Sensor Observations Service (SOS)

The Sensor Observations Service (SOS) provides an API for managing deployed sensors and retrieving sensor data, specifically observation data. When used with the other OGC specifications, SOS provides a range of interoperable capability for discovering, binding to, and interrogating individual sensors, sensor platforms, or networked groups of sensors in real-time, archived or simulated environments.

An SOS organizes collections of related sensor system observations into Observation Offerings (OO), which are analogous to WMS layers in that each OO is intended to be a non-overlapping group of related observations. When grouping observations in an SOS into OO, the goal should be to minimize the likelihood that a valid query request will get an empty response. That is, reasonable queries should usually produce results.

A critical interoperability issue is to define a standard way to refer to phenomena measures by sensors, and the units of measure for those phenomena. Since the range of phenomena and units of measure is large, unknown a-priori, and possibly even unknowable, the solution is to use dictionaries in GML format. Servers and clients can then use URIs to refer to specific entries in a particular dictionary.

The concepts of units of measure and phenomena are defined separately within the SWE initiative. Units of measure are defined based on the schema developed for GML, while a GML-conformant scheme for defining base and derived phenomena has been developed as part of the initiative.

The operations defined in the SOS specification are divided into four profiels: core, enhanced, transactional and entire.

The SOS Core Operations

The three mandatory core operations are GetObservation, DescribeSensor and GetCapabilities.

The GetObservation operation provides access to sensor observations and measurement data through a spatio-temporal query that can be filtered by time, space, sensor and phenomena. The request and response requirements are:

The DescribeSensor operation provides detailed information about the sensors making the measurements, and the platforms that carry the sensors. The request and response requirements are:

The GetCapabilities provides SOS service metadata. The request and response requirements are:

The SOS Transactional Operations

The optional transactional operations are RegisterSensor and InsertObservation.

The RegisterSensor operation allows the client to register a new sensor system with the SOS as part of the transactional profile. Sensor observations can only be inserted via InsertObservation for sensors that have first been registered via this operation.

The InsertObservation operation allows a client to insert new observations into a sensor system.

The SOS Enhanced Operations

The optional enhanced operations are GetResult, GetFeatureOfInterest, GetFeatureOfInterestTime, DescribeFeatureOfInterest, DescribeObservationType and DescribeResultModel.

The GetResult operation allows a client to repeatedly obtain data from the same set of sensors without having to send and receive requests and responses that are redundant except for the timestamp. This operation relies on the creation of an O&M template from a previous call to GetObservation, with the ID of the template used for subsequent GetResult calls rather than a duplicate GetObservation XML document. The response contains only the result portion of the observation, with the other components included by reference to the template.

The GetFeatureOfInterest operation obtains information about features of interest.

The GetFeatureOfInterestTime operation returns the time periods for which the SOS will return data.

The DescribeFeatureOfInterest operation returns the XML schema for the specified GML feature of interest advertised in the GetCapabilities document.

The DescribeObservationType operation returns the XML schema that describes the Observation type returned for a particular phenomenon.

The DescribeResultModel operation returns the schema for the result element returned when the client asks for the result model by the ResultName.

Configuring MapServer as an SOS Server

The configuration procedure described in:

Sensor Observation Server with MapServer - http://mapserver.gis.umn.edu/docs/howto/sos_server

is summarized in the following.

A MapServer can be configured to act as an SOS server, with each instance of an SOS server requiring a separate MapFile. The MapFile is configured for SOS duty by adding some parameters and metadata entries. The required or strongly recommended parameters and metadata at the MAP level are:

NAME - PROJECTION - The required WEB object METADATA are: sos_title - sos_onlineresource - This specifies the URL used to access the SOS server. sos_srs - EPSG projection code denoting the projection of the offering. Required unless PROJECTION object is defined using "init=epsg:...

The required or strongly recommended parameters and metadata at the LAYER level are:

NAME - PROJECTION - The required LAYER object METADATA are: sos_offering_id - The ID of the offering. sos_observedproperty_id - The ID of the observed property. sos_describesensor_url - A temporary measure until the describe sensor is generated from MapServer. For DescribeSensor requests, it will redirect to the URL defined by: "sos_describesensor_url" "http://some.edu/url/NS01EE0014.xml" sos_procedure - A unique sensor ID, with one per layer.

An example of an SOS server MapFile that meets the bare minimum requirements is:

NAME SOS_DEMO STATUS ON SIZE 300 300 EXTENT -66 44 -62 45 UNITS METERS SHAPEPATH "./data/" IMAGECOLOR 255 255 0 SYMBOLSET "./etc/symbols.sym" IMAGETYPE png WEB IMAGEPATH "/ms4w/tmp/ms_tmp/" IMAGEURL "/ms_tmp/" METADATA "sos_onlineresource" "http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map" ## REQUIRED "sos_title" "My SOS Demo Server" ## Recommended "sos_srs" "epsg:4326" ## Recommended END END PROJECTION "init=epsg:4326" END LAYER NAME test_sos_layer METADATA "sos_procedure" "NS01EE0014" ## REQUIRED "sos_offering_id" "WQ1289" ## REQUIRED "sos_observedproperty_id" "Water Quality" ## REQUIRED "sos_describesensor_url" "http://some/url/NS01EE0014.xml" ## REQUIRED END TYPE POINT STATUS ON DATA sos_test PROJECTION "init=epsg:4326" END CLASS NAME "water quality" STYLE COLOR 255 0 0 SYMBOL "circle" SIZE 8 END END END END #map

Additional optional metadata are available for the WEB and LAYER objects. The full list of available WEB object SOS metadata is:

ows_schemas_location - The root of the web tree when the family of OGC SOS XML schema files are located. This must be a valid URL where the .xsd files are located if the SOS output is to be validated with a validating XML parser. sos_abstract - A description of the SOS server. sos_accessconstraints - Access constraints imposed on the SOS or data provided. sos_addresstype, sos_address, sos_city, sos_postcode, sos_country - If any of these optional contact address items are provided, then all are required. sos_contactelectronicmailaddress - The email address of a contact. sos_contactperson, sos_contactposition - sos_contactvoicetelephone - A voice telephone number for contact. sos_fees - Information about fees if any, with a default value of none.

A list of optional MAP object SOS metadata is:

sos_[item_name]_alias - An alias for an attribute's name that will be returned with GetObservation requests. sos_[item_name]_componenturl - An associated URL for a component that will be returned with GetObservation requests. sos_observedproperty_name - The name of the observed property. sos_offering_description - A description of the offering. sos_offering_extent - The spatial extent of the offering in minx, miny, maxx, maxy format. An example is: "sos offering_extent" "-66, 43, -62, 45" sos_offering_name - The name of the offering. sos_offering_timeextent - The time extent of the offering, in a "begin/end" format. If the end portion is not specified, it will be set to now. An example is: "sos offering_timeextent" "1990/2003" sos_timeitem - The name of the time field. This is layer specific and should be set in all layers. sos_srs - EPSG projection code denoting the projection of the offering.

Web Services, SOAP and SOS

SOAP Services

One of the goals of the

OOS Tech 2005 Follow-Up Plan - http://twiki.sura.org/twiki/bin/view/Main/OosTechFollowUp

was to promote using and serving data via generic web services as implemented with various open source software packages designed for such tasks. A few definitions are in order at this point:

Web Service - A software system designed to support interoperable machine-to-machine interaction over a network. This commonly refers to services that use SOAP-formatted XML envelopes and have their interfaces described by WSDL.

SOAP - An XML-based message envelope format with bindings to underlying protocols such as HTTP.

WSDL (Web Services Definition Language) - An XML format for describing web services and their bindings to specific protocols.

UDDI (Universal Description Discovery and Integration) - A protocol for publishing and discovering metadata about web services, to enable applications to find them.

Basically, WSDL is used to describe what sort of web service you are offering, e.g. salinity or temperature values, SOAP is used to obtain what is being offered, and UDDI is used to discover if any salinity or temperature values are being offered via SOAP.

The initial OOSTech implementation of web services employed the

Perl SOAP::Lite - http://www.soaplite.com/

implementation of the SOAP protocol, and the work done to implement the

OOS Tech Service Definition - http://twiki.sura.org/twiki/bin/view/Main/OosTechServiceDefinition

used for the

OpenIOOS Interoperability Test Bed - http://twiki.sura.org/twiki/bin/view/Main/InteroperabilityDemo

is documented in the

OpenIOOS SOAP::Lite Cookbook - http://twiki.sura.org/twiki/bin/view/Main/SoapliteCookBook

The following web services were planned to be the initial OpenIOOS offerings:

TABS SOAP Services

Here we detail the steps used to create a simple SOAP service for obtaining temperature values from the TABS buoys. While the eventual goal of such technology is to request such things directory from the sensors or platforms, the TGLO/TABS buoys are not yet ready for that task. We obtain the processed temperatures and velocity components from a GERG-based server on a daily basis, from which flat database files are created for each of the TABS buoys. The SOAP server detailed below makes available the most recent parameter available in a flat database file for a given buoy.

After the installation of the SOAP::Lite package on our server, we created the following server script:

#!/usr/bin/perl

#----------------------------------------------------------------------------------

#  These lines set up a SOAP wrapper around a class.

use SOAP::Transport::HTTP;

SOAP::Transport::HTTP::CGI

#  The next line refers to the class package TABSData below.  Dispatching
#  is the process of determining what class should handle a request, and
#  then loading that class.  The subroutine TABSGetW contains the only
#  class available in this simple example.

#  More than one class package can be specified.  If another module, e.g.
#  TABSModel.pm, were available, the next line would be:
#
#        ->dispatch_to('TABSData','TABSModel')
#
#  A dynamic dispatch method is also available wherein at line such as:
#
#        ->dispatch_to('/usr/soaplite/modules')
#
#  could be specified, with the given directory automatically searched
#  for the requested modules.

-> dispatch_to('TABSData')
-> handle;

#-----------------------------------------------------------------------------------

#  These lines are the package TABSData containing the class TABSGetW.
#  These could also exist in a separate file TABSData.pm in the same
#  or a different directory than TABS-server.pl.  It is only required
#  that the module TABSData.pm be in a location specified within @INC,
#  the Perl variable containing the list of directories to search for
#  library files.

package TABSData;

sub TABSGetW {

my ($class, $parameter) = @_;
my $result = -9999;

#  These lines define the name and location of the archive file containing
#  the buoy data; in this case buoy W.

my $buoy = "W";
my $soap_pre = "soap_archive_";
my $DAT_DIR = "/var/www/html/TABS-DAT";
my $datfile = "$DAT_DIR/$soap_pre$buoy.d";

my %data;

#  Here we open the W archive file and obtain the last line in a
#  hash containing all the individual column values.  For this example
# our data file looks like this:
#                                             Current
#    Date       Time        Lon.       Lat.   Speed  Dir.  Temp.
#2006-04-03T23:00:00   -96.0221   28.33476   0.2299   209   19.66
#2006-04-03T23:30:00   -96.0221   28.33476   0.2043   220   19.62

open(DATAFILE, "<$datfile");
while (<DATAFILE>) {
  @data{'datetime','lon','lat','currentspd','currentdir','temp'} = split(" ",$_);
}
close(DATAFILE);

$result = $data{$parameter};

return ($result);
}
1;

along with a matching client program:

#!/usr/bin/perl -w

use SOAP::Lite;

#  The package name TABSData from the server script is called as
#  part of the following URI to identify the class package
#  desired by the client.

my $uri   = "http://csanady.tamu.edu/TABSData";

#  This identifies the CGI script that supplies access to the
#  desired module, i.e. it is the server script placed in a
#  suitable CGI directory.

my $proxy = "http://csanady.tamu.edu/cgi-bin/TABS-server.pl";

$tglo_response = SOAP::Lite
  -> uri("$uri")
   -> proxy("$proxy")

#  This is where the specific subroutine/class within the module
#  is called.  If multiple values are available - e.g. the six
#  values stored in @data in the server script - then the desired
#  value is requested as a parameter of TABSGetW.

   -> TABSGetW('currentspd');

$result = $tglo_response->result;

print " Result is $result\n";

SOS-Compatible SOAP Services

Eventually, it was decided to align the existing web service activities with the OGC Sensor Web Enablement (SWE) initiative. Therefore, the existing operational SOAP-based XML service definition and demonstration project were to adopt the standards and interoperability demonstrations being developed by the OGC, while attempting to adhere to the CF-1.0 conventions. These activities are documented at:

Moving Toward SWE - http://twiki.sura.org/twiki/bin/view/Main/OOSTechSWE

TGLO/TABS INTEROPERABILITY WITH GNOME MODEL

The Motivation and the Crisis

GNOME (General NOAA Oil Modeling Environment) is the oil spill trajectory model used by HAZMAT responders during an oil spill. When there is a spill, the GNOME model predicts how wind, water currents, and other processes might move and spread the oil. Traditionally, climatological average values of wind and water currents have been used to force the GNOME model, but recently the capability to ingest and use spatially and temporally varying fields of wind and water currents has been incorporated into the model. The required formats for this are detailed in the:

GNOME Data Formats Report - http://response.restoration.noaa.gov/type_topic_entry.php?RECORD_KEY%28entry_topic_type%29=entry_id,topic_id,type_id&entry_id(entry_topic_type)=298&topic_id(entry_topic_type)=1&type_id(entry_topic_type)=3

Part of the long-term goals of the TGLO/TABS program is to enable those in charge of the TGLO oil spill remediation program to force the GNOME model with the TGLO/TABS surface current prediction system model output. The development of this part of the program was accelerated by Hurricane Katrina. The NOAA facilities at Bay St. Louis that had been operationally providing predicted surface current fields for the Gulf of Mexico were unavailable for several weeks after the passage of the hurricane. This created a situation wherein NOAA only had climatological wind and surface currents to force force the GNOME model. We received a request from NOAA to provide wind and surface current fields while the Bay St. Louis facilities were down. We used the information available in the GNOME Data Formats Report to produce GNOME-ready surface current and wind fields within a week of the request. Here we describe the process of and the problems encountered in producing these fields.

http://www.unidata.ucar.edu/software/netcdf/

Both the data and metadata are stored in a machine-independent binary format that conserves storage space. In the case of GNOME fields, the required ASCII format would require huge files for storing and transmitting the needed 2-D (and even larger 3-D) current fields, so we chose to go with the NetCDF format. While there is some "spin-up" time required for a novice to become familiar with the use of NetCDF, we have extensive experience with it. Also, NetCDF has become a de-facto standard in the geoscience community for storing and transmitting array-oriented data, with many data analysis programs and circulation models capable of NetCDF input and/or output. Additionally, our choice was motivated by the fact that the ROMS circulation model, i.e.

ROMS - http://marine.rutgers.edu/po/index.php?model=roms

we employ to produce our Gulf of Mexico prediction simulations uses NetCDF as the default standard for both input and output files.

Metadata Issues

NetCDF and XML

Although our experience with and the use of NetCDF by both the front-end (ROMS) and back-end (GNOME) models would seem to make our task straightforward, a few curveballs were encountered along the way. First of all, while NetCDF is a widely accepted and used standard data encoding format wherein the files that are created are machine independent, the data model consists only of abstract variables, dimensions and attributes. A convention is required to transform the NetCDF format from the abstract to the realm of the concrete. For example, there has been until relatively recently no widely accepted method for specifying the name used for the horizontal velocity components created by circulation models. Examples of terminology that has been used for this variable include "u", "uvel", "u-velocity" and "eastward sea water velocity."

There have been several projects that have attempted to standardize the NetCDF metadata conventions, e.g.

COARDS - http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
CF - http://www.cgd.ucar.edu/cms/eaton/cf-metadata/
GDT - http://www-pcmdi.llnl.gov/drach/GDT_convention.html
SEACOOS - http://marinemetadata.org/content/refs/scncdfstd/

with discussion and information about previous and ongoing work at:

Unidata NetCDF Conventions - http://www.unidata.ucar.edu/software/netcdf/conventions.html

providing an overview of the state-of-the-art. The CF convention seems to be becoming the de facto community standard NetCDF profile.

A further complicating factor is the collision of the XML bulldozer (rollercoaster?) with this work. Not only must work continue on standardizing the names of the variables and attributes that will populate the metadata portion of the file, but also on an XML schema into which these will fit. The most advanced project in this regard - and the one that seems to be gaining wider acceptance - is the CF project, although there are still unresolved issues that need to be addressed, e.g. staggered and unstructured grids, GIS/spatial information content, ontologies, discovery information and others.

The primary document explaining the current plans for incorporating NetCDF and XML is

Encoding netCDF Semantics and/or Data into XML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/NcMLAndGML.htm

which explains the various projects underway:

ncML - An XML representation of NetCDF metadata. NcML is similar to the Common Data Language (CDL) used to provide a text description of a NetCDF dataset, with the obvious difference being that it uses XML rather than ASCII text.

ncML-CS - A variant of ncML with extensions for georeferencing. This extends the core ncML grammar by adding coordinate systems and axis semantics, although the encoding is not GML-based.

ncML-GML - A variant of ncML with GML extensions. This contains all the core NetCDF semantics plus semantics useful for GIS-based applications, with particular attention paid to coordinate systems and projection information.

An appreciation of the similarities and differences among these representations can be obtained by perusing the same header file represented in four different ways:

• NetCDF - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncdump.txt - The text output of the ncdump utility.
• ncML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml.xml - The attribute information in the netCDF file encoded into core ncML.
• ncML-CS - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml-cs.xml - Core ncML plus coordinate system extensions.
• ncML-GML - http://www.unidata.ucar.edu/projects/THREDDS/GALEON/TestDatasets/sst-ncml-gml.xml - ncML-GML semantic information and data in ASCII XML.

Geoscience and XML

There is also much work being done external to NetCDF with XML in the geoscience community. XML-related projects concerning data interoperability in the geosciences include:

Geography Markup Language (GML) - http://opengis.net/gml/ - An XML grammar defined to express geographic features that serves as a modeling language for geographic systems as well as an open interchange format.
Earth Science Markup Language (ESML) - http://esml.itsc.uah.edu/index.jsp - An interchange XML technology that enables data (both structural and semantic) interoperability with applications without enforcing a standard format within the Earth science community. Users write external files using ESML schema to describe the structure of the data file, and applications employ the ESML library to parse this description file and decode the data format.
Climate Science Modeling Language (CSML) - http://ndg.nerc.ac.uk/csml/ - A set of feature types and GML application schema for the standards-based integration of climate-related data from both observations and models.
MarineXML - http://www.iode.org/marinexml/categories.php?category_no=2 - An XML grammar developed to encapsulate marine data, and to provide an efficient means to store, transfer and display marine data.

with the:

Marine Metadata Interoperability - http://marinemetadata.org/

project providing clearinghouse and guidance roles for XML initiatives in the geosciences.

While there are many issues that have to be dealt with before a widely accepted and useful standard is available for facilitating the interoperability of data in the geoscience community, the time limits imposed on our project of converting our ROMS model output into input files for GNOME meant converting the ROMS output "standard" metadata directly into GNOME input "standard" metadata.

Horizontal Grid Issues

The output fields created by the TGLO/TABS ROMS simulations are on staggered, curvilinear grids. A curvilinear grid is an orthogonal grid mesh created over a non-regular area (such as the Gulf of Mexico) that is mathematically transformed into a regular grid for numerical computations. Geometrical simplicity is gained at the cost of extra terms in the difference equations and therefore extra computational time. A staggered grid is one on which the u and v horizontal velocity components and the density variable are placed on three different overlapping grids. This allows the difference equations and computational algorithms to possess various useful mathematical properties, and also provides some computational advantages.

When the work to transform ROMS output files into GNOME input files began, our goal was to transform the staggered, curvilinear ROMS fields in NetCDF files into nonstaggered, regular GNOME fields in NetCDF files. After we had completed this task, we obtained a newer version of the GNOME Data Formats document wherein a standard for NetCDF files containing curvilinear grids for input into GNOME was detailed. Not knowing at the time whether the curvilinear grid support was beta or operational, we decided to maintain our production of regular grids in additional to also providing curvilinear grids. Also, since the use of curvilinear grids is not yet as widespread as regular grids - along with the tools with which to handle them - it was decided that it was simply a good idea to maintain the production of regular grid files.

We initially developed scripts and programs to provide the following GNOME-ready fields:

• ROMS simulated surface current fields on a regular, unstaggered grid;
• ROMS simulated surface current fields on a curvilinear, unstaggered grid;
• NCEP simulated surface wind fields on a regular, unstaggered grid;

• 48-hour forecasts 4 times per day;
• 6-hour nowcasts 4 times per day;
• 24-hour hindcasts once per day.

An accident involving the sinking of a tanker with unusually dense cargo that spread deeper than the surface layer prompted us to offer additional GNOME-ready fields:

• ROMS simulated bottom current fields on a curvilinear, unstaggered grid;
• ROMS simulated 3-D current fields on a curvilinear, unstaggered grid.

The general procedure for tranforming curvilinear grid ROMS output files into regular grid GNOME input files is:

• Spatially average the U and V points to obtain U and V values at the RHO points.
• Calculate the U and V components on a regular lon/lat grid from those on the curvilinear grid using the ANGLE array containing the angle between the curvilinear and regular grid at each RHO point.
• Interpolate the rotated U and V components from the curvilinear grid to a pre-defined regular grid.
• Write the U and V values into a NetCDF file using the metadata required by the GNOME Input Formats report.

Vertical Coordinate Issues

Providing the 3-D current fields exposed another interoperability problem. In additional to using curvilinear coordinates in the horizontal plane, the ROMS model employs what is known as an s-coordinate in the vertical, the details about which can be seen on a page about:

S-Coordinate Design Philosophy - http://www.ocean-modeling.org/docs_main.php?page=s-coordinate

Basically, the s-coordinate is a generalization or extension of the more well known terrain-following sigma coordinates. Just as with the curvilinear coordinates in the horizontal, s- and sigma-coordinates are used to transform the vertical plane from irregular topography to a regular rectangle for computational convenience. While sigma-coordinates employ a linear stretching of the vertical coordinate to increase resolution near the surface, s-coordinates apply a nonlinear stretching that depends on the local water depth. This allows a more uniform vertical resolution near the surface and therefore a better representation of the mixed layer and thermocline.

Data Serving Issues

Static Web Page

After the fields have been appropriately transformed into the required format(s), the issue of how to get them from the producer to the consumer must be addressed. The ideal is to make all needed data available in a timely and seamless manner. The fields for a 48-hour prediction simulation must be made available immediately after the simulation has been finished and the post-processing has been performed, and they must be both easily found and obtained. Our initial approach to the issue of making our GNOME-ready input fields available was to place all of the files in a single directory on one of our net-accessible machines. We then configured the Apache web server to create a web page consisting of a list of all of the files in that directory. A page containing a brief explanation of the contents was then created, with a link to the page containing the file list. The results of this approach can be seen at:

dods.tamu.edu GNOME Files - http://dods.tamu.edu/

where one can see the less than satisfactory nature of this solution in terms of making the files easy to find and obtain.

Dynamic Web Page

Our second attempt at making our GNOME-ready fields available involving creating a custom web page that provides the files in a much more easily understandable format. The second page can be found at:

csanady.tamu.edu GNOME Files - http://csanady.tamu.edu/GNOME/gnome.html

and illustrates the improvements made for easier access. The key changes were:

• creating a table at the top of the page that always contains the most recent files available;
• clearly and succinctly explaining the contents and time-frame of each file in each table entry;
• providing a link to the archives of each different type of file; and
• providing OPeNDAP/DODS-enabled links to each of the files.

This web page is created after each TGLO/TABS ROMS simulation, i.e. four times per day, via a Perl script and an HTML template file. The Perl script is:

#!/usr/bin/perl

use Template::Magic;
use Date::Calc qw(Day_of_Week Week_Number Day_of_Year);
use Date::Calc qw(Add_Delta_Days);
use Date::Calc qw(Add_Delta_DHMS);
use Time::Local;
use Getopt::Declare;

#  Set parameters.

#  The URL pointing to where the files will be served from.
$gnomeurlbase = "http://csanady.tamu.edu/GNOME";
$gnomeurldods = "http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/GNOME";

#  The directory containing the script and template file(s).
$SCRIPTDIR = "/home/baum/TGLO/GNOME";
$GNOMEDIR = "/var/www/html/GNOME";

#  The directory in which the calculations are performed.
$WORKDIR = "/home/baum/TGLO/GNOME";

#  Directory in which the GIF, PS and PDF files are archived.
#$ARCHIVEDIR = "/usr/local/apache2/htdocs/TGLO";
#$ARCHIVEDIR = "/usr/local/apache2/htdocs/TGLO";
$GNOMEARCHIVEDIR = "/bfd6000/TGLO/GNOME";
#$ARCHIVEDIR = "/home/baum/TGLO/TSPLOTS/PS";

#  The prefixes used for the daily and weekly time-series comparison files.

$zed = "0";

#  Calculate today.

$year = (localtime)[5] - 100;
$month = (localtime)[4] + 1;
$mday = (localtime)[3];

#$year = 6;
#$month = 5;
#$mday = 22;

#  Calculate yesterday.

$diff_day = -1;
($y_year, $y_month, $y_mday) = Add_Delta_Days($year, $month, $mday, $diff_day);

$diff_week = -7;
($w_year, $w_month, $w_mday) = Add_Delta_Days($year, $month, $mday, $diff_week);

#  Prepend a zero to date strings (if necessary) for labeling purposes.

if ($y_year < 10) {$y_year = $zed.$y_year;};
if ($y_month < 10) {$y_month = $zed.$y_month;};
if ($y_mday < 10) {$y_mday = $zed.$y_mday;};

if ($w_year < 10) {$w_year = $zed.$w_year;};
if ($w_month < 10) {$w_month = $zed.$w_month;};
if ($w_mday < 10) {$w_mday = $zed.$w_mday;};

$date_yest = "$y_year-$y_month-$y_mday";
$date_last = "$w_year-$w_month-$w_mday";

$yest_human = "$y_month/$y_mday/20$y_year";
$last_human = "$w_month/$w_mday/20$w_year";

#  Now to creating the archived file list.  The general strategy:
#
#  1.  Find and create lists of all of the available archived files.
#  2.  Loop through the daily and weekly files and
#     a.  Create a human readable date string from each file name
#     b.  Create an array of hashes containing the linked date strings
#           and file names
#  3.  Use the {hasharray} {datestring}{filename} {\hasharray} construct
#         from Template::Magic to print these in the template file.

#  Find all the archived daily and weekly graphs in PDF format.

@GNAM_fore_reg_48 = glob "$GNOMEARCHIVEDIR/GNAM-fore-reg-48/*";
@GNAM_hind_reg_24 = glob "$GNOMEARCHIVEDIR/GNAM-hind-reg-24/*";
@GROM_fore_crv_all_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-all-48/*";
@GROM_fore_crv_bot_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-bot-48/*";
@GROM_fore_crv_sfc_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-crv-sfc-48/*";
@GROM_fore_reg_sfc_48 = glob "$GNOMEARCHIVEDIR/GROM-fore-reg-sfc-48/*";
@GROM_hind_crv_all_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-all-24/*";
@GROM_hind_crv_bot_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-bot-24/*";
@GROM_hind_crv_sfc_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-crv-sfc-24/*";
@GROM_hind_reg_sfc_24 = glob "$GNOMEARCHIVEDIR/GROM-hind-reg-sfc-24/*";


#  Create an array of hashes containing date strings and PDF file names.
#    ...for the daily files

# split("\Q - \E",$file);
# GNAM-fore-reg-06-05-16-06-48.nc

$no = 0;
$size = @GROM_fore_reg_sfc_48 -1;
foreach $file (@GROM_fore_reg_sfc_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-reg-sfc-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_reg_sfc_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_reg_sfc_48_latest = "$filelist[$size]";
$GROM_fore_reg_sfc_48_url = "$gnomeurlbase/GROM-fore-reg-sfc-48/";
$tmp = "$gnomeurldods/GROM-fore-reg-sfc-48/$GROM_fore_reg_sfc_48_latest";
$GROM_fore_reg_sfc_48_dods_html = "$tmp".".html";
$GROM_fore_reg_sfc_48_dods_dds = "$tmp".".dds";
$GROM_fore_reg_sfc_48_dods_das = "$tmp".".das";
$GROM_fore_reg_sfc_48_date = "$dates[$size]";

#goto FIN;

$no = 0;
$size = @GROM_fore_crv_sfc_48 -1;
foreach $file (@GROM_fore_crv_sfc_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-sfc-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_sfc_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_sfc_48_latest = "$filelist[$size]";
$GROM_fore_crv_sfc_48_url = "$gnomeurlbase/GROM-fore-crv-sfc-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-sfc-48/$GROM_fore_crv_sfc_48_latest";
$GROM_fore_crv_sfc_48_dods_html = "$tmp".".html";
$GROM_fore_crv_sfc_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_sfc_48_dods_das = "$tmp".".das";
$GROM_fore_crv_sfc_48_date = "$dates[$size]";

$no = 0;
$size = @GROM_fore_crv_bot_48 -1;
foreach $file (@GROM_fore_crv_bot_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-bot-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$bot,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_bot_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_bot_48_latest = "$filelist[$size]";
$GROM_fore_crv_bot_48_url = "$gnomeurlbase/GROM-fore-crv-bot-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-bot-48/$GROM_fore_crv_bot_48_latest";
$GROM_fore_crv_bot_48_dods_html = "$tmp".".html";
$GROM_fore_crv_bot_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_bot_48_dods_das = "$tmp".".das";
$GROM_fore_crv_bot_48_date = "$dates[$size]";

$no = 0;
$size = @GROM_fore_crv_all_48 -1;
foreach $file (@GROM_fore_crv_all_48) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GROM-fore-crv-all-48/\E",$file);
        $filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GROM,$fore,$crv,$all,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#       print " daystring = $daystring   dayfilename = $dayfilename\n";
        $GROM_fore_crv_all_48_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GROM_fore_crv_all_48_latest = "$filelist[$size]";
$GROM_fore_crv_all_48_url = "$gnomeurlbase/GROM-fore-crv-all-48/";
$tmp = "$gnomeurldods/GROM-fore-crv-all-48/$GROM_fore_crv_all_48_latest";
$GROM_fore_crv_all_48_dods_html = "$tmp".".html";
$GROM_fore_crv_all_48_dods_dds = "$tmp".".dds";
$GROM_fore_crv_all_48_dods_das = "$tmp".".das";
$GROM_fore_crv_all_48_date = "$dates[$size]";

$no = 0;
$size = @GNAM_fore_reg_48 -1;
foreach $file (@GNAM_fore_reg_48) {
#  Strip off file path.
	($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GNAM-fore-reg-48/\E",$file);
	$filelist[$no] = $filename;
#	print " no = $no  filename = $filename\n";
#	print " file = $file  filename = $filename\n";
#  Split on the dash to get the date and time information.
	($GNAM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
	$date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
#	print " daystring = $daystring   dayfilename = $dayfilename\n";
	$GNAM_fore_reg_48_hash[$no] = {datestring => $date,  filename => "$filename"};
	$no = $no + 1;
}
$GNAM_fore_reg_48_latest = "$filelist[$size]";
$GNAM_fore_reg_48_url = "$gnomeurlbase/GNAM-fore-reg-48/";
$tmp = "$gnomeurldods/GNAM-fore-reg-48/$GNAM_fore_reg_48_latest";
$GNAM_fore_reg_48_dods_html = "$tmp".".html";
$GNAM_fore_reg_48_dods_dds = "$tmp".".dds";
$GNAM_fore_reg_48_dods_das = "$tmp".".das";
$GNAM_fore_reg_48_date = "$dates[$size]";


$no = 0;
$size = @GNAM_hind_reg_24 - 1;
foreach $file (@GNAM_hind_reg_24) {
#  Strip off file path.
        ($a,$filename) = split("\Q$GNOMEARCHIVEDIR/GNAM-hind-reg-24/\E",$file);
 	$filelist[$no] = $filename;
#  Split on the dash to get the date and time information.
        ($GNAM,$fore,$reg,$yr,$mn,$dy,$hr,$ln) = split("\Q-\E",$filename);
        $date = "$mn/$dy/20$yr $hr"."00 UCT";
	$dates[$no] = $date;
#  Create an entry in an array of hashes.
	$GNAM_hind_reg_24_hash[$no] = {datestring => $date,  filename => "$filename"};
        $no = $no + 1;
}
$GNAM_hind_reg_24_latest = "$filelist[$size]";
$GNAM_hind_reg_24_url = "$gnomeurlbase/GNAM-hind-reg-24/";
$tmp = "$gnomeurldods/GNAM-hind-reg-24/$GNAM_hind_reg_24_latest";
$GNAM_hind_reg_24_dods_html = "$tmp".".html";
$GNAM_hind_reg_24_dods_dds = "$tmp".".dds";
$GNAM_hind_reg_24_dods_das = "$tmp".".das";
$GNAM_hind_reg_24_date = "$dates[$size]";

Template::Magic->new->print("$SCRIPTDIR/gnome_template");

system "cd $WORKDIR; cp gnome.html /bfd6000/TGLO/GNOME";

FIN:

and the HTML template file is:

<HTML>

<HEAD>
<TITLE>GNOME-Ready Input Files from TGLO/TABS ROMS Prediction Simulations</TITLE>
</HEAD>

<FONT SIZE=6>
GNOME-Ready Input Files from TGLO/TABS ROMS Prediction Simulations
</FONT>

<BODY>

<P><HR><P>
<FONT SIZE=5>
Latest Files
</FONT>
<P>
<FONT SIZE=4>
Forecasts
</FONT>
<P>
<FONT SIZE=3>
<TABLE BORDER="4" CELLSPACING="15">
<TR>
<TH>File Description</TH>
<TH>Date and Time</TH>
<TH>Filename</TH>
<TH COLSPAN="3">OPeNDAP</TH>
<TH>Archives</TH>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - surface - regular grid</TD>
<TD>{GROM_fore_reg_sfc_48_date}</TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_url}{GROM_fore_reg_sfc_48_latest}">{GROM_fore_reg_sfc_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_reg_sfc_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_reg_sfc_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - surface - curvilinear grid</TD>
<TD>{GROM_fore_crv_sfc_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_url}{GROM_fore_crv_sfc_48_latest}">{GROM_fore_crv_sfc_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_sfc_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_sfc_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - bottom - curvilinear grid</TD>
<TD>{GROM_fore_crv_bot_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_bot_48_url}{GROM_fore_crv_bot_48_latest}">{GROM_fore_crv_bot_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_bot_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_bot_48">Archive</A></TD>
</TR>
<TR>
<TD>ROMS Surface Currents - 48 hour forecast - 3-D - curvilinear grid</TD>
<TD>{GROM_fore_crv_all_48_date}</TD>
<TD><A HREF="{GROM_fore_crv_all_48_url}{GROM_fore_crv_all_48_latest}">{GROM_fore_crv_all_48_latest}</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_das}">das</A></TD>
<TD><A HREF="{GROM_fore_crv_all_48_dods_html}">html</A></TD>
<TD><A HREF="#GROM_fore_crv_all_48">Archive</A></TD>
</TR>
<TR>
<TD>ETA Winds - 48 hour forecast - regular grid</TD>
<TD>{GNAM_fore_reg_48_date}</TD>
<TD><A HREF="{GNAM_fore_reg_48_url}{GNAM_fore_reg_48_latest}">{GNAM_fore_reg_48_latest}</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_dds}">dds</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_das}">das</A></TD>
<TD><A HREF="{GNAM_fore_reg_48_dods_html}">html</A></TD>
<TD><A HREF="#GNAM_fore_reg_48">Archive</A></TD>
</TR>
</TABLE>
<P>
<FONT SIZE=4>
Hindcasts
</FONT>
<P>
<FONT SIZE=3>
<TABLE BORDER="4" CELLSPACING="15">
<TR>
<TH>File Description</TH>
<TH>Date and Time</TH>
<TH>Filename</TH>
<TH COLSPAN="3">OPeNDAP</TH>
<TH>Archives</TH>
</TR>
<TR>
<TD>ETA Winds - 24 hour hindcast - regular grid</TD>
<TD>{GNAM_hind_reg_24_date}</TD>
<TD><A HREF="{GNAM_hind_reg_24_url}{GNAM_hind_reg_24_latest}">{GNAM_hind_reg_24_latest}</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_dds}">dds</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_das}">das</A></TD>
<TD><A HREF="{GNAM_hind_reg_24_dods_html}">html</A></TD>
<TD><A HREF="#GNAM_hind_reg_24">Archive</A></TD>
</TR>
</TABLE>

<P><HR><P>

<FONT SIZE=5>
Archives
</FONT>

<P><HR><P>
<A NAME="GROM_fore_reg_sfc_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - surface - regular grid
</FONT>
<P>
{GROM_fore_reg_sfc_48_hash}
{datestring}: <A HREF="{GROM_fore_reg_sfc_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_reg_sfc_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_sfc_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - surface - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_sfc_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_sfc_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_sfc_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_bot_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - bottom - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_bot_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_bot_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_bot_48_hash}

<P><HR><P>
<A NAME="GROM_fore_crv_all_48"></A>
<FONT SIZE=4>
ROMS Surface Currents - 48 hour forecast - 3-D - curvilinear grid
</FONT>
<P>
{GROM_fore_crv_all_48_hash}
{datestring}: <A HREF="{GROM_fore_crv_all_48_url}{filename}">{filename}</A><BR>
{/GROM_fore_crv_all_48_hash}

<P><HR><P>
<A NAME="GNAM_fore_reg_48"></A>
<FONT SIZE=4>
ETA Surface Winds - 48 hour forecast - regular grid
</FONT>
<P>
{GNAM_fore_reg_48_hash}
{datestring}: <A HREF="{GNAM_fore_reg_48_url}{filename}">{filename}</A><BR>
{/GNAM_fore_reg_48_hash}

<P><HR><P>
<A NAME="GNAM_hind_reg_24"></A>
<FONT SIZE=4>
ETA Surface Winds - 24 hour hindcast - regular grid
</FONT>
<P>
{GNAM_hind_reg_24_hash}
{datestring}: <A HREF="{GNAM_hind_reg_24_url}{filename}">{filename}</A><BR>
{/GNAM_hind_reg_24_hash}

<P><HR><P>

</BODY>

</HTML>

When the Perl script is executed, the variables and arrays created therein are substituted into their corresponding locations (within curly brackets) in the HTML template file via the use of a Perl module called Template::Magic.

OPeNDAP/DODS

The ultimate goal in serving the GNOME-ready NetCDF files involves the use of software designed for serving these types of files. The

OPeNDAP/DODS - http://www.opendap.org/

project is creating software to make array-based data produced anywhere available to those who wish to use it anywhere, as long as web connections are available at both ends. OPeNDAP (Open-source Project for a Network Data Access Protocol) (the name DODS is now a historical artifact) is built on the client/server model wherein servers provide data and clients allow access to that data. OPeNDAP servers can provide data in many formats including NetCDF, and the available clients - over a dozen to date - can download data in all of those formats.

Particularly interesting and useful for our purposes is the NetCDF client library component, a drop-in replacement for the standard NetCDF library. The standard library is typically used by being compiled into a program or application, which enables them to read and write files in NetCDF format. The ROMS circulation model we use for TGLO/TABS is an example of this. Basically, a NetCDF-enabled program can read a local file in NetCDF format, and access the data in that file natively, e.g. a Fortran program compiled with the library is enabled to read the binary NetCDF-format data in an external file into Fortran arrays, and can write data in Fortran arrays out to a NetCDF file. The OPeNDAP version of the NetCDF library does exactly the same thing, but adds the capability of reading and writing non-local files. If a NetCDF file can be accessed via a web address, then it can be read directly into a program from its location on the web. This is done simply by replacing the file path with a fully qualified URL in the appropriate location in either the program or a data file read by the program.

Although OPeNDAP is already useful in the capacity just described, it has additional capabilities that greatly extend its utility. In addition to being able to download entire NetCDF files from anywhere on the web, OPeNDAP can be used to:

• obtain a subset of the total number of fields in a NetCDF file, e.g. get only the U and V fields in a file also containing salinity, density, temperature, etc. fields;
• obtain a spatial subset of a field, e.g. the lower left 50x50 section of a 100x100 field;
• obtain a temporal subset of a field, e.g. the first 2 months of a year long file; and
• concatenate the contents of temporally contiguous and discrete NetCDF files into a single NetCDF file, i.e. data aggregation.

The OPeNDAP server is implemented as a CGI script, and files are accessed via a URL resembling:

http://csanady.tamu.edu/cgi-bin/nph-dods/

where nph-dods is simply appended to the URL of your CGI bin root address. A quick and simple test to see if the server is working is:

http://csanady.tamu.edu/cgi-bin/nph-dods/version

For demonstration purposes, we will now assume a NetCDF file TGLO-his-06-05-24-00-06.nc located in the subdirectory /TGLO/HIS under the base URL.

The metadata or information about the file can be accessed via appending .das the the URL. The will return the Dataset Attribute Structure, which contains information about the data such as units and variable names. For example,

http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.das

will return something looking like this:

Attributes {
    ntimes {
        String long_name "number of long time-steps";
    }
    ndtfast {
        String long_name "number of short time-steps";
    }
    dt {
        String long_name "size of long time-steps";
        String units "second";
    }
    dtfast {
        String long_name "size of short time-steps";
        String units "second";
...

Appending .asc or .ascii to the URL will return the contents of the entire file to your screen in ASCII format. This can be quite huge for our example file, so we'll use an example of Constraint Expression Syntax to return a limited portion of the file. If we append ?dt to our URL (after .asc) we will obtain only the specified variable. Thus, for:

http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.asc?dt

we would get:

Dataset: TGLO-his-06-05-24-00-06.nc
dt, 200

The Constraint Expression Syntax provides the capabilities to access temporal and spatial subsets of the data contained within NetCDF files. The full capabilities of this syntax are explained in the:

OPeNDAP User Guide - http://www.opendap.org/user/guide-html/

Appending .html to the URL will obtain a Server Dataset Access Form. For example, for:

http://csanady.tamu.edu/cgi-bin/nph-dods/TGLO/HIS/TGLO-his-06-05-24-00-06.nc.html

you will receive a web page resembling:

This HTML form allows you to sample the data at a URL and obtain either an ASCII or a NetCDF file containing the subset you have selected.