A Guide to MBL - a Metcast Request Language

This document describes a language a Metcast client uses to communicate its wishes to a server. MBL phrases constitute the body of a Metcast client request.

The language is somewhat similar to a Knowledge Query Language, which is often used for agent communication. MBL was designed to express a hierarchy of request parameters in a simple and easily parseable way. What parameters are those is up to a particular application. MBL permits a requestor to find out abilities of a server: which requests the server can fulfill and which parameters it demands or accepts. The language has a use clause which lets the two sides agree on a specific vocabulary. MBL is also similar to the language of a PICS rating system [PICS], and DSSSL.
 

  1. Design Principles
    1. The spirit of grid product requests

  2. Words of the Request Language
  3. MBL Syntax
  4. Parameters
     
  5. MBL Cookbook
    1. Synoptic reports, warnings and forecasts
    2. Imagery
    3. Gridded products
    4. Plain-text WMO messages
    5. Miscellaneous products

  6. Examples
    1. Request Language Live
  7. Metcast Table of Contents [a separate document]
    which tells which products are available from a particular Metcast server, and how to request them
     
  8. Fulfilling GRIB requests
  9. MBL Development History [a separate document]

 
 

What should a request message do and
How should it look like

The following is a set of features that, although not as basic as the principles above, are desirable for the request language:

 

The spirit of grid product requests

This is how a Metcast client (e.g., JMV) should formulate requests for gridded products. We will follow the standard, WMO convention of specifying gridded products, as implemented in a GRIB standard. A data grid is characterized by attributes:
 
origin
Identification of an organization or an organizational unit that published the product as well as the reference to a weather model or other software that generated the data: center id, subcenter id, generating process id.
parameter type
"pressure", "pressure reduced to MSL", "visibility", "pseudo-adiabatic potential temperature", etc. (see Table 2 of GRIB)
layer
surface, tropopause level, height above ground, isobar, hPa, between isobars (see Tables 3a and 3)
grid id
a special predefined id or boundaries plus projection plus resolution
time
forecast time, etc

The server needs to know all these parameters, eventually. Some of them can be inferred.

When parameters are presented to a user to choose from, long descriptive names are preferred. When parameters are communicated to a server, short names are obviously better. When a database is queried, small binary numbers are ideal.

 
 

Words of the Request Language

The request language is built around a single syntactic category: an S-expression. Everyone who ever came across LISP or similar code knows what it is all about.The syntax is free-form, clear, and easily understood by both a human being and the computer.

Formally, an S-expression is:

Components of the S-expression are kept separate with white spaces (blanks, HT and newlines). The number of the white spaces is irrelevant, and can be used to improve readability.

An atom is:


 

MBL Syntax

The following describes the structure as well as meaning of MBL phrases and their components. Following R5RS, we describe valid expressions by their templates, for example,
     (st_country_name st-country-name)
where st_country_name is a literal symbol, and st-country-name is a syntax variable, which stands for a fragment of an MBL request (an atom or another expression). A particular form and the meaning for these fragments is explained in the comments that accompany the template. The notation
     thing ...
indicates zero or more occurrences of a thing, and
     thing1 thing2 ...
is to denote one or more occurrences of a thing.
 

General structure of a request

     (area-id parameter ...
               (products product-desc ...) )
where area-id (an HTTP token) is an id of an area: a transaction id. It can be anything a client chooses to identify its request, for example, to match several requests to the corresponding replies. area-ids are specific to a client, so different clients may choose the same ids for their transactions without clashing.

Each product-desc has the following format

     (product-id parameter ...)
where product-id identifies a product being requested. The parameters are arguments that specialize the request. The cookbook section below lists ids of all the typical products as well as their mandatory and optional arguments.

When a Metcast server is about to fulfill a request for a product, it looks for the product arguments. The server first searches the list of the parameters specified in that products' scope. If the search fails the server scans parameters in the global request scope. It makes sense therefore to place the most common arguments (eg, bounding-box) into the global scope, and specify particular or overriding parameters by the corresponding product. See the cookbook for more details.
   

Parameters

A parameter is a phrase
     (parameter-keyword argument ...)
that specializes a request. This phrase may occur either in the global request scope, or in a local scope -- within a product-desc. The same parameter may be specified in several places or several times. If this is the case, the innermost rightmost occurrence takes effect. The following table lists typical parameters and their arguments. Not every parameter in the table is supported by every Metcast server. A server ignores the parameters it does not recognize. A Metcast client can always find out which parameters a particular Metcast server understands, by submitting a Metcast Table of Contents request (see below).

TemplateDefault
Value
Example
attr-constraint
(attr= name val)
(not-attr= name val)
(attr-like name val)
(not-attr-like name val)
  (attr= region "17")
(attr-like lib "coas%")
An attribute constraint is not a proper parameter. It may appear only inside a Channel product request. An attribute constraint limits the set of pieces of data ("things") returned in response to the Channel request. In particular: (attr= name val) selects only those "things" that are annotated by an attribute name with the value val. (attr-like name val) selects only those "things" that are annotated by an attribute name with the value that matches val, which may contain SQL-style wildcards: _ and %. Attribute constraints not-attr= and not-attr-like do the opposite selection.
A Channel request may include several attr-constraints; in which case they are implicitly ANDed together.
 
block_id
(block_id block-id block-id ...)   (block_id "724915")
block-id is a WMO-assigned id of a weather observation station, a string of one to six digits.
 
bounding-box
(bounding-box N-LAT W-LON S-LAT E-LON) Mandatory (bounding-box 70.0N 25.00W -50 175)
(bounding-box 90 -180 -90 180)
Specifies the latitudal and the longitudal spans of the area. Here N-LAT is the latitude of the Northern-most point of the area, S-LAT is the latitude of the Southern-most point, W-LON is the longitude of the Western-most point of the area, and E-LON is the Eastern-most longitude.
There is no default for this parameter. Most of the product requests need it, and will fail if the bounding box cannot be determined. Therefore it makes sense to place this parameter in the global request scope.
 
call_id
(call_id call-letters call-letters... )   (call_id "KMRY" "KSFO")
(call_id "WST9756")
call-letters is a character string: call letters for a land station, call sign or an alphanumerical id for a ship, or a serial number for a buoy. All letters must be in upper case.
 
center-id
(center-id numerical-center-id) 58 (center-id 7)
numerical-center-id is a WMO-assigned numerical id of an originating center (that is, the publisher of a GRIB product). This id is the content of GRIB record's PDS Octet 5.
See Table 0, part 1 of the GRIB standard, or the Metcast Table of Contents. See also a source parameter.
 
depth-max
(depth-max depth-max-limit)   (depth-max 400)
depth-max-limit is the depth, in meters, for the bottommost vertical layer to be returned by a query (e.g., a BTSC product query)
 
depth-min
(depth-min depth-min-limit) 0 (that is, the surface) (depth-min 100)
depth-min-limit is the depth, in meters, for the topmost vertical layer to be returned by a query (e.g., a BTSC product query)
 
grid-id
(grid-id num-grid-id) 240 (grid-id 183)
num-grid-id is a grid identification, which distinguishes between several grids produced by the same model at the same originating center. This id is the content of GRIB record's PDS Octet 7.
Grid Ids 21 through 64 are standardized in Table B of the GRIB standard. All other ids are assigned by an originating center. See also a model parameter, which specifies the same information but in a symbolic form.
 
isobar-p-max
(isobar-p-max isobar-p-max-limit) the current surface pressure (isobar-p-max 950)
isobar-p-max-limit is the pressure, in hPa, for the bottommost vertical layer to be returned by a query (e.g., a UAR product query)
 
isobar-p-min
(isobar-p-min isobar-p-min-limit) highest atmospheric level available (isobar-p-min 100)
isobar-p-min-limit is the pressure, in hPa, for the topmost vertical layer to be returned by a query (e.g., a UAR product query)
 
keywords
(keywords keyword keyword... )   (keywords "alaska" "cyclo%" "summary" "_irep")
a keyword is a case-insensitive sequence of characters that is supposed to occur in the description of a WMO plain-text message. A keyword does not have to be the whole word, and may contain SQL-style wildcards % and _.
 
layer
(layer layer-spec)   (layer entire-atm)
(layer isobar 1013)
(layer height-between-ft 1000.0 0.0)
This phrase is used to ask for a specific level of a product: at a particular height above the ground, at a particular isobar level, at msl (mean sea level), between two isobar layers, etc. See Tables 3 and 3a of the GRIB standard for more details.
 
manops
(manops manop-code manop-code... )   (manops "YRXX84" "FP%" "ABAK2PANC")
manop-code is one of the following:
  • a complete T1T2A1A2ii id of a message (4 through 6 characters, the first four must be uppercase letters and the rest are digits, if present). This is the first field of a WMO Abbreviated message header (see WMO Manual 386). For example, "FABA31".
  • a partial id of a message: an id as above containing SQL-style wildcards % or _. For example, "AX%" "ABAK%".
  • a complete T1T2A1A2ii id of a message with call letters of the issuing station appended, for example, "FAUS1KDFW". This is the string of 8 to 10 characters.
 
max-records
(max-records output-limit)   (max-records 20)
output-limit limits a Metcast query to no more than that many output records.
 
mime-type
(mime-type mime-type-spec) Depends on a product (mime-type "text/html")
specifies the MIME type to tag the product content with. Each product served by a Metcast server has the default mime type associated with it. The client however may override this default.
 
model
(model model-id) NOGAPS-1deg (model NOGAPS)
(model NORAPS-CONUS)
where the model-id identifies a particular computation process that generated requested data. See the Metcast Table of Contents. This parameter is a shorthand for three low-level parameters: process-id, grid-id and resolution. If both are specified, the rightmost is used.
 
modified-since
(modified-since epoch-timestamp) 0 (modified-since 938124054)
where epoch-timestamp is in epoch seconds. If this parameter is present and non-zero, only product(s) that have been modified after that time moment are sent to the client.
A Metcast server implicitly inserts the modified-since parameter into the global request scope based on a IF_MODIFIED_SINCE HTTP header. Therefore a client does not need to mention this parameter in his request. It may make sense however to place a modified-since argument into the scope of a particular product request. For example, a user may say
     (area ... (products (SIGMET (modified-since 0))))
to request all rather than the latest SIGMET warnings, even in a conditional request.
 
msgtype
(msgtype msgtype)   (msgtype "FO")
msgtype is a string of two uppercase characters T1T2 that specify the WMO type of a message, e.g. "WT" for a tropical cyclone warning. See the WMO Manual 386, tables A and B1-B6 for more details.
 
msg_constraint
(msg_constraint a-msg-constraint a-msg-constraint...)   (msg_constraint (call_id "KMKC") (msgtype "AC"))
where a-msg-constraint is one of the following parameters: msgtype, call_id, or keywords.
 
name
(name full-name-of-the-area)    
Not currently supported.
 
process-id
(process-id num-process-id) 58 (process-id 43)
num-process-id is a Generating process ID number, which identifies a weather model that created the GRIB data. The number is allocated by an originating center. This id is the content of GRIB record's PDS Octet 6.
See also the model parameter, which specifies the same information but in a symbolic form.
 
product-dict
(product-dict dict-id)    
 
product-GRIB-code
(product-GRIB-code grib-code-value)   (product-GRIB-code 11)
grib-code-value is a numerical id of a grib product. For example, (product-GRIB-code 1) requests a pressure grid. See Table 2 of the GRIB standard and the Metcast Table of Contents.
This parameter is mandatory if a product with a grib product-id is requested.
 
productname
(productname productname-value)   (productname "N.Americ_visible_FNMOC.jif")
productname-value is the name of a satellite product to retrieve. This is a mandatory parameter for imagery products.
 
projection
(projection label)    
Not currently supported.
 
resolution
(resolution x-res y-res)
(resolution grid-spacing)
1.0 degree (resolution 0.5)
where grid-spacing is a floating point number specifying grid's mesh size in degrees. Currently only the second format is supported.
 
source
(source source-id) FNMOC  
where source-id identifies a center-publisher of the product. This id may either be a symbol -- eg, FNMOC, AFGWC, NWSTG -- or a number. In the latter case, the number is assumed to be 100*num_subcenter_id + numerical_center_id. For example, (source 7) as well as (source NCEP) refer to a US Weather Service - National Met. Center, while (source 107) means NCEP-Re-analysis project; (source 57) refers to a USAF Global weather center, and (source 1057) means a USAF TBD. The numerical_center_id corresponds to a center id as defined in the GRIB document.
The source parameter is equivalent to a pair center-id, subcenter-id parameters. If both are given, the rightmost takes effect.
See Table 0, part 1 of the GRIB standard, or the Metcast Table of Contents.
 
st_constraint
(st_constraint a-st-constraint1 a-st-constraint2...)   (st_constraint (call_id "KMRY"))
where a-st-constraint is one of the following parameters: block_id, call_id, st_name or st_country_code.
 
st_country_code
(st_country_code st-country-code)   (st_country_code "US")
st-country-code is a WMO country code for the station: a string of two uppercase letters.
 
st_country_name
(st_country_name st-country-name)   (st_country_name "UNIT%")
st-country-name is full uppercase name for the station's country. It may contain SQL-style wildcards '_' and '%'.
 
st_name
(st_name st-name)   (st_name "MONTE%")
st-name is the uppercase name for the station. It may contain SQL-style wildcards '_' and '%'.
 
subcenter-id
(subcenter-id num-subcenter-id) 0 (subcenter-id 1)
num-subcenter-id identifies an organizational unit within the center-id that has published a requested GRIB product. This numerical id is the content of GRIB record's PDS Octet 26.
The subcenter-ids are specific to originating centers. See also the source parameter.
 
tau
(tau tau-value1 tau-value2...)   (tau 0)
(tau 0 12 24 36)
The tau-value of zero stands for analysis data. Other tau-values denote forecasts that many hours into the future.
 
use
(use dict-id)    
 
 
valid-at
(valid-at epoch-timestamp)   (valid-at 938124054)
where epoch-timestamp is in epoch seconds. This parameter if present limits the query to only those products (e.g., forecasts or warnings) which are still valid as of epoch-timestamp. This time moment may point into the future as well as into the past.

 
 

Cookbook

A web page Request-Lang-Forms.html lets you type in an arbitrary MBL request and submit it to a Metcast server. You may also execute one of the "canned" requests on the same page -- either as it is, or after arbitrary modifications.

Alas, regulations prohibit offering an unrestricted access even to a development Metcast server. Please direct all inquiries to Mr. Dave Huff.

 

Synoptic products

The products are returned as OMF XML formatted documents, described in Weather Observation Definition Format (OMF)

Product IdParameters
(mandatory are in bold)
MSL bounding-box
modified-since
mime-type
max-records
block_id
call_id
st_name
st_country_code
st_country_name
Querying the Master Stations Library
 
METAR bounding-box
modified-since
mime-type
st_constraint
surface weather observation reports from land stations (METAR, SPECI, SYNOP)
 
SYNSEA bounding-box
modified-since
mime-type
call_id
surface observations from sea stations (moving or drifting or moored)
 
UAR bounding-box
modified-since
mime-type
st_constraint
isobar-p-min
isobar-p-max
Rawinsonde and Pibal Observations (Upper Air Reports)
 
BTSC bounding-box
modified-since
mime-type
call_id
depth-min
depth-max
Bathythermal, Salinity and Current Observations (BATHY, TESAC, TRACKOB)
 
TAF bounding-box
modified-since
mime-type
st_constraint
valid-at
Terminal Aerodrome Forecasts
 
SIGMET bounding-box
modified-since
mime-type
valid-at
SIGMET advisories

   

Imagery products

The products are usually returned in TIFF (or Metoc TIFF: MTIFF, MIF or JIF) formats.

Product IdParameters
(mandatory are in bold)
imagery bounding-box
modified-since
productname
Requesting a satellite image

   

Gridded products

Gridded data are sent in a GRIB format.

Product IdParameters
(mandatory are in bold)
grib bounding-box
modified-since
product-GRIB-code
source
model
center-id
subcenter-id
process-id
grid-id
resolution
layer
tau
Requesting a grid product
 
pressure bounding-box
modified-since
source
model
center-id
subcenter-id
process-id
grid-id
resolution
layer
tau
Requesting a pressure grid. There is a one-to-one correspondence between GRID product names and product-GRIB-codes, see Table 2 of the GRIB standard. For example, the following two requests are equivalent:
(products (pressure (model "NOGAPS") (layer isobar 1000)))

(products (grib (product-GRIB-code 1)
    (model "NOGAPS") (layer isobar 1000)))

   

Plain-text WMO messages

The messages are returned as OMF XML formatted documents, described in Weather Observation Definition Format (OMF)

Product IdParameters
(mandatory are in bold)
METMSG manops
modified-since
mime-type
Requesting one or several messages by their MANOP ids.
 
METMSG bounding-box
modified-since
mime-type
msg_constraint
Requesting one or several messages by Area of Interest and other parameters

   

Miscellaneous products

Product IdTemplate
GlobalWarnings (GlobalWarnings warning-id1 warning-id2 ...)
Free-text warnings and advisories pertaining to the entire globe (or without any specific geographical registration).
warning-id can be:
trop-cyc
An advanced tropical cyclone warning
jots-w
A JOTS (high wind, etc) warning
 
Channel (Channel channel-id attr-constraint ...)
Requesting a thing -- an unstructured document -- from a Metcast Channel. See Metcast Channels for more details.

 
 

Examples

Basic examples

A single grid request

(NORF
   (bounding-box  45.0N 99W 23.0 74W)
   (model NOGAPS)
   (products
     (pressure (layer msl) (tau 24))))

Asking for several gridded products

(some-area
   (bounding-box  90.0 178W -90.0 179E)
   (source FNMOC) (model NOGAPS)(resolution 1.0)
   (products
     (geopotential-height (layer isobar 500)(tau 12))
     (temperature (layer msl))))
(some-area
   (bounding-box  90.0 178W -90.0 179E)
   (source FNMOC) (model NOGAPS)(resolution 1.0)
   (products
     (geopotential-height (layer isobar 500)(tau 12))
     (grib (product-GRIB-code 55) (layer msl) (tau 12) (source 7))
     (grib (product-GRIB-code 155) (layer msl) (tau 24) (source 15557))))

Satellite imagery request

(BIGarea
   (bounding-box 70.1N 25.00W -50 175)
   (products
     (imagery (productname "bayvis.mif"))))

Synoptic requests

(Some-area
   (bounding-box 30.0N 5.00W -5 20)
   (products
     (GlobalWarnings trop-cyc jots-w)
     (METAR)))
(ReqID
   (bounding-box 56.6N 122.0W 10.5 70.5W)
   (products (SIGMET (valid-at 916993601))))

Asking for a report of a specific station or stations

(ReqID (bounding-box 37N 122.0W 36.5 121.5W)
   (products (METAR (st_constraint (block_id "724915")))))

(ID (bounding-box 70.1N 125.00W -50 -70)
    (products (METAR (st_constraint (call_id "KMRY" "KSNS" "KSFO")))))

Requesting a product from a Metcast Channel

(vdu-req
   (products
     (Channel VDU-INCR (attr= region "17")
                       (attr-like lib "coas%"))))

Requesting (a great number of) plain-text WMO Meteorological messages. Note that some bounding-box-es include the International Date Line.

(Some-area
  (bounding-box 30.0N 5.00W -5 20)
(products
  (METMSG (manops "FPUS52"))
  (METMSG (manops "FPUS52" "ABAK2"))
  (METMSG (manops "AB%"))
  (METMSG (manops "ABAK2PANC"))
  (METMSG (manops "FP%" "ABAK2PANC"))
  (METMSG (manops "FP%" "ABAK2PANC" "YRXX84"))
  (METMSG (manops "FP%" "ABAK2PANC" "YRXX__"))
  (METMSG (manops "FP%" "ABAK2PANC" "YRXX84KAWN"))
  (METMSG (manops "ABAK2PANC" "YRXX84KAWN"))
  (METMSG (manops "YIXX84" "ABAK2PANC" "YRXX84"))
  (METMSG (manops "YIXX84" "AB%" "YRXX84") (modified-since 100))
  (METMSG (bounding-box 38 -125 36 -120))
  (METMSG (bounding-box 10 170 -10 -170))
  (METMSG (bounding-box 10 170 -10 -170)
       (msg_constraint (call_id "PHNL")))
  (METMSG (bounding-box 10 170 -10 -170) (manops ())
       (msg_constraint (call_id "PHNL") (msgtype "FP")))
  (METMSG (bounding-box 10 170 -10 -170)
        (msg_constraint (call_id "PHNL") (keywords "forecast" "zone")))
))

(Area51
  (use special-area-51))
(Area51
  (use special-area-51) (user 007)
  (products (TAF)))
as long as you and the server agree on the defaults, special-area-51 and 007 in the case above.
 

Several bounding boxes

An example below requests essentially the same product but from two different weather models: NOGAPS and COAMPS. bounding-box for the latter model has to be adjusted to reflect that model's coarser resolution.

(CAU
   (bounding-box  47.0  -12.0  29.0   40.0)
   (products
     (grib (product-GRIB-code 7) (layer isobar 1000) (tau 0)
         (source 58) (process-id 22) (resolution 0.200))
     (grib
         (bounding-box  47.5  -12.5  27.5   40.0)
         (product-GRIB-code 7) (layer isobar 900) (tau 0) (source NCEP)
         (process-id 80) (resolution 2.500))))
Note the two bounding-boxes. The first one is the default bounding box, which applies to all the products that do not specify their own. The second bounding-box is for the NCEP product only. There is only one products clause (as it should be).

In the following example the area of interest is made of two distinct, non-overlapping geographical regions:

(CAU
   (bounding-box  47.0  -12.0  29.0   40.0)
   (products
     (grib (product-GRIB-code 7) (layer isobar 1000) (tau 0)
         (source 58) (process-id 22) (resolution 0.200))
     (METAR
         (bounding-box  38  -123  36   -121))))
Executing this request brings a grib file for an area in Europe, and METARs for the San Francisco Bay area. This particular request might not be too useful, yet it makes a fine example.

 

 

Fulfilling GRIB requests

Having received a request, the server would parse it and launch a database requestor, passing it one or several lines of parameters through the requestor's stdin. The lines have the following format:
  AREA_ID= user id of a request
  MODIFIED_SINCE=Epoch seconds or 0
  BOUNDING_BOX=Northmost-lat Westmost-lon Southmost-lat Eastmost-lon
  PARAMETER=an id (the number) for a product, per Table 3 of GRIB
  CENTER=three numbers: center-id sub-center-id generating-process-id
  LAYER=one to three numbers, as specified by tables 3a and 3 of GRIB
   For example,
LAYER=100 500
specifies a single layer: 100 stands stands for a "isobar, hPa" level (as per Table 3), and 500 is the pressure value at this particular isobar. In general LAYER= parameter has the same meaning as Octets 10, 11 and 12 of GRIB's Product Description Section (PDS).
  GRID_ID=a grid-special id if present
  TAU=one or several tau
  RESOLUTION=x-res y-res
   The first one is x-resolution, the other is y-resolution. In most of the situations they are the same. Still, the GRIB standard requires separate specifications.

Each line is terminated with CRLF; a blank line signifies the end of the request.

The grid database query application application will read the request line from its stdin (up to and including the blank line, a CRLFCRLF sequence), and write the result of the query onto the stdout. The result may be empty, if there was some problem, or if there were no changes in the requested products since the MODIFIED_SINCE time). If the result is generated, it must contain grid data with appropriate MIME headers, specifically, content-type with at least AREA=some-area parameter. If there are several pieces of information (grib data) found to satisfy the request,the query application should write a multi-part MIME message, of a multipart/mixed content type. The application may also choose to concatenate GRIB records together and send them as a single data file.

The set of grid query specifications above is deliberately patterned after a PDS section of GRIB files. Therefore one can potentially use this technique to serve grid data without any database at all, off a flat directory filled with various GRIB files.

Note if a request has a phrase (tau 0 12 24) then the parameter file would contain
    TAU=0 12 24
and the grib query application is expected to create a multipart/mixed message with three GRIB files.

The reason for this requirement is twofold: to cut down on calls to the query application as products for different taus can be efficiently retrieved within a single DB query. The main reason however is the following. Suppose a user requested a product with tau=012 while the database contains only a 24-hour forecast. It is reasonable to expect that the server - rather than refusing the request - should send the 24-hr forecast product along with a note describing the substitution.

Assume now that the user issues a request with (tau 0 12 24). If the grid query application is called three times separately, the 24-hr product will be sent twice. Keeping in mind the size of 1-deg products, this could be a nuisance. When the requestor gets all the taus, it hopefully can do something more intelligent.



$Id: Request-Lang.html,v 3.4 2007/07/03 01:16:30 oleg Exp oleg $