A Metcast Taker is a publishing server, a web application
server that takes data from a w-shover and publishes
them into a Metcast Channel. To be more precise, the Metcast Taker
receives incoming data and metadata and inserts them into tables of a
Metcast Channels database. The data are stored as a BLOB. The taker
also records the timestamp and the identity of the remote client. For a
virtual channel, processing, translation and perhaps storage of
published data are performed by a separate application -- a virtual
taker. The main Metcast Taker still does common chores of credential
verification, authorization, and record-keeping.
The Metcast taker may also function as a remote CGI script: it may run
on a computer different from the one of an HTTP server. For example,
the taker can be launched by an inetd of a remote
computer, communicating with the HTTP daemon through a TCP or a similar
pipe. The taker is smart enough to figure out the local or remote mode
of its invocation. If the Metcast Taker operates in a remote mode, it
must be represented by a local CGI proxy on a computer that
runs the HTTP daemon. It is this proxy that gets called by the daemon
when it receives a PUT request. The proxy queries its process
environment, determines all necessary CGI information, opens a
connection to the taker (if it had not been already open), sends the CGI
information followed by request's content. The proxy then relays the
remote taker's response to the HTTP server.
A channel is virtual if a channel administrator specified a
particular taker application when he created the channel. The
application -- a sh command line to be precise -- will
process that virtual channel's incoming data. This virtual taker
application is invoked in a manner similar to that of a regular CGI
script. That is, the incoming content is fed onto application's
stdin. All other parameters -- MCHANNELS_ID,
CONTENT_LENGTH, REMOTE_HOST,
REMOTE_USER, etc, are passed via POSIX process
environment.
A virtual channel taker must tell the outcome of its processing by sending a response on its stdout -- again, in a manner similar to that of a CGI application. If a virtual channel taker has successfully handled the content on its own, it should reply:
Status: 201 Created ...etc information... CRLF
( other HTTP-headers if needed CRLF )*
CRLF
This information is handed over to the HTTP daemon that accepted w-shove's connection.
If a virtual channel taker has processed the incoming content and wants the result inserted into a channel, the taker should write on its sdtout:
Status: 200 OK
Content-type: media-type
( Content-length: length )?
( other HTTP-headers if needed CRLF )*
CRLF
Content-itself
The Content-itself is then stored into the channel as if it was received from the original publisher.
All other virtual taker's responses are considered an error, and sent to the HTTP server preceded by a "400 VC taker Error" HTTP response line.
The Metcast Taker logs every publishing request and the results of its processing. To keep logs of potentially concurrently running takers separate, the name or an IP address of the publishing host is used as a suffix in the name of a log file. If a particular log file does not exist, it is created; if it did exist, the new information is appended to it. The logs may therefore grow very big and need periodical pruning/trimming. All log files are created with a prefix LOG:PREFIX, which may include a directory (path) prefix, as in logs/jmvlog-taker. This means that logs are contained in a local directory logs, which may be a symbolic link pointing elsewhere.
For example, a MORIAH report submitted as
w-shove w-shove-rest.conf "text/plain; cid=17" d:\MORIAH\report.txtleaves the following trace in a Metcast Taker log:
----> New remote request
moriah@host.navy.mil[192.XX.XX.XX] => /cgi-bin/oleg/rest/taker-cgi-proxy by WShove/1.2
PUT /d:\transport\XNPS_Ob.mor "text/plain; cid=17" 78
Publishing into virtual channel: launching a taker ./MORIAH-vtaker
4/2 0127 MORIAH report of 78 bytes
virtual channel response: 201 Accepted MORIAH report
as well as a line in HTTP server's access log:
host.navy.mil - moriah [01/Apr/1999:17:16:28 -0800] "PUT /cgi-bin/oleg/rest/taker-cgi-proxy/d:\transport\XNPS_Ob.mor HTTP/1.0" 201 0 "-" "WShove/1.2" 1For more details, see the comments to the Metcast Taker source code,
http://zowie.metnet.navy.mil/~spawar/JMV-TNG/Code/taker.scmand to a sample virtual taker code,
http://zowie.metnet.navy.mil/~spawar/JMV-TNG/Code/Feed-vtaker.pl