Exporting Log Files

The logexport command line utility provides you with the ability to securely perform detailed queries against the Host Integrator logging component. It is capable of performing bulks extracts of messages from the logging component suitable for parsing or importing into third-party applications.

Operation

The logexport utility can be found in the bin directory of the Host Integrator installation. You can run it from this location, or you can add this directory to your PATH for easy access. The utility has a variety of command line switch options.

Help on available options can be obtained with the use of the command line switch -help.

logexport -help

LogExport -query sql [ -delim delim_char ] [ -escape esc_char ]
                  OR
          -purge msgserial

          -server servername
          -ds mgmtservername
          -user username
          -password userpass
          -profile ( Administrator | Developer )
          [ -config filename ]

The utility performs two primary functions:

One of these options must be specified, but they are mutually exclusive. The -server switch is required. This option specifies which log service to contact.

Regardless whether security is enabled on the Host Integrator installation in question, logs are protected from access by unauthorized users. Access to logs always requires a user to authenticate with the management server. The options -ds, -user, -password, and -profile are required. Both administrators and developers can perform queries against the logs, but only administrators can execute message purges.

Querying Messages

The -query option takes a SQL select statement which specifies which messages you want to see. For example:

logexport -server s1 -query "select msgtime, msgtext from messages"

This invocation will contact the logging service (Log Manager Service) running on host "s1" and output the message time and message text of all messages currently in the Host Integrator server logs. The results will be directed to standard out. Any errors that occur will be directed to standard error. Columns of output will be separated with the "|" character.

Along with the -query option, two optional parameters can be specified, -delim and -escape. Each of these options takes a single character. Use the -delim switch to specify the character used to separate columns in the output. It is the "|" character by default. Use the -escape option to specify the character used to escape occurrences of the column delimiter that occur within any column of message data. It is the "\" character by default.

For details on the SQL supported by the Host Integrator logging system, see Understanding the SQL Dialect of the Host Integrator Logging System.

Within the SELECT statement, specify the columns of the message entry desired in the output, and the order they will appear. To specify all columns in the default order, use "*" as the column identifier.

Each message entry has the following columns available:

Name Type Description
msgserial INTEGER Message primary key
msgtime TIMESTAMP Date/Time message
msgid INTEGER Numeric message identifier
sessionid INTEGER Session identifier
requestid INTEGER User request identifier
cltaddr VARCHAR User's network name
user VARCHAR User's name
model VARCHAR Model or pool name
msgtext VARCHAR Text of the message
severity VARCHAR Message severity

The SQL SELECT statement optionally can contain a WHERE clause that allows the user to specify complex matching criteria on the message entries they desire to see.

Purging Messages

If this utility is used to perform a bulk extract of messages from the Host Integrator logging system, you will need the ability to purge messages from the system after they have been safely loaded into another database or application. All messages within a message store are tagged with a unique serial number that can be obtained by specifying the msgserial column in a query. This column is the message's primary key. The -purge option takes a message serial number, and deletes all messages in the specified message store, up to and including the serial number specified. For example:

logexport -server s1 -purge 1234

This example deletes all messages with a msgserial number less than or equal to 1234 on machine "s1".

Advanced Features

Due to the relatively large number of available switches on the logexport utility, you may want to specify run-time arguments from a configuration file. Specify the file containing run-time options on the command line. To employ this feature, create a file containing the desired options, and specify the path to the file as the argument to the -config switch.

logexport -config sample.config

Any switch that can be specified at the command line, with the exception of the -config switch, can be placed in the config file. The format of the file is shown below:

#
# Comments look like this.
#
# Line tags are the same as switch names.  Any number required can 
# be specified in the file.
#
# server
# query
# purge
# delim
# escape
# ds
# user
# password
# profile
# 
# Here are some real entries.
#
server=s1
query=select msgtext from messages

A config file can be specified in combination with other command line switches. If a setting is specified more than once, the value of its final specification is used during run-time. This allows you to create a default configuration file, but override particular values for a given run. For example:

logexport -config myconfig -server somepc

In the above example, logexport will contact server "somepc" regardless of whether "server" is specified in the file "myconfig".

Related Topics
Bullet Building Queries
Bullet Understanding the SQL Dialect of the Host Integrator Logging System
Bullet Setting Logging Properties