Customize logging on the NSWL client system
-
Log filters. Filter log information based on the host IP address, domain name, and host name of the Web servers.
-
Log properties. Each filter has an associated set of log properties. Log properties define how to store the filtered log information.
Sample configuration file
##########
# This is the NSWL configuration file
# Only the default filter is active
# Remove leading # to activate other filters
##########
##########
# Default filter (default on)
# W3C Format logging, new file is created every hour or on reaching 10MB file size,
# and the file name is Exyymmdd.log
##########
Filter default
begin default
logFormat W3C
logInterval Hourly
logFileSizeLimit 10
logFilenameFormat Ex%`{`%y%m%d}t.log
end default
##########
# NetScaler caches example
# CACHE_F filter covers all the transaction with HOST name www.netscaler.com and the listed server ip's
##########
#Filter CACHE_F HOST www.netscaler.com IP 192.168.100.89 192.168.100.95 192.168.100.52 192.168.100.53 ON
##########
# netscaler origin server example
# Not interested in Origin server to Cache traffic transaction logging
##########
#Filter ORIGIN_SERVERS IP 192.168.100.64 192.168.100.65 192.168.100.66 192.168.100.67 192.168.100.225 192.168.100.226 192.168.
100.227 192.168.100.228 OFF
##########
# netscaler image server example
# all the image server logging.
##########
#Filter IMAGE_SERVER HOST www.netscaler.images.com IP 192.168.100.71 192.168.100.72 192.168.100.169 192.168.100.170 192.168.10
0.171 ON
##########
# NCSA Format logging, new file is created every day midnight or on reaching 20MB file size,
# and the file name is /datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log.
# Exclude objects that ends with .gif .jpg .jar.
##########
#begin ORIGIN_SERVERS
# logFormat NCSA
# logInterval Daily
# logFileSizeLimit 40
# logFilenameFormat /datadisk5/ORGIN/log/%v/NS%`{`%m%d%y}t.log
# logExclude .gif .jpg .jar
#end ORIGIN_SERVERS
##########
# NCSA Format logging, new file is created every day midnight or on reaching 20MB file size,
# and the file name is /datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log with log record timestamp as GMT.
##########
#begin CACHE_F
# logFormat NCSA
# logInterval Daily
# logFileSizeLimit 20
# logFilenameFormat /datadisk5/netscaler/log/%v/NS%`{`%m%d%y}t.log
# logtime GMT
#end CACHE_F
##########
# W3C Format logging, new file on reaching 20MB and the log file path name is
# atadisk6/netscaler/log/server's ip/Exmmyydd.log with log record timestamp as LOCAL.
##########
#begin IMAGE_SERVER
# logFormat W3C
# logInterval Size
# logFileSizeLimit 20
# logFilenameFormat /datadisk6/netscaler/log/%AEx%`{`%m%d%y}t
# logtime LOCAL
#end IMAGE_SERVER
##########
# Virtual Host by Name firm, can filter out the logging based on the host name by,
##########
#Filter VHOST_F IP 10.101.2.151 NETMASK 255.255.255.0
#begin VHOST_F
# logFormat W3C
# logInterval Daily
# logFileSizeLimit 10
logFilenameFormat /ns/prod/vhost/%v/Ex%`{`%m%d%y}t
#end VHOST_F
########## END FILTER CONFIGURATION ##########
Create filters
| Parameter | Specifies | |
|---|---|---|
| filterName | Name of the filter. The filter name can include alphanumeric characters and cannot be longer than 59 characters. Filter names longer than 59 characters are truncated to 59 characters. | |
| Host name | Host name of the server for which the transactions are being logged. | |
IP ip |
IP address of the server for which transactions are to be logged (for example, if the server has multiple domains that have one IP address). | |
IP ip 2...ip n |
Multiple IP addresses (for example, if the server domain has multiple IP addresses). | |
| ip6 IP | IPv6 address of the server for which transactions are to be logged. | |
| IP ip NETMASK mask | IP addresses and netmask combination to be used on a subnet. | |
| `ON | OFF` | Enable or disable the filter to log transactions. If no argument is selected, the filter is enabled (ON). |
-
filter <filterName> <HOST name> | [IP<ip> ] | [IP<ip 2...ip n> ] | <IP ip NETMASK mask> [ON | OFF] -
filter <filterName> <HOST name> | [IP6 ip/<prefix length>] [ON | OFF]
Create a filter for a virtual server
filter <filterName> <VirtualServer IP address>
Filter F1 HOST www.netscaler.com ON
Filter F2 HOST www.netscaler.com IP 192.168.100.151 ON
Filter F3 HOST www.netscaler.com IP 192.168.100.151 192.165.100.152 ON
Filter F4 IP 192.168.100.151
Filter F5 IP 192.168.100.151 HOST www.netscaler.com OFF
Filter F6 HOST www.netscaler.com HOST www.xyz.com HOST www.abcxyz.com IP 192.168.100.200 ON
Filter F7 IP 192.250.100.0 NETMASK 255.255.255.0
Filter F8 HOST www.xyz.com IP 192.250.100.0 NETMASK 255.255.255.0 OFF
For creating filters for servers having IPv6 addresses.
Filter F9 2002::8/112 ON
Filter F10 HOST www.abcd.com IP6 2002::8 ON
Specify log properties
BEGIN and ends with END as illustrated in the following example:
BEGIN <filtername>
logFormat ...
logFilenameFormat ...
logInterval ...
logFileSize ....
logExclude ....
logTime ….
END
-
LogFormat specifies the Web server logging feature that supports NCSA, W3C Extended, and custom log file formats.
logformat property is w3c. To override, enter custom or NCSA in the configuration file, for example:
LogFormat NCSA
-
LogInterval specifies the intervals at which new log files are created. Use one of the following values:
-
Hourly: A file is created every hour.
-
Daily: A file is created every day at midnight. Default value.
-
Weekly: A file is created every Sunday at midnight.
-
Monthly: A file is created on the first day of the month at midnight.
-
None: A file is created only once, when Web server logging starts.</span>
-
LogInterval Daily
loginterval property so that a file is created only when the log file size limit is reached.
LogFileSizeLimit is 10 MB.
LogFileSizeLimit 35
-
LogFilenameFormat specifies the file name format of the log file. The name of the file can be of the following types:
-
Static: Specifies a constant string that contains the absolute path and file name.Dynamic: Specifies an expression containing the following format:
-
Server IP address
-
Date (%{format}t)
-
URL suffix (%x)
-
Host name (%v)
-
-
LogFileNameFormat Ex%`{`%m%d%y}t.log
LogInterval size
LogFileSize 100
LogFileNameFormat Ex%`{`%m%d%y}t
-
LogExclude prevents logging of transactions with the specified file name extensions.
LogExclude.html
-
NCSA log file format: LOCAL
-
W3C log file format: GMT.
Understand NCSA and W3C log formats
-
NCSA Common Log Format
-
W3C Extended Log Format
NCSA Common log format
Client_IP_address -User_Name [Date:Time -TimeZone] "Method Object HTTP_version" HTTP_StatusCode BytesSent
log.conf file.
| Argument | Specifies |
|---|---|
| Client_IP_address | The IP address of the client computer. |
| User Name | The user name. |
| Date | The date of the transaction. |
| Time | The time when the transaction was completed. |
| Time Zone | The time zone (Greenwich Mean Time or local time). |
| Method | The request method (for example; GET, POST). |
| Object | The URL. |
| HTTP_version | The version of HTTP used by the client. |
| HTTP_StatusCode | The status code in the response. |
| Bytes Sent | The number of bytes sent from the server. |
W3C extended log format
%`{`%Y-%m-%d%H:%M:%S}t %a %u %S %A %p %m %U %q %s %j %J %T %H %+{user-agent}i %+{cookie} i%+{referer}i
logFormat W3C %`{`%Y-%m-%d%H:%M:%S}t %m %U
#Version: 1.0
#Fields: date time cs-method cs-uri
#Date: 12-Jun-2001 12:34
2001-06-12 12:34:23 GET /sports/football.html 2001-06-12 12:34:30
GET /sports/football.html
Entries
Directives
#Version: 1.0
#Fields: time cs-method cs-uri
#Date: 12-Jan-1996 00:00:00
00:34:23 GET /sports/football.html
12:21:16 GET /sports/football.html
12:45:52 GET /sports/football.html
12:57:34 GET /sports/football.html
Fields
-
identifier: Relates to the transaction as a whole.
-
prefix-identifier: Relates to information transfer between parties defined by the value prefix.
-
prefix (header): Specifies the value of the HTTP header field header for transfer between parties defined by the value prefix. Fields specified in this manner always have the type.
| Prefix | Specifies |
|---|---|
| c | Client |
| s | Server |
| r | Remote |
| cs | Client to server |
| sc | Server to client |
| sr | Server to remote server (prefix used by proxies) |
| rs | Remote server to server (prefix used by proxies) |
| x | Application-specific identifier |
Referer field in the reply.
Identifiers
| Identifier | Description |
|---|---|
| date | The date on which the transaction was done. |
| time | The time when the transaction is done. |
| time-taken | The time taken (in seconds) for the transaction to complete. |
| bytes | The number of bytes transferred. |
| cached | Records whether a cache hit has occurred. A zero indicates a cache misses. |
| Identifier | Description |
|---|---|
| IP | The IP address and the port number. |
| DNS | The DNS name. |
| status | The status code. |
| comment | The comment returned with a status code. |
| method | The method. |
| url | The URL. |
| url-stem | The stem portion of the URL. |
| url-query | The query portion of the URL. |
| Field | Description |
|---|---|
| Date | The date on which the transaction is done. |
| Time | The time when the transaction is done. |
| Client IP | The IP address of the client. |
| User Name | The user name. |
| Service Name | The service name, which is always HTTP. |
| Server IP | The server IP address. |
| Server Port | The server port number |
| Method | The request method (for example; GET, POST). |
| Url Stem | The URL stem. |
| Url Query | The query portion of the URL. |
| HTTP Status | The status code in the response. |
| Bytes Sent | The number of bytes sent to the server (request size, including HTTP headers). |
| Bytes Received | The number of bytes received from the server (response size, including HTTP headers). |
| Time Taken | The time taken for a transaction to complete, in seconds. |
| Protocol Version | The version number of HTTP being used by the client. |
| User Agent | The User-Agent field in the HTTP protocol. |
| Cookie | The Cookie field of the HTTP protocol. |
Referer |
The Referer field of the HTTP protocol. |
Create a custom log format
Create a custom log format by using the NSWL library
-
Windows: The nswl.lib library in the
\ns\bindirectory on the system manager host computer. -
Solaris: The
libnswl.alibrary inusr/local/netscaler/bin.
-
Add the following two C functions defined by the system in a C source file:ns_userDefFieldName(): This function returns the string that must be added as a custom field name in the log record.ns_userDefFieldVal(): This function implements the custom field value, then returns it as a string that must be added at the end of the log record.
-
Compile the file into an object file.
-
Link the object file with the NSWL library (and optionally, with third party libraries) to form a new NSWL executable.
-
Add a %d string at the end of the logFormat string in the configuration file (log.conf).
##########
# A new file is created every midnight or on reaching 20MB file size,
# and the file name is
/datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log and create
digital
#signature field for each record.
BEGIN CACHE_F
logFormat custom "%a - "%{user-agent}i" [%d/%B/%Y %T -%g] "%x"
%s %b%{referrer}i "%{user-agent}i" "%{cookie}i" %d "
logInterval Daily
logFileSizeLimit 20
logFilenameFormat
/datadisk5/netscaler/log/%v/NS%`{`%m%d%y}t.log
END CACHE_F
Create a Custom log format manually
LogFormat Custom ""%a - "%{user-agent}i" %[%d/%m/%Y]t %U %s %b %T"
-
The string can contain the "c" type control characters \n and \t to represent new lines and tabs.
-
Use the Esc key with literal quotes and backslashes.
" *. /: < > ? \ |
%<ASCII value of character in hexadecimal>.
Create Apache log formats
referer}i" "%{user-agent}i"
Referer Log: LogFormat custom "%{referer}i" -\> %U
Arguments for defining a custom log format
| Argument | Specifies |
|---|---|
| %a | Remote IPv4 address. |
| %A | Local IPv4 address. |
| %a6 | Remote IPv6 address. |
| %A6 | Local IPv6 address. |
| %B | Bytes sent, excluding the HTTP headers (response size). |
| %b | Bytes received, excluding the HTTP headers (request size). |
| %d | User-defined field. |
| %K | Client port information. |
| %e1 | Value of the first custom HTTP request header. |
| %e2 | Value of the second custom HTTP request header. |
| %E1 | Value of the first custom HTTP response header. |
| %E2 | Value of the second custom HTTP response header. Note: For instructions on how to export customHTTP headers, see Configuring the NetScaler for Web Server Logging |
| %g | Greenwich Mean Time offset (for example, -0800 for Pacific Standard Time). |
| %h | IPv4 address of a remote host. |
| %h6 | IPv6 address of a remote host. |
| H | Request protocol. |
| % {Foobar}i | Contents of the Foobar: header line(s) in the request sent to the server. The system supports the User -Agent, Referer, and cookie headers. The + after the % in this format informs the logging client to use the + as a word separator. |
| %j | Bytes received, including headers(request size). |
| %J | Bytes sent, including headers (response size). |
| %l | Remote log name (from identd, if supplied). |
| %m | Request method. |
| %M | Time taken to serve the request (in microseconds). |
| % {Foobar}o | Contents of Foobar: header line(s) in the reply. USER -AGENT, Referrer, and cookie headers (including set cookie headers) are supported. |
| %p | Canonical port of the server serving the request. |
| %P | The admin partition. |
| %q | Query string (prefixed with a question mark (?) if a query string exists). |
| %r | First line of the request. |
| %s | Requests that were redirected internally, this is the status of the original request. |
| %t | Time, in common log format (Standard English time format). |
| % {format}t | Time, in the form given by format, must be in the strftime(3) format. For format descriptions, see Time Format Definition. |
| %T | Time taken to serve the request, in seconds. |
| %u | Remote user (from auth; may be bogus if return status (%s) is 401). |
| %U | URL path requested. |
| %v | Canonical name of the server serving the request. |
| %V6 | Virtual server IPv6 address in the system, if load balancing, content switching, and/or cache redirection is used. |
| %D | Prints the HTTP transaction ID. |
| %L | Transaction time in milliseconds. |
| %R | HTTP Reason string mapped to status code. |
| %f | Source port logging. |
| %V | Virtual Server IPv4 address. |
-
The virtual IP address of HTTP transaction is displayed as "-" in the following scenarios:
-
Transparent mode is enabled
-
Integrated Caching is enabled, where NetScaler responds to client from Cache
-
An error response is generated
-
-
For instructions on how to export custom HTTP headers, see Configuring the NetScaler for Web Server Logging.
%+{user-agent}i, and if the user agent value is NetScaler system Web Client, then the information is logged as NetScaler system+Web+Client. An alternative is to use double quotation marks. For example, "%{user-agent}i" logs it as "NetScaler system Web Client." Do not use the \<Esc\> key on strings from %.. .r, %. . .i and, %. . .o. It complies with the requirements of the Common Log Format. Clients can insert control characters into the log. Therefore, you must take care when working with raw log files.
Time format definition
%{format}t string described in the Custom Log Format table. Values within brackets () show the range of values that appear. For example, in the %d description in the following table shows %d ranges from 1 to 31.
| Argument | Specifies |
|---|---|
| %% | The same as %. |
| %a | The abbreviated name of the weekday for the locale. |
| %A | The full name of the weekday for the locale |
| %b | The abbreviated name of the month for the locale. |
| %B | The full name of the month for the locale. |
| %C | The century number (the year divided by 100 and truncated to an integer as a decimal number [1,99]); single digits are preceded by a 0. |
| %d | User-defined field. |
| %K | The century number (the year divided by 100 and truncated to an integer as a decimal number [1,99]); single digits are preceded by a 0. |
| %e | The day of month [1,31]; single digits are preceded by a blank. |
| %h | The abbreviated name of the month for the locale. |
| %H | The hour (24-hour clock) [0,23]; single digits are preceded by a 0. |
| %I | The hour (12-hour clock) [1,12]; single digits are preceded by a 0. |
| %j | The number of the day in the year [1,366]; single digits are preceded by 0. |
| %k | The hour (24-hour clock) [0,23]; single digits are preceded by a blank. |
| %l | The hour (12-hour clock) [1,12]; single digits are preceded by a blank. |
| %m | The number of the month in the year [1,12]; single digits are preceded by a 0. |
| %M | The minute [00,59]; leading 0 is permitted but not required. |
| %n | Inserts a new line. |
| %p | The equivalent of either a.m. or p.m. for the locale. |
| %r | The appropriate time representation in 12-hour clock format with %p |
| %S | The seconds [00,61]; the range of values is [00,61] rather than [00,59] to allow for the occasional leap second and for the double leap second. |
| %3 | The milliseconds [000,999]; the range of values is [000,999]. |
| %6 | The microseconds [000000,999999]; the range of values is [000000,999999]. |
| %9 | The nanoseconds [000000000,999999999]; the range of values is [000000000,999999999]. |
| %t | Inserts a tab. |
| %u | The day of the week as a decimal number [1,7]. 1 represents Sunday, 2 represents Tuesday and so on. |
| %U | The number of the week in the year as a decimal number [00,53], with Sunday as the first day of week 1. |
%U and %W (and also between modified conversions %OU and %OW) is the day considered to be the first day of the week. Week number 1 is the first week in January (starting with a Sunday for %U, or a Monday for %W). Week number 0 contains the days before the first Sunday or Monday in January for %U and %W.
Display server logs
logfilenameformat as STDOUT.