BigFix Logs and Monitoring

Blog Post Written By Dan Powers

Monitoring BigFix

The purpose of this blog is to show various aspects of the BigFix infrastructure that should be monitored for detecting issues before they become problems. This blog does not provide nor give explicit examples of which monitoring tools to use, but examples of what should be monitored so it can be adapted to a organizations monitoring standards / tool set.

This blog assumes that the audience reading this is familiar with BigFix and how it works, however we will start with an overview of the various components.

 

The BigFix Enterprise Server (BES) is the main component that runs the environment. This server communicates to the relays, client and the internet to download content (from IBM) and other sources such as Adobe and Microsoft to obtain patches.

Relays are used in remote locations for communications to endpoints and network bandwidth constraints. These are important components because if relays are not health, then the endpoints under them are not getting or sending data to the BigFix server.

Clients: the clients or devices being managed themselves, which communicate to the parent relay. For the most part the health of these agents is simply communication to the relay and ensuring that the BES Client (Agent) is running.

NOTE: For the BES server and all relays basic monitoring that your company has established like CPU / MEMORY / DISK space should be accepted and added to the default monitoring profile.

BigFix Enterprise Server (BES)

The BES server is the most important component to monitor, below are a number of options to monitor on the BES server itself.

1. Network Accessibility: A network “probe/ping” to the server to ensure it is on the network and responding. The BES server at the very least needs to be reachable by all Top level relays.

2. Services: The following services should be running always:

a. BigFix Root Server: Handles all incoming connections
b. BigFix FillDB: Puts information from the clients to the database
c. BigFix GatherDB: put new Fixlet information into the database
d. BigFix Client: the client on the BES server should be running
e. BigFIx Web Reports: (optional) usually installed on the BES server, should be running

3. BufferDir: The bufferdir temporarily stores reports from the clients before being sent into the database

a. Default Location: c:\Program Files\BigFix Enterprise\BES Server\FillDBDAta\Bufferdir
b. Becomes FULL if it has 3MB of file or more than 10,000 files (default)
c. Monitor this location and alert when 2.5 MB full, don’t monitor more than once every 5-10 minutes

4. Database: The database is central core of information coming and going from the BigFix environment. Various components can be monitored based on DB (usually MS-SQL)

a. MSSQL Server service is running
b. SQL Server Agent is running
c. Additional SQL Server checks your DB team has established

5. Ensure BigFix server is getting up0to0date Fixlet Information: The BES will periodically gather from the main Fixlet server to get the latest data from IBM

a. Each Fixlet message site that your BigFix server subscribes to has to “GatherURL”. For example “Patches for Windows” site has a GatherURL of http://sync.bigfix.com/cgi-bin/bfgather/bessecurity. This is then mirrored to the relays with a URL : “http://YOURBESERVERNAME:52311/cgi-bin/bfenterprise/besgathermirror.exe?url=http//sync.bigfix.com/cgi-bin/bfgather/bessecuirty – Checking the logs file with a log monitoring tool looking for “Gather errors”.

6. DISK I/O

a. BigFix does a lot of READ/WRITE on the disks and relays. Monitor under normal operations to get a baseline and alert when too high.

7. FIN_WAIT2

a. On the BES server and relays check for the FIN_WAIT2 state on port 52311, there will be some of these on the BES server and relays, however, monitor for normal count and alert when high.

BigFix Relay Server(s)

BigFix relays are important to monitor because it moves data between the BigFix server and the clients, both in sending new content and actions to the client and from the client back up to the BES database. Many of the same components on the BES server can be applied to the relays.

1. Network Accessibility: A network “probe/ping” to the server to ensure it is on the network and responding. The relay should be reachable by all client it services.

2. Services: The following services should be running always:

a. BigFix Relay: Handles all incoming connections
b. BigFix Client: the client on the BES relay should be running

3. BufferDir: The bufferdir temporarily stores reports from the clients before being sent up to the next relay/BES server

a. Default Location: c:\Program Files\BigFix Enterprise\BES Server\FillDBDAta\Bufferdir
b. Becomes FULL if it has 3MB of file or more than 10,000 files (default)
c. Monitor this location and alert when 2.5 MB full, don’t monitor more than once every 5-10 minutes

4. DISK I/O

a. BigFix does a lot of READ/WRITE on the disks and relays. Monitor under normal operations to get a baseline and alert when too high.

5. FIN_WAIT2

a. On the BES server and relays check for the FIN_WAIT2 state on port 52311, there will be some of these on the BES server and relays, however monitor for normal count and alert when high.

 

Logfiles

This section denotes various logfiles on the BES server / Relays and services like the Web Reports system. Where these logs are, how to adjust the verbosity etc.

We want to look at your logs, example below of the RELAY log file on both BES server and relays showing issues:

• MON, 18 APR 2016 01:19:59 -0400 – /CGI-BIN/BFENTERPRISE/CLIENTREGISTER.EXE (1460) – GETEXPECTEDVERSIONOFPARENT ERROR: HTTP ERROR 28: TIMEOUT WAS REACHED: CONNECTION TIMED OUT AFTER 10000 MILLISECONDS
• MON, 18 APR 2016 01:19:59 -0400 – /CGI-BIN/BFENTERPRISE/CLIENTREGISTER.EXE (2180) – GETEXPECTEDVERSIONOFPARENT ERROR: HTTP ERROR 28: TIMEOUT WAS REACHED: CONNECTION TIMED OUT AFTER 10000 MILLISECONDS

THE FOLLOWING CAN HAPPEN, BUT IF YOU HAVE MULTIPLE OF THESE IN A SHORT PERIOD OF TIME, THEN ALERT

• TUE, 19 APR 2016 00:23:57 -0400 – 1736 – POSTRESULTSFORWARDER (HTTP ERROR 503): PARENT RELAY IS BUSY, BACKING OFF.
• TUE, 19 APR 2016 00:24:05 -0400 – 1736 – POSTRESULTSFORWARDER (HTTP ERROR 503): PARENT RELAY IS BUSY, BACKING OFF.
• TUE, 19 APR 2016 00:24:14 -0400 – 1736 – POSTRESULTSFORWARDER (HTTP ERROR 503): PARENT RELAY IS BUSY, BACKING OFF.
• TUE, 19 APR 2016 00:24:29 -0400 – 1736 – POSTRESULTSFORWARDER (HTTP ERROR 503): PARENT RELAY IS BUSY, BACKING OFF.

The log above is showing up on relays when a problem exists with the parent relay, telling a lower level relay to back off.

Client

Client Logs

The BigFix Client will record its current activity into a log file with the current date as the file name in the format “[year][month][day].log”. If an active log reaches 512K in size it will be moved to a backup (.bkg) file and a new log will be started for the current day. If the log reaches 512K again the backup will overwrite the existing backup. Both the active and backup logs will be deleted after ten days.

Here are the default locations of the BigFix Client logs for each operating system:

• Windows: \Program Files\BigFix Enterprise\BES Client\__BESData\__Global\Logs
• Unix/Linux: /var/opt/BESClient/__BESData/__Global/Logs
• Mac: /Library/Application Support/Bigfix/BES Agent/__BESData/__Global/Logs

Client logs can be collected using one of the following methods:

1. Manually remote into the endpoint machine, copy/compress the logs directory, and transfer or FTP it to a network file share.
2. Collect client logs via Client Diagnostics.

Client Debug Logging

Client debug logging (also called EMsg [extended message] logging) can be activated on a client to verbosely trace every operation a client performs. When setting the debug detail level, always set it to 10000.

Enabling/Disabling Client Debug logging:

Method 1: Take action on the following tasks in the BES Support site:

Task # 157: BES Client Setting: Enable Debug Logging (specify 10000 as the debug detail level)
Task # 196: BES Client Setting: Disable Debug Logging

Method 2: Create/Remove the following custom client settings via the BigFix Console. Select the computer(s) > right click > choose “Edit Computer Settings”. To activate, create the following three client settings:

_BESClient_EMsg_Detail – This setting will enable the BigFix Client debug log level that will give extended information about BigFix Client activity. Set the value to 10000

_BESClient_EMsg_File – Set this settings value to the full path of the file to store the extended messages to. The value should be the full path to the log (For example: C:\temp\BESClientEMsg.log)

Note: The _BESClient_EMsg_Detail setting must be greater than 0 to use this option.

_BESClient_EMsg_EvalLog – Set this settings value to 1 to enable EMsg (debug) logging of content identifiers in advance of relevance evaluation. This can be useful in helping to identify a relevance expression that may be crashing a client.

Method 3: Manually create/remove debug logging settings directly on the endpoint machine:

On Windows endpoints, add/remove the following registry keys and values:

o [HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_Detail] set to “value”=”10000”
o [HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_File] set to “value”=”C:\Program Files (x86)\BigFix Enterprise\BES Client\besclientdebug.log”
o [HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_EvalLog] set to “value”=”1”

On Non-Windows endpoints, add/remove the following lines in the config file located at /var/opt/BESClient/besclient.config:

[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_File]
value = /var/log/BESClient/besclientdebug.log
[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_Detail]
value = 10000
[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_EvalLog]
value = 1

Restart the client service after adding/removing the settings for them to take effect.

Client debug logging can be collected using one of the following methods:

1. Manually remote into the endpoint machine, copy/compress the besclientdebug.log file, and transfer or FTP it to a network file share.
2. Collect client logs via Client Diagnostics.

Client Usage Profiler

The client usage profiler will log time spent on evaluating content. This information easily allows you to see which pieces of content consume most of a client’s time during its evaluation cycle. After the usage profiler has been enabled, the Client tracks the top 100 activities that take the longest time and stores them in files (in the client directory) with the following naming convention “usageprofiler.txt.xxxx”

Enabling/Disabling Client Debug logging:

Method 1: Take action on the following tasks in the BES Support site:
Task # 361: TROUBLESHOOTING: Enable BES Client Usage Profiler
Task # 418: TROUBLESHOOTING: Disable BES Client Usage Profiler

Method 2: Create/Remove the following custom client settings via the BigFix Console. Select the computer(s) > right click > choose “Edit Computer Settings”. To activate, create the following three client settings:

_BESClient_Resource_TrackingMaxFiles – Set to 24 ( 24 hours of tracking )
_BESClient_Resource_TrackingCount – Set to 100
_BESClient_Resource_TrackingCycleSeconds – Set to 3600
_BESClient_Resource_TrackingFile – Set to C:\Program Files (x86)\BigFix Enterprise\BES Client\usageprofiler.txt for Windows endpoints and /var/opt/BESClient/usageprofiler.txt for non-Windows endpoints.

Method 3: Manually create/remove debug logging settings directly on the endpoint machine:

On Windows endpoints, add/remove the following registry keys and values:

o HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingMaxFiles] set to “value”=”24”
o [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingCount] set to “value”=”100”
o [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingCycleSeconds] set to “value”=”3600”
o [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingFile] set to “value”=”C:\Program Files (x86)\BigFix Enterprise\BES Client\usageprofiler.txt”

On Non-Windows endpoints, add/remove the following lines in the config file located at /var/opt/BESClient/besclient.config:

[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingMaxFiles]
value = 24
[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingCount]
value = 100
[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingCycleSeconds]
value = 3600
[Software\BigFix\EnterpriseClient\Settings\Client\_BESClient_Resource_TrackingFile]
value = “/var/opt/BESClient/usageprofiler.txt”
Restart the client service after adding/removing the settings for them to take effect.

As a best practice, collect client usage data over an entire 24-hour period. This will allow the usage profiler to generate enough data samples to be useful in your analysis of the problem.

The following is a small example of a usage file that contains unusually long evaluation times. An efficient piece of content would typically take less that a second to evaluate. Items taking longer than a second should be further investigated.

Start :Thu, 30 Feb 2013 15:21:40 -0800
Elapsed Time: 02:48:48
Tracking: Top 100
Samples: 99
Elapsed Evaluation Time: 18:18:32

Time in Seconds: Site.Item ID:Activity

1. 16478.126: actionsite.2147453787:Analysis Relevance
2. 16478.124: actionsite.2146599782:Analysis Relevance
3. 16478.119: actionsite.2142786964:Analysis Relevance
4. 16478.118: BES Support.521:Verify Fixlet Relevance

Client Dump Files

If the client process on a Window’s endpoint is stopping or crashing; dump files might get generated and could prove useful in determining the cause of the agent’s stop/crash. Dump files output to the client’s BES Client directory on the endpoint, for example to:

C:\Program Files (x86)\BigFix Enterprise\BES Client\BESClient.dmp

Client dump files can be collected using one of the following methods:

Manually remote into the endpoint machine, copy/compress the dump files generated, and transfer or FTP them to a network file share.

Collect client logs via Client Diagnostics.

Client Performance Counters

Performance counter properties that can be deployed to measure client evaluation performance and track the top 10 longest evaluating pieces of content on a client.

Client Diagnostics

The client diagnostics action collects the following information and performs the following tasks on the endpoint:

Windows:

• Gathers a variety of information about the local computer (name, OS, DC, user, drive space, language, etc.)
• Gathers a variety of information about the BigFix Client (masthead info, service status, path, license info, settings info, etc.)
• Checks each Fixlet site to ensure that it is gathered on the latest version
• Collects all retrieved properties in a QnA format
• Analyzes log files
• Checks for consistency across BigFix Client version files and collect the upgrade history
• Performs a test to determine the distance to all BigFix Relays (use –r command line option)
• Performs a test to determine the distance to all BigFix Relays (must run the tool with a command line argument ‘–runrelayselector’)
• Summarizes all warnings and errors
• Saves a copy of the ~__BESData folder and the BigFix Client registry keys
• Creates a human readable XML output of the client’s Fixlet history (SiteData)
• Saves a copy of the Client Usage Profiler log and EMsg (client debug logging) log if they exist
• Zips up the data for easy collection
• Collection of BES Client Diagnostics for Windows via Task # 353 happens through the deployment and execution of a Windows BESClientDiagnostics.exe executable (current version 3.0.2.31); which can be downloaded here for manual execution.

Note: The BESClientDiagnostics.exe executable is also assisted by the SiteDataReader.exe and ParseRelaysDotDat.exe executables during the deployment of the tool via the task. The SiteDataReader assists in unpacking the site data from the client while the ParseRelaysDotDat converts the relays.dat file into human readable text. Each of the tools can be run separately if you are collecting client diagnostics manually using only the BESClientDiagnostics.exe.

ParseRelaysDotDat.exe

SiteDataReader.exe

Manual Instructions: You can manually execute this tool on a Windows endpoint to collect diagnostics by remoting into the Windows machine, downloading the client diagnostics .exe to the Windows machine, and then double clicking on it. The tool will output a folder of information in the same directory as the tool. Zip this folder if information up and copy it off of the machine to a network share.

The ClientDiagnostics tool can also be run via a command line.

Command line usage:

Usage: clientdiagnostics.exe [options]

Command Line Options:

–nolistsettings: Don’t list all the settings

–nolistproperties: Don’t list all the properties

–nologanalysis: Don’t analyze the BES Client logs

–runrelayselector: Run a test relay selector to debug relay selection problems

–minimaltests: Run only the minimum set of tests (no properties, no log analysis, no settings)

–help: displays these options

Non-Windows:

A zip of the client endpoint’s __BESData directory

Collecting the uploaded data from the BigFix Server:

The data will be uploaded to the BigFix server machine to a subfolder within the \BES Server\UploadManagerData\BufferDir\sha1 directory. The sub folder will be the last two digits of the endpoint’s computer id and then the computer id, for example: \BES Server\UploadManagerData\BufferDir\sha\12\342912

To determine the computer id of a computer, you can look for the Computer ID property on the computer’s property Summary sheet.

Notes:
Read the description tab within the deployment tasks for additional information.

On Windows systems, during action execution, a small DOS window will be displayed in the task bar.

The BESClientDiagnostic.exe executable only works on Microsoft Windows machines. There is no equivalent version of the tool for non-Windows machines at this time.

If the size of the resulting zip file generated by the tool is larger than 15 MB, it will not be uploaded to the Bigfix Server. The action script can be modified to set a higher limit by modifying the _BESClient_ArchiveManager_MaxArchiveSize setting to a higher value in the action script from the Action Script tab right before deploying the action.

After executing the above task, the “TROUBLESHOOTING: Run BES Client Diagnostics – Cleanup” task can be used to remove the diagnostics results files from the BigFix client machine.

By default, the maximum size of the Upload Manager directory on a BigFix relay is 20 MB. The BES Support site contains the following fixlet message that will alert you if a BigFix relays Upload Manager directories are at maximum capacity: Task # 343 “WARNING: Upload Manager Directory Full – BES Server / Relay”. Refer to the fixlet for further information.

Console

By default, logging is not enabled for the TEM Console. However, you may enable logging by setting the following registry keys.

Console logging is not enabled by default because the log can grow to be quite large. Best practice recommends you turn on the log file for debugging purposes only and then deactivate it afterwards (by removing the settings) to disable further logging.

To activate console logging create the following two REG_SZ values in the [HKEY_CURRENT_USER\Software\BigFix\Enterprise Console] registry key.

Name: DebugOut
Type: REG_SZ
Value: (path and filename of the log to generate. i.e. c:\console.log)

Name: EnabledLogs (Note: beginning in TEM version 6.0.19)
Type: REG_SZ
Value: all (Note: the all value is only available in TEM 7.1+)

If you do not want to log all stream types (using the all value), the following stream types can help to refine which specific logging streams are output to the log (by putting them in the EnabledLogs value as a semicolon-delimited list):

critical
debug
memory
performance
timing

Please verify that the log file is created and written to. You do not need to restart the TEM Console for the file to appear.

Server

FillDB.log

FillDB Debug Log — Used to troubleshoot issues with agent reports or FillDB connection errors. By default, log will show “critical” log messages

• Windows:

o Default location: “FillDBData” folder of the TEM Server (e.g., “C:\program files\BigFix Enterprise\BES Server\FillDBData\FillDB.log”)
o Log controlled by registry key

– HKLM\Software\BigFix\Enterprise Server\FillDB
– “EnabledLogs”

– default: “critical”
– “all” or a subset of “critical; debug; database”

• Linux:

o Default location: “FillDBData” folder of the TEM Server (“/var/opt/BESServer/FillDBData/FillDB.log”)
o Log controlled by configuration file BESServer.config

– “EnabledLogs”

– default: “critical”
– “all” or a subset of “critical; debug; database”

FillDB Performance Log — Used to measure and troubleshoot FillDB performance issues with the server/database. Disabled by default.

• Windows:

o Log controlled by registry key:

– HKLM\Software\BigFix\Enterprise Server\FillDB
– “PerformanceDataPath”
– Example: “C:\program files\BigFix Enterprise\BES Server\FillDBData\FillDBPerf.log”

o Requires FillDB service restart after registry key change to activate.
o To enable rotating capability for FillDB.log, you must create the following registry value and set it to the maximum number of bytes that the file can reach before rotating.
The maximum number of rotating files is 10 and is not configurable.

– “LogFileSizeLimit” (of DWORD type) value under the FillDB registry key for your FillDB.log rotating file
For example, if you want your FillDB.log to rotate across 10 files, each one being at most 100 KB, you must set “LogFileSizeLimit” to 100000.

• Linux:

o Log controlled by configuration file BESServer.config

– “PerformanceDataPath”
– Example: “C:\program files\BigFix Enterprise\BES Server\FillDBData\FillDBPerf.log”

o Requires FillDB service restart after configuration file change to activate.

o To enable rotating capability for FillDB.log, you must create the following registry value and set it to the maximum number of bytes that the file can reach before rotating.
The maximum number of rotating files is 10 and is not configurable.

– “LogFileSizeLimit” (of DWORD type) value in the BESServer.config file
For example, if you want your FillDB.log to rotate across 10 files, each one being at most 100 KB, you must set “LogFileSizeLimit” to 100000.

BESRelay.log

• Relay Log — This log contains information on the activities of the BES Root Server service component of the server, along with debugging information for top level exceptions on any of the debugging threads.
• Default enabled in non-verbose mode named “BESRelay.log” on root server (“C:\program files\BigFix Enterprise\BES Server\BESRelay.log”, “/var/log/BESRelay.log”) and Relays (“C:\program files\BigFix Enterprise\BES Relay\logfile.txt”).
• Verbose logging can be enabled to output more details
• To enable rotating capability, you must create the following registry values and set it to the maximum number of bytes that the file can reach before rotating.

The maximum number of rotating files is 10 and is not configurable.

–  Windows:
“value” (of REG_SZ type) under HKEY_LOCAL_MACHINE\…\BigFix\EnterpriseClient\Settings\Client\_BESRelay_HTTPServer_LogFileSizeLimit
–  Linux:
“value” (of REG_SZ type) in the BESServer.config file

For example, if you want your BESRelay.log to rotate across 10 files, each one being at most 100 KB, you must set “value” to “100000”

GatherDB.log

• The GatherDB service logs its output to the GatherDB log located on the server:

o Windows: \Program Files (x86)\BigFix Enterprise\BES Server\GatherDBData
o Linux: The GatherDB service connects to the database and processes (imports) the external fixlet sites, that are gathered by the BES Root Server and BES Gather services from the sync.bigfix.com Internet site, into the database.

• Typical log messages indicating a successful import of a newly gathered content site:
Thu, 04 Sep 2014 08:12:29 -0700 — Beginning import of version 2072 of site Enterprise Security
Thu, 04 Sep 2014 08:14:08 -0700 — Import of version 2072 of site Enterprise Security completed successfully

Server Audit.Log

The server keeps and audit log at BigFix Enterprise\BES Server\server_audit.log and logs the following types of audit events:

• AuditStream() << “approver \”” << checkResult.user << “\” approved an activity performed by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was made a reader for custom site \”” << parameters.siteName.GetString() << “\”” << “by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was removed as a reader from custom site \”” << parameters.siteName.GetString() << “\”” << “by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was made a writer for custom site \”” << parameters.siteName.GetString() << “\”” << “by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was removed as a writer from custom site \”” << parameters.siteName.GetString() << “\”” << “by ” << “user “{name}” ({id})”;
• AuditStream() << “role “{name}”” << ” was created by ” << “user “{name}” ({id})”;
• AuditStream() << “role “{name}”” << ” was deleted by ” << “user “{name}” ({id})”;
• AuditStream() << “role “{name}”” << ” has been given ” << ( it->second == UserRoleSitePrivileges::SiteWriter ? “write” : it->second == UserRoleSitePrivileges::SiteReader ? “read” : “ownership” ) << ” privileges on ” << SiteAuditText( it->first ) << ” by ” << UserAuditText( user );
• AuditStream() << “user “{name}” ({id})” << ” has been added to ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “ldap group “{name}” (DN={dn})” << ” has been added to ” << “role “{name}””;
• AuditStream() << “site {site}” << ” removed from ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “site {site}” << ” added to ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” added to ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” removed from ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was assigned to ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” was removed from ” << “role “{name}”” << ” by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” privileges updated by ” << “user “{name}” ({id})” << “: ” << PrivilegesAuditText( iface.db, userInfo, isMasterOperator, canCreateCustomContent, showOtherUsersActions, unmanagedAssetPrivilege, loginPermission, approverRoleID );
• AuditStream() << “ldap user \”{name}\” (DN={dn})” << ” created by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” password changed by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” changed their own password”;
• AuditStream() << “user “{name}” ({id})” << ” was removed by ” << “user “{name}” ({id})”;
• AuditStream() << “user “{name}” ({id})” << ” management rights updated by ” << “user “{name}” ({id})”;

The following were added in 8.2 Patch #3:

• AuditStream() << oldUserAuditText << ” converted to ldap user ” << “ldap user \”{name}\” (DN={dn})” << ” by ” << initiator;
• AuditStream() << “role “{name}”” << ” was modified by ” << “user “{name}” ({id})” << “: ” << PrivilegesAuditText( iface.db, role.UserRolePrivileges() );
• AuditStream() << “user “{name}” ({id})” << ” created by ” << “user “{name}” ({id})” << “: ”
• AuditStream() << “ldap user \”{name}\” (DN={dn})” << ” created based on membership of role(s): ” << roleNames.Ref();

If the size of this log file increases too much, it might happen a server failure. To avoid that, its maximum size can be set with option “_Audit_Logging_LogMaxSize” expressed in bytes. The default value is 100MB. When the size limit is reached, the file is renamed with the format “server_audit.YYYYMMDDHHMM.log”. To enable the new setting, the user must restart the server. Important: The renamed log files are never deleted; to free space, the user must delete manually the unnecessary files.

Relay

logfile.txt

• This log contains information on the activities of the BESRelay service component of the relay, along with debugging information for top level exceptions on any of the debugging threads. Default enabled in non-verbose mode named “logfile.txt” on relay machine (“C:\program files\BigFix Enterprise\BES Relay\logfile.txt”) and the root server (“C:\program files\BigFix Enterprise\BES Server\BESRelay.log”).
• Verbose logging can be enabled to output more details

Take the following steps to enable (debug level) verbose logging on the main IEM server or relay to log for all operations performed by the BES Root Server or BES Relay service processes.
Verbose data is output to the \Program Files\BigFixEnterprise\BES Server\BESRelay.log file on the server and \Program Files\BigFixEnterprise\BES Relay\logfile.txt on a relay

The logging is enabled by creating a client setting.
Enabling logging through the IEM console:
1. Login to the console as a master console operator.
2. Right click on the IEM Server or relay computer in the console.
3. Select Edit Computer Settings….
4. Check in the list to see if the _BESRelay_Log_Verbose setting has already been created. If it has, Edit it and change its value to 1 (to enable it).
5. If the setting has not been created, click Add to create it. Enter _BESRelay_Log_Verbose for the setting name and 1 for the setting value to enable the verbose logging.

6. Click OK. An action named “Change Multiple Settings” is taken targeted at the IEM server or relay machine.
7. After the action has completed successfully (and the setting has been applied), restart the BES Root Server service if on the IEM server or restart the BES Relay service if on the relay. You can take action on Task # 447: Restart Service in the BES Support site to do this).

Enabling logging manually through the registry (Windows)
1. Remote into the main IEM server or relay machine.
2. Open up the registry editor (regedit)
3. Add the setting (_BESRelay_Log_Verbose) as a key to the client section of the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\BigFix\EnterpriseClient\Settings\Client
4. Create a REG_SZ value named value.
5. Set the value to 1

6. Restart the BES Root Server service if on the server or the BES Relay service if on a relay.

Enabling logging manually through the settings file (Linux)
1. Remote into the relay endpoint and edit /var/opt/BESClient/besclient.config, and add the following lines:

[Software\BigFix\EnterpriseClient\Settings\Client\_BESRelay_Log_Verbose]
effective date = [Enter Current Date Time In Standard Format]
value = 1

The effective current date time must be in a format similar to the following:
Wed, 06 Jun 2012 11:00:00 -0700

2. Stop and restart the relay service

/etc/init.d/besrelay stop
/etc/init.d/besrelay start

Note: to disable verbose logging, set the setting to 0.

Warning: Leave the verbose logging on only as long enough to troubleshoot the issue you are having in order to conserve on disk space and processing resources.

Note: You may specify a file size at which the log begins rotation. Beginning in version 9.0, the size is 50*1024*1024 (52,428,800)
_BESRelay_HTTPServer_LogFileSizeLimit
Value is in bytes

If this is non-zero, it is the file size at which the log begins rotation.

Also Note: A maximum of 10 rotated log files will be maintained in addition to the active log file. (Eg. logfile.txt, logfile.txt_0, logfile.txt_1, …, logfile.txt_9.)

Web Reports

Web Reports logs — Used to troubleshoot web reports issues

• HKLM\Software\BigFix\Enterprise Server\BESReports

• “LogOn” (dword)

o 0 for log off, 1 for on

• “EnabledLogs” (string)

o “all”
o or a subset of “critical; debug; memory; performance; timing; database”

• “LogPath” (string)

o full path to log file (i.e. “C:\program files\BigFix Enterprise\BES Server\WR.log”)

• “Debug” (string)
• “1” will display more detail for errors inside the web reports user interface.

PREV

SKOUT Education Series: Managed SIEM vs MSSP | A Cyber 101 Resource

NEXT

SKOUT Education Series: Medical Lab Provider Breaches | A Cyber 101 Resource

WRITTEN BY: