plaso.parsers.text_plugins package

Submodules

plaso.parsers.text_plugins.android_logcat module

Text parser plugin for Android logcat files.

Android logcat can have a number of output forms, however this particular parser only supports the ‘threadtime’ and ‘time’ formats.

In addition, support for the format modifiers: - uid - usec - UTC | zone - year

class plaso.parsers.text_plugins.android_logcat.AndroidLogcatEventData[source]

Bases: EventData

Android logcat event data.

component_tag

the tag that indicates the system component from which the logcat line originates.

Type

str

file_offset

the file offset of where the log message was parsed.

Type

int

message

the log message.

Type

str

pid

process identifier (PID) that created the logcat line.

Type

int

priority

a character in the set {V, D, I, W, E, F, S}, which is ordered from lowest to highest priority.

Type

str

thread_identifier

thread identifier (TID) that created the logcat line.

Type

int

user_identifier

the user identifier (UID) or Android ID of the logged process.

Type

int

DATA_TYPE = 'android:logcat'
class plaso.parsers.text_plugins.android_logcat.AndroidLogcatTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Android logcat files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Android logcat file'
ENCODING = 'utf-8'
NAME = 'android_logcat'

plaso.parsers.text_plugins.apache_access module

Text parser plugin for Apache access log (access.log) files.

Parser based on the two default apache formats, common and combined log format defined in https://httpd.apache.org/docs/2.4/logs.html

class plaso.parsers.text_plugins.apache_access.ApacheAccessEventData[source]

Bases: EventData

Apache access event data.

http_request_referer

http request referer header information.

Type

str

http_request

first line of http request.

Type

str

http_request_user_agent

http request user agent header information.

Type

str

http_response_bytes

http response bytes size without headers.

Type

int

http_response_code

http response code from server.

Type

int

ip_address

IPv4 or IPv6 addresses.

Type

str

port_number

canonical port of the server serving the request.

Type

int

remote_name

remote logname (from identd, if supplied).

Type

str

server_name

canonical hostname of the server serving the request.

Type

str

user_name

logged user name.

Type

str

DATA_TYPE = 'apache:access'
class plaso.parsers.text_plugins.apache_access.ApacheAccessLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Apache access log (access.log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Apache access log (access.log) file'
NAME = 'apache_access'

plaso.parsers.text_plugins.apt_history module

Text parser plugin for Advanced Packaging Tool (APT) History log files.

class plaso.parsers.text_plugins.apt_history.APTHistoryLogEventData[source]

Bases: EventData

APT History log event data.

command

command executed

Type

str

error

reported error.

Type

str

packages

list of packages being affected.

Type

str

requester

user requesting the activity.

Type

str

DATA_TYPE = 'apt:history:line'
class plaso.parsers.text_plugins.apt_history.APTHistoryLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Advanced Packaging Tool (APT) History log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Advanced Packaging Tool (APT) History log file'
ENCODING = 'utf-8'
NAME = 'apt_history'

plaso.parsers.text_plugins.aws_elb_access module

Text parser plugin for AWS ELB access logs.

The AWS documentation is not clear about the meaning of the “target_port_list” field. The assumption is that it refers to a list of possible backend instances’ IP addresses that could receive the client’s request. This parser stores the “target_port_list” data in the “destination_list” attribute of an EventData object.

Also see:

https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/access-log-collection.html https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-access-logs.html

class plaso.parsers.text_plugins.aws_elb_access.AWSELBEventData[source]

Bases: EventData

AWS Elastic Load Balancer access log event data.

actions_executed

The actions taken when processing the request.

Type

str

alpn_back_end_protocol

The application protocol negotiated with the target, in string format. If no ALPN policy is configured in the TLS listener, no matching protocol is found, or no valid protocol list is sent, this value is set to -. (only for network load balancer logs)

Type

str

alpn_client_preference_list

The value of the application_layer_protocol_negotiation extension in the client hello message. This value is URL-encoded. Each protocol is enclosed in double quotes and protocols are separated by a comma. If no ALPN policy is configured in the TLS listener, no valid client hello message is sent, or the extension is not present, this value is set to -. The string is truncated if it is longer than 256 bytes. (only for network load balancer logs)

Type

str

alpn_front_end_protocol

The application protocol negotiated with the client, in string format. If no ALPN policy is configured in the TLS listener, no matching protocol is found, or no valid protocol list is sent, this value is set to -. (only for network load balancer logs)

Type

str

chosen_cert_arn

The ARN of the certificate presented to the source.

Type

str

chosen_cert_serial

Reserved for future use. This value is always set to -. (only for network load balancer logs)

Type

str

classification

The classification for desync mitigation.

Type

str

classification_reason

The classification reason code.

Type

str

connection_time

The total time for the connection to complete, from start to closure, in milliseconds. (only for network load balancer logs)

Type

str

destination_group_arn

The Amazon Resource Name (ARN) of the destination group.

Type

str

destination_ip_address

The IP address of the destination that processed this request.

Type

str

destination_list

A space-delimited list of IP addresses and ports for the destinations that processed this request.

Type

str

destination_port

The port of the destination that processed this request.

Type

int

destination_processing_time

The total duration from the time the load balancer sent the request to a destination until the destination started to send the response headers.

Type

str

destination_status_code

The status code of the response from the destination.

Type

int

destination_status_code_list

A space-delimited list of status codes.

Type

str

domain_name

The SNI domain provided by the source during the TLS handshake.

Type

str

error_reason

The error reason code, enclosed in double quotes.

Type

str

handshake_time

The total time for the handshake to complete after the TCP connection is established, including client-side delays, in milliseconds. This time is included in the connection_time field. (only for network load balancer logs)

Type

str

incoming_tls_alert

The integer value of TLS alerts received by the load balancer from the client, if present. (only for network load balancer logs)

Type

str

listener

The resource ID of the TLS listener for the connection. (only for network load balancer logs)

Type

str

matched_rule_priority

The priority value of the rule that matched the request.

Type

int

received_bytes

The size of the request, in bytes, received from the source.

Type

int

redirect_url

The URL of the redirect destination.

Type

str

request_processing_time

The total duration from the time the load balancer received the request until the time it sent the request to a destination.

Type

str

request_type

The type of request or connection.

Type

str

resource_identifier

The resource ID of the load balancer.

Type

str

response_processing_time

The total processing duration.

Type

str

sent_bytes

The size of the response, in bytes, sent to the source.

Type

int

ssl_cipher

The SSL cipher of the HTTPS listener.

Type

str

ssl_protocol

The SSL protocol of the HTTPS listener.

Type

str

source_ip_address

The IP address of the requesting source.

Type

str

source_port

The port of the requesting source.

Type

int

tls_cipher

The cipher suite negotiated with the client, in OpenSSL format. If TLS negotiation does not complete, this value is set to -. (only for network load balancer logs)

Type

str

tls_named_group

Reserved for future use. This value is always set to -. (only for network load balancer logs)

Type

str

tls_protocol_version

The TLS protocol negotiated with the client, in string format. If TLS negotiation does not complete, this value is set to -. (only for network load balancer logs)

Type

str

trace_identifier

The contents of the X-Amzn-Trace-Id header.

Type

str

user_agent

A User-Agent string.

Type

str

version

The version of the log entry. (only for network load balancer logs)

Type

str

DATA_TYPE = 'aws:elb:access'
class plaso.parsers.text_plugins.aws_elb_access.AWSELBTextPlugin[source]

Bases: TextPlugin

Text parser plugin for AWS ELB access log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'AWS ELB Access log file'
ENCODING = 'utf-8'
NAME = 'aws_elb_access'

plaso.parsers.text_plugins.confluence_access module

Text plugin for Confluence access log (conf_access_log[DATE].log) files.

Also see:

https://confluence.atlassian.com/doc/configure-access-logs-1044780567.html

class plaso.parsers.text_plugins.confluence_access.ConfluenceAccessEventData[source]

Bases: EventData

Confluence access event data.

forwarded_for

request X-FORWARDED-FOR header value.

Type

str

http_request_method

HTTP request method.

Type

str

http_request_referer

HTTP request referer header information.

Type

str

http_request_uri

HTTP request URI.

Type

str

http_request_user_agent

HTTP request user agent header information.

Type

str

http_response_bytes

HTTP response bytes size without headers.

Type

int

http_response_code

HTTP response code from server.

Type

int

http_version

HTTP request version.

Type

str

process_duration

time taken to process the request in milliseconds.

Type

int

remote_name

remote hostname or IP address

Type

str

thread_name

name of the thread that handled the request.

Type

str

user_name

response X-AUSERNAME header value.

Type

str

DATA_TYPE = 'confluence:access'
class plaso.parsers.text_plugins.confluence_access.ConfluenceAccessTextPlugin[source]

Bases: TextPlugin

Text plugin for Confluence access log (conf_access_log[DATE].log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Confluence access log (access.log) file'
NAME = 'confluence_access'

plaso.parsers.text_plugins.dpkg module

Text parser plugin for Debian package manager log (dpkg.log) files.

Information updated 02 September 2016.

An example:

2016-08-03 15:25:53 install base-passwd:amd64 <none> 3.5.33

Log messages are of the form:

YYYY-MM-DD HH:MM:SS startup type command Where type is:

archives (with a command of unpack or install) packages (with a command of configure, triggers-only, remove or purge)

YYYY-MM-DD HH:MM:SS status state pkg installed-version

YYYY-MM-DD HH:MM:SS action pkg installed-version available-version Where action is:

install, upgrade, configure, trigproc, disappear, remove or purge.

YYYY-MM-DD HH:MM:SS conffile filename decision Where decision is install or keep.

class plaso.parsers.text_plugins.dpkg.DpkgEventData[source]

Bases: EventData

Dpkg event data.

body

body of the log line.

Type

str

DATA_TYPE = 'dpkg:line'
class plaso.parsers.text_plugins.dpkg.DpkgTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Debian package manager log (dpkg.log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Debian package manager log (dpkg.log) file'
ENCODING = 'utf-8'
NAME = 'dpkg'

plaso.parsers.text_plugins.iis module

Text parser plugin for Microsoft IIS log files.

class plaso.parsers.text_plugins.iis.IISEventData[source]

Bases: EventData

IIS log event data.

Content of a sent or received cookie.

Type

str

cs_host

HTTP host header name.

Type

str

cs_referrer

Site that referred to the requested site.

Type

str

cs_uri_query

URI query that was requested.

Type

str

cs_username

Username of the authenticated user that accessed the server, where anonymous users are indicated by a hyphen.

Type

str

dest_ip

IP address of the server that generated the logged activity.

Type

str

dest_port

Server port number.

Type

str

http_method

HTTP request method, such as GET or POST.

Type

str

http_status

HTTP status code that was returned by the server.

Type

str

protocol_version

HTTP protocol version that was used.

Type

str

received_bytes

Number of bytes received and processed by the server.

Type

str

requested_uri_stem

File requested, such as index.php or Default.htm

Type

str

s_computername

Name of the server that generated the logged activity.

Type

str

sc_substatus

HTTP substatus error code that was returned by the server.

Type

str

sc_win32_status

Windows status code of the server.

Type

str

sent_bytes

Number of bytes sent by the server.

Type

str

source_ip

IP address of the client that made the request.

Type

str

s_sitename

Service name and instance number that was running on the client.

Type

str

time_taken

Time taken, in milliseconds, to process the request.

Type

str

user_agent

User agent that was used.

Type

str

DATA_TYPE = 'iis:log:line'
class plaso.parsers.text_plugins.iis.WinIISTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Microsoft IIS log files.

COMMENT = {'#' {{{'Date:' {{W:(0-9){4} Suppress:('-')} W:(0-9){2} Suppress:('-')} W:(0-9){2} {{W:(0-9){2} Suppress:(':')} W:(0-9){2} Suppress:(':')} W:(0-9){2}} | {'Fields:' SkipTo:(LineEnd)}} | SkipTo:(LineEnd)}}
CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Microsoft IIS log file'
DATE_METADATA = {'Date:' {{W:(0-9){4} Suppress:('-')} W:(0-9){2} Suppress:('-')} W:(0-9){2} {{W:(0-9){2} Suppress:(':')} W:(0-9){2} Suppress:(':')} W:(0-9){2}}
DATE_TIME = {{W:(0-9){4} Suppress:('-')} W:(0-9){2} Suppress:('-')} W:(0-9){2} {{W:(0-9){2} Suppress:(':')} W:(0-9){2} Suppress:(':')} W:(0-9){2}
ENCODING = 'utf-8'
FIELDS_METADATA = {'Fields:' SkipTo:(LineEnd)}
LOG_LINE_6_0 = {{W:(0-9){4} Suppress:('-')} W:(0-9){2} Suppress:('-')} W:(0-9){2} {{W:(0-9){2} Suppress:(':')} W:(0-9){2} Suppress:(':')} W:(0-9){2} {W:(%&()+-;=?A-Z_a-z) | '-'} {{IPv4 address | IPv6 address} | '-'} {W:(-0-9A-Za-z) | '-'} {W:(%&()+-;=?A-Z_a-z) | '-'} {W:(%&()+-;=?A-Z_a-z) | '-'} {W:(0-9){1,6} | '-'} {W:(-0-9A-Za-z) | '-'} {{IPv4 address | IPv6 address} | '-'} {W:(%&()+-;=?A-Z_a-z) | '-'} {W:(0-9) | '-'} {W:(0-9) | '-'} {W:(0-9) | '-'}
NAME = 'winiis'
PORT = {W:(0-9){1,6} | '-'}
QUERY = {W:("%-)+-?A-~) | '-'}
URI = {W:(%&()+-;=?A-Z_a-z) | '-'}
USERNAME = {W:(-.0-9A-Za-z) | '-'}

plaso.parsers.text_plugins.interface module

This file contains the interface for text plugins.

class plaso.parsers.text_plugins.interface.PyparsingLineStructure(name, expression)[source]

Bases: object

Line structure.

expression

pyparsing expression to parse the line structure.

Type

pyparsing.ParserElement

name

name to identify the line structure.

Type

str

weight

number of times the line structure was successfully used.

Type

int

ParseString(string)[source]

Parses a string.

Parameters

string (str) – string to parse.

Returns

parsed tokens or None if the string could not

be parsed.

Return type

pyparsing.ParseResults

class plaso.parsers.text_plugins.interface.TextPlugin[source]

Bases: BasePlugin

The interface for text plugins.

abstract CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Text file'
ENCODING = None
NAME = 'text_plugin'
Process(parser_mediator, file_object=None, **kwargs)[source]

Extracts events from a text log file.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • file_object (Optional[dfvfs.FileIO]) – a file-like object.

plaso.parsers.text_plugins.ios_logd module

Text parser plugin for iOS sysdiagnose logd files (logd.0.log).

class plaso.parsers.text_plugins.ios_logd.IOSSysdiagnoseLogdData[source]

Bases: EventData

iOS sysdiagnose logd event data.

body

body of the event line.

Type

str

logger

name of the process that generated the event.

Type

str

DATA_TYPE = 'ios:sysdiagnose:logd:line'
class plaso.parsers.text_plugins.ios_logd.IOSSysdiagnoseLogdTextPlugin[source]

Bases: TextPlugin

Text parser plugin for iOS sysdiagnose logd files (logd.0.log).

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'iOS sysdiagnose logd file'
NAME = 'ios_logd'

plaso.parsers.text_plugins.mac_appfirewall module

Text plugin for MacOS Application firewall log (appfirewall.log) files.

class plaso.parsers.text_plugins.mac_appfirewall.MacAppFirewallLogEventData[source]

Bases: EventData

MacOS Application firewall log (appfirewall.log) file event data.

action

action.

Type

str

agent

agent that save the log.

Type

str

computer_name

name of the computer.

Type

str

process_name

name of the entity that tried to do the action.

Type

str

status

saved status action.

Type

str

DATA_TYPE = 'mac:appfirewall:line'
class plaso.parsers.text_plugins.mac_appfirewall.MacAppFirewallTextPlugin[source]

Bases: TextPlugin, YearLessLogFormatHelper

Text plugin for MacOS Application firewall log (appfirewall.log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'MacOS Application firewall log (appfirewall.log) file'
ENCODING = 'utf-8'
NAME = 'mac_appfirewall_log'

plaso.parsers.text_plugins.mac_securityd module

Text parser plugin for MacOS security daemon (securityd) log files.

Also see:

https://opensource.apple.com/source/Security/Security-55471/sec/securityd

class plaso.parsers.text_plugins.mac_securityd.MacOSSecuritydLogEventData[source]

Bases: EventData

MacOS securityd log event data.

caller

caller, consists of two hex numbers.

Type

str

facility

facility.

Type

str

level

priority level.

Type

str

message

message.

Type

str

security_api

name of securityd function.

Type

str

sender

name of the sender.

Type

str

sender_pid

process identifier of the sender.

Type

int

DATA_TYPE = 'mac:securityd:line'
class plaso.parsers.text_plugins.mac_securityd.MacOSSecuritydLogTextPlugin[source]

Bases: TextPlugin, YearLessLogFormatHelper

Text parser plugin for MacOS security daemon (securityd) log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'MacOS security daemon (securityd) log file'
DATE_TIME = Group:({W:(A-Za-z){3} W:(0-9){1,2}} {{W:(0-9){2} Suppress:(':')} W:(0-9){2} Suppress:(':')} W:(0-9){2})
ENCODING = 'utf-8'
NAME = 'mac_securityd'

plaso.parsers.text_plugins.mac_wifi module

Text parser plugin for MacOS Wifi log (wifi.log) files.

class plaso.parsers.text_plugins.mac_wifi.MacWifiLogEventData[source]

Bases: EventData

Mac Wifi log event data.

action

known WiFI action, for example connected to an AP, configured, etc. If the action is not known, the value is the message of the log (text variable).

Type

str

agent

name and identifier of process that generated the log message.

Type

str

function

name of function that generated the log message.

Type

str

text

log message

Type

str

DATA_TYPE = 'mac:wifilog:line'
class plaso.parsers.text_plugins.mac_wifi.MacWifiLogTextPlugin[source]

Bases: TextPlugin, YearLessLogFormatHelper

Text parser plugin MacOS Wifi log (wifi.log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'MacOS Wifi log (wifi.log) file'
ENCODING = 'utf-8'
NAME = 'mac_wifi'
THREE_DIGITS = W:(0-9){3}
THREE_LETTERS = W:(A-Za-z){3}

plaso.parsers.text_plugins.popcontest module

Text parser plugin for popularity contest log files.

Information updated 20 january 2014. From Debian Package Popularity Contest Avery Pennarun <apenwarr@debian.org>

From ‘https://www.unix.com/man-page/Linux/8/popularity-contest’:

The popularity-contest command gathers information about Debian pack- ages installed on the system, and prints the name of the most recently used executable program in that package as well as its last-accessed time (atime) and last-attribute-changed time (ctime) to stdout.

When aggregated with the output of popularity-contest from many other systems, this information is valuable because it can be used to deter- mine which Debian packages are commonly installed, used, or installed and never used. This helps Debian maintainers make decisions such as which packages should be installed by default on new systems.

The resulting statistic is available from the project home page https://popcon.debian.org

Normally, popularity-contest is run from a cron(8) job, /etc/cron.daily/popularity-contest, which automatically submits the results to Debian package maintainers (only once a week) according to the settings in /etc/popularity-contest.conf and /usr/share/popularity- contest/default.conf.

From ‘https://popcon.ubuntu.com/README’:

The popularity-contest output looks like this:

POPULARITY-CONTEST-0 TIME:914183330 ID:b92a5fc1809d8a95a12eb3a3c8445 914183333 909868335 grep /bin/fgrep 914183333 909868280 findutils /usr/bin/find 914183330 909885698 dpkg-awk /usr/bin/dpkg-awk 914183330 909868577 gawk /usr/bin/gawk […more lines…] END-POPULARITY-CONTEST-0 TIME:914183335

The first and last lines allow you to put more than one set of popularity-contest results into a single file and then split them up easily later.

The rest of the lines are package entries, one line for each package installed on your system. They have the format:

<atime> <ctime> <package-name> <mru-program> <tag>

<package-name> is the name of the Debian package that contains <mru-program>. <mru-program> is the most recently used program, static library, or header (.h) file in the package.

<atime> and <ctime> are the access time and creation time of the <mru-program> on your disk, respectively, represented as the number of seconds since midnight GMT on January 1, 1970 (i.e. in Unix time_t format). Linux updates <atime> whenever you open the file; <ctime> was set when you first installed the package.

<tag> is determined by popularity-contest depending on <atime>, <ctime>, and the current date. <tag> can be RECENT-CTIME, OLD, or NOFILES.

RECENT-CTIME means that atime is very close to ctime; it’s impossible to tell whether the package was used recently or not, since <atime> is also updated when <ctime> is set. Normally, this happens because you have recently upgraded the package to a new version, resetting the <ctime>.

OLD means that the <atime> is more than a month ago; you haven’t used the package for more than a month.

NOFILES means that no files in the package seemed to be programs, so <atime>, <ctime>, and <mru-program> are invalid.’

REMARKS. The parser will generate events solely based on the <atime> field and not using <ctime>, to reduce the generation of (possibly many) useless events all with the same <ctime>. Indeed, that <ctime> will be probably get from file system and/or package management logs. The <ctime> will be reported in the log line.

class plaso.parsers.text_plugins.popcontest.PopularityContestEventData[source]

Bases: EventData

Popularity Contest event data.

mru

recently used app/library from package.

Type

str

package

installed packaged name, which the mru belongs to.

Type

str

record_tag

popularity context tag.

Type

str

DATA_TYPE = 'popularity_contest:log:event'
class plaso.parsers.text_plugins.popcontest.PopularityContestSessionEventData[source]

Bases: EventData

Popularity Contest session event data.

details

version and host architecture.

Type

str

hostid

host uuid.

Type

str

session

session number.

Type

int

status

session status, either “start” or “end”.

Type

str

DATA_TYPE = 'popularity_contest:session:event'
class plaso.parsers.text_plugins.popcontest.PopularityContestTextPlugin[source]

Bases: TextPlugin

Text parser plugin for popularity contest log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Popularity Contest log file'
ENCODING = 'utf-8'
NAME = 'popularity_contest'

plaso.parsers.text_plugins.santa module

Text file parser plugin for Santa log files.

class plaso.parsers.text_plugins.santa.SantaExecutionEventData[source]

Bases: EventData

Santa execution event data.

action

action recorded by Santa.

Type

str

certificate_common_name

certificate common name.

Type

str

certificate_hash

SHA256 hash for the certificate associated with the executed process.

Type

str

decision

if the process was allowed or blocked.

Type

str

gid

group identifier associated with the executed process.

Type

str

group

group name associated with the executed process.

Type

str

long_reason

further explanation behind Santa decision to execute or block a process.

Type

str

mode

Santa execution mode, for example Monitor or Lockdown.

Type

str

pid

process identifier for the process.

Type

str

pid_version

the process identifier version extracted from the Mach audit token. The version can sed to identify process identifier rollovers.

Type

str

ppid

parent process identifier for the executed process.

Type

str

process_arguments

executed process with its arguments.

Type

str

process_hash

SHA256 hash for the executed process.

Type

str

process_path

process file path.

Type

str

reason

reason behind Santa decision to execute or block a process.

Type

str

uid

user identifier associated with the executed process.

Type

str

user

user name associated with the executed process.

Type

str

DATA_TYPE = 'santa:execution'
class plaso.parsers.text_plugins.santa.SantaFileSystemEventData[source]

Bases: EventData

Santa file system event data.

action

event type recorded by Santa.

Type

str

file_new_path

new file path and name for RENAME events.

Type

str

file_path

file path and name for WRITE/DELETE events.

Type

str

gid

group identifier associated with the executed process.

Type

str

group

group name associated with the executed process.

Type

str

pid

process identifier for the process.

Type

str

pid_version

the process identifier version extracted from the Mach audit token. The version can be used to identify process identifier rollovers.

Type

str

ppid

parent process identifier for the executed process.

Type

str

process_path

process file path.

Type

str

process

process name.

Type

str

uid

user identifier associated with the executed process.

Type

str

user

user name associated with the executed process.

Type

str

DATA_TYPE = 'santa:file_system_event'
class plaso.parsers.text_plugins.santa.SantaMountEventData[source]

Bases: EventData

Santa mount event data.

action

event type recorded by Santa.

Type

str

appearance

disk appearance date.

Type

str

bsd_name

disk BSD name.

Type

str

bus

device protocol.

Type

str

dmg_path

DMG file path.

Type

str

fs

disk volume kind.

Type

str

model

disk model.

Type

str

mount

disk mount point.

Type

str

serial

disk serial.

Type

str

volume

disk volume name.

Type

str

DATA_TYPE = 'santa:diskmount'
class plaso.parsers.text_plugins.santa.SantaProcessExitEventData[source]

Bases: EventData

Santa process exit event data.

action

action recorded by Santa.

Type

str

gid

group identifier associated with the executed process.

Type

str

pid

process identifier for the process.

Type

str

pid_version

the process identifier version extracted from the Mach audit token. The version can be used to identify process identifier rollovers.

Type

str

ppid

parent process identifier for the executed process.

Type

str

uid

user identifier associated with the executed process.

Type

str

DATA_TYPE = 'santa:process_exit'
class plaso.parsers.text_plugins.santa.SantaTextPlugin[source]

Bases: TextPlugin

Text file parser plugin for Santa log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Santa log (santa.log) file'
ENCODING = 'utf-8'
NAME = 'santa'

plaso.parsers.text_plugins.selinux module

Text parser plugin for SELinux audit log (audit.log) files.

audit.log log line example:

type=AVC msg=audit(1105758604.519:420): avc: denied { getattr } for pid=5962 comm=”httpd” path=”/home/auser/public_html” dev=sdb2 ino=921135

Where msg=audit(1105758604.519:420) contains the number of seconds since January 1, 1970 00:00:00 UTC and the number of milliseconds after the dot for example: “seconds: 1105758604, milliseconds: 519”.

The number after the timestamp (420 in the example) is a ‘serial number’ that can be used to correlate multiple logs generated from the same event.

class plaso.parsers.text_plugins.selinux.SELinuxLogEventData[source]

Bases: EventData

SELinux log event data.

audit_type

audit type.

Type

str

body

body of the log line.

Type

str

pid

process identifier (PID) that created the SELinux log line.

Type

int

DATA_TYPE = 'selinux:line'
class plaso.parsers.text_plugins.selinux.SELinuxTextPlugin[source]

Bases: TextPlugin

Text parser plugin for SELinux audit log (audit.log) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'SELinux audit log (audit.log) file'
NAME = 'selinux'

plaso.parsers.text_plugins.setupapi module

Text parser plugin for Windows SetupAPI log files.

The format is documented at: https://docs.microsoft.com/en-us/windows-hardware/drivers/install/setupapi-text-logs

class plaso.parsers.text_plugins.setupapi.SetupAPILogEventData[source]

Bases: EventData

SetupAPI log event data.

entry_type

log entry type, for examaple “Device Install - PCIVEN_104C&DEV_8019&SUBSYS_8010104C&REV_003&61aaa01&0&38” or “Sysprep Respecialize - {804b345a-ffd7-854c-a1b5-ca9598907846}”.

Type

str

exit_status

the exit status of the logged operation.

Type

str

DATA_TYPE = 'setupapi:log:line'
class plaso.parsers.text_plugins.setupapi.SetupAPILogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Windows SetupAPI log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Windows SetupAPI log file'
NAME = 'setupapi'

plaso.parsers.text_plugins.skydrivelog1 module

Text parser plugin for SkyDrive version 1 log files.

class plaso.parsers.text_plugins.skydrivelog1.SkyDriveLog1TextPlugin[source]

Bases: TextPlugin

Text parser plugin for SkyDrive version 1 log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'OneDrive (or SkyDrive) version 1 log file'
ENCODING = 'utf-8'
NAME = 'skydrive_log_v1'
class plaso.parsers.text_plugins.skydrivelog1.SkyDriveOldLogEventData[source]

Bases: EventData

SkyDrive version 1 log event data.

log_level

log level.

Type

str

source_code

source file and line number that generated the log message.

Type

str

text

log message.

Type

str

DATA_TYPE = 'skydrive:log:old:line'

plaso.parsers.text_plugins.snort_fastlog module

Text parser plugin for Snort3/Suricata fast-log alert log files.

Snort3/Suricata fast.log format is a single line log format as shown below. The following variants are known:

Snort3:

Month/Day-Hour:Minute:Second:FractionOfSeconds [**] [sid] “string” [**] [Classification: string] [Priority : int] {protocol} SOURCE_IP:SOURCE_PORT -> DESTINATION_IP:DESTINATION_PORT

Suricata:

Year/Month/Day-Hour:Minute:Second:FractionOfSeconds [**] [sid] string [**] [Classification: string] [Priority : int] {protocol} SOURCE_IP:SOURCE_PORT -> DESTINATION_IP:DESTINATION_PORT

Also see:

https://suricata.readthedocs.io/en/suricata-6.0.0/configuration/suricata-yaml.html#line-based-alerts-log-fast-log

class plaso.parsers.text_plugins.snort_fastlog.SnortFastAlertEventData[source]

Bases: EventData

Snort3/Suricata fast-log alert event data.

classification

classification of the alert.

Type

str

destination_ip

destination IP-address.

Type

str

destination_port

destination TCP/UDP port number.

Type

int

message

message associated with the alert.

Type

str

priority

priorty, ranging from 1 (high) to 4 (very low).

Type

int

rule_identifier

identifier of the Snort3/Suricata rule that generated the alert.

Type

str

source_ip

source IP-address.

Type

str

source_port

source TCP/UDP port number.

Type

int

DATA_TYPE = 'snort:fastlog:alert'
class plaso.parsers.text_plugins.snort_fastlog.SnortFastLogTextPlugin[source]

Bases: TextPlugin, YearLessLogFormatHelper

Text parser plugin for Snort3/Suricata fast-log alert log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Snort3/Suricata fast-log alert log (fast.log) file'
NAME = 'snort:fastlog:alert'

plaso.parsers.text_plugins.sophos_av module

Text parser plugin for Sophos anti-virus logs (SAV.txt) files.

Also see:

https://support.sophos.com/support/s/article/KB-000033745?language=en_US

class plaso.parsers.text_plugins.sophos_av.SophosAVLogEventData[source]

Bases: EventData

Sophos anti-virus log event data.

text

Sophos anti-virus log message.

Type

str

DATA_TYPE = 'sophos:av:log'
class plaso.parsers.text_plugins.sophos_av.SophosAVLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Sophos anti-virus logs (SAV.txt) files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Sophos anti-virus log file (SAV.txt) file'
ENCODING = 'utf-16-le'
NAME = 'sophos_av'

plaso.parsers.text_plugins.vsftpd module

Text parser plugin for vsftpd log files.

class plaso.parsers.text_plugins.vsftpd.VsftpdEventData[source]

Bases: EventData

vsftpd Log event data.

text

vsftpd log message.

Type

str

DATA_TYPE = 'vsftpd:log'
class plaso.parsers.text_plugins.vsftpd.VsftpdLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for vsftpd log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'vsftpd log file'
NAME = 'vsftpd'

plaso.parsers.text_plugins.winfirewall module

Text parser plugin for Windows Firewall Log files.

class plaso.parsers.text_plugins.winfirewall.WinFirewallEventData[source]

Bases: EventData

Windows Firewall event data.

action

action taken.

Type

str

protocol

IP protocol.

Type

str

source_ip

source IP address.

Type

str

dest_ip

destination IP address.

Type

str

source_port

TCP or UDP source port.

Type

int

dest_port

TCP or UDP destination port.

Type

int

size

size of ???

Type

int

flags

TCP flags.

Type

str

tcp_seq

TCP sequence number.

Type

int

tcp_ack

TCP ACK ???

Type

int

tcp_win

TCP window size ???

Type

int

icmp_type

ICMP type.

Type

int

icmp_code

ICMP code.

Type

int

info

???

Type

str

path

???

Type

str

DATA_TYPE = 'windows:firewall:log_entry'
class plaso.parsers.text_plugins.winfirewall.WinFirewallLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for Windows Firewall Log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'Windows Firewall log file'
ENCODING = 'ascii'
NAME = 'winfirewall'

plaso.parsers.text_plugins.xchatlog module

Text parser plugin for XChat log files.

Information updated 24 July 2013.

The parser applies to XChat log files. Despite their apparent simplicity it’s not straightforward to manage every possible case. XChat tool allows users to specify how timestamp will be encoded (using the strftime function), by letting them specify additional separators. This parser will accept only the simplest default English form of an XChat log file, as the following:

**** BEGIN LOGGING AT Mon Dec 31 21:11:55 2001
dec 31 21:11:55 --> You are now talking on #gugle
dec 31 21:11:55 --- Topic for #gugle is plaso, nobody knows what it means
dec 31 21:11:55 Topic for #gugle set by Kristinn
dec 31 21:11:55 --- Joachim gives voice to fpi
dec 31 21:11:55 *   XChat here
dec 31 21:11:58 <fpi> ola plas-ing guys!
dec 31 21:12:00 <Kristinn> ftw!

It could be managed the missing month/day case too, by extracting the month/day information from the header. But the parser logic would become intricate, since it would need to manage day transition, chat lines crossing the midnight. From there derives the last day of the year bug, since the parser will not manage that transition.

Moreover the strftime is locale-dependent, so month names, footer and headers can change, even inside the same log file. Being said that, the following will be the main logic used to parse the log files (note that the first header must be ‘**** BEGIN …’ otherwise file will be skipped).

1) Check for ‘****’ 1.1) If ‘BEGIN LOGGING AT’ (English) 1.1.1) Extract the YEAR 1.1.2) Generate new event start logging 1.1.3) set parsing = True 1.2) If ‘END LOGGING’ 1.2.1) If parsing, set parsing=False 1.2.2) If not parsing, log debug 1.2.3) Generate new event end logging 1.3) If not BEGIN|END we are facing a different language and we don’t now which language! If parsing is True, set parsing=False and log debug 2) Not ‘****’ so we are parsing a line 2.1) If parsing = True, try to parse line and generate event 2.2) If parsing = False, skip until next good header is found

References http://xchat.org

class plaso.parsers.text_plugins.xchatlog.XChatLogEventData[source]

Bases: EventData

XChat Log event data.

nickname

nickname.

Type

str

text

text sent by nickname or other text (server, messages, etc.).

Type

str

DATA_TYPE = 'xchat:log:line'
class plaso.parsers.text_plugins.xchatlog.XChatLogTextPlugin[source]

Bases: TextPlugin, YearLessLogFormatHelper

Text parser plugin for XChat log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'XChat log file'
ENCODING = 'utf-8'
NAME = 'xchatlog'

plaso.parsers.text_plugins.xchatscrollback module

Text parser plugin for XChat scrollback log files.

Information updated 06 September 2013.

Besides the logging capability, the XChat IRC client has the option to record the text for opened tabs. So, when rejoining a particular channel and/or a particular conversation, XChat will display the last messages exchanged. This artifact could be present, if not disabled, even if normal logging is disabled.

From the XChat FAQ (http://xchat.org/faq):

Q: ‘How do I keep text from previous sessions from being displayed when I join a channel?’ R: ‘Starting in XChat 2.8.4, XChat implemented the Scrollback feature which displays text from the last time you had a particular tab open. To disable this setting for all channels, Go to Settings -> Preferences -> Logging and uncheck Display scrollback from previous session. In XChat 2.8.6, XChat implemented both Per Channel Logging, and Per Channel Scrollbacks. If you are on 2.8.6 or newer, you can disable loading scrollback for just one particular tab name by right clicking on the tab name, selecting Settings, and then unchecking Reload scrollback’

The log file format differs from logging format, but it’s quite simple ‘T 1232315916 Python interface unloaded’ <T><space><decimal timestamp><space><text>< >

The time reported in the log is the number of seconds since January 1, 1970 00:00:00 UTC (from source code, time(0)). The <text> part could contain some ‘decorators’ (bold, underline, colors indication, etc.), so the parser should strip those control fields.

References http://xchat.org

class plaso.parsers.text_plugins.xchatscrollback.XChatScrollbackEventData[source]

Bases: EventData

XChat Scrollback line event data.

nickname

nickname.

Type

str

text

text sent by nickname service messages.

Type

str

DATA_TYPE = 'xchat:scrollback:line'
class plaso.parsers.text_plugins.xchatscrollback.XChatScrollbackLogTextPlugin[source]

Bases: TextPlugin

Text parser plugin for XChat scrollback log files.

CheckRequiredFormat(parser_mediator, text_file_object)[source]

Check if the log record has the minimal structure required by the plugin.

Parameters
  • parser_mediator (ParserMediator) – mediates interactions between parsers and other components, such as storage and dfVFS.

  • text_file_object (dfvfs.TextFile) – text file.

Returns

True if this is the correct parser, False otherwise.

Return type

bool

DATA_FORMAT = 'XChat scrollback log file'
ENCODING = 'utf-8'
LOG_LINE = {{Suppress:('T') W:(0-9)} SkipTo:(LineEnd)}
MSG_ENTRY = {[{'<' SkipTo:('>')} '>'] SkipTo:(LineEnd)}
MSG_ENTRY_NICK = [{'<' SkipTo:('>')} '>']
MSG_ENTRY_TEXT = SkipTo:(LineEnd)
MSG_NICK = SkipTo:('>')
MSG_NICK_END = '>'
MSG_NICK_START = '<'
NAME = 'xchatscrollback'
STRIPPER = {Suppress:(W:(, 0-9){1,3}) | Suppress:(())}

Module contents

Imports for the text parser.