Output and formatting

The plaso tools psort.py and psteal.py can output events in multiple formats using several output modules.

Output modules

Plaso supports several output formats:

Name Description
dynamic Output events to a delimiter (comma by default) separated value output format, that supports a dynamic selection of fields.
elastic Output events to an ElasticSearch database. Requires elasticsearch-py.
json Output events to JSON format.
json_line Output events to JSON line format.
l2tcsv Output events to log2timeline.pl legacy CSV format, with 17 fixed fields. Also see: l2tcsv output format
l2ttln Output events to log2timeline.pl extended TLN format, with 7 fixed field.
null Do not output events.
rawpy Output events in "raw" (or native) Python format.
timesketch Output events to a Timesketch ElasticSearch database. Requires elasticsearch-py.
tln Output events to TLN format, with 5 fixed fields. Also see: TLN.

Dynamic output module fields

The dynamic output module defines the following command line options to specify which fields should be represented in the output, namely --fields and --additional_fields. The name of the fields typically map 1-to-1 to the names of attributes of the event data. However there are “special” fields that are composed at runtime.

Name Description
date The date of the event
datetime The date and time of the event in ISO 8601 format
description The event message string as defined by the message formatter
description_short The short event message string as defined by the message formatter
display_name Human readable representation of the path specification
filename The "filename" attribute if present in the event data, otherwise derived from the path specification
host The hostname derived by pre-processing
hostname The hostname derived by pre-processing
inode The "inode" attribute if present in the event data, otherwise derived from the file system identifer (such as inode, mft entry) in the path specification
macb MACB (Modification, Access, Change, Birth) group representation
message The event message string as defined by the message formatter
message_short The short event message string as defined by the message formatter
source The short event source as defined by the message formatter
sourcetype The event source as defined by the message formatter
source_long The event source as defined by the message formatter
tag The labels defined by event tags
time The time of the event
timestamp_desc Indication of what the event time represents such as Creation Time or Program Execution Duration
timezone Time zone indicator
type Indication of what the event time represents such as Creation Time or Program Execution Duration
user The username derived by pre-processing
username The username derived by pre-processing
zone Time zone indicator

Output fields that are not part of the event data but of the data stream the event data originates from.

Name Description
file_entropy Byte entropy of the data stream content. This is a value ranging from 0.0 to 8.0, where 8.0 indicates the distribution of byte values is highly random.
md5_hash MD5 hash of the data stream content.
sha1_hash SHA-1 hash of the data stream content.
sha256_hash SHA-256 hash of the data stream content.
yara_match Names of the Yara rules that matched the data stream content.

Output field formatting

Source fields

As of Plaso 20200916 the value of the long and short source fields are defined in data/sources.config. This file contains 3 tab separated values:

  • data_type; event data type.

  • short_source; short source identifier that corresponds with the l2tcsv and tln source field.

  • source; source identifier that corresponds with the l2tcsv sourcetype field.

Message formatting

In log2timeline.pl the l2tcsv format introduced the desc and short fields that provide a description of the field, the interpreted results or the content of the corresponding log line.

In Plaso the dynamic format extended the idea of the desc field, to provide a formatted message field. That allow to provide more extensive formatting such as supporting Windows Event Log message strings.

Formatter configuration file format

As of version 20200227 Plaso supports formatter configuration files.

Note that the format of these configuration files is subject to change.

An event formatter is defined as a set of attributes:

  • “data_type”; required event data type.

  • “enumeration_helpers”; optional enumeration helpers.

  • “message”; required formatter message string, for a basic type, or list of messages string pieces, for a conditional type.

  • “separator”; optional conditional message string piece separator, the default is a single space.

  • “short_message”; required formatter short message string, for a basic type, or list of short messages string pieces, for a conditional type.

  • “type”; required event formatter type either “basic” or “conditional”.

For example:

---
type: 'basic'
data_type: 'bash:history:command'
message: 'Command executed: {command}'
short_message: '{command}'
---
type: 'conditional'
data_type: 'syslog:cron:task_run'
message:
- 'Cron ran: {command}'
- 'for user: {username}'
- 'pid: {pid}'
separator: ', '
short_message:
- '{body}'

Enumeration helpers

Enumeration helpers can be defined to map a value of an event attribute to a more descriptive value, for example mapping 100 to BEGIN_SYSTEM_CHANGE in the example below.

type: 'conditional'
data_type: 'windows:restore_point:info'
enumeration_helpers:
- input_attribute: 'restore_point_event_type'
  output_attribute: 'restore_point_event_type'
  default_value: 'UNKNOWN'
  values:
    100: 'BEGIN_SYSTEM_CHANGE'
    101: 'END_SYSTEM_CHANGE'
    102: 'BEGIN_NESTED_SYSTEM_CHANGE'
    103: 'END_NESTED_SYSTEM_CHANGE'
- input_attribute: 'restore_point_type'
  output_attribute: 'restore_point_type'
  default_value: 'UNKNOWN'
  values:
    0: 'APPLICATION_INSTALL'
    1: 'APPLICATION_UNINSTALL'
    10: 'DEVICE_DRIVER_INSTALL'
    12: 'MODIFY_SETTINGS'
    13: 'CANCELLED_OPERATION'
message:
- '{description}'
- 'Event type: {restore_point_event_type}'
- 'Restore point type: {restore_point_type}'
short_message:
- '{description}'

enumeration helpers are defined as a set of attributes:

  • “input_attribute”; required name of the attribute which the value that needs to be mapped is read from.

  • “output_attribute”; required name of the attribute which the mapped value is written to.

  • “default_value”; optional default value if there is no corresponding mapping in “values”.

  • “values”; required value mappings, contains key value pairs.

Flags helpers

Flags helpers can be defined to map a value of an event attribute to a more descriptive value, for example mapping 0x00000040 to FinderInfoModified in the example below.

type: 'conditional'
data_type: 'macos:fseventsd:record'
flags_helpers:
- input_attribute: 'flags'
  output_attribute: 'flag_values'
  # The include header sys/fsevents.h defines various FSE constants, e.g.
  # #define FSE_CREATE_FILE          0
  # The flag values correspond to: FLAG = 1 << CONSTANT
  values:
    0x00000000: 'None'
    0x00000001: 'Created'
    0x00000002: 'Removed'
    0x00000004: 'InodeMetadataModified'
    0x00000008: 'Renamed'
    0x00000010: 'Modified'
    0x00000020: 'Exchange'
    0x00000040: 'FinderInfoModified'
    0x00000080: 'DirectoryCreated'
    0x00000100: 'PermissionChanged'
    0x00000200: 'ExtendedAttributeModified'
    0x00000400: 'ExtendedAttributeRemoved'
    0x00001000: 'DocumentRevision'
    0x00004000: 'ItemCloned'
    0x00080000: 'LastHardLinkRemoved'
    0x00100000: 'IsHardLink'
    0x00400000: 'IsSymbolicLink'
    0x00800000: 'IsFile'
    0x01000000: 'IsDirectory'
    0x02000000: 'Mount'
    0x04000000: 'Unmount'
    0x20000000: 'EndOfTransaction'
message:
- '{path}'
- 'Flag Values: {flag_values}'
- 'Flags: 0x{flags:08x}'
- 'Event Identifier: {event_identifier}'
short_message:
- '{path}'
- '{flag_values}'

Change log

  • 20200227 Added support for formatter configuration files.

  • 20200822 Added support for enumeration helpers.

  • 20200904 Added support for flags helpers.

  • 20200916 Removed source types from formatters.