Anuket Project

PCIe RAS Executed Tests

Owned by Former user (Deleted)
Feb 01, 2018

Test Environment details:

  • Bare Metal,  Ubuntu 16.04.2 LTS

Repo/branch used:

Tests precondition:

  • AER injection tool installed.
  • Collectd installed.
  • Exec/python collectd plugin configured.

PCIe non native OS error injection

Collectd configuration:

<Plugin pcie_errors>
  Source "sysfs"
  AccessDir "/sys/bus/pci"
  ReportMasked false
  PersistentNotifications false
  ReadLog false
  LogFile "/var/log/syslog"
</Plugin>

 

Table#1: PCIe non native OS error injection test cases

#
Test Summary
Steps
Expected
Status
Comments
1PCIe plugin notifications upon collectd start
  1. Collected initial configuration. Start collectd.
  2. Verify notifications dispatched by PCIe plugin.
  1. Collectd started.
  2. Basic/AER PCIe errors dispatched after collectd start for all related (network NIC's) PCIe devices.

Pass

SA: No clear limits between messages in collectd/exec plugin notifications for exec plugin.
2PCIe plugin dispatches notifications after every collectd restart
  1. Collectd initial configuration. Start collectd.
  2. Inject PCIe error.
  3. Clear PCIe error.
  4. Restart collectd.
  5. Repeat steps #2-#4 multiple times.

2. Notification about PCIe error is dispatched as set.

3. Notification about PCIe error is dispatched as cleared.

4. Collectd is started, no warnings/errors in syslog.

PCIe errors dispatched after collectd start for all related (network NIC's) PCIe devices.

Pass  
3PCIe plugin upon raised notification and collectd restart
  1. Collectd initial configuration. Start collectd.
  2. Inject PCIe error.
  3. Restart collectd.
  4. Clear PCIe error.
  5. Repeat steps #2-#4 multiple times.
2. Notification about PCIe error is dispatched as set.

3. Collectd is started, no warnings/errors in syslog. Notification about PCIe error is dispatched as set.

4. Notification about PCIe error is dispatched as cleared.

Pass 
4

PCIe plugin results upon collectd recovery (kill collectd process)

  1. Collectd initial configation. Start collectd.
  2. Inject PCIe error.
  3. Clear PCIe error.
  4. Kill collectd.
  5. Start collectd if collectd not autostarted.
  6. Repeat steps#2-#4 at least one more time.

 

2. Notification about basic/AER PCIe error is dispatched as set.

3. Notification about basic/AER PCIe error is dispatched as cleared.

4. / 5. Collectd is killed and started.

Note: if collectd is configured as service it may be auto started.

Pass 
5PCIe plugin upon commented whole "Plugin pcie_errors" section
  1. Comment whole pcie_errors section including <Plugin pcie_errors>, </Plugin> lines. Start collectd.
  2. Inject basic PCIe error.
  1. Collectd started.
    PCIE plugin loaded with default parameters.
  2. Notification about basic/AER PCIe error is dispatched.

Fail

Internal JIRA filed

SA: Readout's summary - default non native OS support. So, default configuration should be available (Ubuntu is target OS).
6PCIe plugin upon commented content of "Plugin pcie_errors" (<Plugin pcie_errors>...</Plugin>)
  1. Comment content of "Plugin pcie_errors", like:
    <Plugin pcie_errors>
      # ...
    </Plugin>
  2. Inject basic PCIe error.
  1. Collectd started. PCIE plugin loaded with default parameters.
  2. Notification about basic/AER PCIe error is dispatched.

Fail

Internal JIRA filed

SA: Readout's summary - default non native OS support. So, default configuration should be available (Ubuntu is target OS).
7PCIe plugin upon 'Source' parameter changed
  1. Configure collectd:
    <Plugin pcie_errors>
      Source "sysfs"
      AccessDir "/sys/bus/pci"
    </Plugin>
  2. Inject basic/AER PCIe error.
  3. Clear injected PCIe error.
  4. Configure collectd, restart collectd:
    <Plugin pcie_errors>
       Source "proc"
       AccessDir "/proc/bus/pci"
    </Plugin>
  5. Inject basic/AER PCIe error.
  6. Clear injected PCIe error.
  7. Configure collectd, restart collectd:
    <Plugin pcie_errors>
        Source "off"
    </Plugin>
  8. Inject basic/AER PCIe error.
  9. Clear injected PCIe error.
  1. Collectd started.
  2. Notification about basic/AER PCIe error is dispatched as set.
  3. Notification about basic/AER PCIe error is dispatched as cleared.
  4. Collectd started.
  5. Notification about basic/AER PCIe error is dispatched as set.
  6. Notification about basic/AER PCIe error is dispatched as cleared.
  7. Collectd started, plugin not loaded.
  8. No notification about basic/AER PCIe error is dispatched.
Pass 
8PCIe plugin upon incorrect device location given ('AccessDir')
  1. Collectd initial configuration. Start collectd.
  2. Inject basic/AER PCIe error.
  3. Change location to existing path to invalid ("/sys/bus", "/proc/cpuinfo"), restart collectd.
  4. Inject basic/AER PCIe error.
  5. Clear injected PCIe error.
  6. Change location to non existing path, restart collectd.
  7. Inject basic/AER PCIe error.
  8. Clear injected PCIe error.
  1. Collectd started.
  2. Notification about basic/AER PCIe error is dispatched.
  3. Collectd started, pcie_errors plugin loaded. Warning about PCIe devices read failure logged.
  4. No notification about basic/AER PCIe error is dispatched.
  5. PCIe error is cleared (lspci -s 05:00.0 -vv | grep -e "DevSta:" -A5 -e "Capabilities: \[100").
  6. Collectd started, plugin not loaded.
  7. No notification about basic/AER PCIe error is dispatched.

Fail

Internal JIRA filed

 
9PCIe plugin errors upon 'ReportMasked' parameter changed
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe error:
    # cat ue_unsupreq
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x00100000
    # ./aer-inject ue_unsupreq
  3. Clear PCIe error (setpci -s 05:00.0 0x104.l=00100000).
  4. Set 'ReportMasked true' in collectd.conf. Restart collectd.
  5. Inject ue_unsupreq AER PCIe error.
  6. Clear PCIe error.
  1. Collectd started.
  2. Notification about AER PCIe error is not dispatched. Status is changed:
    # setpci -s 05:00.0 0x104.l
    00100000
    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
  3. Notification about AER PCIe error is dispatched as cleared (00000000, UnsupReq-).
  4. Collectd started.
  5. Notification about AER PCIe error is dispatched as set.
  6. Notification about AER PCIe error is dispatched as cleared. 
Pass 
10PCIe plugin dispatches events according to 'PersistentNotifications' parameter
  1. Collectd initial configuration. Start collectd.
  2. Inject basic/AER PCIe error.
  3. Clear injected errors.
  4. Set 'PersistentNotifications true' in collectd.conf. Restart collectd.
  5. Inject basic/AER PCIe error.
  6. Clear injected errors.
  1. Collectd started.
  2. Notification about PCIe error 'set'  dispatched only once when error injected.
  3. Notification about PCIe error 'cleared' is dispatched only once when error cleared.
  4. Collectd started.
  5. Notification about PCIe error 'set' is dispatched every time interval after error was injected.
  6. Notification about PCIe error 'cleared' is dispatched every time interval after error was injected.
Pass 
11PCIe plugin dispatches events every interval when 'PersistentNotifications true'
  1. Set 'PersistentNotifications true' in collectd.conf. Change time interval to 1 second. Restart collectd.
  2. Inject basic/AER PCIe error.
  3. Clear injected errors.
  4. Repeat test with time interval in range 2-60 seconds.
  1. Collectd started.
  2. Notification about PCIe error 'set' is dispatched every time interval after error was injected.
  3. Notification about PCIe error 'cleared' is dispatched every time interval after error was injected.
Pass 
12

PCIE plugin upon 'ReadLog' parameter changed

  1. Set 'ReadLog true'. Restart collectd.
  2. Inject basic/AER PCIe error.
  3. Set 'ReadLog false'. Restart collectd.
  4. Clear/inject PCIe errors.
  1. Collectd started.
  2. No notification about PCIe error dispatched as syslog is parsed.
  3. After collectd restart notifications appeared about PCIe errors injected.
  4. Notification about PCIe error dispatched once injected/cleared.
Pass 
13

PCIE plugin upon 'ReadLog true' parameter upon collectd start and error injected

  1. Set 'ReadLog true' in collectd.conf. Start collectd.
  2. Inject a PCIe error.
  3. Restart colectd.
  4. Clear injected error.
  1. Collectd started.
  2. PCIe error is raised (lspci/setpci) but not dispatched as syslog is parsed.
  3. No notifications after collectd started as non native OS support configured with syslog read option.
    Observed: Upon start up all errors are read from default configuration as well as injected prior PCIe error, but syslog doesn't contain any PCIe error.
  4. Error is cleared, notification is not dispatched.

Fail

Internal JIRA filed

 
14PCIe plugin AER Corrected errors notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe error:
    # cat ce_badtlp
    AER PCI_ID 05:00.0 COR_STATUS 0x00000040
    # ./aer-inject ce_badtlp
  3. Clear injected errors:
    setpci -s 05:00.0 0x110.l=0x40
  4. Repeat test with other PCIe correctable errors: Receiver Error Status (0x00000001), Bad DLLP Status (0x00000080), REPLAY_NUM Rollover (0x00000100), etc.
  1. Collectd started.
  2. Verify register status by setpci -s 05:00.0 0x110.l:
    00000040
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10": CESta:  RxErr- BadTLP+ BadDLLP- Rollover- Timeout- NonFatalErr-
    Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:WARNING
    Time:1491226583.117
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:correctable
    CorrectableError set: Bad TLP Status
  3. Verify register status by setpci -s 05:00.0 0x110.l:
    00000000
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10": CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
    Notification with correct timestamp, title, severity, address is dispatched as cleared.
Pass 
15PCIe plugin AER Uncorrected non fatal errors notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe uncorrectable fatal error:
    # cat ue_acsviol
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x00200000
    # ./aer-inject ue_acsviol
  3. Clear injected errors: setpci -s 05:00.0 0x104.l=0x00200000
  1. Collectd started.
  2. Verify register status by setpci -s 05:00.0 0x104.l:
    00200000
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10":
    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol+
    Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:WARNING
    Time:1491229811.884
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:non_fatal
    Uncorrectable(non_fatal)Error set: ACS Violation
  3. Verify register status by setpci -s 05:00.0 0x104.l: 00000000
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10": UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
    Notification with correct timestamp, title, severity, address is dispatched as cleared.
Pass 
16PCIe plugin AER Uncorrected fatal error notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe uncorrectable fatal error:
    # cat ue_dlp
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x10
    # ./aer-inject ue_dlp
  3. Clear injected errors:
    setpci -s 05:00.0 0x104.l=0x10
  4. Repeat test with other PCIe Uncorrectable non fatal errors: Surprise Down (0x00000020), Poisoned TLP (0x00001000), Flow Control Protocol (0x00002000), etc.
  1. Collectd started.
  2. Verify register status by setpci -s 05:00.0 0x104.l:
    00000010
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10":
    UESta:  DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
    Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:FAILURE
    Time:1491227475.743
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:fatal
    Uncorrectable(fatal)Error set: Data Link Protocol
  3. Verify register status by setpci -s 05:00.0 0x104.l:
    00000000
    Verify status by lspci -s 05:00.0 -vv | grep -A9 "Capabilities: \[10": UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
    Notification with correct timestamp, title, severity, address is dispatched as cleared.
Pass 
17PCIe plugin basic errors notification 
  1. Collectd initial configuration. Start collectd.
  2. Check PCIe devices for active errors (lspci -s 81:00.0 -vv | grep "DevSta").
  3. Get errors and try to clear, e.g.:
    Get: setpci -s 81:00.0 0xAA.w
    Clear: setpci -s 81:00.0 0xAA.w=<read_value>.
  4. Restart collectd.
  1. Collectd started.
  2. Found similar to:
    DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
  3. Notification with correct timestamp, title, severity, address is dispatched as cleared.
  4. PCIe error cleared in previous step is not notified after collectd restart.
Pass 

 


PCIe native OS error injection

System configuration:

  • Turn on native OS driver support

Collectd configuration:

<Plugin pcie_errors>
  Source "off"
  AccessDir "/sys/bus/pci"
  ReportMasked false
  PersistentNotifications false
  ReadLog true
  FirstFullRead false
  LogFile "/var/log/syslog"
  <MsgPattern "aer syslog">
    <Match>
      Name "root port"
      Regex "pcieport (.*): AER:"
      IsMandatory true
    </Match>
  </MsgPattern>
</Plugin>


Table#2: PCIe native OS error injection test cases

#
Test Summary
Steps
Expected
Status
Comments
1PCIe plugin notifications upon collectd start
  1. Collected initial configuration. Start collectd.
  2. Verify notifications dispatched by PCIe plugin.
  1. Collectd started.
  2. PCIe errors dispatched after collectd started for all related (network NIC's) PCIe devices.

Pass

 
2PCIe plugin dispatched notification after every collectd restart
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe error.
  3. Restart collectd.
  4. Repeat steps #2-#3 multiple times.

2. Notification about PCIe error is dispatched. Injected AER PCIe error is cleared

3. Collectd is started, no warnings/errors in syslog.

4. PCIe errors dispatched after collectd start for all related (network NIC's) PCIe devices.

Pass  
3

PCIE plugin upon 'ReadLog' parameter changed

  1. Change initial collectd.conf by changing 'ReadLog true'. Start collectd.
  2. Inject basic/AER PCIe error.
  3. Set 'ReadLog false'. Restart collectd.
  4. Inject basic/AER PCIe error.
  1. Collectd started.
  2. Notification about PCIe error dispatched from syslog. PCIe error is cleared.
  3. Collectd started, PCIe plugin unloaded.
  4. Notification about PCIe error dispatched from syslog. PCIe error is cleared.
Pass 
4

PCIE plugin notification fields when 'FirstFullRead true'

  1. Change initial collectd.conf by changing 'FirstFullRead false'.
  2. Start collectd.

2. Collectd started with PCIe error plugin.

Notification fields are same as fields read from syslog.

Fail

Internal JIRA filed

 

5

PCIE plugin upon 'FirstFullRead' parameter changed

  1. Collectd initial configuration. Start collectd
  2. Inject basic/AER PCIe error.
  3. Change initial collectd.conf by changing 'FirstFullRead false'. Start collectd.
  4. Inject basic/AER PCIe error.
  1. Collectd started. All notifications are read from syslog.
  2. Notification about PCIe error dispatched from syslog. PCIe error is cleared.
  3. Collectd started. No notifications are read from collectd.
  4. Notification about PCIe error dispatched from syslog. PCIe error is cleared.
Pass 
6PCIe plugin errors upon 'ReportMasked' parameter changed
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe error:
    # cat ue_unsupreq
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x00100000
    # ./aer-inject ue_unsupreq
  3. Set 'ReportMasked true' in collectd.conf. Restart collectd.
  4. Inject ue_unsupreq AER PCIe error.
  1. Collectd started.
  2. Notification about AER PCIe error is not dispatched.
    Message recorded in syslog: "kernel: The uncorrectable error(s) is masked by device"
  3. Collectd started.
  4. Notification about AER PCIe error is not dispatched.
    Message recorded in syslog: "kernel: The uncorrectable error(s) is masked by device"

Fail

Internal JIRA filed

SA: Update the documentation accordingly.

Step#4 - PCIe error is not dispatched as masked by HW.

Syslog is updated by AER injection tool: "kernel: The uncorrectable error(s) is masked by device"

7PCIe plugin dispatches events according to 'PersistentNotifications' parameter
  1. Collectd initial configuration. Start collectd.
  2. Inject basic/AER PCIe error.
  3. Set 'PersistentNotifications true' in collectd.conf. Restart collectd.
  4. Inject basic/AER PCIe error.
  1. Collectd started.
  2. Notification about PCIe error dispatched only once when error injected.
  3. Collectd started.
  4. Notification about PCIe error 'set' is dispatched every time interval after error was injected.

Fail

Internal JIRA filed

SA: Update the documentation accordingly

'PersistentNotifications' and syslog case is not applicable.

8PCIe plugin dispatches events every interval when 'PersistentNotifications true'
  1. Set 'PersistentNotifications true' in collectd.conf. Change time interval to 1 second. Restart collectd.
  2. Inject basic/AER PCIe error.
  3. Clear injected errors.
  4. Repeat test with time interval in range 2-60 seconds.
  1. Collectd started.
  2. Notification about PCIe error 'set' is dispatched every time interval after error was injected.
  3. Notification about PCIe error 'cleared' is dispatched every time interval after error was injected.

Blocked

Internal JIRA filed

 
9

PCIE plugin upon 'ReadLog' parameter changed

  1. Set 'ReadLog true'. Restart collectd.
  2. Inject PCIe error.
  3. Set 'ReadLog false'. Restart collectd.
  4. Inject PCIe error.
  1. Collectd started.
  2. Notification about PCIe error dispatched from syslog.
  3. Collectd started. PCIe plugin unloaded as no source given.
  4. No notification about PCIe error dispatched as not logged to syslog.
Pass 
10PCIe plugin AER Corrected errors notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe error:
    # cat ce_badtlp
    AER PCI_ID 05:00.0 COR_STATUS 0x00000040
    # ./aer-inject ce_badtlp
  1. Collectd started.
  2. Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:WARNING
    Time:1491414816.213
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:correctable
    rootport: 0000:00:03.0
    errortype: Data Link Layer
    id:0500(Receiver ID)
    AERcorrectable error reported in log
Pass

 

11PCIe plugin AER Uncorrected non fatal errors notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe uncorrectable fatal error:
    # cat ue_acsviol
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x00200000
    # ./aer-inject ue_acsviol
  1. Collectd started.
  2. Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:WARNING
    Time:1491415276.673
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:non_fatal
    rootport: 0000:00:03.0
    errortype: Transaction Layer
    id:0500(Receiver ID)
    AERnon_fatal error reported in log
Pass 
12PCIe plugin AER Uncorrected fatal error notifications
  1. Collectd initial configuration. Start collectd.
  2. Inject AER PCIe uncorrectable fatal error:
    # cat ue_dlp
    AER PCI_ID 05:00.0 UNCOR_STATUS 0x10
    # ./aer-inject ue_dlp
  1. Collectd started.
  2. Notification with correct timestamp, title, severity, address is dispatched as set:
    Severity:FAILURE
    Time:1491415541.938
    Host:silpixa00398942
    Plugin:pcie_errors
    PluginInstance:0000:05:00.0
    Type:pcie_error
    TypeInstance:fatal
    rootport: 0000:00:03.0
    errortype: Unaccessible
    id:0500(Unregistered Agent ID)
    AERfatal error reported in log

Pass 
13PCIe plugin notification read from non existent syslog file location
  1. Change syslog location to non-existent. Start collectd.
  2. Change syslog location to valid. Restart collectd.
  3. Inject AER PCIe error.
  1. Collectd started. Error message recorded to syslog:
    "utils_tail: stat (/var/llog/syslog) failed: No such file or directory"
    Notification "Failedto read from log file" is dispatched.
  2. Collectd started.
  3. Notification about AER PCIe error is dispatched from syslog.
Pass 
14PCIe plugin notifications of basic PCIe errors caused by AER UE fatal error injection
  1.  Change collectd.conf file, restart collectd:
    <LoadPlugin pcie_errors>
      Interval 0.005
    </LoadPlugin>
    <Plugin pcie_errors>
      Source "sysfs"
      AccessDir "/sys/bus/pci"
      ReportMasked false
      PersistentNotifications false
      ReadLog true
      FirstFullRead false
      LogFile "/var/log/syslog"
    </Plugin>
  2. Inject AER Uncorrected fatal error (like, ue_dlp).
  3. Change interval to 1 second.
  4. Inject AER Uncorrected fatal error (like, ue_dlp).
  1. Collectd started.
  2. Notifications about PCIe AER Uncorrected fatal error is dispatched.
  3. Notifications about basic PCIe errors are dispatched.
Pass 
15PCIe plugin dispatches notification after single negative match

  1. Add following to initial collectd.conf and restart collectd:
    <MsgPattern "aer syslog">
        <Match>
            Name "root port"
            Regex "pcieport (.*)"
            IsMandatory false
        </Match>
    </MsgPattern>

  2. Inject AER PCIe error. 
  1. Collectd started.
  2. Notification is not sent because "IsMandatory false", so all is taken by default.

Fail

Internal JIRA filed

SA: Current implementation forcibly turns mandatory fields for first and last match section. Why first and last match must be mandatory?
16PCIe plugin dispatches notification after simple positive match for the same line
  1. Add following to initial collectd.conf and restart collectd:
    <Match>
        Name "root port"
        Regex "pcieport (.*)"
        IsMandatory true
    </Match>
    <Match>
        Name "root port"
        Regex "AER : (.*)"
        IsMandatory false
    </Match>
    <Match>
        Name "root port"
        Regex "(.*) Corrected"
        IsMandatory true
    </Match>
  2. Inject AER PCIe error.
  1. Collectd started.
  2. Notification is sent because "IsMandatory false" and such match not found.
Pass  
17

PCIe plugin dispatches notification after simple negative match for the same line

  1. Add following to initial collectd.conf and restart collectd:
    <Match>
        Name "root port"
        Regex "pcieport (.*)"
        IsMandatory true
    </Match>
    <Match>
        Name "root port"
        Regex "AER : (.*)"
        IsMandatory true
    </Match>
    <Match>
        Name "root port"
        Regex "(.*) Corrected"
        IsMandatory true
    </Match>
  2. Inject AER PCIe error.
  1. Collectd started.
  2. Notification is not sent because "IsMandatory true" and such match not found. (At least one mandatory match not found - message not sent)
Pass