Knowledgebase
Sorry - either this article does not exist or you haven't been given permission to view it.
Knowledgebase

SOP - How to get SEL and Sensor Reading

SOP- How to get Supermicro System/Components Serial Number

SOP - How to get Crash Dump

SOP - How to use SUM to activate Keys

See attached

 Ipmitool is the standard adopted tool of most customers to query and execute ipmi commands.  It works on HP, Dell, Supermicro, and many other manufacturers.

The most useful function is to query the event logs with “sel elist”.  Example:

[root@lab-twin2]ipmitool -U root -P root -H 172.24.203.102 sel elist

   1 | 06/15/2015 | 22:32:07 | Unknown #0xff |  | Asserted

   2 | 06/15/2015 | 22:34:28 | Unknown #0xff |  | Asserted

   3 | 06/16/2015 | 18:20:30 | Unknown #0xff |  | Asserted

   4 | 06/16/2015 | 18:20:54 | Unknown #0xff |  | Asserted

   5 | 06/16/2015 | 18:21:23 | Physical Security #0x51 | General Chassis intrusion | Asserted

   6 | 06/16/2015 | 18:46:03 | Unknown #0xff |  | Asserted

   7 | 06/16/2015 | 18:46:24 | Unknown #0xff |  | Asserted

For all CPU issues, we need to gather the raw data, and or crash dump logs and send it back to SMC for investigation.

  • List the Sel Logs for the system.
  • Check if there are Configuration Errors, IERR, CATERR

Run command line to list events.

  • ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U username -P passwd sel list
  • Output from the command prompt
    • C:\ipmi>ipmitool -I lanplus -H 172.31.37.31 -U ADMIN -P ADMIN sel list
    • 1 | 03/21/2019 | 22:36:31 | Processor #0xff | IERR | Asserted
    • 2 | 09/19/2019 | 21:07:39 | Processor | Configuration Error | Asserted
    • 3 | 09/19/2019 | 21:07:39 | Processor | Configuration Error | Asserted
    • 4 | 09/19/2019 | 21:07:39 | Processor | Configuration Error | Asserted
    • 5 | 09/19/2019 | 21:07:39 | Processor | Configuration Error | Asserted

 

  • If there is “Configuration Error” in the SEL list, then execute the raw command to get the raw data and send it back to SMC to debug.
    • ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U username -P passwd sel writeraw hex
    • Send hex to Supermicro to decode

 

  • If there is “IERR / CATERR” in the SEL list, then download and send back the crashdump logs to SMC to debug.
    • Please NOTE: If there is No CATERR in the SEL Log entries, then that means there is No crashdump file available.
    • Crash dump logs are are stored in the BMC memory. If you pull AC power then the crashdump will be gone.

  • There are two ways we can download the CPU crashdump.
    • BMC WebGUI
      • Go to “Maintenance” => “Troubleshooting”
  • Click on the “Download” button to download the crash dump and send it to Supermicro’s Service Desk Engineer.
    • Do Not click on the “Dump” button as it will overwrite the current crash dump.

  • We can also download the CPU crashdump with command line but we need to use SMCIPMITool.

Execute command:

  • # ./SMCIPMITool <IPMI IP> <User> <Password> ipmi oem generalfiledownload 19 crash.txt

 

  • If you get and (Exception:Prepare download file failed(CCh)) message, it means that there is No crashdump to download.

  • An example valid crash dump will look like (See Below):
    • Once you have downloaded the crash dump, send it to a Supermicro Service Desk Engineer through the onsite ticket.

##### ipmitool raw command for CPU issues with No Sel Log entries ####

The SEL Logs most of the time will logs any hardware issues that the system is having, but there are some cases where there is nothing is the SEL Logs, but the system is behaving not normal. Here are some cases where the system doesn’t have anything in the SEL Logs, but reboots randomly.

  • System randomly reboots and there is nothing reported in the SEL Logs.
    • We need to execute these IPMI RAW Commands and send back to Supermicro to help debug.

  

 ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0xb0 0x0

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0xb1 0x0

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0xb2 0x0

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0x03 0x03

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0x04 0x03

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0x05 0x03

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0xfc 0x02

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0x00 0x03

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> -b 0 -t 0x2c raw 0x30 0x26 0x57 0x1 0x0 0x4 0x6 0x4 0x9f 0x00 0x01 0x03

  • System hangs, does not respond and there is nothing reported in the SEL Logs.
    • We need to execute these IPMI RAW Commands and send back to Supermicro to help debug.

 

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x54 0x00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x55 0x00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x56 0x00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x57 0x00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x53 00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0x52 00

ipmitool -I lanplus -H <IPMI IP> -U <Username> -P <Password> raw 0x30 0x70 0xc3 0xD4 00

Most popular articles 
 
Newest articles 
 
Introducing SMCI’s Data Center Solution Engineering (DCSE) Service and how you can leverage their subject matter expertise to increase Data Center Operations Efficiency (DCOE) ,while reducing the Total Cost to the Environment (TCE) and Total Cost of Ownership (TCO). As a business leader you are constantly seeking new opportunities to deliver additional services, maximize investments in tandem with positive social and environmental outcomes, while meeting fiscal responsibility. As a valued SMCI’s customer DCSE can provide a complimentary DCOE analysis and results consultation. If would like to schedule a call with you to discuss this and how DCSE can add value. We can be reached at 1.877.275.3125 or dcse@supermicro.com. Also, If you are interested in DCSE products and services please go to https://www.supermicro.com/products/rack/dcse/, call 1.877.275.3125 or email dcse@supermicro.com

data_center