Syslog Configuration
Syslog is a standard for logging system messages. Logging helps for fault notification, network forensics, and security auditing.
OcNOS supports logging messages to a syslog server in addition to logging to a file or on the VTY terminal (ssh/telnet connection) and on the TTY serial console device. OcNOS messages can be logged to a local syslog server (the system on which OcNOS executes) into /var/log/messages by default as well as to one or multiple remote syslog servers (maximum of 8 remote syslog server is supported). Remote syslog servers can either be configured with IPv4 addresses or host names.
Support for In-band management over Default VRF
OcNOS shall stream logs to remote syslog server through the interfaces associated with management VRF by default. Also OcNOS provides configurable option to stream the logs through interfaces associated with default VRF. At any point of time OcNOS shall stream logs through only one VRF.
Topology
.
Syslog sample topology
Enabling rsyslog
#configure terminal | Enter configure mode. |
config)#feature rsyslog vrf management | Enable syslog feature on default or management VRF. By default this feature runs on the management VRF. |
config)#exit | Exit configuration mode |
Logging to a File
The below configurations enable debug logs for a particular protocol. In this case, OSPF is shown.
#debug ospf all | This enables the debugging on OSPF. |
#configure terminal | Enter configure mode |
(config)#router ospf 1 | Enable OSPF process 1 |
(config-router)#exit | Exit router mode |
(config)#feature rsyslog | Enable syslog feature on default or management VRF. By default this feature runs on the management VRF. |
(config)#logging level ospf 7 | This enable debug messages for OSPF module. This is configurable either if default of management VRF. |
(config)#logging logfile ospf1 7 | This creates the log file where the logs will be saved. The path of the file will be in the directory /log/ospf1. Log File size 4096-4194304 bytes. |
(config)#exit | Exit configure mode |
To verify this, do some OSPF configuration and view the messages in the log file or with the show logging logfile command.
Validation Commands
#show logging logfile
File logging : enabled File Name : /log/ospf1 Size : 419430400 Severity : (7)
2019 Jan 05 20:10:52.202 : OcNOS : OSPF : INFO : NSM Message Header
2019 Jan 05 20:10:52.202 : OcNOS : OSPF : INFO : VR ID: 0
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : VRF ID: 0
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : Message type: NSM_MSG_LINK_ADD
(5)
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : Message length: 232
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : Message ID: 0x00000000
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : NSM Interface
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : Interface index: 100001
2019 Jan 05 20:10:52.203 : OcNOS : OSPF : INFO : Name: po1
2019 Jan 05 20:10:52.204 : OcNOS : OSPF : INFO : Flags: 536875010
2019 Jan 05 20:10:52.204 : OcNOS : OSPF : INFO : Status: 0x00000804
2019 Jan 05 20:10:52.204 : OcNOS : OSPF : INFO : Metric: 1
2019 Jan 05 20:10:52.207 : OcNOS : OSPF : INFO : MTU: 1500
2019 Jan 05 20:10:52.207 : OcNOS : OSPF : INFO : Type: L3
2019 Jan 05 20:10:52.207 : OcNOS : OSPF : INFO : HW type: 9
2019 Jan 05 20:10:52.208 : OcNOS : OSPF : INFO : HW len: 6
2019 Jan 05 20:10:52.209 : OcNOS : OSPF : INFO : HW address: ecf4.bb5c.a2b0
2019 Jan 05 20:10:52.210 : OcNOS : OSPF : INFO : Bandwidth: 0.000000
2019 Jan 05 20:10:52.211 : OcNOS : OSPF : INFO : Interface lacp key flag 0
2019 Jan 05 20:10:52.212 : OcNOS : OSPF : INFO : Interface lacp aggregator upda
te flag 0
#show logging level
Facility Default Severity Current Session Severity
nsm 3 3
ripd 3 3
ospfd 3 7
ospf6d 3 3
isisd 3 3
hostpd 3 3
ldpd 2 2
rsvpd 2 2
mribd 2 2
pimd 2 2
authd 2 2
mstpd 2 2
imi 2 2
onmd 2 2
oamd 2 2
vlogd 2 2
vrrpd 2 2
ribd 2 2
bgpd 3 3
l2mribd 2 2
lagd 2 2
sflow 2 2
pservd 2 2
Logging to the Console
#configure terminal | Enter configure mode. |
(config)#logging level ospf 7 | This enable debug messages for OSFP module. |
(config)#logging console 7 | This enables the console logs. |
(config)#debug ospf | This enables the debugging on OSPF configurations. |
(config)#router ospf | Enabling ospf for process 1. |
(config-router)#exit | Exit router mode. |
(config)#exit | Exit configure mode. |
To verify this, do some OSPF configuration and view the messages in the console.
Validation
#show logging console
Console logging : enabled Severity: (debugging)
#show logging level
Facility Default Severity Current Session Severity
nsm 3 3
ripd 3 3
ospfd 3 7
ospf6d 3 3
isisd 3 3
hostpd 3 3
ldpd 2 2
mribd 2 2
pimd 2 2
authd 2 2
mstpd 2 2
imi 2 2
onmd 2 2
oamd 2 2
vlogd 2 2
vrrpd 2 2
ribd 2 2
bgpd 3 3
l2mribd 2 2
lagd 2 2
sflow 2 2
pservd 2 2
Logging to Remote Server
#configure terminal | Enter configure mode. |
(config)#logging level bgp 7 | This enable debug messages for BGP module. |
(config)#logging remote server 10.16.2.1 vrf management | Redirects the log messages to the remote server configured. |
(config)#debug bgp | This enables the debugging on BGP configurations. |
(config)#router bgp 1 | Enabling BGP process 1. |
(config-router)#exit | Exit router mode. |
(config)#exit | Exit configure mode. |
Logging to Remote Server via User-Defined VRF
#configure terminal | Enter configure mode. |
(config)#ip vrf snmp-vrf | Create a user-defined VRF called snmp-vrf |
(config)#commit | Commit the candidate configuration to the running configuration |
(config)#logging level bgp 7 | This enable debug messages for BGP module. |
(config)#logging remote server 10.16.2.1 vrf management | Redirects the log messages to the server configured over the management VRF |
(config)#logging remote server 172.18.19.22 7 vrf snmp-vrf | Redirects the log messages to the server configured over the User defined VRF snmp-vrf |
(config)#debug bgp all | This enables the debugging on BGP configurations. |
(config)#router bgp 1 | Enabling BGP process 1. |
(config-router)#exit | Exit router mode. |
(config)#commit | Commit the candidate configuration to the running configuration |
(config)#exit | Exit configure mode. |
Validation
#show logging server
Remote Servers:
10.16.2.1
severity: (debugging)
facility: local7
VRF: management
172.18.19.22
severity: Operator (debug-detailed)
facility: local7
authpriv: Enabled
VRF : snmp-vrf
#show logging level
Facility Default Severity Current Session Severity
nsm 3 3
ripd 3 3
ospfd 3 3
ospf6d 3 3
isisd 3 3
hostpd 3 3
ldpd 2 2
rsvpd 2 2
mribd 2 2
pimd 2 2
authd 2 2
mstpd 2 2
imi 2 2
onmd 2 2
oamd 2 2
vlogd 2 2
vrrpd 2 2
ribd 2 2
bgpd 3 7
l2mribd 2 2
lagd 2 2
sflow 2 2
pservd 2 2
Validation
#show logging server
Remote Servers:
10.16.2.1
severity: (debugging)
facility: local7
VRF: management
#show logging level
Facility Default Severity Current Session Severity
nsm 3 3
ripd 3 3
ospfd 3 3
ospf6d 3 3
isisd 3 3
hostpd 3 3
ldpd 2 2
mribd 2 2
pimd 2 2
authd 2 2
mstpd 2 2
imi 2 2
onmd 2 2
oamd 2 2
vlogd 2 2
vrrpd 2 2
ribd 2 2
bgpd 3 7
l2mribd 2 2
lagd 2 2
sflow 2 2
pservd 2 2
Configuration
Note: The configuration to support multiple logging servers is listed below. Maximum 4 remote syslog server is supported.
#configure terminal | Enter Configure mode. |
(config)#hostname OcNOS | Configuring the hostname of the device |
(config)#feature rsyslog vrf management | Enable feature on default or management VRF. By default this feature runs on the management VRF. |
(config)#logging level all 7 | Enables debug messages for all modules. |
(config)# logging remote server 10.12.17.10 5 vrf management | Redirects the log messages to the server configured. (Configuring 1 logging server).Configuring with log severity level as 5.By default severity level 7 is considered if no specific levels configured. |
(config)# logging remote server 10.12.17.16 5 vrf management | Redirects the log messages to the server configured. (Configuring 2 logging server). Configuring with log severity level as 5. By default severity level 7 is considered if no specific levels configured. |
(config)# logging remote server 10.12.17.11 7 vrf management | Redirects the log messages to the server configured. (Configuring 3 logging server). Configuring with log severity level as 7. By default severity level 7 is considered if no specific levels configured. |
(config)# logging remote server 10.12.28.22 7 vrf management | Redirects the log messages to the server configured. (Configuring 4 logging server). Configuring with log severity level as 7. By default severity level 7 is considered if no specific levels configured. |
(config)#exit | Exit configure mode. |
Validation
OcNOS # show running-config logging
<snippet of show running-config logging output ...>
feature rsyslog vrf management
logging remote server 10.12.17.10 5 vrf management
logging remote server 10.12.17.16 5 vrf management
logging remote server 10.12.17.11 7 vrf management
logging remote server 10.12.28.22 7 vrf management
OcNOS # show logging server
Remote Servers:
10.12.17.10
severity: Operator (informational)
facility: local7
VRF : management
10.12.17.16
severity: Operator (informational)
facility: local7
VRF : management
10.12.17.11
severity: Operator (debug-detailed)
facility: local7
VRF : management
10.12.28.22
severity: Operator (debug-detailed)
facility: local7
VRF : management
Remote machine Syslog Configuration:
Provided below are the changes required for rsyslog configuration on a debian system. Please refer to respective operating system official sites for more information
cat /etc/rsyslog.conf
$ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so # provides kernel logging support (previously done by rklogd)
$ModLoad immark.so # provides --MARK-- message capability
$ModLoad imudp.so
$UDPServerRun 514
$ModLoad imtcp.so
$InputTCPServerRun 514
# Logs will be placed in separate folders based on hostnames and process modules in the provided path
$template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs
& ~
$template precise,"%msg%\n"
*.* /var/log/messages
auth,authpriv.* /var/log/auth.log
Save the changes and restart the rsyslog services to bring the changes in effect.
Monitoring Logging Server:
Provided below are the sample outputs collected from one of the remote logging server.
root@localhost:~# cd /var/log/
Different folders l get created based on hostnames inthe defined location in rsyslog.conf
root@localhost:/var/log# ls -lt
drwx------ 2 root root 4096 Nov 18 03:02 Leaf1
drwx------ 2 root root 4096 Nov 15 07:24 10.12.56.112-leaf5
drwx------ 2 root root 4096 Nov 15 05:40 10.12.56.109-leaf2
drwx------ 2 root root 4096 Nov 15 01:26 Bingo1
drwx------ 2 root root 4096 Nov 14 06:07 Leaf2
drwx------ 2 root root 4096 Nov 11 04:57 R1-LEAF1
drwx------ 2 root root 4096 Nov 8 06:46 leaf2
drwx------ 2 root root 4096 Nov 8 03:38 R7-LEAF4
drwx------ 2 root root 4096 Nov 8 01:30 LEAF1
drwx------ 2 root root 4096 Nov 8 01:18 leaf3
drwx------ 2 root root 4096 Nov 7 07:56 OcNOS
drwx------ 2 root root 4096 Nov 6 23:58 mgmt-sw-3k
drwx------ 2 root root 4096 Nov 4 21:51 R5-LEAF3
Check under OcNOS folder
root@localhost:/var/log/OcNOS# ls –ltr
Different log files get created based on process name under folder based on hostname.
-rw-r--r-- 1 root root 444 Oct 25 02:20 PSERV.log
-rw-r--r-- 1 root root 328 Oct 30 05:05 ONMD.log
-rw-r--r-- 1 root root 174 Oct 30 05:37 usermod.log
-rw-r--r-- 1 root root 498 Oct 30 07:55 SFLOW.log
-rw-r--r-- 1 root root 486 Oct 30 07:55 RIP.log
-rw-r--r-- 1 root root 486 Oct 30 07:55 LAG.log
-rw-r--r-- 1 root root 492 Oct 30 07:55 VRRP.log
-rw-r--r-- 1 root root 486 Oct 30 07:55 PIM.log
-rw-r--r-- 1 root root 504 Oct 30 07:55 OSPFv3.log
-rw-r--r-- 1 root root 492 Oct 30 07:55 OSPF.log
-rw-r--r-- 1 root root 498 Oct 30 07:55 IS-IS.log
-rw-r--r-- 1 root root 504 Oct 30 07:55 802.1X.log
-rw-r--r-- 1 root root 492 Oct 30 07:56 MSTP.log
-rw-r--r-- 1 root root 483 Oct 30 07:56 HSL.log
-rw-r--r-- 1 root root 486 Oct 30 07:56 RIB.log
-rw-r--r-- 1 root root 492 Oct 30 07:56 MRIB.log
-rw-r--r-- 1 root root 8709 Nov 2 11:22 OAM.log
-rw-r--r-- 1 root root 17959 Nov 2 11:23 NSM.log
-rw-r--r-- 1 root root 12178 Nov 2 11:23 BGP.log
-rw-r--r-- 1 root root 74488 Nov 3 07:41 CMM.log
-rw-r--r-- 1 root root 4128 Nov 3 08:17 login.log
-rw-r--r-- 1 root root 5265 Nov 3 08:17 HOSTP.log
-rw-r--r-- 1 root root 21982 Nov 3 08:17 CML.log
-rw-r--r-- 1 root root 28094411 Nov 3 08:17 CMLSH.log
-rw-r--r-- 1 root root 278619 Nov 3 08:19 sshd.log
-rw-r--r-- 1 root root 695277 Nov 3 08:20 CRON.log