System auditing allows to save information in logs about different interactions of the system (like crashes, logins, reports of services, etc.) and use to investigate incidents on the system.
“La información sobre tráfico, navegación y localización se tratará durante un periodo que puede oscilar entre 3 y 12 meses para cumplir con las obligaciones de mantenimiento de la seguridad de las redes o prevención del fraude.”
“En cumplimiento de la Ley 25/2007, se conservarán los datos de tráfico y localización 12 meses que podrán ser solicitados mediante requerimiento judicial.”
Which basically translates to: we save the data (traffic, navigation and location) for one year in case of fraud and may be requested by court order.
You can read more about Data Retention in this wikipedia post.
As administrators we should do the same, because if our system has been hacked is our duty to report it to the suitable institution. Or in case an incident happened, it’s useful to check exactly what happened.
Linux kernel and daemons usually generate diferent type of message that are stored in logs in
They are useful to :
- Have a trace of its operation.
- Detect errors, malfunction warnings or critical situations.
- Immediately detect an attack
- Check for incorrect uses of the resources
But, can be bad because:
- It generates so much data that it can be used for DDoS
- Can be difficult to analyze the data for the vastness of it. (Solved with logrotate).
Logs in general are owned by the root user and the root group, and haven’t permissions assigned for others.
Run the following commands
apt install acct accton # This creates /var/log/account/pacct
And that’s it!.
pacct file is like the database for accounting.
The daemon in charge of receiving the messages from other services, peripherals and programs is called rsyslog. It stores it in files as declared in
/etc/rsyslog.conf (global file which defines parameters like the default permissions for all log files) and
By default logs are stored in
/var/log. But we can send them to:
- A remote location changing the resulting log file with the server
@log.myserver.com. Would be a good idea since if a cyber criminal tries to remove his footprints, he/she won’t be able to.
- Piping to another process
- Another user
- A text console (
It’s best to read logs with
logger command allows us to interact directly with syslogd.
|5||syslog||Messages generated internally by syslogd|
|6||lpr||Line printer subsystem|
|7||news||Network news subsystem|
|16–23||local0 – local7||Locally used facilities|
These are like groups by which rsyslog creates log files. Example of
# First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #cron.* /var/log/cron.log #daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log #lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log #user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # #mail.info -/var/log/mail.info #mail.warn -/var/log/mail.warn mail.err /var/log/mail.err
|0||Emergency||emerg||panic||System is unusable|
|1||Alert||alert||Action must be taken immediately|
|5||Notice||notice||Normal but significant conditions|
Both tables from Wikipedia.
Knowing the facilities and the severities, we can understand how the
50-default.config is build, lines are formed by two columns separated by tabs:
- 1º Column have the facility and the severity separated with a dot.
- 2º Column refers to the file where the log is going to be saved.
I said here that log files tend to increase and can DDoS with depleting space, well we can obliterate the Hard Drive Disk (NOT RECOMMENDED) with the following:
/etc/rsyslog.confand add the following line at the end of the file:
/etc/rsyslog.conf and add:
Reboot rsyslog service:
systemctl restart rsyslog
To send a message to syslog we use:
logger -p local6.info "My first log message"
Check the file is created.
tail -f /var/log/myPersonalLogFile
This is used a lot in scripts. For example:
logger -t myTag -f /var/log/myPersonalLogFile "My second message"
Will result in something like:
April 28 20:58:10 n0nuser myTag: My second message
When logs grow up in size, log rotate is in charge of dividing old logs, compress them (generally with gzip) and rename them to be manageable.
We can configurate logrotate to do its tasks weekly, monthly… You can see the default config file (
# see "man logrotate" for details # rotate log files weekly weekly # use the adm group by default, since this is the owning group # of /var/log/syslog. su root adm # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # use date as a suffix of the rotated file #dateext # uncomment this if you want your log files compressed #compress # packages drop log rotation information into this directory include /etc/logrotate.d # system-specific logs may be also be configured here.
This script to rotate logs for a week can be executed everyday with cron:
#!/bin/bash DAYS=7 FILENAME="pacct" FILENAME2="pacc" cd /var/log/account if [ -f "$FILENAME.$DAYS" ]; then rm "$FILENAME.$DAYS" fi for (( i=$DAYS; i<=1; i-- )) do mv "$FILENAME.$($DAYS-1)" "$FILENAME.$DAYS" done mv "$FILENAME" "$FILENAME.1" cat /dev/null > "$FILENAME2"
accton: Enable accounting.
sa (summary accounting): Get statistics from used commands.
ac: Print statistics about users’ connect time.
last: Show a listing of last logged in users.
lastb: Show a listing of last users that failed to log in.
lastlog: Show a listing of when was the last login for every user. Reads
w: Show who is logged on and what they are doing.
finger: Displays information about the system users.
who: Show who is logged in. Reads
Check how many login tries have been made in our system:
tail -f /var/log/auth.log
Check which are the biggest files/folders:
du -h /var/log
Users that have been connected the most time:
ac -p ac -p | head -1 # To grab the user with most time ac -p | tail -n 1 # To grab the user with the least time
5 commands that has been executed the most (in order):
sa -nd sa -nd | head -6 | tail -n 5 # Takes the 4 most used commands
5 commands that have consumed the most CPU time:
sa -ak | head -6 | tail -n 5 # -k to order by CPU time average memory usage
Check syslog messages in live:
tail -f /var/log/messages
I’ll add more tools here with the time.