[rsyslog] Refresh of the 'debops.rsyslog' role

parent 025efb2f
......@@ -198,6 +198,20 @@ General
- The persistent configuration stored on the Ansible Controller has been
refactored and does not use multiple separate tasks to handle the JSON files.
:ref:`debops.rsyslog` role
''''''''''''''''''''''''''
- The role has been refreshed and uses the custom Ansible filter plugins to
manage the :command:`rsyslog` configuration files. The default configuration
was rearranged, the :file:`/etc/rsyslog.conf` configuration file has the
default contents that come with the Debian package and can be configured by
the role. The configuration model has been redesigned; any changes in the
configuration of the role set in the Ansible inventory need to be reviewed
before applying the new version.
- The ``rsyslog`` APT package and its service can be cleanly removed from the
host, either via the role or by uninstalling the package itself.
Removed
~~~~~~~
......
debops.rsyslog - Manage syslog daemon using Ansible
Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
Copyright (C) 2015-2017 DebOps <https://debops.org/>
Copyright (C) 2015-2020 Maciej Delmanowski <[email protected]>
Copyright (C) 2015-2020 DebOps <https://debops.org/>
SPDX-License-Identifier: GPL-3.0-only
This Ansible role is part of DebOps.
......
This diff is collapsed.
---
# Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2017 DebOps <https://debops.org/>
# Copyright (C) 2015-2020 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2020 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
- name: Check and restart rsyslogd
command: rsyslogd -N 1
notify: [ 'Restart rsyslogd' ]
- name: Restart rsyslogd
service:
name: 'rsyslog'
......
This diff is collapsed.
#!{{ ansible_python['executable'] }}
# Copyright (C) 2020 Maciej Delmanowski <[email protected]>
# Copyright (C) 2020 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
# {{ ansible_managed }}
from __future__ import print_function
from json import loads, dumps
import subprocess
import os
def cmd_exists(cmd):
return any(
os.access(os.path.join(path, cmd), os.X_OK)
for path in os.environ["PATH"].split(os.pathsep)
)
output = {'installed': cmd_exists('rsyslogd')}
try:
version_stdout = subprocess.check_output(
["rsyslogd", "-v"]
).decode('utf-8').strip()
for line in version_stdout.split('\n'):
if line.lower().startswith('rsyslogd '):
output['version'] = line.split()[1].rstrip(',')
except Exception:
pass
print(dumps(output, sort_keys=True, indent=4))
{# Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2017 DebOps <https://debops.org/>
{# Copyright (C) 2015-2020 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2020 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
#}
# {{ ansible_managed }}
# /etc/rsyslog.conf Configuration file for rsyslog.
#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
{% for section in rsyslog__combined_configuration_sections | parse_kv_config %}
{% if section.name|d() and section.state|d('present') not in [ 'absent', 'init', 'ignore' ] %}
{% set section_config = (rsyslog__combined_configuration | parse_kv_config
| selectattr("section", "equalto", section.name) | list) %}
{% if section_config|d() %}
{% if section.state != 'hidden' %}
{% if ('present' in (section_config | map(attribute='state') | list)) %}
{% set section_title = (' ' + ((section.title | d(section.name)) | upper) + ' ') %}
{% set section_width = section_title | length + 8 %}
{{ '' }}
{{ '' }}
{{ ("{:#^" + section_width|string + "}").format('#') }}
{{ ("{:#^" + section_width|string + "}").format(section_title) }}
{{ ("{:#^" + section_width|string + "}").format('#') }}
{{ '' }}
{% endif %}
{% else %}
{{ '' }}
{% endif %}
{% for element in section_config %}
{% if element.state|d('present') not in [ 'absent', 'ignore', 'init' ] %}
{% if element.comment|d() %}
{% if not loop.first %}
{{ '' }}
{% endif %}
{{ element.comment | regex_replace('\n$', '') | comment }}
{% endif %}
{% if element.raw|d() %}
{% if element.state|d('present') == 'comment' %}
{{ '{}'.format(element.raw | regex_replace('\n$', '') | comment(prefix='', decoration='#', postfix='')) -}}
{% else %}
{{ '{}'.format(element.raw | regex_replace('\n$', '')) }}
{% endif %}
{% endif %}
{% endif %}
{% endfor %}
{% endif %}
{% endif %}
{% endfor %}
{# Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2017 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
#}
# {{ ansible_managed }}
{% macro print_config(element) %}
{% if element is mapping %}
{% if element.state is undefined or element.state != 'absent' %}
{% if element.comment|d() %}
{{ element.comment | comment }}
{% endif %}
{% if element.selector|d() and element.action|d() %}
{% set rsyslog__tpl_selector = element.selector | trim %}
{{ "%-30s %s" | format(rsyslog__tpl_selector, element.action) }}
{% elif element.options|d() %}
{{ element.options }}
{% endif %}
{% endif %}
{% elif element is string %}
{{ element }}
{% else %}
{% for thing in element %}
{{ print_config(thing) -}}
{% endfor %}
{% endif %}
{% endmacro %}
{% if item.sections|d() %}
{% for section in item.sections %}
{{ print_config(section) -}}
{% endfor %}
{% else %}
{{ print_config(item) -}}
{% endif %}
{# Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2017 DebOps <https://debops.org/>
{# Copyright (C) 2015-2020 Maciej Delmanowski <[email protected]>
# Copyright (C) 2015-2020 DebOps <https://debops.org/>
# SPDX-License-Identifier: GPL-3.0-only
#}
# {{ ansible_managed }}
{#
# ==== template for rsyslog created via debops.rsyslog role ====
#
# List of parameters supported by this template:
#
# - item.name: ''
# Name of the template. Required.
#
# - item.comment: ''
# Comment to the template, which you want to see on the top of the
# template file. Optional.
#
# - item.options: |
# Text block with value mapping specified in the template format, check
# rsyslog documentation or examples if not sure about syntax. Required.
#
# - item.state: 'present'
# If this parameter is defined and 'absent', template file will be removed
# from the rsyslog configuration. Optional.
#}
{% if item.comment|d() %}
{{ item.comment | regex_replace('\n$', '') | comment }}
{% endif %}
{% if item.raw|d() %}
{% if item.state|d('present') == 'comment' %}
{{ '{}'.format(item.raw | regex_replace('\n$', '') | comment(prefix='', decoration='#', postfix='')) -}}
{% else %}
{{ '{}'.format(item.raw | regex_replace('\n$', '')) }}
{% endif %}
{% elif item.options|d() %}
{% for element in item.options %}
{% if element.state|d('present') not in [ 'absent', 'ignore', 'init' ] %}
{% if element.comment|d() %}
{% if not loop.first %}
{% if item.name is defined and item.name %}
{% if item.comment is defined and item.comment %}
# {{ item.comment }}
{% endif %}
template(
name="{{ item.name }}"
{% if item.options is defined and item.options %}
{{ item.options | indent(8) | regex_replace("(?m)^\s*$", "") }}
{% endif %}
)
{% endif %}
{% endif %}
{{ element.comment | regex_replace('\n$', '') | comment }}
{% endif %}
{% if element.state|d('present') == 'comment' %}
{{ '{}'.format(element.raw | regex_replace('\n$', '') | comment(prefix='', decoration='#', postfix='')) -}}
{% else %}
{{ '{}'.format(element.raw | regex_replace('\n$', '')) }}
{% endif %}
{% endif %}
{% endfor %}
{% endif %}
.. Copyright (C) 2015-2017 Maciej Delmanowski <[email protected]>
.. Copyright (C) 2015-2017 DebOps <https://debops.org/>
.. Copyright (C) 2015-2020 Maciej Delmanowski <[email protected]>
.. Copyright (C) 2015-2020 DebOps <https://debops.org/>
.. SPDX-License-Identifier: GPL-3.0-only
Getting started
......@@ -19,27 +19,37 @@ generated, although with slightly longer log rotation. If the operating system
is Debian, ``rsyslog`` will be run on a privileged ``root`` account; if the
system is Ubuntu, an unprivileged ``syslog`` account will be used by default.
The ``rsyslog`` configuration is stored in :file:`/etc/rsyslog.d/`, most of the
configuration located in :file:`/etc/rsyslog.conf` has been moved to the directory
and put in separate files (old configuration is preserved in a diverted file).
Configuration filename extensions
---------------------------------
The configuration order is important, and to aid support of configuration from
other roles, ``debops.rsyslog`` includes configuration files with different
filename extensions at certain parts of the configuration:
other roles, the :file:`/etc/rsyslog.conf` configuration file includes other
configuration files with different filename extensions at certain points of the
configuration:
:file:`/etc/rsyslog.d/*.input`
These files define configuration of the `rsyslog input modules`__ which can
be used as data sources.
.. __: https://www.rsyslog.com/doc/v8-stable/configuration/modules/idx_input.html
:file:`/etc/rsyslog.d/*.template`
These configuration files can be used to create custom templates used by
``rsyslog`` in different parts of the configuration.
:file:`/etc/rsyslog.d/*.conf`
These files are included by default. They are meant to be used for
configuration of the local system logs, the extension is used to preserve
compatibility with Debian package conventions.
:file:`/etc/rsyslog.d/*.template`
These configuration files can be used to create custom templates used by
``rsyslog`` in different parts of the configuration.
:file:`/etc/rsyslog.d/*.output`
These files define configuration of the `rsyslog output modules`__ which can
be used as targets by various local and remote rulesets defined later on.
:file:`/etc/rsyslog.d/*.system`
.. __: https://www.rsyslog.com/doc/v8-stable/configuration/modules/idx_output.html
:file:`/etc/rsyslog.d/*.ruleset`
These configuration files are meant to be used to define log matching rules
specific to a given system, to store logs in different files.
......@@ -50,63 +60,53 @@ filename extensions at certain parts of the configuration:
modules. This way the local (system) logs and remote logs from other hosts
can be managed separately and shouldn't mix with each other.
Quick start: log forwarding
---------------------------
To enable log forwarding, you will want to configure a few variables differently
in your Ansible inventory. The quick and dirty setup described here assumes
that you want to forward logs over UDP without any encryption, so it should
only be used for testing if remote logs work. For more advanced configuration
check the :ref:`rsyslog__forward` documentation.
First, on the host that should receive the remote logs, for example in
:file:`ansible/inventory/host_vars/logs.example.org/rsyslog.yml`, configure
variables:
.. code-block:: yaml
`Log forwarding`__ tells :command:`rsyslogd` server to send all or specific syslog
messages to another syslog server(s). The :ref:`debops.rsyslog` role is
tailored for configuring log forwarding over TLS to a central syslog server
using `DNS SRV resource records`__.
# Enable network input channels and storage of remote logs in filesystem
rsyslog__capabilities: [ 'network', 'remote-files' ]
.. __: https://www.rsyslog.com/sending-messages-to-a-remote-syslog-server/
.. __: https://tools.ietf.org/html/draft-schoenw-opsawg-nm-srv-03
# Specify which subnets can send remote logs through the firewall
rsyslog__host_allow: [ '192.0.2.0/24', '2001:db8::/32' ]
The role checks if the ``_syslog._tcp.{{ rsyslog__domain }}`` DNS SRV resource
record exists. If it's found, the host is not configured to receive logs via
:envvar:`rsyslog__remote_enabled` variable and the :ref:`debops.pki` role has
been configured on the host, the :ref:`debops.rsyslog` will generate
configuration for each target server that will send syslog messages over TLS to
port 6514 by default. This configuration can be found and changed in the
:envvar:`rsyslog__default_forward` and the :envvar:`rsyslog__default_rules`
variables.
# Mask log forwarding configuration defined elsewhere
rsyslog__forward: []
rsyslog__group_forward: []
rsyslog__host_forward: []
# Or, alternatively, forward logs to a different host
rsyslog__host_forward: [ '*.* @other.{{ ansible_domain }}' ]
Quick start: receiving remote logs
----------------------------------
This will prepare a given central log storage host to receive logs from other
systems on specified subnets, and store them in :file:`/var/log/remote/` directory.
The role does not configure :command:`rsyslogd` service to receive log messages
from the network by default. To enable this, you can specify a list of allowed
IP addresses and/or CIDR subnets which are allowed to send syslog messages
using the :envvar:`rsyslog__allow`, :envvar:`rsyslog__group_allow` and/or
:envvar:`rsyslog__host_allow` variables. Defining these in the inventory will
tell the role to configure :command:`rsyslog` to accept remote logs and store
them in subdirectories under the :file:`/var/log/remote/` directory. The
:ref:`debops.ferm` and the :ref:`debops.logrotate` roles will be used to
configure the IPTables firewall and log rotation respectively.
Now, you can enable log forwarding for all hosts in your inventory (in
:file:`ansible/inventory/group_vars/all/rsyslog.yml`) or only for a specific group
(in :file:`ansible/inventory/group_vars/logged/rsyslog.yml`), using:
This behaviour is controlled by the :envvar:`rsyslog__remote_enabled` variable.
.. code-block:: yaml
rsyslog__forward: [ '*.* @logs.{{ ansible_domain }}' ]
This will forward logs on all hosts in the inventory over unencrypted UDP to
a specified host. Due to above "masking" of the variables on the host inventory
level, the log server should not create an infinite loop which forwards logs to
itself. The ``debops.rsyslog`` role does not handle such case automatically, so
you need to make sure this doesn't happen by accident.
The role by default supports more advanced setups like forwarding logs over TCP
using encrypted TLS connections, but these require more extensive configuration
from different Ansible roles. You should read the rest of the
``debops.rsyslog`` documentation to see how you can enable these features.
Example inventory
-----------------
To enable the ``debops.rsyslog`` role on a given host or group of hosts, you
need to add that host to the ``[debops_service_rsyslog]`` Ansible inventory
group:
The :ref:`debops.rsyslog` role is included by default in the DebOps
:file:`common.yml` playbook and does not need to be specifically enabled.
To enable the ``debops.rsyslog`` role on a given host or group of hosts not
managed by DebOps, you need to add that host to the
``[debops_service_rsyslog]`` Ansible inventory group:
.. code-block:: none
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment