We are using an event-handler with on-startup-config write to push the configuration to our langley service which push the config in the central GIT repository.
To execute an bash command in an different vrf context use „ip netns exec ns-<VRF>“ before the command. See the example:
event-handler ARCHIVE_LANGLEY trigger on-startup-config action bash sudo ip netns exec ns-MANAGEMENT curl https://10.1.1.1/langley/put/$(hostname)/1 --upload-file /mnt/flash/startup-config
Von ARISTA wird ein Python Paket bereitgestellt um über die eAPI mit dem Switch arbeiten zu können. Dies ist eine Qickanleitung.
pip install pyeapi
Auf dem Arista Switch muss die API Schnittstelle aktiviert werden.
!
management api http-commands
protocol http
protocol https
no shutdown
!
vrf MANAGEMENT
no shutdown
!
#!/usr/bin/env python3
import pyeapi
import os, ssl
username = "apiuser"
password = "supersecret"
ip = "10.1.1.1"
# redhat ssl verification patch
if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
ssl._create_default_https_context = ssl._create_unverified_context
conn = pyeapi.client.connect(
transport='https',
host=ip,
username=username,
password=password,
port=443,
timeout=60)
node = pyeapi.client.Node(conn)
print(node.running_config)
#!/usr/bin/env python3
import pyeapi
import os, ssl
username = "apiuser"
password = "supersecret"
ip = "10.1.1.1"
# redhat ssl verification patch
if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
ssl._create_default_https_context = ssl._create_unverified_context
conn = pyeapi.client.connect(
transport='https',
host=ip,
username=username,
password=password,
port=443,
timeout=60)
node = pyeapi.client.Node(conn)
result = node.enable(commands=["show running-config"], encoding="text")
print result[0]['result']['output']
The title says everything …
From webserver:
openssl s_client -connect example.org:443 | openssl x509 -noout -text | grep DNS:
And from file:
openssl x509 -noout -text -in cert.pem | grep DNS:
Gitolite ist ein einfaches Script um einen GIT Repo Server aufzusetzen. Zusammen mit Ansible nutze ich es um z.B. Zonenfiles auf Bind9 Server zu verteilen. Das werde ich in diesem Beispiel zeigen. Ansible und Gitolite laufen auf dem gleichen Server.
apt install gitolite3
Während der Installation wird nach dem Public SSH Key des Admins gefragt. Dieser Key ist später berechtigt das gitolite-admin Repo welches die Konfiguration der Repos beinhaltet zu benutzen.
git clone gitolite3@10.1.1.1:gitolite-admin
gitolite-admin/conf/gitolite.conf
repo gitolite-admin
RW+ = admin
repo testing
RW+ = @all
repo dns
option hook.post-update = dns-deploy
RW+ = admin
Hier wurde bereits ein post-update Hook definiert. Es müssen aber noch weitere Einstellungen angepasst werden damit dieser funktioniert.
Neue User legt man an in dem Verzeichnis gitolite-admin/keydir <username>.pub mit dem entsprechenden SSH Key anlegt. Die Usernamen können dann in der gitolite.conf verwendet werden. Ausführliche Doku zu gitolite -> http://gitolite.com/gitolite/index.html
apt install software-properties-common apt-add-repository ppa:ansible/ansible apt update apt install ansible
Ich verwende auf den Zielsystemen eigene User um Dinge per Ansible zu deployen. Im Beispiel heist der „vader“.
[dns-servers] ns1.example.com ns2.example.com
--- ansible_ssh_private_key_file: /etc/ansible/sshkey/id_rsa ansible_ssh_user: vader ansible_sudo_pass: lukeiamyourfather
mkdir -p /etc/ansible/sshkey ssh-keygen -t rsa -f /etC/ansible/sshkey/id_rsa
Den SSH Key ohne Passwort ablegen.
Anpassen der Verzeichnis Rechte:
chmod 600 /etc/ansible/sshkey chmod 600 /etc/ansible/sshkey/id_rsa chmod 644 /etc/ansible/sshkey/id_rsa.pub chown root:root -R /etc/ansible/sshkey
Auf den Zielsystemen muss jeweils der User vader mit dem Passwort eingerichtet werden und der Public Key des SSH Keys muss als authorized_key hinterlegt werden. Der User muss in der sudo Gruppe sein.
Bei folgenden Zeilen Kommentierung entfernen.
LOCAL_CODE => "$ENV{HOME}/local",
# allow repo-specific hooks to be added
'repo-specific-hooks',
mkdir -p /var/lib/gitolite3/local/hooks/repo-specific
sudo /opt/ansible/dns.sh
Owner und Rechte ändern
chown gitolite3:gitolite3 dns-deploy chmod +x dns-deploy
Das Hook Script liegt in /opt/ansible:
mkdir -p /opt/ansible
#!/bin/bash
REPO=/opt/ansible/dns
if [ ! -d $REPO ]
then
ssh-agent bash -c 'ssh-add /etc/ansible/sshkey/id_rsa; git clone gitolite3@10.1.1.1:dns'
else
cd $REPO
ssh-agent bash -c 'ssh-add /etc/ansible/sshkey/id_rsa; git pull'
fi
cd $REPO
ansible-playbook dns.yaml
# End
Das Hook Script klont das Repo dns nach /opt/ansible/dns und führt das ansible playbook aus dem Repo aus.
Anpassen der Rechte
chown root:root /opt/ansible/dns.sh chmod 700 /opt/ansible/dns.sh
Den gitolite3 User berechtigen das Script ohne Passwort per Sudo auszuführen
gitolite3 ALL = (root) NOPASSWD: /opt/ansible/dns.sh
git clone gitolite3@10.1.1.1:dns cd dns
---
- hosts: dns-servers
tasks:
- name: create or update named.conf.local
become: true
register: named
template:
src: /opt/ansible/dns/conf/named.conf.local.j2
dest: /etc/bind/named.conf.local
vars:
zones:
- lanbugs.de
- name: create or update zone files
become: true
register: zones
template: src={{ item }} dest=/etc/bind/{{ item | basename | regex_replace('\.j2','') }} owner=root group=root mode=0644
with_fileglob:
- /opt/ansible/dns/conf/db.*.j2
- name: restart bind
become: true
service:
name: bind9
state: restarted
when: zones.changed or named.changed
mkdir conf
{% for zone in zones %}
zone "{{ zone }}" {
type master;
file "/etc/bind/db.{{ zone}}";
};
{% endfor %}
; BIND db file for lanbugs.de
$TTL 86400
{% include "zone_header.j2" %}
@ 1800 IN TXT "v=spf1 a mx ptr ~all"
test 1800 IN A 1.2.3.4
$ORIGIN lanbugs.de.
@ IN SOA ns1.example.com. admin.example.com. (
2018111101 ; serial number YYMMDDNN
28800 ; Refresh
7200 ; Retry
864000 ; Expire
86400 ; Min TTL
)
@ 1800 IN NS ns1.example.com.
@ 1800 IN NS ns2.example.com.
@ 1800 IN MX 10 mail1.example.com.
@ 1800 IN MX 20 mail2.example.com.
git add -A git commit -m "initial commit" git push Counting objects: 3, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 311 bytes | 311.00 KiB/s, done. Total 3 (delta 1), reused 0 (delta 0) remote: Identity added: /etc/ansible/sshkey/id_rsa (Ansible vader) remote: From 10.1.1.1:dns remote: 63bf7b0..63d2dc5 master -> origin/master remote: Updating 63bf7b0..63d2dc5 remote: Fast-forward remote: conf/db.lanbugs.de.j2 | 1 - remote: 1 file changed, 1 deletion(-) remote: remote: PLAY [dns-servers] ************************************************************* remote: remote: TASK [Gathering Facts] ********************************************************* remote: ok: [ns1.example.com] remote: ok: [ns2.example.com] remote: remote: TASK [create or update named.conf.local] *************************************** remote: ok: [ns1.example.com] remote: ok: [ns2.example.com] remote: remote: TASK [create or update zone files] ********************************************* remote: changed: [ns1.example.com] => (item=/opt/ansible/dns/conf/db.lanbugs.de.j2) remote: changed: [ns2.example.comc] => (item=/opt/ansible/dns/conf/db.lanbugs.de.j2) remote: remote: TASK [restart bind] ************************************************************ remote: changed: [ns1.example.com] remote: changed: [ns2.example.com] remote: remote: PLAY RECAP ********************************************************************* remote: ns1.example.com : ok=4 changed=2 unreachable=0 failed=0 remote: ns2.example.com : ok=4 changed=2 unreachable=0 failed=0 remote: To 10.1.1.1:/dns 63bf7b0..63d2dc5 master -> master
Have fun 🙂
This is a basic configuration for DCB and PFC to use with Windows 2016 and „storage spaces direct“ on Dell EMC S4xxx switches. CoS class is set to 4.
conf t int vlan 30 description S2D ip address 10.10.10.10 255.255.255.0 mtu 9216 no shut exit dcbx enable policy-map type qos TRUSTMAP class class-trust trust dot1p exit exit class-map type network-qos PFCMAP match qos-group 4 exit policy-map type network-qos QOSMAP class PFCMAP pause buffer-size 100 pause-threshold 50 resume-threshold 10 pfc-cos 4 exit exit system qos service-policy input type qos TRUSTMAP exit interface range ethernet 1/1/1-1/1/12 no flowcontrol receive no flowcontrol transmit switchport mode trunk switchport trunk allowed vlan 30 service-policy input type network-qos QOSMAP mtu 9216 priority-flow-control mode on exit exit
Config on Windows:
# Clear previous configuration Get-NetQosTrafficClass | Remove-NetQosTrafficClass Get-NetQosPolicy | Remove-NetQosPolicy # Disable the DCBx setting Set-NetQosDcbxSetting -Willing $false -Confirm $true # create Qos policies and tag each type of traffic with the relevant priority # SMB PFC priority = 4 New-NetQosPolicy 'SMB' -NetDirectPortMatchCondition 445 -PriorityValue8021Action 4 # Set VLAN ID for SMB-NICs Get-NetAdapter | ? Name -like '*SMB*' | Set-NetAdapterAdvancedProperty -RegistryKeyword 'VlanID' -RegistryValue 30 -Verbose Get-NetAdapter | ? Name -like '*SMB*' | Get-NetAdapterAdvancedProperty -RegistryKeyword 'VlanID' # Enable Priotity Flow Control (PFC) on a specific priority (5). Disable for others.... Enable-NetQosFlowControl -Priority 4 Disable-NetQosFlowControl 0,1,2,3,5,6,7 # Enable QoS on SMB-NICs Get-NetAdapter | ? Name -like '*SMB*'| Enable-NetAdapterQos
With this script you can parse the postfix databases / lookuptables which are in berkley database format.
Das Skript verwende ich um aus der Plesk Datenbank alle gültigen E-Mail Adressen für einen Frondend Mailserver zu exportieren.
TG-Notify ist eine von mir geschriebene Erweiterung für Check_MK welche Notifications über Telegram ermöglicht und über Callback Acknowledge und Downtimes setzen kann für Hosts/Services.
Sourcecode: https://github.com/lanbugs/tg-notify
Status/ Version: 0.1 beta
Download als MKP: folgt
Lizenz: GPLv2
Vorteile:
Geplant / Future:
TG-Notify ist in Python geschrieben und kommt mit den Standard Libraries von Python 2.7 aus die von Check_MK in der OMD Umgebung bereitgestellt werden.
Verwendete Python Libraries:
Bestandteile von TG-Notify:
Verzeichnisse -> Dateien:
Message an BotFather schicken mit „/newbot“, anschließend Namen definieren für neuen Bot z.B. „TGnotifyDEMO_bot“. Der BotFather schickt anschlieend den Token für die HTTP API von Telegram.
Damit die Callbacks ACKs und Downtimes setzen können muss ein Automation User eingerichtet werden. Der User muss als Contact Group die gleichen hanben wie die User sonst kann er keine ACKs/Downtimes setzen. Alternativ muss das Userprofil angepasst werden. Für den Automation User sind die Notifications zu deaktivieren!
Hier ist der Token vom BotFather einzugeben und der Automation User sowie die URL zum Check_MK System.
; ; Configuration for TG-Notify ; [Telegram] ; URL mit / url = https://api.telegram.org/ ; Token with bot at the beginning, without / at the end token = bot123456789:AAAAAbBbbbCCcCccDDddDdDEeEeEEEEeE-FFF [Database] ; Relative to OMD_ROOT without / at the beginning path = local/lib/tg_notify/ file = tg.db [Check_MK] url = http://localhost/dev1/check_mk/ automation_user = a_test automation_pass = ATPQTVBKJYGSDUAHYDQF
Die Funktion ist in tg_admin eingebaut, dazu einfach „tg_admin –initialize-database“ aufrufen.
OMD[dev1]:~$ tg_admin --initialize-database
Die Funktion ist in tg_admin eingebaut, dazu einfach „tg_admin –generate-cronjobs“ aufrufen.
OMD[dev1]:~$ tg_admin --create-cronjobs
TG-Notify hat seine eigene Userdatenbank, es müssen die Check_MK User <-> Chat_ID von Telegram TG-Notify erst bekannt gemacht werden.
Dazu vom Handy ein „open“ an den Bot senden und das Kommando „tg_admin –show-agent“ aufrufen. Es wird die entsprechende Chat_ID angezeigt die der User hat. Anschließend den User mit „tg_admin -c -u hdampf -i 123456789“ anlegen. Aktuelle User können mit „tg_admin -s“ abgefragt werden.
OMD[dev1]:~/local/etc$ tg_admin --show-agent ---------------------------------------------------------------- First Name: Hans Last Name: Dampf CHAT ID: 123456789 ---------------------------------------------------------------- OMD[dev1]:~/local/etc$ tg_admin -c -u hdampf -i 319651791 OMD[dev1]:~/local/etc$ tg_admin -s Current User: Username Chat_ID ------------------------ hdampf 123456789
Nach der Installation von TG-Notify umbedingt einen „omd restart“ durchführen damit das neue Notifications Script gefunden wird von Check_MK. Die Regel wie gewohnt erstellen, hier im Beispiel ist es eine sehr einfach gehaltene, es sind alle bekannten Optionen möglich.
Ich habe in meinem Testsystem Hosts (10.44.4.5, 10.44.4.6) angelegt welche es nicht gibt um Meldungen zu generieren. Mit „tg_admin –show-notify-history“ kann man sich die letzten Notifications ansehen. Für jede Notification wird eine Callback_ID generiert. Die ID dient dazu wenn vom Telegram Client ein Callback aufgeführt wird z.B. Acknowledge eines Services den Callback eindeutig zu identifizieren.
Im Callback wird dann z.B. $$$CB$$$WCA38KSKU9XTODCO:ack gesendet für den Acknowledge von Host 10.44.4.5 / Service PING.
OMD[dev1]:~$ tg_admin --show-notify-history ID Callback ID Host Service Datum Chat_ID Username ---------------------------------------------------------------------------------------------------------------------------------- 232 WCA38KSKU9XTODCO 10.44.4.5 PING 2017-08-21 13:01:50 123456789 cmkadmin 233 T46ETYJ1CH7MV2ZS 10.44.4.5 Check_MK Discovery 2017-08-21 13:01:54 123456789 cmkadmin 234 ADZJPKLMO0SV63WU 10.44.4.5 None 2017-08-21 13:02:02 123456789 cmkadmin 235 4WLYX43CGRJJQNHX 10.44.4.4 None 2017-08-21 13:12:13 123456789 cmkadmin 236 6OMRDN6DQ1KAIWY4 10.44.4.5 None 2017-08-21 13:12:16 123456789 cmkadmin 237 BUQ0HEAKPTFMLPED 10.44.4.5 Check_MK Discovery 2017-08-21 13:31:11 123456789 cmkadmin 238 3QP6YL48RNWGJV4V 10.44.4.5 PING 2017-08-21 13:40:00 123456789 cmkadmin 239 7BKRA2ATVRHESFWH 10.44.4.5 None 2017-08-21 13:46:14 123456789 cmkadmin 240 1MZ2X50KJ55ZXA2W 10.44.4.4 None 2017-08-21 13:46:16 123456789 cmkadmin 241 N3RUH9EGYGJ314UU 10.44.4.6 PING 2017-08-21 13:47:16 123456789 cmkadmin 242 0273U5QDMDH2RFQA 10.44.4.6 Check_MK Discovery 2017-08-21 13:47:18 123456789 cmkadmin 243 JZFJEUYOZAJ7U0VF 10.44.4.6 None 2017-08-21 13:47:29 123456789 cmkadmin 244 0YSN5ZG4DW80NGT1 10.44.4.6 PING 2017-08-21 13:48:13 123456789 cmkadmin 245 D436YY9GE3MLH00W 10.44.4.6 Check_MK Discovery 2017-08-21 13:48:14 123456789 cmkadmin
Die empfangenen Callbacks werden in der SQL Datenbank gespeichert und werden hier von dem Programm tg_callback verarbeitet und die entsprechenden Aufrufe gegen Check_MK durchgeführt. Mit „tg_admin –show-callbacks“ lassen sich die angeforderten Callbacks und der Status anzeigen.
OMD[dev1]:~$ tg_admin --show-callbacks Callback ID Host Service Datum Command Executed ---------------------------------------------------------------------------------------------------------------------------------- OWQML2LFDZM8LIB3 localhost Interface 1 2017-08-18 23:58:49 ack 1 C6V7O11LAF65F5Q5 localhost Check_MK Discovery 2017-08-20 12:23:13 ack 1 P81F439ZX54ET4SG 10.44.4.4 None 2017-08-20 12:53:15 ack 1 CQHDP8OJFE4VTNHE 10.44.4.4 None 2017-08-20 13:04:06 ack 1 ZZ3OAVP41OPWE1PG localhost OMD dev1 Notification Spooler 2017-08-20 13:08:46 ack 1 2XKFHEVFGLB389QX 10.44.4.4 None 2017-08-20 13:34:00 down24h 1 916IUVV6O66GBDUA 10.44.4.4 None 2017-08-20 13:49:45 ack 1 916IUVV6O66GBDUA 10.44.4.4 None 2017-08-20 13:49:46 down24h 1 ZPAZ4OJ63CJ7IEM7 localhost OMD dev1 Notification Spooler 2017-08-21 11:57:28 ack 1 ZPAZ4OJ63CJ7IEM7 localhost OMD dev1 Notification Spooler 2017-08-21 11:57:29 down24h 1 ADZJPKLMO0SV63WU 10.44.4.5 None 2017-08-21 13:12:02 ack 1 T46ETYJ1CH7MV2ZS 10.44.4.5 Check_MK Discovery 2017-08-21 13:31:02 ack 1 WCA38KSKU9XTODCO 10.44.4.5 PING 2017-08-21 13:39:49 down24h 1 ADZJPKLMO0SV63WU 10.44.4.5 None 2017-08-21 13:46:04 down24h 1 N3RUH9EGYGJ314UU 10.44.4.6 PING 2017-08-21 13:48:02 ack 1 0273U5QDMDH2RFQA 10.44.4.6 Check_MK Discovery 2017-08-21 13:48:03 down24h 1
Die ersten Notifications sind am Client angekommen:
Jeder einzelne Service lässt sich ACKen oder für 24h auf Downtime setzen. Klickt man die Buttons an dauert es max. 1 Minute und TG-Notify anwortet das er den Callback empfangen hat.
This tool exports every hardware asset from an Cisco device with a serial number. You can export the list as table or CSV.
Download: https://gist.github.com/lanbugs/4dbed5e0e8a7d5b6d29c4ea9b9e93bb2
>python cisco_inventory.py -h
usage: cisco_inventory.py [-h] -H HOST -v SNMP_VERSION [-C SNMP_COMMUNITY]
[-u SNMP_USER] [-A SNMP_AUTH] [-a SNMP_AUTH_METHOD]
[-X SNMP_PRIVACY] [-x SNMP_PRIVACY_METHOD]
[-L SNMP_SECURITY] [--csv]
Cisco inventory grabber Version 0.1 Written by Maximilian Thoma 2018
optional arguments:
-h, --help show this help message and exit
-H HOST WLC IP address
-v SNMP_VERSION SNMP version, valid are 2 or 3
-C SNMP_COMMUNITY SNMP Community (only v2)
-u SNMP_USER SNMP user (v3)
-A SNMP_AUTH SNMP auth password (v3)
-a SNMP_AUTH_METHOD SNMP auth method, valid are MD5 or SHA (v3)
-X SNMP_PRIVACY SNMP privacy password (v3)
-x SNMP_PRIVACY_METHOD
SNMP privacy method, valid are AES or DES (v3)
-L SNMP_SECURITY SNMP security level, valid are no_auth_or_privacy,
auth_without_privacy or auth_with_privacy (v3)
--csv Result should be CSV
Demo;
>python cisco_inventory.py -H 10.10.10.33 -v 3 -u snmpuser -A snmpauth -a MD5 -X snmpencr -x DES -L auth_with_privacy | Description | Class | Name | HWRev | FWRev | SWRev | Serial | Manufactor | Model | FRU? | |--------------------------------------------------------+-------------+----------------------------------------+---------+-------------+------------+------------------+---------------------+----------------+--------| | 1000BaseSX | - | GigabitEthernet1/4/12 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/4/11 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/4/10 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/4/11 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/3/8 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/3/9 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/3/6 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet1/3/7 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseSX | - | GigabitEthernet2/3/10 | V01 | | | FNS11111111 | CISCO | GLC-SX-MMD | true | | 1000BaseX (SFP) with 12 SFP Ports Jumbo Frame Support | module | Switch1 Linecard 4 (virtual slot 4) | V02 | | | FNS11111111 | Cisco | WS-X4612-SFP-E | true | | 1000BaseX (SFP) with 12 SFP Ports Jumbo Frame Support | module | Switch1 Linecard 3 (virtual slot 3) | V02 | | | FNS11111111 | Cisco | WS-X4612-SFP-E | true | | Cisco Systems, Inc. WS-C4506-E 6 slot switch | chassis | Switch1 System | V03 | | | FNS11111111 | Cisco | WS-C4506-E | false | | FanTray | fan | Switch2 FanTray 1 | V04 | | | FNS11111111 | Cisco | WS-X4596-E | true | | Power Supply ( AC 1400W ) | powerSupply | Switch2 Power Supply 1 | V04 | | | FNS11111111 | Cisco Systems, Inc. | PWR-C45-1400AC | true | | Power Supply ( AC 1400W ) | powerSupply | Switch2 Power Supply 2 | V04 | | | FNS11111111 | Cisco Systems, Inc. | PWR-C45-1400AC | true | | SFP-10Gbase-SR | - | TenGigabitEthernet1/1/1 | V03 | | | FNS11111111 | CISCO-FINISAR | SFP-10G-SR | true | | SFP-10Gbase-SR | - | TenGigabitEthernet1/1/2 | V03 | | | FNS11111111 | CISCO-FINISAR | SFP-10G-SR | true | | Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) with 4 SFP Ports | module | Switch1 Supervisor 1 (virtual slot 1) | V01 | 15.0(1r)SG3 | 03.06.03.E | FNS11111111 | Cisco | WS-X45-SUP7L-E | true |
Script:
#!/usr/bin/env python
# Need following pip packages
# - easysnmp
# - tabulate
# Checkout blog article to tool
# https://lanbugs.de/netzwerktechnik/hersteller/cisco/commandline-tool-for-exporting-cisco-hardware-inventory-via-snmp/
from easysnmp import Session
import argparse
from tabulate import tabulate
from operator import itemgetter
from pprint import pprint
def main():
####
# ARGS
####
description = """
Cisco inventory grabber\nVersion 0.1\nWritten by Maximilian Thoma 2018
"""
aparser = argparse.ArgumentParser(description=description)
aparser.add_argument('-H', dest='host', help='WLC IP address', required=True)
aparser.add_argument('-v', dest='snmp_version', help='SNMP version, valid are 2 or 3', required=True)
aparser.add_argument('-C', dest='snmp_community', help='SNMP Community (only v2)')
aparser.add_argument('-u', dest='snmp_user', help='SNMP user (v3)')
aparser.add_argument('-A', dest='snmp_auth', help='SNMP auth password (v3)')
aparser.add_argument('-a', dest='snmp_auth_method', help='SNMP auth method, valid are MD5 or SHA (v3)')
aparser.add_argument('-X', dest='snmp_privacy', help='SNMP privacy password (v3)')
aparser.add_argument('-x', dest='snmp_privacy_method', help='SNMP privacy method, valid are AES or DES (v3)')
aparser.add_argument('-L', dest='snmp_security',
help='SNMP security level, valid are no_auth_or_privacy, auth_without_privacy or auth_with_privacy (v3)')
aparser.add_argument('--csv', dest='csv', help='Result should be CSV', action='store_true')
args = aparser.parse_args()
####
# Setup SNMP connection
####
if args.snmp_version == "2":
try:
snmp = Session(hostname=args.host, version=2, use_numeric=True)
except Exception as e:
print e
if args.snmp_version == "3":
try:
snmp = Session(
hostname=args.host,
version=3,
security_level=args.snmp_security,
security_username=args.snmp_user,
auth_protocol=args.snmp_auth_method,
auth_password=args.snmp_auth,
privacy_protocol=args.snmp_privacy_method,
privacy_password=args.snmp_privacy,
use_numeric=True
)
except Exception as e:
print e
####
# Init Data Buffer
####
inventory = {}
inv_print = []
####
# SNMP Walk inventory
####
port = {
0: "-",
1: "other",
2: "unknown",
3: "chassis",
4: "backplane",
5: "container",
6: "powerSupply",
7: "fan",
8: "sensor",
9: "module",
10: "port",
11: "stack",
12: "cpu"
}
## Get inventory
result_ids = snmp.walk(".1.3.6.1.2.1.47.1.1.1.1")
def stripper(string):
if "NoneType" not in str(type(string)):
return string.strip()
else:
return string
for r in result_ids:
if r.oid_index in inventory:
element_id = r.oid.replace(".1.3.6.1.2.1.47.1.1.1.1.","")
if element_id == "16": # fru
fru = "true" if "1" in r.value else "false"
inventory[r.oid_index][element_id] = fru
elif element_id == "5": # class
classx = port[int(r.value)] if len(r.value) is 1 else port[0]
inventory[r.oid_index][element_id] = classx
else: # everything else
inventory[r.oid_index][element_id] = r.value
else:
element_id = r.oid.replace(".1.3.6.1.2.1.47.1.1.1.1.","")
inventory[r.oid_index] = {}
inventory[r.oid_index][element_id] = r.value
for elements, values in inventory.iteritems():
if len(values['11']) >= 1:
#print elements
#2 entPhysicalDescr
#3 entPhysicalVendorType
#4 entPhysicalContainedIn
#5 entPhysicalClass
#6 entPhysicalParentRelPos
#7 entPhysicalName
#8 entPhysicalHardwareRev
#9 entPhysicalFirmwareRev
#10 entPhysicalSoftwareRev
#11 entPhysicalSerialNum
#12 entPhysicalMfgName
#13 entPhysicalModelName
#14 entPhysicalAlias
#15 entPhysicalAssetID
#16 entPhysicalIsFRU
inv_print.append([stripper(values.get('2')),
stripper(values.get('5')),
stripper(values.get('7')),
stripper(values.get('8')),
stripper(values.get('9')),
stripper(values.get('10')),
stripper(values.get('11')),
stripper(values.get('12')),
stripper(values.get('13')),
stripper(values.get('16'))])
####
# Sort table
####
inv = sorted(inv_print, key=itemgetter(0))
####
# Result
####
if args.csv is True:
print 'Description;Class;Name;HWRev;FWRev;SWRev;Serial;Manufactor;Model;FRU?'
for line in inv:
print ';'.join(str(l) for l in line)
else:
print tabulate(inv, headers=['Description', 'Class', 'Name', 'HWRev', 'FWRev', 'SWRev', 'Serial', 'Manufactor', 'Model', 'FRU?'], tablefmt="orgtbl")
if __name__ == "__main__":
main()
This tool exports all current clients on all APs assigned to the WLC. You can export as CSV or as table.
Download: https://gist.github.com/lanbugs/6dc874ad0736424c5c7808f01ec78f96
$ python cisco_ap_clients_grabber.py -h
usage: cisco_ap_clients_grabber.py [-h] -H HOST -v SNMP_VERSION
[-C SNMP_COMMUNITY] [-u SNMP_USER]
[-A SNMP_AUTH] [-a SNMP_AUTH_METHOD]
[-X SNMP_PRIVACY] [-x SNMP_PRIVACY_METHOD]
[-L SNMP_SECURITY] [--csv]
Cisco AP clients grabber Version 0.1 Written by Maximilian Thoma 2017
optional arguments:
-h, --help show this help message and exit
-H HOST WLC IP address
-v SNMP_VERSION SNMP version, valid are 2 or 3
-C SNMP_COMMUNITY SNMP Community (only v2)
-u SNMP_USER SNMP user (v3)
-A SNMP_AUTH SNMP auth password (v3)
-a SNMP_AUTH_METHOD SNMP auth method, valid are MD5 or SHA (v3)
-X SNMP_PRIVACY SNMP privacy password (v3)
-x SNMP_PRIVACY_METHOD
SNMP privacy method, valid are AES or DES (v3)
-L SNMP_SECURITY SNMP security level, valid are no_auth_or_privacy,
auth_without_privacy or auth_with_privacy (v3)
--csv Result should be CSV
Demo:
>python cisco_ap_clients_grabber.py -H 1.1.1.1 -v 3 -u username -A authpass -a MD5 -X privpass -x DES -L auth_with_privacy | IP | MAC | Name | SSID | SSID MAC | |---------------+-------------------+------------------------------------+--------------+-------------------| | 10.10.10.1 | C0:FF:EE:C0:FF:EE | host/client0001.m.local | workstations | DE:AD:BE:EF:00:00 | | 10.10.10.2 | C0:FF:EE:C0:FF:EE | host/client0002.m.local | workstations | DE:AD:BE:EF:00:00 | | 10.10.10.3 | C0:FF:EE:C0:FF:EE | host/client0003.m.local | workstations | DE:AD:BE:EF:00:00 | | 10.10.10.4 | C0:FF:EE:C0:FF:EE | host/client0004.m.local | workstations | DE:AD:BE:EF:00:00 | | 10.10.10.5 | C0:FF:EE:C0:FF:EE | host/client0005.m.local | workstations | DE:AD:BE:EF:00:00 | | 10.10.45.200 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.201 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.202 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.203 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.204 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.205 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 | | 10.10.45.206 | C0:FF:EE:C0:FF:EE | | guests | DE:AD:BE:EF:00:00 |
Script:
#!/usr/bin/env python
# Need following pip packages
# - easysnmp
# - tabulate
# Checkout blog article to tool
# https://lanbugs.de/netzwerktechnik/commandline-tool-to-export-current-registered-users-at-aps-from-an-cisco-wireless-lan-controller-wlc/
from easysnmp import Session
import argparse
from tabulate import tabulate
from operator import itemgetter
def main():
####
# ARGS
####
description = """
Cisco AP clients grabber\nVersion 0.1\nWritten by Maximilian Thoma 2018
"""
aparser = argparse.ArgumentParser(description=description)
aparser.add_argument('-H', dest='host', help='WLC IP address', required=True)
aparser.add_argument('-v', dest='snmp_version', help='SNMP version, valid are 2 or 3', required=True)
aparser.add_argument('-C', dest='snmp_community', help='SNMP Community (only v2)')
aparser.add_argument('-u', dest='snmp_user', help='SNMP user (v3)')
aparser.add_argument('-A', dest='snmp_auth', help='SNMP auth password (v3)')
aparser.add_argument('-a', dest='snmp_auth_method', help='SNMP auth method, valid are MD5 or SHA (v3)')
aparser.add_argument('-X', dest='snmp_privacy', help='SNMP privacy password (v3)')
aparser.add_argument('-x', dest='snmp_privacy_method', help='SNMP privacy method, valid are AES or DES (v3)')
aparser.add_argument('-L', dest='snmp_security',
help='SNMP security level, valid are no_auth_or_privacy, auth_without_privacy or auth_with_privacy (v3)')
aparser.add_argument('--csv', dest='csv', help='Result should be CSV', action='store_true')
args = aparser.parse_args()
####
# Setup SNMP connection
####
if args.snmp_version == "2":
try:
snmp = Session(hostname=args.host, version=2, use_numeric=True)
except Exception as e:
print e
if args.snmp_version == "3":
try:
snmp = Session(
hostname=args.host,
version=3,
security_level=args.snmp_security,
security_username=args.snmp_user,
auth_protocol=args.snmp_auth_method,
auth_password=args.snmp_auth,
privacy_protocol=args.snmp_privacy_method,
privacy_password=args.snmp_privacy,
use_numeric=True
)
except Exception as e:
print e
####
# Init Data Buffer
####
inventory = []
longids = []
####
# SNMP Walk Clients inventory
####
## Get longids for clients
result_longids = snmp.walk(".1.3.6.1.4.1.14179.2.1.4.1.1")
for rl in result_longids:
longids.append(rl.oid.replace(".1.3.6.1.4.1.14179.2.1.4.1.1.", "") + "." + rl.oid_index)
## Collect informations
for id in longids:
# Client MAC
result_mac = snmp.get(".1.3.6.1.4.1.14179.2.1.4.1.1." + id)
mac = ":".join(["%02s" % hex(ord(m))[2:] for m in result_mac.value]).replace(' ', '0').upper()
# Client Name
name = snmp.get(".1.3.6.1.4.1.14179.2.1.4.1.3." + id).value
# Client IP
ip = snmp.get(".1.3.6.1.4.1.14179.2.1.4.1.2." + id).value
# SSID
ssid = snmp.get(".1.3.6.1.4.1.14179.2.1.4.1.7." + id).value
# SSID MAC
result_ssid_mac = snmp.get(".1.3.6.1.4.1.14179.2.1.4.1.4." + id)
ssid_mac = ":".join(["%02s" % hex(ord(m))[2:] for m in result_ssid_mac.value]).replace(' ', '0').upper()
inventory.append([ip, mac, name, ssid, ssid_mac])
###
# Sort table
####
inv = sorted(inventory, key=itemgetter(0))
####
# Result
####
if args.csv is True:
print 'IP;MAC;Name;SSID;SSID MAC'
for ip, mac, name, ssid, ssid_mac in inv:
print '%s;%s;%s;%s;%s' % (ip, mac, name, ssid, ssid_mac)
else:
print tabulate(inv, headers=["IP", "MAC", "Name", "SSID", "SSID MAC"], tablefmt="orgtbl")
if __name__ == "__main__":
main()
This tool exports the complete AP inventory from an Cisco WLC. You can create an CSV export or an table.
Download: https://gist.github.com/lanbugs/e86042c0b2afaf7166637a9aa9711cb6
$ python cisco_wlc_ap_grabber.py -h
usage: cisco_wlc_ap_grabber.py [-h] -H HOST -v SNMP_VERSION
[-C SNMP_COMMUNITY] [-u SNMP_USER]
[-A SNMP_AUTH] [-a SNMP_AUTH_METHOD]
[-X SNMP_PRIVACY] [-x SNMP_PRIVACY_METHOD]
[-L SNMP_SECURITY] [--csv]
Cisco AP WLC inventory grabber Version 0.1 Written by Maximilian Thoma 2017
optional arguments:
-h, --help show this help message and exit
-H HOST WLC IP address
-v SNMP_VERSION SNMP version, valid are 2 or 3
-C SNMP_COMMUNITY SNMP Community (only v2)
-u SNMP_USER SNMP user (v3)
-A SNMP_AUTH SNMP auth password (v3)
-a SNMP_AUTH_METHOD SNMP auth method, valid are MD5 or SHA (v3)
-X SNMP_PRIVACY SNMP privacy password (v3)
-x SNMP_PRIVACY_METHOD
SNMP privacy method, valid are AES or DES (v3)
-L SNMP_SECURITY SNMP security level, valid are no_auth_or_privacy,
auth_without_privacy or auth_with_privacy (v3)
--csv Result should be CSV
Demo result:
>python cisco_wlc_ap_grabber.py -H 1.1.1.1 -v 3 -u username -A authpass -a MD5 -X privpass -x DES -L auth_with_privacy | Name | IP | MAC | Model | Serialnumber | |-------------+---------------+-------------------+-------------------+----------------| | dexxx-acp01 | 10.10.100.1 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp02 | 10.11.100.2 | 58:97:1E:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp03 | 10.11.100.3 | 34:DB:FD:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111 | | dexxx-acp04 | 10.11.100.4 | 58:97:1E:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp05 | 10.11.100.5 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp06 | 10.11.100.6 | 40:F4:EC:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp07 | 10.11.100.7 | 0C:27:24:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111 | | dexxx-acp08 | 10.11.100.8 | 00:78:88:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp09 | 10.11.100.9 | 40:F4:EC:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp10 | 10.11.100.10 | 44:AD:D9:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111 | | dexxx-acp11 | 10.11.100.11 | 50:0F:80:C0:FF:EE | AIR-AP2802I-E-K9 | FDW11111111 | | dexxx-acp12 | 10.11.100.12 | 74:A0:2F:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp13 | 10.11.100.13 | 0C:85:25:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp14 | 10.11.100.14 | 74:A0:2F:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp15 | 10.11.100.15 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp16 | 10.11.100.16 | C0:62:6B:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111 | | dexxx-acp17 | 10.11.100.17 | 0C:27:24:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111 | | dexxx-acp18 | 10.11.100.18 | CC:16:7E:C0:FF:EE | AIR-CAP1702I-E-K9 | FCW11111111 | | dexxx-acp19 | 10.11.100.19 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp20 | 10.11.100.20 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp21 | 10.11.100.21 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp22 | 10.11.100.22 | E4:AA:5D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp23 | 10.11.100.23 | F4:CF:E2:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp24 | 10.11.100.24 | 00:FE:C8:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp25 | 10.11.100.25 | F4:CF:E2:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp26 | 10.11.100.26 | 00:FE:C8:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp27 | 10.11.100.27 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp28 | 10.11.100.28 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 | | dexxx-acp29 | 10.11.100.29 | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111 |
Script:
#!/usr/bin/env python
# Need following pip packages
# - easysnmp
# - tabulate
# Checkout blog article to tool
# https://lanbugs.de/netzwerktechnik/hersteller/cisco/cli-tool-export-ap-inventory-from-an-cisco-wireless-lan-controller-wlc/
from easysnmp import Session
import argparse
from tabulate import tabulate
from operator import itemgetter
def main():
####
# ARGS
####
description = """
Cisco AP WLC inventory grabber\nVersion 0.1\nWritten by Maximilian Thoma 2017
"""
aparser = argparse.ArgumentParser(description=description)
aparser.add_argument('-H', dest='host', help='WLC IP address', required=True)
aparser.add_argument('-v', dest='snmp_version', help='SNMP version, valid are 2 or 3', required=True)
aparser.add_argument('-C', dest='snmp_community', help='SNMP Community (only v2)')
aparser.add_argument('-u', dest='snmp_user', help='SNMP user (v3)')
aparser.add_argument('-A', dest='snmp_auth', help='SNMP auth password (v3)')
aparser.add_argument('-a', dest='snmp_auth_method', help='SNMP auth method, valid are MD5 or SHA (v3)')
aparser.add_argument('-X', dest='snmp_privacy', help='SNMP privacy password (v3)')
aparser.add_argument('-x', dest='snmp_privacy_method', help='SNMP privacy method, valid are AES or DES (v3)')
aparser.add_argument('-L', dest='snmp_security',
help='SNMP security level, valid are no_auth_or_privacy, auth_without_privacy or auth_with_privacy (v3)')
aparser.add_argument('--csv', dest='csv', help='Result should be CSV', action='store_true')
args = aparser.parse_args()
####
# Setup SNMP connection
####
if args.snmp_version == "2":
try:
snmp = Session(hostname=args.host, version=2, use_numeric=True)
except Exception as e:
print e
if args.snmp_version == "3":
try:
snmp = Session(
hostname=args.host,
version=3,
security_level=args.snmp_security,
security_username=args.snmp_user,
auth_protocol=args.snmp_auth_method,
auth_password=args.snmp_auth,
privacy_protocol=args.snmp_privacy_method,
privacy_password=args.snmp_privacy,
use_numeric=True
)
except Exception as e:
print e
####
# Init Data Buffer
####
inventory = []
longids = []
####
# SNMP Walk AP Inventory
####
## Get longids for APs
result_longids = snmp.walk(".1.3.6.1.4.1.14179.2.2.1.1.1")
for rl in result_longids:
longids.append(rl.oid.replace(".1.3.6.1.4.1.14179.2.2.1.1.1.", "") + "." + rl.oid_index)
## Collect informations
for id in longids:
# MAC
result_mac = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.1." + id)
mac = ":".join(["%02s" % hex(ord(m))[2:] for m in result_mac.value]).replace(' ', '0').upper()
# Name
name = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.3." + id).value
# IP
ip = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.19." + id).value
# SN
sn = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.17." + id).value
# Model
model = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.16." + id).value
inventory.append([name, ip, mac, model, sn])
###
# Sort table
####
inv = sorted(inventory, key=itemgetter(0))
####
# Result
####
if args.csv is True:
print 'Name;IP;MAC;Model;Serialnumber'
for name, ip, mac, model, sn in inv:
print '%s;%s;%s;%s;%s' % (name, ip, mac, model, sn)
else:
print tabulate(inv, headers=["Name", "IP", "MAC", "Model", "Serialnumber"], tablefmt="orgtbl")
if __name__ == "__main__":
main()
This a pipe backend for PowerDNS to MongoDB written in Python.
For SOA records:
{
"name":"example.org",
"type":"SOA",
"content":"",
"ttl": 300,
"primary": "ns1.example.org",
"mail": "admin.example.org",
"serial": 2018030311,
"refresh": 86400,
"retry": 7200,
"expire": 3600000,
"nttl": 3600
}
For standard records:
{
"name":"www.example.org",
"type":"A",
"ttl": 300,
"content": "1.1.1.1"
}
apt install pdns-server pdns-backend-pipe
pip install pymongo
You can download it from GitHub – https://github.com/lanbugs/powerdns_mongodb_backend
Parts of the code based on: https://gist.github.com/sokratisg/10069682
#!/usr/bin/env python
import sys
from pymongo import MongoClient
# Config
mongo_host = "127.0.0.1"
mongo_port = 27017
mongo_db = "dns"
mongo_collation = "records"
class Lookup(object):
ttl = 30
def __init__(self, query):
(_type, qname, qclass, qtype, _id, ip) = query
self.has_result = False
qname_lower = qname.lower()
self.results = []
self.results.append('LOG\t%s-%s-%s-%s-%s-%s' % (_type, qname, qclass, qtype, _id, ip))
self.has_result = True
client = MongoClient(mongo_host, mongo_port, connect=False)
db = client[mongo_db]
coll = db[mongo_collation]
if qtype == "ANY":
records = coll.find({"name": qname_lower})
else:
records = coll.find({"type": qtype, "name": qname_lower})
if records:
for record in records:
if record['type'] == "SOA":
"""
{
"name":"example.org",
"type":"SOA",
"content":"",
"ttl": 300,
"primary": "ns1.example.org",
"mail": "admin.example.org",
"serial": 2018030311,
"refresh": 86400,
"retry": 7200,
"expire": 3600000,
"nttl": 3600
}
"""
try:
self.results.append(
'DATA\t%s\t%s\t%s\t%s\t-1\t%s\t%s\t%s\t%s\t%s\t%s\t%s' % ( qname_lower,
qclass,
qtype,
record['ttl'],
record['primary'],
record['mail'],
record['serial'],
record['refresh'],
record['retry'],
record['expire'],
record['nttl']
))
self.has_result = True
except:
self.results.append('LOG\t %s SOA Record currupt maybe fields are missing.' % qname_lower)
else:
"""
{
"name":"www.example.org",
"type":"A",
"ttl": 300,
"content": "1.1.1.1"
}
"""
self.results.append('DATA\t%s\t%s\t%s\t%d\t-1\t%s' % (
qname_lower, qclass, record['type'], record['ttl'], record['content']))
self.has_result = True
def str_result(self):
if self.has_result:
return '\n'.join(self.results)
else:
return ''
class DNSbackend(object):
def __init__(self, filein, fileout):
self.filein = filein
self.fileout = fileout
self._process_requests()
def _fprint(self, message):
self.fileout.write(message + '\n')
self.fileout.flush()
def _process_requests(self):
first_time = True
while 1:
rawline = self.filein.readline()
if rawline == '':
return
line = rawline.rstrip()
if first_time:
if line == 'HELO\t1':
self._fprint('OK\tPython backend ready.')
else:
rawline = self.filein.readline()
sys.exit(1)
first_time = False
else:
query = line.split('\t')
if len(query) != 6:
self._fprint('LOG\tPowerDNS sent unparseable line')
self._fprint('FAIL')
else:
lookup = Lookup(query)
if lookup.has_result:
pdns_result = lookup.str_result()
self._fprint(pdns_result)
self._fprint('END')
if __name__ == "__main__":
infile = sys.stdin
outfile = sys.stdout
try:
DNSbackend(infile, outfile)
except:
raise
Don`t forget to make the backend.py script executable to the world.
# Here come the local changes the user made, like configuration of # the several backends that exist. launch=pipe pipe-command=/opt/pdns/backend.py
Launch the PowerDNS service in monitor mode
root@pdnsdev:~# /etc/init.d/pdns monitor Jun 20 21:10:11 Reading random entropy from '/dev/urandom' Jun 20 21:10:11 Loading '/usr/lib/x86_64-linux-gnu/pdns/libpipebackend.so' Jun 20 21:10:11 [PIPEBackend] This is the pipe backend version 4.0.0-alpha2 reporting Jun 20 21:10:11 Loading '/usr/lib/x86_64-linux-gnu/pdns/libbindbackend.so' Jun 20 21:10:11 [bind2backend] This is the bind backend version 4.0.0-alpha2 reporting Jun 20 21:10:11 This is a standalone pdns Jun 20 21:10:11 UDP server bound to 0.0.0.0:53 Jun 20 21:10:11 Unable to enable timestamp reporting for socket Jun 20 21:10:11 UDPv6 server bound to [::]:53 Jun 20 21:10:11 TCP server bound to 0.0.0.0:53 Jun 20 21:10:11 TCPv6 server bound to [::]:53 Jun 20 21:10:11 PowerDNS Authoritative Server 4.0.0-alpha2 (C) 2001-2016 PowerDNS.COM BV Jun 20 21:10:11 Using 64-bits mode. Built using gcc 5.3.1 20160330. Jun 20 21:10:11 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. Jun 20 21:10:11 Set effective group id to 118 Jun 20 21:10:11 Set effective user id to 112 Jun 20 21:10:11 Creating backend connection for TCP % Jun 20 21:10:11 Backend launched with banner: OK Python backend ready. Jun 20 21:10:11 [bindbackend] Parsing 0 domain(s), will report when done Jun 20 21:10:11 [bindbackend] Done parsing domains, 0 rejected, 0 new, 0 removed Jun 20 21:10:11 About to create 3 backend threads for UDP Jun 20 21:10:11 Backend launched with banner: OK Python backend ready. Jun 20 21:10:11 Backend launched with banner: OK Python backend ready. Jun 20 21:10:11 Done launching threads, ready to distribute questions Jun 20 21:10:11 Backend launched with banner: OK Python backend ready.
Do some querys
user@pdnsdev:~$ dig example.org @127.0.0.1 SOA ; <<>> DiG 9.10.3-P4-Ubuntu <<>> example.org @127.0.0.1 SOA ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63891 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1680 ;; QUESTION SECTION: ;example.org. IN SOA ;; ANSWER SECTION: example.org. 300 IN SOA ns1.example.org. admin.example.org. 2018030311 86400 7200 3600000 3600 ;; Query time: 13 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Thu Jun 20 21:20:22 CEST 2018 ;; MSG SIZE rcvd: 86 user@pdnsdev:~$ dig www.example.org @127.0.0.1 ; <<>> DiG 9.10.3-P4-Ubuntu <<>> www.example.org @127.0.0.1 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 896 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1680 ;; QUESTION SECTION: ;www.example.org. IN A ;; ANSWER SECTION: www.example.org. 300 IN A 1.1.1.1 ;; Query time: 26 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Thu Jun 20 21:20:44 CEST 2018 ;; MSG SIZE rcvd: 60
Have fun 🙂
– Kurze Zusammenfassung, ausführlich siehe Quelle am Ende des Artikels –
Python Virtual Environment aufsetzen
pip install virtualenv virtualenv projekt source projekt/bin/activate pip install uwsgi flask
Mini Flask Applikation projekt.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello world!"
WSGI Startfile wsgi.py
from projekt import app
if __name__ == "__main__":
app.run()
Virtual Environment verlassen
deactivate
uWSGI Configfile erstellen
[uwsgi] module = wsgi:app master = true processes = 5 socket = /opt/projekt/projekt.sock chmod-socket = 660 vacuum = true die-on-term = true
systemd File erstellen /etc/systemd/system/projekt.service
[Unit] Description=uWSGI instance to serve projekt After=network.target [Service] User=projekt Group=www-data WorkingDirectory=/opt/projekt Environment="PATH=/opt/projekt/bin" ExecStart=/opt/projekt/bin/uwsgi --ini projekt.ini [Install] WantedBy=multi-user.target
uWSGI Projekt Service starten
sudo systemctl start projekt sudo systemctl enable projekt
NGINX Config erstellen – /etc/nginx/sites-available/projekt
server {
listen 80;
server_name 10.10.10.10;
location / {
include uwsgi_params;
uwsgi_pass unix:/opt/projekt/projekt.sock;
}
}
NGINX Config aktivieren
ln -s /etc/nginx/sites-available/projekt /etc/nginx/sites-enabled systemctl restart nginx
Verifizieren NGINX und uWSGI
dev@dev1:~$ ps aux | grep projekt
projekt 17175 0.0 2.1 71216 21812 ? Ss 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17178 0.0 1.7 71216 17316 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17179 0.0 1.7 71216 17316 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17180 0.0 1.6 71216 16504 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17181 0.0 1.7 71472 17848 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17182 0.0 1.6 71216 16504 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
dev@dev1:~$ tail -f /var/log/syslog
Jun 19 19:44:42 dev1 uwsgi[17175]: [pid: 17178|app: 0|req: 1/3] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:42 2018] GET / => generated 40 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 79 bytes (1 switches on core 0)
Jun 19 19:44:43 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 1/4] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:43 2018] GET / => generated 40 bytes in 4 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:44 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 2/5] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:44 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:45 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 3/6] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:45 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Quelle / ausführlicher Artikel: https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-ubuntu-16-04
Per Default ist bei einer MongoDB Instanz keine Authentication aktiviert. Hier der kurze Weg.
Admin Account anlegen
devusr@testsystem:~# mongo
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.5
> use admin
switched to db admin
> db.createUser({user: "admin", pwd: "geheimes_passwort", roles: [{ role: "root", db: "admin" }]})
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
> quit()
Authentication in mongod.conf aktivieren (Ubuntu: /etc/mongod.conf)
... # network interfaces net: port: 27017 bindIp: 127.0.0.1 security: authorization: enabled ...
Mongod neustarten
service mongod restart
Erster Test: User anlegen für Test DB
> use test
switched to db test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
2018-06-18T08:05:53.448+0200 E QUERY [thread1] Error: couldn't add user: not authorized on test to execute command { createUser: "devuser", pwd: "xxx", roles: [ { role: "readWrite", db: "test" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 600000.0 }, $db: "test" } :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype.createUser@src/mongo/shell/db.js:1437:15
@(shell):1:1
>
Sieht gut aus, es geht nicht 😉
Anmelden und User für Test DB anlegen
use admin
db.auth("admin","geheimes_passwort")
1
use test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
Successfully added user: {
"user" : "devuser",
"roles" : [
{
"role" : "readWrite",
"db" : "test"
}
]
}
>
Authentifizierung mit pymongo in Python Code
>>> from pymongo import MongoClient
>>> uri = "mongodb://devuser:secure_pwd@localhost/test?authSource=test"
>>> client = MongoClient(uri)
>>> db = client.test
>>> collection = db.foo
>>> collection.insert_one({"foo":"bar"})
<pymongo.results.InsertOneResult object at 0x7fa764f2a998>
>>> collection.find_one()
{u'_id': ObjectId('5b274db939d9c0683b47c0e2'), u'foo': u'bar'}
>>>
Quellen / Weitere Informationen:
Enable Authentication – https://docs.mongodb.com/manual/tutorial/enable-authentication/
Built-In Roles – https://docs.mongodb.com/manual/core/security-built-in-roles/
pymongo authentication – http://api.mongodb.com/python/current/examples/authentication.html
Ein paar Notizen zu Python und MongoDB 🙂 MongoDB ist eine NoSQL Datenbank, weitere Infos -> Wikipedia 🙂
Aktuelle MongoDB Version installieren (auf Ubuntu 16.04)
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5 echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list apt install apt-transport-https apt update apt-get install -y mongodb-org
siehe auch: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
Python Erweiterung pymongo für MongoDB installieren mit pip
python -m pip install pymongo
siehe auch: https://api.mongodb.com/python/current/
Verbindung zur MongoDB
from pymongo import MongoClient
client = MongoClient('localhost', 27017)
Connect zur Datenbank test
db = client.test
Collection myCollection in Datenbank test verbinden
coll = db.myCollection
Einen Datensatz anlegen in einer Collection
post = {"author":"Fritz Fuchs", "book":"Hasenjagd"}
>>> coll.insert_one(post)
<pymongo.results.InsertOneResult object at 0x7fd0136ce3f8>
Mehrere Datensätze in einer Collection anlegen
>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7"}]
>>> result = coll.insert_many(posts)
>>> result.inserted_ids
[ObjectId('5b259b2c39d9c04f7b43af01'), ObjectId('5b259b2c39d9c04f7b43af02'), ObjectId('5b259b2c39d9c04f7b43af03')]
Datensatz suchen
>>> coll.find_one({"author":"Fritz Fuchs"})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Datensatz mit Regex suchen
>>> coll.find_one({"author":{"$regex": "^Fritz.*"}})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Datensatz mit ObjectId abrufen
>>> from bson.objectid import ObjectId
>>> coll.find_one({"_id":ObjectId("5b25998139d9c04f7b43aefe")})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Mehrere Datensätze abrufen z.B. mit Regex
>>> for post in coll.find({"author":{"$regex": "^Fritz.*"}}):
... print post
...
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a8939d9c04f7b43aeff'), u'book': u'Hasenjagd 2', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a9039d9c04f7b43af00'), u'book': u'Hasenjagd 3', u'author': u'Fritz Fuchs'}
Index erzeugen für Collection z.B. Username ist Unique
>>> import pymongo
>>> db.users.create_index([('user_id', pymongo.ASCENDING)], unique=True)
u'user_id_1'
>>> new_users = [{'user_id':'max'},{'user_id':'fritz'}]
>>> db.users.insert_many(new_users)
<pymongo.results.InsertManyResult object at 0x7fd0136cee18>
>>> new_user = {'user_id':'max'}
>>> db.users.insert_one(new_user)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 683, in insert_one
session=session),
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 599, in _insert
bypass_doc_val, session)
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 580, in _insert_one
_check_write_command_response(result)
File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 207, in _check_write_command_response
_raise_last_write_error(write_errors)
File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 188, in _raise_last_write_error
raise DuplicateKeyError(error.get("errmsg"), 11000, error)
pymongo.errors.DuplicateKeyError: E11000 duplicate key error collection: test.users index: user_id_1 dup key: { : "max" }
ObjectId als String
>>> new_user = {'user_id':'maxx'}
>>> result = db.users.insert_one(new_user)
>>> result
<pymongo.results.InsertOneResult object at 0x7fd0122410e0>
>>> result.inserted_id
ObjectId('5b259ead39d9c04f7b43af07')
>>> str(result.inserted_id)
'5b259ead39d9c04f7b43af07'
String ObjectId zu ObjectId Object wandeln und für Suche verwenden
>>> from bson.objectid import ObjectId
>>> str_obj = '5b259ead39d9c04f7b43af07'
>>> users.find_one({'_id':ObjectId(str_obj)})
{u'_id': ObjectId('5b259ead39d9c04f7b43af07'), u'user_id': u'maxx'}
Datensatz löschen
>>> users.delete_one({'_id':ObjectId(str_obj)})
<pymongo.results.DeleteResult object at 0x7fd0136cee18>
Datensatz aktualisieren
>>> new_user = {'user_id':'maxx', 'name':'Hans Wurst'}
>>> users.insert_one(new_user)
>>> change = {'name': 'Fritz Fritz'}
>>> users.update_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')}, {'$set':change} )
<pymongo.results.UpdateResult object at 0x7fd0136ced40>
>>> users.find_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')})
{u'_id': ObjectId('5b25a14f39d9c04f7b43af08'), u'user_id': u'maxx', u'name': u'Fritz Fritz'}
Bereich bei Suche
>>> client = MongoClient()
>>> db = client.huu
>>> c = db.test
>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5", "boo":1}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6", "boo":5}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7", "boo":10}]
>>> c.insert_many(posts)
>>> for post in c.find({"boo": {"$lt": 6}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b54'), u'book': u'Hasenjagd 5', u'boo': 1}
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}
>>> for post in c.find({"boo": {"$gt": 6}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b56'), u'book': u'Hasenjagd 7', u'boo': 10}
>>> for post in c.find({"boo": {"$lt": 6, "$gt": 3}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}
Quelle / weitere Beispiele: http://api.mongodb.com/python/current/tutorial.html
Hier die einfachste Variante um in sicheres Passwort Hashing in Python umzusetzen.
Dazu wird die passlib verwendet. Diese kann per pip installiert werden, passlib ist für Python 2.x und 3.x kompatibel.
pip install passlib
Passwort hashen:
>>> # passlib laden >>> from passlib.hash import pbkdf2_sha256 >>> >>> # das passwort zum hashen >>> password = "EinsuperGeheimesPasswort" >>> >>> # ein falsches passwort zum testen >>> password2 = "FalschesPasswort" >>> >>> hash = pbkdf2_sha256.hash(password) >>> # hash ausgeben >>> hash '$pbkdf2-sha256$29000$IQSgdE6p1VoL4fwfQwjBWA$9sy/3NJmX1jP.3kYwgmG96zVpBxoVA5yKA6pB.T5Mrw'
Passwort verifizieren:
>>> # richtiges passwort >>> pbkdf2_sha256.verify(password, hash) True >>> # falsches passwort >>> pbkdf2_sha256.verify(password2, hash) False >>>
Wem pbkdf2_sha256 zu kompliziert ist kann das auch beim Import mit einem Alias versehen.
>>> from passlib.hash import pbkdf2_sha256 as pwhash >>> hash = pwhash.hash(password) >>> hash '$pbkdf2-sha256$29000$Z2zNWYsxBiDEmFPK2XsvZQ$WQ8Urv1d4AmBV4v.vFV01uHeZZ7ya52VuYbLkEHEsWE' >>> pwhash.verify(password, hash) True
Weitere Infos:
Die Palo Alto Jungs haben eine exzellente XML API implementiert die für die Firewalls als auch Panorama usw. funktioniert.
Um auf die API Dokumentation zuzugreifen einfach https://<firewall_ip>/api aufrufen.
Der User muss API Rechte haben.
Hier ein Beispiel zum Auslesen der ARP Tabelle.
Ich habe den SSL Zertifikatscheck in Python deaktiviert in meiner Lab Umgebung. Bitte nicht in Produktionsumgebungen verwenden!
#!/usr/bin/env python
# required pip packages: lxml, beautifulsoup4, tabulate
from bs4 import BeautifulSoup as BS
import urllib2
import ssl
import urllib
from tabulate import tabulate
username = "api_test"
password = "supersecurepassword"
firewall = "192.168.111.1"
#####################
# SSL ignore Cert Check (Lab Environment only!)
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
#####################
# Req function
def req_api(command, key):
url = "https://%s/api/?key=%s&type=op&cmd=" % (firewall, key) + urllib.quote_plus(command)
res = urllib2.urlopen(url, context=ctx)
return res.read()
#####################
# API Key request
req_api_url = "https://%s/api/?type=keygen&user=%s&password=%s" % (firewall, username, password)
res_api_key = urllib2.urlopen(req_api_url, context=ctx)
soup = BS(res_api_key.read(), "lxml")
key = soup.find('key').text
#####################
# Get arp entrys
soup = BS(req_api("<show><arp><entry name = 'all'/></arp></show>", key), "lxml")
arp_buffer = []
for e in soup("entry"):
arp_buffer.append([e.status.text, e.ip.text, e.ttl.text, e.interface.text, e.port.text, e.mac.text])
print "#" * 120 + "\n> ARP Cache Count: %s\n" %(len(arp_buffer)) + "#" * 120
print tabulate(arp_buffer, headers=['Status', 'IP', 'TTL', 'Interface', 'Port', 'MAC'], tablefmt='orgtbl')
Ergebnis:
>.\palo_xml.py ######################################################################################################################## > ARP Cache Count: 14 ######################################################################################################################## | Status | IP | TTL | Interface | Port | MAC | |----------+---------------+-------+-------------+-------------+-------------------| | c | 192.168.0.1 | 1560 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.2 | 1058 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.5 | 1797 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.11 | 1779 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.12 | 1581 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.13 | 1787 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.30 | 1055 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.33 | 1542 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.40 | 1121 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.42 | 173 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.44 | 1799 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.45 | 910 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.52 | 1788 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee | | c | 192.168.0.57 | 1671 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
I have configured an active/active cluster with 2 PA-5220 in routed mode (dynamic routing with OSPF) in different datacenters. The problem was that the firewalls not synced thier session tables vice versa. I have controlled the HA links several times all was configured well.
I have tried to debug the problem, so i looked at the cluster state on the CLI:
admin@node1(active-primary)> show high-availability state-synchronization -------------------------------------------------------------------------------- State Synchronization Status: Complete -------------------------------------------------------------------------------- state synchronization to peer device enabled: yes -------------------------------------------------------------------------------- state synchronization messages processed since system up message enable version sent received -------------------------------------------------------------------------------- session setup yes 9 24036024 810298 session teardown yes 9 24296403 822660 session update yes 9 117885229 5838204 predict session add yes 9 33302 1581 predict session delete yes 9 32947 1168 predict session update yes 9 15960 284 ARP update no 1 0 0 ARP delete no 1 0 0 MAC update no 1 0 0 MAC delete no 1 0 0 IPSec sequence number update yes 3 0 0 ND update no 1 0 0 ND delete no 1 0 0 DoS Aggregate entry update yes 1 0 0 DoS Class Tbl IP update yes 1 0 0 DoS Class Tbl IP delete yes 1 0 0 DoS Block Tbl IP update yes 1 0 0 DoS Block Tbl IP delete yes 1 0 0 A/A session setup yes 9 24038236 810298 A/A session statistics yes 9 0 0 A/A packet forward using HA2 yes 9 0 0 Return MAC Update yes 1 0 0 Return MAC Delete yes 1 0 0 V6 Return MAC Update yes 1 0 0 V6 Return MAC Delete yes 1 0 0 HA2 monitor message yes 1 489636 488960 predict session modify yes 9 0 0 --------------------------------------------------------------------------------
You can see that the firewall creates sessions and updates and has send it and recieved it.
But if you look at the global counters you will see the following:
admin@node2(active-secondary)> show counter global filter severity error Global counters: Elapsed time since last sampling: 49.87 seconds name value rate severity category aspect description -------------------------------------------------------------------------------- flow_rcv_dot1q_tag_err 54 0 drop flow parse Packets dropped: 802.1q tag not configured flow_no_interface 54 0 drop flow parse Packets dropped: invalid interface flow_policy_nofwd 5 0 drop flow session Session setup: no destination zone from forwarding flow_tcp_non_syn_drop 37977 0 drop flow session Packets dropped: non-SYN TCP without session match flow_fwd_l3_ttl_zero 1953 0 drop flow forward Packets dropped: IP TTL reaches zero flow_fwd_l3_noroute 35 0 drop flow forward Packets dropped: no route flow_fwd_l3_noarp 7 0 drop flow forward Packets dropped: no ARP flow_fwd_zonechange 1171 0 drop flow forward Packets dropped: forwarded to different zone flow_fwd_notopology 6292 0 drop flow forward Packets dropped: no forwarding configured on interface flow_xmt_platform_encap_err 426 0 drop flow offload Packets dropped: Platform encapsulation error flow_predict_hash_insert_failure 1429 0 error flow pktproc Predict session has insert failure flow_host_decap_err 15 0 drop flow mgmt Packets dropped: decapsulation error from control plane flow_host_service_deny 9399 0 drop flow mgmt Device management session denied flow_fpga_ingress_exception_err 1314289 7 drop flow offload Packets dropped: receive ingress exception error from offload processor flow_fpga_egress_exception_err 1457 0 drop flow offload Packets dropped: receive egress exception error from offload processor flow_fpp_sess_bind_ack_flow_state_error 1850 0 drop flow offload FPP Sess bind ACK flow state verification error ctd_filter_decode_failure_zip 30 0 error ctd pktproc Number of decode filter failure for zip ctd_filter_decode_failure_qpdecode 4 0 error ctd pktproc Number of decode filter failure for qpdecode ha_err_xmt_l2 52 0 error ha system HA sync transmit error: link layer info unavailable ha_err_state 11536 0 error ha system Packets dropped: invalid HA state ha_err_decap 463971 2 error ha system Packets dropped: HA message decapsulation error ha_err_decap_intf 1816 0 error ha system Packets dropped: HA message decapsulation error because interface not match ha_err_decap_proto 462155 2 error ha system Packets dropped: HA message protocol decapsulation error ha_err_msg_payload 154264121 958 error ha system Packets dropped: HA message payload processing error ha_err_session_update 51758863 201 error ha system Packets dropped: HA session update error ha_aa_pktfwd_err_rcv_no_interface 4169 0 drop ha aa Active/Active: packets received on the non-configured local interface -------------------------------------------------------------------------------- Total counters shown: 26 --------------------------------------------------------------------------------
The counter „ha_err_msg_payload“ had an rate of 958 per second.
ha_err_msg_payload 154264121 958 error ha system Packets dropped: HA message payload processing error
Description: „Packets dropped: HA message payload processing error“ – I have nothing found at PaloAlto pages to that error so i had to reasearch it on my own…
The solution …
After a while i recognized that some sessions are syncronized. What was the difference.
I have on both sites LACP portchannels + subinterfaces for different VLANs. As usual I give the interface for eg. VLAN 500 the interface ID 500 => AE1.500. But in my szenario I had different VLANs in the datacenters for the same security zone. So on Node1 the interface name was AE1.500 (VLAN 500) and in the other datacenter on Node2 AE1.1500 (VLAN 1500). We have for each security zone a VRF-lite setup, the VLAN 500 in datacenter1 is in the same VRF like VLAN 1500 in datacenter2. I thought that the session table matching is only applied on the security zone, but in Active/Active mode it is nessesary that the interface name is equal on both sites! The VLAN ID can be different but the interface name must be equal. The network where the session syncronization worked accidentally has the same VLAN on both sites and also the same subinterface ID. After changing the subinterface id the sync works perfectly.
Per Default ist die Boot Partition ca. 500M groß, da sich hier durch die Updates gerne alte Kernel ansammeln ist es ratsam zwischendurch aufzuräumen.
Beispiel einer abgebrochenen Installation, Meldung ist „No space left on device“
linux-image-4.4.0-116-generic (4.4.0-116.140) wird eingerichtet ... Running depmod. update-initramfs: deferring update (hook will be called later) The link /initrd.img is a dangling linkto /boot/initrd.img-4.4.0-116-generic vmlinuz(/boot/vmlinuz-4.4.0-116-generic ) points to /boot/vmlinuz-4.4.0-116-generic (/boot/vmlinuz-4.4.0-116-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-116-generic.postinst line 491. Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-116-generic /boot/vmlinuz-4.4.0-116-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-116-generic /boot/vmlinuz-4.4.0-116-generic update-initramfs: Generating /boot/initrd.img-4.4.0-116-generic W: mdadm: /etc/mdadm/mdadm.conf defines no arrays. gzip: stdout: No space left on device E: mkinitramfs failure cpio 141 gzip 1 update-initramfs: failed for /boot/initrd.img-4.4.0-116-generic with 1. run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1 Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-116-generic.postinst line 1052. dpkg: Fehler beim Bearbeiten des Paketes linux-image-4.4.0-116-generic (--configure): Unterprozess installiertes post-installation-Skript gab den Fehlerwert 2 zurück
Bei ubuntuusers gibt es einen nützlichen Artikel wie man am besten seine alten Kernel entsorgt.
Hier die Kurzfassung, weitere Details bitte direkt bei ubuntuusers nachlesen.
Aktuellen Kernel feststellen:
max@kronos01:~$ uname -a Linux kronos01 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Alte Kernel in Textdatei schreiben, prüfen ob aktueller Kernel dabei ist wenn ja entfernen:
max@kronos01:~$ dpkg -l 'linux-[ihs]*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\([-0-9]*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | tee zu_entfernende_Kernel
max@kronos01:~$ cat zu_entfernende_Kernel linux-headers-4.4.0-101 linux-headers-4.4.0-101-generic linux-headers-4.4.0-103 linux-headers-4.4.0-103-generic linux-headers-4.4.0-104 linux-headers-4.4.0-104-generic linux-headers-4.4.0-108 linux-headers-4.4.0-108-generic linux-headers-4.4.0-81 linux-headers-4.4.0-81-generic linux-headers-4.4.0-93 linux-headers-4.4.0-93-generic linux-headers-4.4.0-97 linux-headers-4.4.0-97-generic linux-headers-4.4.0-98 linux-headers-4.4.0-98-generic linux-image-4.4.0-101-generic linux-image-4.4.0-103-generic linux-image-4.4.0-104-generic linux-image-4.4.0-108-generic linux-image-4.4.0-81-generic linux-image-4.4.0-93-generic linux-image-4.4.0-97-generic linux-image-4.4.0-98-generic linux-image-extra-4.4.0-101-generic linux-image-extra-4.4.0-103-generic linux-image-extra-4.4.0-104-generic linux-image-extra-4.4.0-108-generic linux-image-extra-4.4.0-93-generic linux-image-extra-4.4.0-97-generic linux-image-extra-4.4.0-98-generic
Alte Kernel löschen und File löschen:
max@kronos01:~$ cat zu_entfernende_Kernel | xargs sudo apt-get -y purge max@kronos01:~$ rm zu_entfernende_Kernel
Quelle: https://wiki.ubuntuusers.de/Skripte/Alte_Kernel_entfernen/
I have found a nice help for fast rsync transfers over ssh. You can reach 40MB/s on gigabit network, depends on disk speed, SSD are faster.
rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>
More details & source: https://gist.github.com/KartikTalwar/4393116