Schlagwort-Archive: python

Commandline Tool for exporting Cisco hardware inventory via SNMP

This tool exports every hardware asset from an Cisco device with a serial number. You can export the list as table or CSV.

Download: https://gist.github.com/lanbugs/4dbed5e0e8a7d5b6d29c4ea9b9e93bb2

>python cisco_inventory.py -h
usage: cisco_inventory.py [-h] -H HOST -v SNMP_VERSION [-C SNMP_COMMUNITY]
                          [-u SNMP_USER] [-A SNMP_AUTH] [-a SNMP_AUTH_METHOD]
                          [-X SNMP_PRIVACY] [-x SNMP_PRIVACY_METHOD]
                          [-L SNMP_SECURITY] [--csv]

Cisco inventory grabber Version 0.1 Written by Maximilian Thoma 2018

optional arguments:
  -h, --help            show this help message and exit
  -H HOST               WLC IP address
  -v SNMP_VERSION       SNMP version, valid are 2 or 3
  -C SNMP_COMMUNITY     SNMP Community (only v2)
  -u SNMP_USER          SNMP user (v3)
  -A SNMP_AUTH          SNMP auth password (v3)
  -a SNMP_AUTH_METHOD   SNMP auth method, valid are MD5 or SHA (v3)
  -X SNMP_PRIVACY       SNMP privacy password (v3)
  -x SNMP_PRIVACY_METHOD
                        SNMP privacy method, valid are AES or DES (v3)
  -L SNMP_SECURITY      SNMP security level, valid are no_auth_or_privacy,
                        auth_without_privacy or auth_with_privacy (v3)
  --csv                 Result should be CSV

Demo;

>python cisco_inventory.py -H 10.10.10.33 -v 3 -u snmpuser -A snmpauth -a MD5 -X snmpencr -x DES -L auth_with_privacy
| Description                                            | Class       | Name                                   | HWRev   | FWRev       | SWRev      | Serial           | Manufactor          | Model          | FRU?   |
|--------------------------------------------------------+-------------+----------------------------------------+---------+-------------+------------+------------------+---------------------+----------------+--------|
| 1000BaseSX                                             | -           | GigabitEthernet1/4/12                  | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/4/11                  | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/4/10                  | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/4/11                  | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/3/8                   | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/3/9                   | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/3/6                   | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet1/3/7                   | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseSX                                             | -           | GigabitEthernet2/3/10                  | V01     |             |            | FNS11111111      | CISCO               | GLC-SX-MMD     | true   |
| 1000BaseX (SFP) with 12 SFP Ports Jumbo Frame Support  | module      | Switch1 Linecard 4 (virtual slot 4)    | V02     |             |            | FNS11111111      | Cisco               | WS-X4612-SFP-E | true   |
| 1000BaseX (SFP) with 12 SFP Ports Jumbo Frame Support  | module      | Switch1 Linecard 3 (virtual slot 3)    | V02     |             |            | FNS11111111      | Cisco               | WS-X4612-SFP-E | true   |
| Cisco Systems, Inc. WS-C4506-E 6 slot switch           | chassis     | Switch1 System                         | V03     |             |            | FNS11111111      | Cisco               | WS-C4506-E     | false  |
| FanTray                                                | fan         | Switch2 FanTray 1                      | V04     |             |            | FNS11111111      | Cisco               | WS-X4596-E     | true   |
| Power Supply ( AC 1400W )                              | powerSupply | Switch2 Power Supply 1                 | V04     |             |            | FNS11111111      | Cisco Systems, Inc. | PWR-C45-1400AC | true   |
| Power Supply ( AC 1400W )                              | powerSupply | Switch2 Power Supply 2                 | V04     |             |            | FNS11111111      | Cisco Systems, Inc. | PWR-C45-1400AC | true   |
| SFP-10Gbase-SR                                         | -           | TenGigabitEthernet1/1/1                | V03     |             |            | FNS11111111      | CISCO-FINISAR       | SFP-10G-SR     | true   |
| SFP-10Gbase-SR                                         | -           | TenGigabitEthernet1/1/2                | V03     |             |            | FNS11111111      | CISCO-FINISAR       | SFP-10G-SR     | true   |
| Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) with 4 SFP Ports | module      | Switch1 Supervisor 1 (virtual slot 1)  | V01     | 15.0(1r)SG3 | 03.06.03.E | FNS11111111      | Cisco               | WS-X45-SUP7L-E | true   |

 

Script:

#!/usr/bin/env python

# Need following pip packages
# - easysnmp
# - tabulate

# Checkout blog article to tool
# https://lanbugs.de/netzwerktechnik/hersteller/cisco/commandline-tool-for-exporting-cisco-hardware-inventory-via-snmp/


from easysnmp import Session
import argparse
from tabulate import tabulate
from operator import itemgetter
from pprint import pprint

def main():
    ####
    # ARGS
    ####
    description = """
    Cisco inventory grabber\nVersion 0.1\nWritten by Maximilian Thoma 2018
    """

    aparser = argparse.ArgumentParser(description=description)
    aparser.add_argument('-H', dest='host', help='WLC IP address', required=True)
    aparser.add_argument('-v', dest='snmp_version', help='SNMP version, valid are 2 or 3', required=True)
    aparser.add_argument('-C', dest='snmp_community', help='SNMP Community (only v2)')
    aparser.add_argument('-u', dest='snmp_user', help='SNMP user (v3)')
    aparser.add_argument('-A', dest='snmp_auth', help='SNMP auth password (v3)')
    aparser.add_argument('-a', dest='snmp_auth_method', help='SNMP auth method, valid are MD5 or SHA (v3)')
    aparser.add_argument('-X', dest='snmp_privacy', help='SNMP privacy password (v3)')
    aparser.add_argument('-x', dest='snmp_privacy_method', help='SNMP privacy method, valid are AES or DES (v3)')
    aparser.add_argument('-L', dest='snmp_security',
                         help='SNMP security level, valid are no_auth_or_privacy, auth_without_privacy or auth_with_privacy (v3)')
    aparser.add_argument('--csv', dest='csv', help='Result should be CSV', action='store_true')
    args = aparser.parse_args()

    ####
    # Setup SNMP connection
    ####

    if args.snmp_version == "2":
        try:
            snmp = Session(hostname=args.host, version=2, use_numeric=True)

        except Exception as e:
            print e

    if args.snmp_version == "3":
        try:
            snmp = Session(
                hostname=args.host,
                version=3,
                security_level=args.snmp_security,
                security_username=args.snmp_user,
                auth_protocol=args.snmp_auth_method,
                auth_password=args.snmp_auth,
                privacy_protocol=args.snmp_privacy_method,
                privacy_password=args.snmp_privacy,
                use_numeric=True
            )

        except Exception as e:
            print e

    ####
    # Init Data Buffer
    ####

    inventory = {}
    inv_print = []

    ####
    # SNMP Walk inventory
    ####

    port = {
    0: "-",
    1: "other",
    2: "unknown",
    3: "chassis",
    4: "backplane",
    5: "container",
    6: "powerSupply",
    7: "fan",
    8: "sensor",
    9: "module",
    10: "port",
    11: "stack",
    12: "cpu"
    
    }

    ## Get inventory
    result_ids = snmp.walk(".1.3.6.1.2.1.47.1.1.1.1")

    def stripper(string):
        if "NoneType" not in str(type(string)):
            return string.strip()
        else:
            return string


    for r in result_ids:
        
        if r.oid_index in inventory:
            element_id = r.oid.replace(".1.3.6.1.2.1.47.1.1.1.1.","")

            if element_id == "16": # fru
                fru = "true" if "1" in r.value else "false"
                inventory[r.oid_index][element_id] = fru

            elif element_id == "5": # class
                classx = port[int(r.value)] if len(r.value) is 1 else port[0]
                inventory[r.oid_index][element_id] = classx

            else: # everything else
                inventory[r.oid_index][element_id] = r.value
            
        else:
            element_id = r.oid.replace(".1.3.6.1.2.1.47.1.1.1.1.","")
            inventory[r.oid_index] = {}
            inventory[r.oid_index][element_id] = r.value
        
    
    
    for elements, values in inventory.iteritems():
        
        if len(values['11']) >= 1:        
            #print elements
            
            #2 entPhysicalDescr
            #3 entPhysicalVendorType
            #4 entPhysicalContainedIn
            #5 entPhysicalClass
            #6 entPhysicalParentRelPos
            #7 entPhysicalName
            #8 entPhysicalHardwareRev
            #9 entPhysicalFirmwareRev
            #10 entPhysicalSoftwareRev
            #11 entPhysicalSerialNum
            #12 entPhysicalMfgName
            #13 entPhysicalModelName
            #14 entPhysicalAlias
            #15 entPhysicalAssetID
            #16 entPhysicalIsFRU
            
            inv_print.append([stripper(values.get('2')),
                              stripper(values.get('5')),
                              stripper(values.get('7')),
                              stripper(values.get('8')),
                              stripper(values.get('9')),
                              stripper(values.get('10')),
                              stripper(values.get('11')),
                              stripper(values.get('12')),
                              stripper(values.get('13')),
                              stripper(values.get('16'))])



    ####
    # Sort table
    ####

    inv = sorted(inv_print, key=itemgetter(0))

    ####
    # Result
    ####

    if args.csv is True:

        print 'Description;Class;Name;HWRev;FWRev;SWRev;Serial;Manufactor;Model;FRU?'
        
        for line in inv:
            print ';'.join(str(l) for l in line)

    else:
        print tabulate(inv, headers=['Description', 'Class', 'Name', 'HWRev', 'FWRev', 'SWRev', 'Serial', 'Manufactor', 'Model', 'FRU?'], tablefmt="orgtbl")


if __name__ == "__main__":
    main()

 

CLI Tool – Export AP inventory from an Cisco Wireless LAN Controller (WLC)

This tool exports the complete AP inventory from an Cisco WLC. You can create an CSV export or an table.

Download: https://gist.github.com/lanbugs/e86042c0b2afaf7166637a9aa9711cb6

$ python cisco_wlc_ap_grabber.py -h
usage: cisco_wlc_ap_grabber.py [-h] -H HOST -v SNMP_VERSION
                               [-C SNMP_COMMUNITY] [-u SNMP_USER]
                               [-A SNMP_AUTH] [-a SNMP_AUTH_METHOD]
                               [-X SNMP_PRIVACY] [-x SNMP_PRIVACY_METHOD]
                               [-L SNMP_SECURITY] [--csv]

Cisco AP WLC inventory grabber Version 0.1 Written by Maximilian Thoma 2017

optional arguments:
  -h, --help            show this help message and exit
  -H HOST               WLC IP address
  -v SNMP_VERSION       SNMP version, valid are 2 or 3
  -C SNMP_COMMUNITY     SNMP Community (only v2)
  -u SNMP_USER          SNMP user (v3)
  -A SNMP_AUTH          SNMP auth password (v3)
  -a SNMP_AUTH_METHOD   SNMP auth method, valid are MD5 or SHA (v3)
  -X SNMP_PRIVACY       SNMP privacy password (v3)
  -x SNMP_PRIVACY_METHOD
                        SNMP privacy method, valid are AES or DES (v3)
  -L SNMP_SECURITY      SNMP security level, valid are no_auth_or_privacy,
                        auth_without_privacy or auth_with_privacy (v3)
  --csv                 Result should be CSV

Demo result:

>python cisco_wlc_ap_grabber.py -H 1.1.1.1 -v 3 -u username -A authpass -a MD5 -X privpass -x DES -L auth_with_privacy

| Name        | IP            | MAC               | Model             | Serialnumber   |
|-------------+---------------+-------------------+-------------------+----------------|
| dexxx-acp01 | 10.10.100.1   | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp02 | 10.11.100.2   | 58:97:1E:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp03 | 10.11.100.3   | 34:DB:FD:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111    |
| dexxx-acp04 | 10.11.100.4   | 58:97:1E:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp05 | 10.11.100.5   | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp06 | 10.11.100.6   | 40:F4:EC:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp07 | 10.11.100.7   | 0C:27:24:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111    |
| dexxx-acp08 | 10.11.100.8   | 00:78:88:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp09 | 10.11.100.9   | 40:F4:EC:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp10 | 10.11.100.10  | 44:AD:D9:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111    |
| dexxx-acp11 | 10.11.100.11  | 50:0F:80:C0:FF:EE | AIR-AP2802I-E-K9  | FDW11111111    |
| dexxx-acp12 | 10.11.100.12  | 74:A0:2F:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp13 | 10.11.100.13  | 0C:85:25:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp14 | 10.11.100.14  | 74:A0:2F:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp15 | 10.11.100.15  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp16 | 10.11.100.16  | C0:62:6B:C0:FF:EE | AIR-LAP1142N-E-K9 | FCZ11111111    |
| dexxx-acp17 | 10.11.100.17  | 0C:27:24:C0:FF:EE | AIR-CAP1602I-E-K9 | FGL11111111    |
| dexxx-acp18 | 10.11.100.18  | CC:16:7E:C0:FF:EE | AIR-CAP1702I-E-K9 | FCW11111111    |
| dexxx-acp19 | 10.11.100.19  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp20 | 10.11.100.20  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp21 | 10.11.100.21  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp22 | 10.11.100.22  | E4:AA:5D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp23 | 10.11.100.23  | F4:CF:E2:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp24 | 10.11.100.24  | 00:FE:C8:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp25 | 10.11.100.25  | F4:CF:E2:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp26 | 10.11.100.26  | 00:FE:C8:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp27 | 10.11.100.27  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp28 | 10.11.100.28  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |
| dexxx-acp29 | 10.11.100.29  | 64:F6:9D:C0:FF:EE | AIR-CAP1702I-E-K9 | FCZ11111111    |

Script:

#!/usr/bin/env python

# Need following pip packages
# - easysnmp
# - tabulate

# Checkout blog article to tool
# https://lanbugs.de/netzwerktechnik/hersteller/cisco/cli-tool-export-ap-inventory-from-an-cisco-wireless-lan-controller-wlc/


from easysnmp import Session
import argparse
from tabulate import tabulate
from operator import itemgetter


def main():
    ####
    # ARGS
    ####
    description = """
    Cisco AP WLC inventory grabber\nVersion 0.1\nWritten by Maximilian Thoma 2017
    """

    aparser = argparse.ArgumentParser(description=description)
    aparser.add_argument('-H', dest='host', help='WLC IP address', required=True)
    aparser.add_argument('-v', dest='snmp_version', help='SNMP version, valid are 2 or 3', required=True)
    aparser.add_argument('-C', dest='snmp_community', help='SNMP Community (only v2)')
    aparser.add_argument('-u', dest='snmp_user', help='SNMP user (v3)')
    aparser.add_argument('-A', dest='snmp_auth', help='SNMP auth password (v3)')
    aparser.add_argument('-a', dest='snmp_auth_method', help='SNMP auth method, valid are MD5 or SHA (v3)')
    aparser.add_argument('-X', dest='snmp_privacy', help='SNMP privacy password (v3)')
    aparser.add_argument('-x', dest='snmp_privacy_method', help='SNMP privacy method, valid are AES or DES (v3)')
    aparser.add_argument('-L', dest='snmp_security',
                         help='SNMP security level, valid are no_auth_or_privacy, auth_without_privacy or auth_with_privacy (v3)')
    aparser.add_argument('--csv', dest='csv', help='Result should be CSV', action='store_true')
    args = aparser.parse_args()

    ####
    # Setup SNMP connection
    ####

    if args.snmp_version == "2":
        try:
            snmp = Session(hostname=args.host, version=2, use_numeric=True)

        except Exception as e:
            print e

    if args.snmp_version == "3":
        try:
            snmp = Session(
                hostname=args.host,
                version=3,
                security_level=args.snmp_security,
                security_username=args.snmp_user,
                auth_protocol=args.snmp_auth_method,
                auth_password=args.snmp_auth,
                privacy_protocol=args.snmp_privacy_method,
                privacy_password=args.snmp_privacy,
                use_numeric=True
            )

        except Exception as e:
            print e

    ####
    # Init Data Buffer
    ####

    inventory = []
    longids = []

    ####
    # SNMP Walk AP Inventory
    ####

    ## Get longids for APs
    result_longids = snmp.walk(".1.3.6.1.4.1.14179.2.2.1.1.1")

    for rl in result_longids:
        longids.append(rl.oid.replace(".1.3.6.1.4.1.14179.2.2.1.1.1.", "") + "." + rl.oid_index)

    ## Collect informations

    for id in longids:
        # MAC
        result_mac = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.1." + id)
        mac = ":".join(["%02s" % hex(ord(m))[2:] for m in result_mac.value]).replace(' ', '0').upper()

        # Name
        name = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.3." + id).value

        # IP
        ip = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.19." + id).value

        # SN
        sn = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.17." + id).value

        # Model
        model = snmp.get(".1.3.6.1.4.1.14179.2.2.1.1.16." + id).value

        inventory.append([name, ip, mac, model, sn])

    ###
    # Sort table
    ####

    inv = sorted(inventory, key=itemgetter(0))

    ####
    # Result
    ####

    if args.csv is True:

        print 'Name;IP;MAC;Model;Serialnumber'

        for name, ip, mac, model, sn in inv:
            print '%s;%s;%s;%s;%s' % (name, ip, mac, model, sn)

    else:
        print tabulate(inv, headers=["Name", "IP", "MAC", "Model", "Serialnumber"], tablefmt="orgtbl")


if __name__ == "__main__":
    main()

 

 

PowerDNS MongoDB Backend

This a pipe backend for PowerDNS to MongoDB written in Python.

Requirements

  • pymongo python library
  • powerdns min. version 4.x

The MongoDB schema (for example DB: dns / Collection: records)

For SOA records:

{
"name":"example.org",
"type":"SOA",
"content":"",
"ttl": 300,
"primary": "ns1.example.org",
"mail": "admin.example.org",
"serial": 2018030311,
"refresh": 86400,
"retry": 7200,
"expire": 3600000,
"nttl": 3600 
}


For standard records:

{
"name":"www.example.org",
"type":"A",
"ttl": 300,
"content": "1.1.1.1"
}

Install PowerDNS

apt install pdns-server pdns-backend-pipe

Install pymongo Library

pip install pymongo

The Backend Script (for example: /opt/pdns/backend.py)

You can download it from GitHub – https://github.com/lanbugs/powerdns_mongodb_backend

Parts of the code based on: https://gist.github.com/sokratisg/10069682

#!/usr/bin/env python

import sys
from pymongo import MongoClient

# Config
mongo_host = "127.0.0.1"
mongo_port = 27017
mongo_db = "dns"
mongo_collation = "records"


class Lookup(object):
    ttl = 30

    def __init__(self, query):
        (_type, qname, qclass, qtype, _id, ip) = query
        self.has_result = False
        qname_lower = qname.lower()

        self.results = []

        self.results.append('LOG\t%s-%s-%s-%s-%s-%s' % (_type, qname, qclass, qtype, _id, ip))
        self.has_result = True

        client = MongoClient(mongo_host, mongo_port, connect=False)
        db = client[mongo_db]
        coll = db[mongo_collation]

        if qtype == "ANY":
            records = coll.find({"name": qname_lower})
        else:
            records = coll.find({"type": qtype, "name": qname_lower})

        if records:
            for record in records:
                if record['type'] == "SOA":
                    """
                    {
                     "name":"example.org",
                     "type":"SOA",
                     "content":"",
                     "ttl": 300,
                     "primary": "ns1.example.org",
                     "mail": "admin.example.org",
                     "serial": 2018030311,
                     "refresh": 86400,
                     "retry": 7200,
                     "expire": 3600000,
                     "nttl": 3600 
                    }
                    """
                    try:
                        self.results.append(
                            'DATA\t%s\t%s\t%s\t%s\t-1\t%s\t%s\t%s\t%s\t%s\t%s\t%s' % ( qname_lower,
                                                                                       qclass,
                                                                                       qtype,
                                                                                       record['ttl'],
                                                                                       record['primary'],
                                                                                       record['mail'],
                                                                                       record['serial'],
                                                                                       record['refresh'],
                                                                                       record['retry'],
                                                                                       record['expire'],
                                                                                       record['nttl']
                                                                                     ))
                        self.has_result = True
                    except:
                        self.results.append('LOG\t %s SOA Record currupt maybe fields are missing.' % qname_lower)
                else:
                    """
                    {
                     "name":"www.example.org",
                     "type":"A",
                     "ttl": 300,
                     "content": "1.1.1.1"
                    }
                    """
                    self.results.append('DATA\t%s\t%s\t%s\t%d\t-1\t%s' % (
                    qname_lower, qclass, record['type'], record['ttl'], record['content']))
                    self.has_result = True

    def str_result(self):
        if self.has_result:
            return '\n'.join(self.results)
        else:
            return ''


class DNSbackend(object):

    def __init__(self, filein, fileout):
        self.filein = filein
        self.fileout = fileout

        self._process_requests()

    def _fprint(self, message):
        self.fileout.write(message + '\n')
        self.fileout.flush()

    def _process_requests(self):
        first_time = True

        while 1:
            rawline = self.filein.readline()

            if rawline == '':
                return

            line = rawline.rstrip()

            if first_time:
                if line == 'HELO\t1':
                    self._fprint('OK\tPython backend ready.')
                else:
                    rawline = self.filein.readline()
                    sys.exit(1)
                first_time = False
            else:
                query = line.split('\t')
                if len(query) != 6:
                    self._fprint('LOG\tPowerDNS sent unparseable line')
                    self._fprint('FAIL')
                else:
                    lookup = Lookup(query)
                    if lookup.has_result:
                        pdns_result = lookup.str_result()
                        self._fprint(pdns_result)
                    self._fprint('END')


if __name__ == "__main__":
    infile = sys.stdin
    outfile = sys.stdout

    try:
        DNSbackend(infile, outfile)

    except:
        raise

Don`t forget to make the backend.py script executable to the world.

PowerDNS Configuration /etc/powernds/pdns.d/pdns.local.conf

# Here come the local changes the user made, like configuration of
# the several backends that exist.

launch=pipe
pipe-command=/opt/pdns/backend.py

Try…

Launch the PowerDNS service in monitor mode

root@pdnsdev:~# /etc/init.d/pdns monitor        
Jun 20 21:10:11 Reading random entropy from '/dev/urandom'
Jun 20 21:10:11 Loading '/usr/lib/x86_64-linux-gnu/pdns/libpipebackend.so'
Jun 20 21:10:11 [PIPEBackend] This is the pipe backend version 4.0.0-alpha2 reporting
Jun 20 21:10:11 Loading '/usr/lib/x86_64-linux-gnu/pdns/libbindbackend.so'
Jun 20 21:10:11 [bind2backend] This is the bind backend version 4.0.0-alpha2 reporting
Jun 20 21:10:11 This is a standalone pdns
Jun 20 21:10:11 UDP server bound to 0.0.0.0:53
Jun 20 21:10:11 Unable to enable timestamp reporting for socket
Jun 20 21:10:11 UDPv6 server bound to [::]:53
Jun 20 21:10:11 TCP server bound to 0.0.0.0:53
Jun 20 21:10:11 TCPv6 server bound to [::]:53
Jun 20 21:10:11 PowerDNS Authoritative Server 4.0.0-alpha2 (C) 2001-2016 PowerDNS.COM BV
Jun 20 21:10:11 Using 64-bits mode. Built using gcc 5.3.1 20160330.
Jun 20 21:10:11 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
Jun 20 21:10:11 Set effective group id to 118
Jun 20 21:10:11 Set effective user id to 112
Jun 20 21:10:11 Creating backend connection for TCP
% Jun 20 21:10:11 Backend launched with banner: OK      Python backend ready.
Jun 20 21:10:11 [bindbackend] Parsing 0 domain(s), will report when done
Jun 20 21:10:11 [bindbackend] Done parsing domains, 0 rejected, 0 new, 0 removed
Jun 20 21:10:11 About to create 3 backend threads for UDP
Jun 20 21:10:11 Backend launched with banner: OK        Python backend ready.
Jun 20 21:10:11 Backend launched with banner: OK        Python backend ready.
Jun 20 21:10:11 Done launching threads, ready to distribute questions
Jun 20 21:10:11 Backend launched with banner: OK        Python backend ready.

Do some querys

user@pdnsdev:~$ dig example.org @127.0.0.1 SOA            

; <<>> DiG 9.10.3-P4-Ubuntu <<>> example.org @127.0.0.1 SOA
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63891
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1680
;; QUESTION SECTION:
;example.org.                   IN      SOA

;; ANSWER SECTION:
example.org.            300     IN      SOA     ns1.example.org. admin.example.org. 2018030311 86400 7200 3600000 3600

;; Query time: 13 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Jun 20 21:20:22 CEST 2018
;; MSG SIZE  rcvd: 86

user@pdnsdev:~$ dig www.example.org @127.0.0.1

; <<>> DiG 9.10.3-P4-Ubuntu <<>> www.example.org @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 896
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1680
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        300     IN      A       1.1.1.1

;; Query time: 26 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Jun 20 21:20:44 CEST 2018
;; MSG SIZE  rcvd: 60

Have fun 🙂

 

 

Flask, uWSGI und Nginx auf Ubuntu 16.04

– Kurze Zusammenfassung, ausführlich siehe Quelle am Ende des Artikels –

Python Virtual Environment aufsetzen

pip install virtualenv

virtualenv projekt

source projekt/bin/activate

pip install uwsgi flask


Mini Flask Applikation projekt.py

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello world!"

WSGI Startfile wsgi.py

from projekt import app

if __name__ == "__main__":
    app.run()

Virtual Environment verlassen

deactivate

uWSGI Configfile erstellen

[uwsgi]
module = wsgi:app

master = true
processes = 5

socket = /opt/projekt/projekt.sock
chmod-socket = 660
vacuum = true

die-on-term = true

systemd File erstellen  /etc/systemd/system/projekt.service

[Unit]
Description=uWSGI instance to serve projekt
After=network.target

[Service]
User=projekt
Group=www-data
WorkingDirectory=/opt/projekt
Environment="PATH=/opt/projekt/bin"
ExecStart=/opt/projekt/bin/uwsgi --ini projekt.ini

[Install]
WantedBy=multi-user.target

uWSGI Projekt Service starten

sudo systemctl start projekt
sudo systemctl enable projekt

NGINX Config erstellen – /etc/nginx/sites-available/projekt

server {
    listen 80;
    server_name 10.10.10.10;

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/opt/projekt/projekt.sock;
    }
}

NGINX Config aktivieren

ln -s /etc/nginx/sites-available/projekt /etc/nginx/sites-enabled
systemctl restart nginx

Verifizieren NGINX und uWSGI

dev@dev1:~$ ps aux | grep projekt
projekt 17175  0.0  2.1  71216 21812 ?        Ss   19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17178  0.0  1.7  71216 17316 ?        S    19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17179  0.0  1.7  71216 17316 ?        S    19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17180  0.0  1.6  71216 16504 ?        S    19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17181  0.0  1.7  71472 17848 ?        S    19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17182  0.0  1.6  71216 16504 ?        S    19:26   0:00 /opt/projekt/bin/uwsgi --ini projekt.ini

dev@dev1:~$ tail -f /var/log/syslog
Jun 19 19:44:42 dev1 uwsgi[17175]: [pid: 17178|app: 0|req: 1/3] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:42 2018] GET / => generated 40 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 79 bytes (1 switches on core 0)
Jun 19 19:44:43 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 1/4] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:43 2018] GET / => generated 40 bytes in 4 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:44 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 2/5] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:44 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:45 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 3/6] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:45 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)

Quelle / ausführlicher Artikel: https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-ubuntu-16-04

 

MongoDB Authentication aktivieren

Per Default ist bei einer MongoDB Instanz keine Authentication aktiviert. Hier der kurze Weg.

Admin Account anlegen

devusr@testsystem:~# mongo
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.5
> use admin
switched to db admin
> db.createUser({user: "admin", pwd: "geheimes_passwort", roles: [{ role: "root", db: "admin" }]})
Successfully added user: {
        "user" : "admin",
        "roles" : [
                {
                        "role" : "root",
                        "db" : "admin"
                }
        ]
}
> quit()

Authentication in mongod.conf aktivieren (Ubuntu: /etc/mongod.conf)

...
# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1

security:
  authorization: enabled
...

Mongod neustarten

service mongod restart

Erster Test: User anlegen für Test DB

> use test
switched to db test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
2018-06-18T08:05:53.448+0200 E QUERY    [thread1] Error: couldn't add user: not authorized on test to execute command { createUser: "devuser", pwd: "xxx", roles: [ { role: "readWrite", db: "test" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 600000.0 }, $db: "test" } :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype.createUser@src/mongo/shell/db.js:1437:15
@(shell):1:1
>

Sieht gut aus, es geht nicht 😉

Anmelden und User für Test DB anlegen

use admin
db.auth("admin","geheimes_passwort")
1
use test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
Successfully added user: {
        "user" : "devuser",
        "roles" : [
                {
                        "role" : "readWrite",
                        "db" : "test"
                }
        ]
}
>

Authentifizierung mit pymongo in Python Code

>>> from pymongo import MongoClient
>>> uri = "mongodb://devuser:secure_pwd@localhost/test?authSource=test"                                     
>>> client = MongoClient(uri)
>>> db = client.test
>>> collection = db.foo
>>> collection.insert_one({"foo":"bar"})
<pymongo.results.InsertOneResult object at 0x7fa764f2a998>
>>> collection.find_one()
{u'_id': ObjectId('5b274db939d9c0683b47c0e2'), u'foo': u'bar'}
>>>

Quellen / Weitere Informationen:

Enable Authentication – https://docs.mongodb.com/manual/tutorial/enable-authentication/

Built-In Roles – https://docs.mongodb.com/manual/core/security-built-in-roles/

pymongo authentication – http://api.mongodb.com/python/current/examples/authentication.html

MongoDB und Python

Ein paar Notizen zu Python und MongoDB 🙂 MongoDB ist eine NoSQL Datenbank, weitere Infos -> Wikipedia 🙂

Aktuelle MongoDB Version installieren (auf Ubuntu 16.04)

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
apt install apt-transport-https
apt update
apt-get install -y mongodb-org

siehe auch: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Python Erweiterung pymongo für MongoDB installieren mit pip

python -m pip install pymongo

siehe auch: https://api.mongodb.com/python/current/

Verbindung zur MongoDB

from pymongo import MongoClient

client = MongoClient('localhost', 27017)

Connect zur Datenbank test

db = client.test

Collection myCollection in Datenbank test verbinden 

coll = db.myCollection

Einen Datensatz anlegen in einer Collection

post = {"author":"Fritz Fuchs", "book":"Hasenjagd"}
>>> coll.insert_one(post)
<pymongo.results.InsertOneResult object at 0x7fd0136ce3f8>

Mehrere Datensätze in einer Collection anlegen

>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7"}]
>>> result = coll.insert_many(posts)
>>> result.inserted_ids
[ObjectId('5b259b2c39d9c04f7b43af01'), ObjectId('5b259b2c39d9c04f7b43af02'), ObjectId('5b259b2c39d9c04f7b43af03')]

Datensatz suchen

>>> coll.find_one({"author":"Fritz Fuchs"}) 
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}

Datensatz mit Regex suchen

>>> coll.find_one({"author":{"$regex": "^Fritz.*"}})     
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}

Datensatz mit ObjectId abrufen

>>> from bson.objectid import ObjectId
>>> coll.find_one({"_id":ObjectId("5b25998139d9c04f7b43aefe")})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}

Mehrere Datensätze abrufen z.B. mit Regex

>>> for post in coll.find({"author":{"$regex": "^Fritz.*"}}):
...     print post
... 
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a8939d9c04f7b43aeff'), u'book': u'Hasenjagd 2', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a9039d9c04f7b43af00'), u'book': u'Hasenjagd 3', u'author': u'Fritz Fuchs'}

Index erzeugen für Collection z.B. Username ist Unique

>>> import pymongo
>>> db.users.create_index([('user_id', pymongo.ASCENDING)], unique=True)
u'user_id_1'
>>> new_users = [{'user_id':'max'},{'user_id':'fritz'}]     
>>> db.users.insert_many(new_users)
<pymongo.results.InsertManyResult object at 0x7fd0136cee18>                   
>>> new_user = {'user_id':'max'}      
>>> db.users.insert_one(new_user) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 683, in insert_one
    session=session),
  File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 599, in _insert
    bypass_doc_val, session)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 580, in _insert_one
    _check_write_command_response(result)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 207, in _check_write_command_response
    _raise_last_write_error(write_errors)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 188, in _raise_last_write_error
    raise DuplicateKeyError(error.get("errmsg"), 11000, error)
pymongo.errors.DuplicateKeyError: E11000 duplicate key error collection: test.users index: user_id_1 dup key: { : "max" }

ObjectId als String

>>> new_user = {'user_id':'maxx'} 
>>> result = db.users.insert_one(new_user) 
>>> result
<pymongo.results.InsertOneResult object at 0x7fd0122410e0>
>>> result.inserted_id
ObjectId('5b259ead39d9c04f7b43af07')
>>> str(result.inserted_id)   
'5b259ead39d9c04f7b43af07'

String ObjectId zu ObjectId Object wandeln und für Suche verwenden

>>> from bson.objectid import ObjectId
>>> str_obj = '5b259ead39d9c04f7b43af07'
>>> users.find_one({'_id':ObjectId(str_obj)})
{u'_id': ObjectId('5b259ead39d9c04f7b43af07'), u'user_id': u'maxx'}

Datensatz löschen

>>> users.delete_one({'_id':ObjectId(str_obj)})    
<pymongo.results.DeleteResult object at 0x7fd0136cee18>

Datensatz aktualisieren

>>> new_user = {'user_id':'maxx', 'name':'Hans Wurst'}
>>> users.insert_one(new_user)
>>> change = {'name': 'Fritz Fritz'}
>>> users.update_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')}, {'$set':change} )
<pymongo.results.UpdateResult object at 0x7fd0136ced40>
>>> users.find_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')})                                                            
{u'_id': ObjectId('5b25a14f39d9c04f7b43af08'), u'user_id': u'maxx', u'name': u'Fritz Fritz'}

Bereich bei Suche

>>> client = MongoClient()         
>>> db = client.huu
>>> c = db.test
>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5", "boo":1}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6", "boo":5}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7", "boo":10}] 
>>> c.insert_many(posts)

>>> for post in c.find({"boo": {"$lt": 6}}).sort("author"):
...     print post
... 
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b54'), u'book': u'Hasenjagd 5', u'boo': 1}
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}

>>> for post in c.find({"boo": {"$gt": 6}}).sort("author"): 
...     print post
... 
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b56'), u'book': u'Hasenjagd 7', u'boo': 10}

>>> for post in c.find({"boo": {"$lt": 6, "$gt": 3}}).sort("author"): 
...     print post
... 
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}

 

Quelle / weitere Beispiele: http://api.mongodb.com/python/current/tutorial.html

 

Passwort Hashing in Python

Hier die einfachste Variante um in sicheres Passwort Hashing in Python umzusetzen.

Dazu wird die passlib verwendet. Diese kann per pip installiert werden, passlib ist für Python 2.x und 3.x kompatibel.

pip install passlib

Passwort hashen:

>>> # passlib laden
>>> from passlib.hash import pbkdf2_sha256
>>>
>>> # das passwort zum hashen 
>>> password = "EinsuperGeheimesPasswort"
>>> 
>>> # ein falsches passwort zum testen
>>> password2 = "FalschesPasswort"
>>> 
>>> hash = pbkdf2_sha256.hash(password)
>>> # hash ausgeben
>>> hash
'$pbkdf2-sha256$29000$IQSgdE6p1VoL4fwfQwjBWA$9sy/3NJmX1jP.3kYwgmG96zVpBxoVA5yKA6pB.T5Mrw'

Passwort verifizieren:

>>> # richtiges passwort
>>> pbkdf2_sha256.verify(password, hash)
True
>>> # falsches passwort
>>> pbkdf2_sha256.verify(password2, hash)
False
>>> 

Wem pbkdf2_sha256 zu kompliziert ist kann das auch beim Import mit einem Alias versehen.

>>> from passlib.hash import pbkdf2_sha256 as pwhash
>>> hash = pwhash.hash(password)
>>> hash
'$pbkdf2-sha256$29000$Z2zNWYsxBiDEmFPK2XsvZQ$WQ8Urv1d4AmBV4v.vFV01uHeZZ7ya52VuYbLkEHEsWE'
>>> pwhash.verify(password, hash)
True

 

Weitere Infos:

https://passlib.readthedocs.io/en/stable/

Palo Alto Networks – XML API verwenden mit Python

Die Palo Alto Jungs haben eine exzellente XML API implementiert die für die Firewalls als auch Panorama usw. funktioniert.

Um auf die API Dokumentation zuzugreifen einfach https://<firewall_ip>/api aufrufen.

Der User muss API Rechte haben.

Hier ein Beispiel zum Auslesen der ARP Tabelle.

Ich habe den SSL Zertifikatscheck in Python deaktiviert in meiner Lab Umgebung. Bitte nicht in Produktionsumgebungen verwenden!

#!/usr/bin/env python

# required pip packages: lxml, beautifulsoup4, tabulate

from bs4 import BeautifulSoup as BS
import urllib2
import ssl
import urllib
from tabulate import tabulate

username = "api_test"
password = "supersecurepassword"
firewall = "192.168.111.1"

#####################
# SSL ignore Cert Check (Lab Environment only!)

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

#####################
# Req function

def req_api(command, key):
    url = "https://%s/api/?key=%s&type=op&cmd=" % (firewall, key) + urllib.quote_plus(command)
    res = urllib2.urlopen(url, context=ctx)
    return res.read()

#####################
# API Key request


req_api_url = "https://%s/api/?type=keygen&user=%s&password=%s" % (firewall, username, password)
res_api_key = urllib2.urlopen(req_api_url, context=ctx)
soup = BS(res_api_key.read(), "lxml")
key = soup.find('key').text

#####################
# Get arp entrys

soup = BS(req_api("<show><arp><entry name = 'all'/></arp></show>", key), "lxml")

arp_buffer = []

for e in soup("entry"):
    arp_buffer.append([e.status.text, e.ip.text, e.ttl.text, e.interface.text, e.port.text, e.mac.text])

print "#" * 120 + "\n> ARP Cache Count: %s\n" %(len(arp_buffer)) + "#" * 120
print tabulate(arp_buffer, headers=['Status', 'IP', 'TTL', 'Interface', 'Port', 'MAC'], tablefmt='orgtbl')

Ergebnis:

>.\palo_xml.py
########################################################################################################################
> ARP Cache Count: 14
########################################################################################################################
| Status   | IP            |   TTL | Interface   | Port        | MAC               |
|----------+---------------+-------+-------------+-------------+-------------------|
| c        | 192.168.0.1   |  1560 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.2   |  1058 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.5   |  1797 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.11  |  1779 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.12  |  1581 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.13  |  1787 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.30  |  1055 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.33  |  1542 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.40  |  1121 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.42  |   173 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.44  |  1799 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.45  |   910 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.52  |  1788 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |
| c        | 192.168.0.57  |  1671 | ethernet1/3 | ethernet1/3 | c0:ff:ee:c0:ff:ee |

 

Python: Arbeiten mit YAML Files

YAML ist eine vereinfachte Auszeichnungssprache zur Datenserialisierung. Das Format ist einfach gehalten und ist eine schöne alternative zu XML, JSON usw. Ist auch ein super Format für Configfiles als alternative zu INI Files.

Einfaches YAML File:

variable_str: value
variable_num: 33

list:
  - a
  - b
  - c
  - d

dictionary:
  - list1:
    - a
    - b
    - c
  - list2:
    - a
    - b
    - c

block: |
    block ..................
    block ..................
    block ..................
    block ..................
    block ..................
    block ..................
    block ..................

Testscript:

import yaml
from pprint import pprint

with open("test3.yaml", "r") as f:
    x = yaml.load(f.read())

print "RAW Data:"
pprint(x)
print "----"
print "Block:"
print x['block']
print "Dictionary:"
print x['dictionary']
print "List:"
print x['list']
print "Var String:"
print x['variable_str']
print "Var Num:"
print x['variable_num']

Ausgabe:

RAW Data:
{'block': 'block ..................\nblock ..................\nblock ..................\nblock ..................\nblock ..................\nblock ..................\nblock ..................\n',
 'dictionary': [{'list1': ['a', 'b', 'c']}, {'list2': ['a', 'b', 'c']}],
 'list': ['a', 'b', 'c', 'd'],
 'variable_num': 33,
 'variable_str': 'value'}
----
Block:
block ..................
block ..................
block ..................
block ..................
block ..................
block ..................
block ..................

Dictionary:
[{'list1': ['a', 'b', 'c']}, {'list2': ['a', 'b', 'c']}]
List:
['a', 'b', 'c', 'd']
Var String:
value
Var Num:
33

Python Dictionarys / Listen etc. „dumpen“ in ein YAML File:

import yaml

t = {
    "bubu": {
            "a": "a",
            "b": "b",
            "c": "c",
            "d": "d",
            "e": "e",
            },
    "baba": {
            "a": "a",
            "b": "b",
            "c": "c",
            "d": "d",
            "e": "e",
            }
    }

with open('foobar.yaml', 'w') as f:
    yaml.dump(t, f, default_flow_style=False)

Ausgabe:

bubu:
  a: a
  b: b
  c: c
  d: d
  e: e
baba:
  a: a
  b: b
  c: c
  d: d
  e: e

 

 

Python: Entfernen von Steuerzeichen in ASCII Strings

Bin heute wieder mal über ein Problem mit ASCII Strings gestolpert. Im String war ein CTRL+C welches als \x03 in der ASCII Tabelle ist. Ich bin auf eine Seite gestoßen welche für alle möglichen Script und Programmiersprachen Codesnippets bereitstellt um Steuerzeichen zu entfernen.

ASCII Tabelle bei Wikipedia: https://de.wikipedia.org/wiki/American_Standard_Code_for_Information_Interchange

Hier für Python:

stripped = lambda s: "".join(i for i in s if 31 < ord(i) < 127)
 
print(stripped("\ba\x00b\n\rc\fd\xc3"))

Output:

abcd

Quelle: https://rosettacode.org/wiki/Strip_control_codes_and_extended_characters_from_a_string

Alternativ kann man auch einzelne ASCII Codes ersetzen mit der Regex Library

Beispiel:

import re

data = "ab\x03cd"
data_clean = re.sub(r"\x03","",data)

print data

abcd

Viel Spaß 🙂

Cisco/Python: Backup der Konfiguration bei write Event auf externen Server

Diverse Cisco Geräte können bei einem write Event die Konfiguration an einen anderen Server z.B. über HTTP pushen.

Cisco Config:

archive
 path http://1.2.3.4/cisco_config/put/$h-$t
 write-memory

Apache /etc/httpd/conf.d/zzz_cisco_backup.conf:

WSGIDaemonProcess cisco_backup user=apache group=apache threads=10
WSGIPythonPath /opt/cisco_backup/web_root
WSGIScriptAlias /cisco_backup /opt/cisco_backup/web_root/cisco_backup.wsgi

<Directory /opt/cisco_backup/web_root>
WSGIProcessGroup cisco_backup
WSGIApplicationGroup %{GLOBAL}
WSGIScriptReloading On
Order deny,allow
Allow from all

<Files cisco_backup.py>
Require all granted
</Files>
<Files cisco_backup.wsgi>
Require all granted
</Files>

</Directory>

cisco_backup.wsgi File:

import sys

sys.path.append("/opt/cisco_backup/web_root")

from cisco_backup import app as application

cisco_backup.py File:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from flask import Flask
from flask import request

app = Flask(__name__)

@app.route("/put/<cfg>", methods=['PUT'])
def get_config(cfg):
   with open('/opt/cisco_config/incoming_configs/%s' % cfg, "wb") as f:
      f.write(request.data)
   return "ok"

if __name__ == "__main__":
    app.run()

Viel Spaß 😉

Python: Snippet – E-Mail versenden, alternative zu Mailer

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import smtplib
from email.mime.text import MIMEText


def postmaster(mfrom, mto, msubject, message, smtphost):

    msg = MIMEText(message.encode("utf-8"))
    msg['Subject'] = msubject
    msg['From'] = mfrom
    msg['To'] = mto

    s = smtplib.SMTP(smtphost)
    s.sendmail(msg['From'], msg['To'], msg.as_string())
    s.quit()

 

Python: Snippet: SSH shell on Cisco devices

Mit dem Snippet können Kommandos auf einer Cisco Shell via SSH ausgeführt werden.

#!/usr/bin/env python

import paramiko
import sys


def send_string_and_wait_for_string(command, wait_string, should_print):
    shell.send(command)
  
    receive_buffer = ""

    while not wait_string in receive_buffer:
        receive_buffer += shell.recv(1024)

    if should_print:
        print receive_buffer

    return receive_buffer

client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect("10.62.62.10", username="testuser", password="testpasswd", look_for_keys=False, allow_agent=False)

shell = client.invoke_shell()
send_string_and_wait_for_string("", "#", False)
send_string_and_wait_for_string("terminal length 0\n", "#", False)
output=send_string_and_wait_for_string("show logging\n", "#", False)
print output
client.close()

Mehr Infos / Quelle: http://blog.timmattison.com/archives/2014/06/25/automating-cisco-switch-interactions/

Python: Snippet/Experiment – Syslog Server mit globalen und Host Filtern

Der Code ist nicht fertig und war mal ein Labor Versuch. Es lassen sich globale und Host Filter setzen wo diese zutreffen werden die Logs in ein extra File geschrieben.

Config file:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Config definition


class CFG:

    def __init__(self):

        # Path for logfiles
        self.syslogpath = "/home/mthoma/_dev/syslog/log/"

        # Listner Port
        self.port = 3702

        # Listner address
        self.host = "0.0.0.0"

        # Global Filter
        self.global_filter = {
            "filter": [
                ".*FOOBAR.*",
                ".*COFFEE.*"
            ]
        }

        # Host Filter
        self.host_filter = {
            "10.201.11.33": {
                "filter": [
                    ".*MACFLAP.*",
                    ".*BUBU.*",
                ]
            },
        }

Syslog Server:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Load config class
from config import CFG

# Load common classes
import re
import logging
import SocketServer
import socket
import os

# Load configuration file
C = CFG()

formatter = logging.Formatter('%(message)s')

def setup_logger(name, log_file, level=logging.INFO):
    handler = logging.FileHandler(log_file)
    handler.setFormatter(formatter)
    
    logger = logging.getLogger(name)
    logger.setLevel(level)
    logger.addHandler(handler)
    
    return logger


class SyslogUDPHandler(SocketServer.BaseRequestHandler):

    def handle(self):
        data = bytes.decode(self.request[0].strip())
        sockets = self.request[1]

        ip = str(self.client_address[0])
        
    # Try to resolve reverse record via DNS
        try:
            name, alias, addresslist = socket.gethostbyaddr(ip)
        except:
            name = ip
        
    # Set path
        path = C.syslogpath+name+"/"
        
    # Create path if not exist
        try:
            os.stat(path)
        except:
            os.mkdir(path)
        
        logger = setup_logger('normal_log', path+"log")
        logger.info(str(data))
        
        logger_sp = setup_logger('special_log', path+"spec")
        
        if ip in C.host_filter:
            filters = options['filter'] + C.global_filter['filter']
            filter_join = "|".join(filters)
            
            if re.match(r"%s" % filter_join, str(data)):
                logger_sp.info(str(data))
        else:
            filters = C.global_filter['filter']
            filter_join = "|".join(filters)
            
            if re.match(r"%s" % filter_join, str(data)):
                logger_sp.info(str(data))
                
        
        
        print "%s : " % self.client_address[0], str(data)

        logging.info(str(data))



if __name__ == "__main__":

    try:
        server = SocketServer.UDPServer((C.host,C.port), SyslogUDPHandler)
        server.serve_forever(poll_interval=0.5)

    except (IOError, SystemExit):
        raise

    except KeyboardInterrupt:
        print "Crtl+C Pressed. Shutting down."

 

Python: Snippet Multiprocessing mit Ergebnis

Beispiel für Parallelisierung von Jobs mit Ergebnis welche als Liste zurückgegeben werden.

#!/usr/bin/env python
# -*- encoding: utf-8; py-indent-offset: 4 -*-

import os
from multiprocessing import Pool


def worker(job):
    x, y = job
    result = x ** y
    return os.getpid(), result
  
if __name__ == '__main__':
    jobs = [(1, 2), (3, 4), (5, 6), (11, 12), (13, 14), (15, 16), (21, 22), (23, 24), (25, 26)]
    
    result_buffer = []
  
    pool = Pool(processes=5)
    
    for job in jobs:
        result_buffer.append(pool.apply_async(worker, args=(job,)))
    
    pool.close()
    pool.join()
  
    results = [r.get() for r in result_buffer]

    print results
  
    for pid, result in results:
        print "working pid was: %s" % pid
        print "result is: %s" % result
        print "---"

Beispiel Ergebnis:

$python mp_with_result.py

[(7992, 1), (7992, 81), (7992, 15625), (7992, 3138428376721L), (7992, 3937376385699289L), (7992, 6568408355712890625L), (7992, 122694327386105632949003612841L), (7992, 480250763996501976790165756943041L), (7992, 2220446049250313080847263336181640625L)]
working pid was: 7992
result is: 1
---
working pid was: 7992
result is: 81
---
working pid was: 7992
result is: 15625
---
working pid was: 7992
result is: 3138428376721
---
working pid was: 7992
result is: 3937376385699289
---
working pid was: 7992
result is: 6568408355712890625
---
working pid was: 7992
result is: 122694327386105632949003612841
---
working pid was: 7992
result is: 480250763996501976790165756943041
---
working pid was: 7992
result is: 2220446049250313080847263336181640625
---

 

Python: Experiment/Snippet – Komprimieren und löschen von Logfiles nach X Tagen

Ein Ansatz für Logverzeichnisse im Format /log/<yyyy>/<mm>/<dd>/<div. logsfiles>

#!/usr/bin/env python

import gzip
import shutil
import os
import datetime
import time

#############################################
# Config
#############################################

# Path of Logfiles
# Structure is /opt/log/<YYYY>/<MM>/<DD>/
gpath='/opt/log/'

# hold logs for x days
hold_time=180



#############################################

def get_immediate_subdirectories(a_dir):
    return [name for name in os.listdir(a_dir) if os.path.isdir(os.path.join(a_dir, name))]

def delete_files(f):
    # delete file if older than hold time
    nowx = time.time()

    for file in os.listdir(f):    
        if os.stat(f+file).st_mtime < nowx - hold_time * 86400:
            f_path = f+file
            print "delete %s " % f_path
            os.remove(f_path)
            
    try:
        os.rmdir(f)
    except:
        pass
            
    
    
def compress_files(lpath):
    # Compress files
    print "Working on: " + lpath
    obj = os.listdir(lpath)
    for f in obj:
        if os.path.isfile(lpath+f) and ".gz" not in f:
            with open(lpath+f,'rb') as f_in:
                with gzip.open(lpath+f+".gz",'wb') as f_out:
                    shutil.copyfileobj(f_in, f_out)
                    os.remove(lpath+f)
            
#compress everything which ist older than now

now = datetime.datetime.now()
years = get_immediate_subdirectories(gpath)

for year in years:

    # delete empty directories
    if not os.listdir(gpath+year):
        os.rmdir(gpath+year)
    else:

        months = get_immediate_subdirectories(gpath+year)
    
        for month in months:

            # delete empty directories
            if not os.listdir(gpath+year+"/"+month):
                os.rmdir(gpath+year+"/"+month)

            else:
                days = get_immediate_subdirectories(gpath+year+"/"+month)
                        
                # Remove current day from compressing & cleaning
                if month == str(now.month) and year == str(now.year):

                    if len(str(now.day)):
                        now_day = "0%s" % now.day
                    else:
                        now_day = str(now.day)

                    days.remove(now_day)
        
                for day in days:
                    # delete empty directories
                    if not os.listdir(gpath+year+"/"+month+"/"+day+"/"):
                        os.rmdir(gpath+year+"/"+month+"/"+day+"/")
                    else:
                        # compress all files in folder
                        compress_files(gpath+year+"/"+month+"/"+day+"/")
                        
                        # delete old files
                        delete_files(gpath+year+"/"+month+"/"+day+"/")

 

Python: Snippet – Kaputten UTF-8 String reparieren

Ich habe aus der Datenbank einen String zurückbekommen der UTF-8 war aber falsch kodiert zurückgegeben wurde. So wurde aus Geschäftsstelle -> Gesch├ñftsstelle

Folgendes Snippet kann einen kaputten UTF-8 String neu auf UTF-8 kodieren:

name_kaputt = 'Gesch\xc3\xa4ftsstelle'

name = ''.join(chr(ord(c)) for c in name_kaputt).decode("utf-8")

print name_kaputt
print name

Ergebnis:

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> name_kaputt = 'Gesch\xc3\xa4ftsstelle'
>>> name = ''.join(chr(ord(c)) for c in name_kaputt).decode("utf-8")
>>> print name_kaputt
Geschäftsstelle
>>> print name
Geschäftsstelle