Mit diesem Befehl lassen sich die installieren Pakete in eine Liste exportieren.
sudo dpkg-query -f '${binary:Package}\n' -W > packages_list.txt
Mit xargs lässt sich apt install z.B. auf einem neuen System füttern.
sudo xargs -a packages_list.txt apt install
We are using an event-handler with on-startup-config write to push the configuration to our langley service which push the config in the central GIT repository.
To execute an bash command in an different vrf context use „ip netns exec ns-<VRF>“ before the command. See the example:
event-handler ARCHIVE_LANGLEY trigger on-startup-config action bash sudo ip netns exec ns-MANAGEMENT curl https://10.1.1.1/langley/put/$(hostname)/1 --upload-file /mnt/flash/startup-config
Gitolite ist ein einfaches Script um einen GIT Repo Server aufzusetzen. Zusammen mit Ansible nutze ich es um z.B. Zonenfiles auf Bind9 Server zu verteilen. Das werde ich in diesem Beispiel zeigen. Ansible und Gitolite laufen auf dem gleichen Server.
apt install gitolite3
Während der Installation wird nach dem Public SSH Key des Admins gefragt. Dieser Key ist später berechtigt das gitolite-admin Repo welches die Konfiguration der Repos beinhaltet zu benutzen.
git clone gitolite3@10.1.1.1:gitolite-admin
gitolite-admin/conf/gitolite.conf
repo gitolite-admin
RW+ = admin
repo testing
RW+ = @all
repo dns
option hook.post-update = dns-deploy
RW+ = admin
Hier wurde bereits ein post-update Hook definiert. Es müssen aber noch weitere Einstellungen angepasst werden damit dieser funktioniert.
Neue User legt man an in dem Verzeichnis gitolite-admin/keydir <username>.pub mit dem entsprechenden SSH Key anlegt. Die Usernamen können dann in der gitolite.conf verwendet werden. Ausführliche Doku zu gitolite -> http://gitolite.com/gitolite/index.html
apt install software-properties-common apt-add-repository ppa:ansible/ansible apt update apt install ansible
Ich verwende auf den Zielsystemen eigene User um Dinge per Ansible zu deployen. Im Beispiel heist der „vader“.
[dns-servers] ns1.example.com ns2.example.com
--- ansible_ssh_private_key_file: /etc/ansible/sshkey/id_rsa ansible_ssh_user: vader ansible_sudo_pass: lukeiamyourfather
mkdir -p /etc/ansible/sshkey ssh-keygen -t rsa -f /etC/ansible/sshkey/id_rsa
Den SSH Key ohne Passwort ablegen.
Anpassen der Verzeichnis Rechte:
chmod 600 /etc/ansible/sshkey chmod 600 /etc/ansible/sshkey/id_rsa chmod 644 /etc/ansible/sshkey/id_rsa.pub chown root:root -R /etc/ansible/sshkey
Auf den Zielsystemen muss jeweils der User vader mit dem Passwort eingerichtet werden und der Public Key des SSH Keys muss als authorized_key hinterlegt werden. Der User muss in der sudo Gruppe sein.
Bei folgenden Zeilen Kommentierung entfernen.
LOCAL_CODE => "$ENV{HOME}/local",
# allow repo-specific hooks to be added
'repo-specific-hooks',
mkdir -p /var/lib/gitolite3/local/hooks/repo-specific
sudo /opt/ansible/dns.sh
Owner und Rechte ändern
chown gitolite3:gitolite3 dns-deploy chmod +x dns-deploy
Das Hook Script liegt in /opt/ansible:
mkdir -p /opt/ansible
#!/bin/bash
REPO=/opt/ansible/dns
if [ ! -d $REPO ]
then
ssh-agent bash -c 'ssh-add /etc/ansible/sshkey/id_rsa; git clone gitolite3@10.1.1.1:dns'
else
cd $REPO
ssh-agent bash -c 'ssh-add /etc/ansible/sshkey/id_rsa; git pull'
fi
cd $REPO
ansible-playbook dns.yaml
# End
Das Hook Script klont das Repo dns nach /opt/ansible/dns und führt das ansible playbook aus dem Repo aus.
Anpassen der Rechte
chown root:root /opt/ansible/dns.sh chmod 700 /opt/ansible/dns.sh
Den gitolite3 User berechtigen das Script ohne Passwort per Sudo auszuführen
gitolite3 ALL = (root) NOPASSWD: /opt/ansible/dns.sh
git clone gitolite3@10.1.1.1:dns cd dns
---
- hosts: dns-servers
tasks:
- name: create or update named.conf.local
become: true
register: named
template:
src: /opt/ansible/dns/conf/named.conf.local.j2
dest: /etc/bind/named.conf.local
vars:
zones:
- lanbugs.de
- name: create or update zone files
become: true
register: zones
template: src={{ item }} dest=/etc/bind/{{ item | basename | regex_replace('\.j2','') }} owner=root group=root mode=0644
with_fileglob:
- /opt/ansible/dns/conf/db.*.j2
- name: restart bind
become: true
service:
name: bind9
state: restarted
when: zones.changed or named.changed
mkdir conf
{% for zone in zones %}
zone "{{ zone }}" {
type master;
file "/etc/bind/db.{{ zone}}";
};
{% endfor %}
; BIND db file for lanbugs.de
$TTL 86400
{% include "zone_header.j2" %}
@ 1800 IN TXT "v=spf1 a mx ptr ~all"
test 1800 IN A 1.2.3.4
$ORIGIN lanbugs.de.
@ IN SOA ns1.example.com. admin.example.com. (
2018111101 ; serial number YYMMDDNN
28800 ; Refresh
7200 ; Retry
864000 ; Expire
86400 ; Min TTL
)
@ 1800 IN NS ns1.example.com.
@ 1800 IN NS ns2.example.com.
@ 1800 IN MX 10 mail1.example.com.
@ 1800 IN MX 20 mail2.example.com.
git add -A git commit -m "initial commit" git push Counting objects: 3, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 311 bytes | 311.00 KiB/s, done. Total 3 (delta 1), reused 0 (delta 0) remote: Identity added: /etc/ansible/sshkey/id_rsa (Ansible vader) remote: From 10.1.1.1:dns remote: 63bf7b0..63d2dc5 master -> origin/master remote: Updating 63bf7b0..63d2dc5 remote: Fast-forward remote: conf/db.lanbugs.de.j2 | 1 - remote: 1 file changed, 1 deletion(-) remote: remote: PLAY [dns-servers] ************************************************************* remote: remote: TASK [Gathering Facts] ********************************************************* remote: ok: [ns1.example.com] remote: ok: [ns2.example.com] remote: remote: TASK [create or update named.conf.local] *************************************** remote: ok: [ns1.example.com] remote: ok: [ns2.example.com] remote: remote: TASK [create or update zone files] ********************************************* remote: changed: [ns1.example.com] => (item=/opt/ansible/dns/conf/db.lanbugs.de.j2) remote: changed: [ns2.example.comc] => (item=/opt/ansible/dns/conf/db.lanbugs.de.j2) remote: remote: TASK [restart bind] ************************************************************ remote: changed: [ns1.example.com] remote: changed: [ns2.example.com] remote: remote: PLAY RECAP ********************************************************************* remote: ns1.example.com : ok=4 changed=2 unreachable=0 failed=0 remote: ns2.example.com : ok=4 changed=2 unreachable=0 failed=0 remote: To 10.1.1.1:/dns 63bf7b0..63d2dc5 master -> master
Have fun 🙂
With this script you can parse the postfix databases / lookuptables which are in berkley database format.
Das Skript verwende ich um aus der Plesk Datenbank alle gültigen E-Mail Adressen für einen Frondend Mailserver zu exportieren.
– Kurze Zusammenfassung, ausführlich siehe Quelle am Ende des Artikels –
Python Virtual Environment aufsetzen
pip install virtualenv virtualenv projekt source projekt/bin/activate pip install uwsgi flask
Mini Flask Applikation projekt.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello world!"
WSGI Startfile wsgi.py
from projekt import app
if __name__ == "__main__":
app.run()
Virtual Environment verlassen
deactivate
uWSGI Configfile erstellen
[uwsgi] module = wsgi:app master = true processes = 5 socket = /opt/projekt/projekt.sock chmod-socket = 660 vacuum = true die-on-term = true
systemd File erstellen /etc/systemd/system/projekt.service
[Unit] Description=uWSGI instance to serve projekt After=network.target [Service] User=projekt Group=www-data WorkingDirectory=/opt/projekt Environment="PATH=/opt/projekt/bin" ExecStart=/opt/projekt/bin/uwsgi --ini projekt.ini [Install] WantedBy=multi-user.target
uWSGI Projekt Service starten
sudo systemctl start projekt sudo systemctl enable projekt
NGINX Config erstellen – /etc/nginx/sites-available/projekt
server {
listen 80;
server_name 10.10.10.10;
location / {
include uwsgi_params;
uwsgi_pass unix:/opt/projekt/projekt.sock;
}
}
NGINX Config aktivieren
ln -s /etc/nginx/sites-available/projekt /etc/nginx/sites-enabled systemctl restart nginx
Verifizieren NGINX und uWSGI
dev@dev1:~$ ps aux | grep projekt
projekt 17175 0.0 2.1 71216 21812 ? Ss 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17178 0.0 1.7 71216 17316 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17179 0.0 1.7 71216 17316 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17180 0.0 1.6 71216 16504 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17181 0.0 1.7 71472 17848 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
projekt 17182 0.0 1.6 71216 16504 ? S 19:26 0:00 /opt/projekt/bin/uwsgi --ini projekt.ini
dev@dev1:~$ tail -f /var/log/syslog
Jun 19 19:44:42 dev1 uwsgi[17175]: [pid: 17178|app: 0|req: 1/3] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:42 2018] GET / => generated 40 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 79 bytes (1 switches on core 0)
Jun 19 19:44:43 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 1/4] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:43 2018] GET / => generated 40 bytes in 4 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:44 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 2/5] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:44 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Jun 19 19:44:45 dev1 uwsgi[17175]: [pid: 17179|app: 0|req: 3/6] 10.10.10.1 () {46 vars in 834 bytes} [Wed Jun 19 19:44:45 2018] GET / => generated 40 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 79 bytes (2 switches on core 0)
Quelle / ausführlicher Artikel: https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-ubuntu-16-04
Per Default ist bei einer MongoDB Instanz keine Authentication aktiviert. Hier der kurze Weg.
Admin Account anlegen
devusr@testsystem:~# mongo
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.5
> use admin
switched to db admin
> db.createUser({user: "admin", pwd: "geheimes_passwort", roles: [{ role: "root", db: "admin" }]})
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
> quit()
Authentication in mongod.conf aktivieren (Ubuntu: /etc/mongod.conf)
... # network interfaces net: port: 27017 bindIp: 127.0.0.1 security: authorization: enabled ...
Mongod neustarten
service mongod restart
Erster Test: User anlegen für Test DB
> use test
switched to db test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
2018-06-18T08:05:53.448+0200 E QUERY [thread1] Error: couldn't add user: not authorized on test to execute command { createUser: "devuser", pwd: "xxx", roles: [ { role: "readWrite", db: "test" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 600000.0 }, $db: "test" } :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype.createUser@src/mongo/shell/db.js:1437:15
@(shell):1:1
>
Sieht gut aus, es geht nicht 😉
Anmelden und User für Test DB anlegen
use admin
db.auth("admin","geheimes_passwort")
1
use test
> db.createUser({user: "devuser", pwd: "secure_pwd", roles: [{ role: "readWrite", db: "test" }]})
Successfully added user: {
"user" : "devuser",
"roles" : [
{
"role" : "readWrite",
"db" : "test"
}
]
}
>
Authentifizierung mit pymongo in Python Code
>>> from pymongo import MongoClient
>>> uri = "mongodb://devuser:secure_pwd@localhost/test?authSource=test"
>>> client = MongoClient(uri)
>>> db = client.test
>>> collection = db.foo
>>> collection.insert_one({"foo":"bar"})
<pymongo.results.InsertOneResult object at 0x7fa764f2a998>
>>> collection.find_one()
{u'_id': ObjectId('5b274db939d9c0683b47c0e2'), u'foo': u'bar'}
>>>
Quellen / Weitere Informationen:
Enable Authentication – https://docs.mongodb.com/manual/tutorial/enable-authentication/
Built-In Roles – https://docs.mongodb.com/manual/core/security-built-in-roles/
pymongo authentication – http://api.mongodb.com/python/current/examples/authentication.html
Ein paar Notizen zu Python und MongoDB 🙂 MongoDB ist eine NoSQL Datenbank, weitere Infos -> Wikipedia 🙂
Aktuelle MongoDB Version installieren (auf Ubuntu 16.04)
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5 echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list apt install apt-transport-https apt update apt-get install -y mongodb-org
siehe auch: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
Python Erweiterung pymongo für MongoDB installieren mit pip
python -m pip install pymongo
siehe auch: https://api.mongodb.com/python/current/
Verbindung zur MongoDB
from pymongo import MongoClient
client = MongoClient('localhost', 27017)
Connect zur Datenbank test
db = client.test
Collection myCollection in Datenbank test verbinden
coll = db.myCollection
Einen Datensatz anlegen in einer Collection
post = {"author":"Fritz Fuchs", "book":"Hasenjagd"}
>>> coll.insert_one(post)
<pymongo.results.InsertOneResult object at 0x7fd0136ce3f8>
Mehrere Datensätze in einer Collection anlegen
>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6"}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7"}]
>>> result = coll.insert_many(posts)
>>> result.inserted_ids
[ObjectId('5b259b2c39d9c04f7b43af01'), ObjectId('5b259b2c39d9c04f7b43af02'), ObjectId('5b259b2c39d9c04f7b43af03')]
Datensatz suchen
>>> coll.find_one({"author":"Fritz Fuchs"})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Datensatz mit Regex suchen
>>> coll.find_one({"author":{"$regex": "^Fritz.*"}})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Datensatz mit ObjectId abrufen
>>> from bson.objectid import ObjectId
>>> coll.find_one({"_id":ObjectId("5b25998139d9c04f7b43aefe")})
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
Mehrere Datensätze abrufen z.B. mit Regex
>>> for post in coll.find({"author":{"$regex": "^Fritz.*"}}):
... print post
...
{u'_id': ObjectId('5b25998139d9c04f7b43aefe'), u'book': u'Hasenjagd', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a8939d9c04f7b43aeff'), u'book': u'Hasenjagd 2', u'author': u'Fritz Fuchs'}
{u'_id': ObjectId('5b259a9039d9c04f7b43af00'), u'book': u'Hasenjagd 3', u'author': u'Fritz Fuchs'}
Index erzeugen für Collection z.B. Username ist Unique
>>> import pymongo
>>> db.users.create_index([('user_id', pymongo.ASCENDING)], unique=True)
u'user_id_1'
>>> new_users = [{'user_id':'max'},{'user_id':'fritz'}]
>>> db.users.insert_many(new_users)
<pymongo.results.InsertManyResult object at 0x7fd0136cee18>
>>> new_user = {'user_id':'max'}
>>> db.users.insert_one(new_user)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 683, in insert_one
session=session),
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 599, in _insert
bypass_doc_val, session)
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 580, in _insert_one
_check_write_command_response(result)
File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 207, in _check_write_command_response
_raise_last_write_error(write_errors)
File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 188, in _raise_last_write_error
raise DuplicateKeyError(error.get("errmsg"), 11000, error)
pymongo.errors.DuplicateKeyError: E11000 duplicate key error collection: test.users index: user_id_1 dup key: { : "max" }
ObjectId als String
>>> new_user = {'user_id':'maxx'}
>>> result = db.users.insert_one(new_user)
>>> result
<pymongo.results.InsertOneResult object at 0x7fd0122410e0>
>>> result.inserted_id
ObjectId('5b259ead39d9c04f7b43af07')
>>> str(result.inserted_id)
'5b259ead39d9c04f7b43af07'
String ObjectId zu ObjectId Object wandeln und für Suche verwenden
>>> from bson.objectid import ObjectId
>>> str_obj = '5b259ead39d9c04f7b43af07'
>>> users.find_one({'_id':ObjectId(str_obj)})
{u'_id': ObjectId('5b259ead39d9c04f7b43af07'), u'user_id': u'maxx'}
Datensatz löschen
>>> users.delete_one({'_id':ObjectId(str_obj)})
<pymongo.results.DeleteResult object at 0x7fd0136cee18>
Datensatz aktualisieren
>>> new_user = {'user_id':'maxx', 'name':'Hans Wurst'}
>>> users.insert_one(new_user)
>>> change = {'name': 'Fritz Fritz'}
>>> users.update_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')}, {'$set':change} )
<pymongo.results.UpdateResult object at 0x7fd0136ced40>
>>> users.find_one({'_id':ObjectId('5b25a14f39d9c04f7b43af08')})
{u'_id': ObjectId('5b25a14f39d9c04f7b43af08'), u'user_id': u'maxx', u'name': u'Fritz Fritz'}
Bereich bei Suche
>>> client = MongoClient()
>>> db = client.huu
>>> c = db.test
>>> posts = [{"author":"Fritz Fuchs", "book":"Hasenjagd 5", "boo":1}, {"author":"Fritz Fuchs", "book":"Hasenjagd 6", "boo":5}, {"author":"Fritz Fuchs", "book":"Hasenjagd 7", "boo":10}]
>>> c.insert_many(posts)
>>> for post in c.find({"boo": {"$lt": 6}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b54'), u'book': u'Hasenjagd 5', u'boo': 1}
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}
>>> for post in c.find({"boo": {"$gt": 6}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b56'), u'book': u'Hasenjagd 7', u'boo': 10}
>>> for post in c.find({"boo": {"$lt": 6, "$gt": 3}}).sort("author"):
... print post
...
{u'author': u'Fritz Fuchs', u'_id': ObjectId('5b25bb9539d9c05186997b55'), u'book': u'Hasenjagd 6', u'boo': 5}
Quelle / weitere Beispiele: http://api.mongodb.com/python/current/tutorial.html
Per Default ist die Boot Partition ca. 500M groß, da sich hier durch die Updates gerne alte Kernel ansammeln ist es ratsam zwischendurch aufzuräumen.
Beispiel einer abgebrochenen Installation, Meldung ist „No space left on device“
linux-image-4.4.0-116-generic (4.4.0-116.140) wird eingerichtet ... Running depmod. update-initramfs: deferring update (hook will be called later) The link /initrd.img is a dangling linkto /boot/initrd.img-4.4.0-116-generic vmlinuz(/boot/vmlinuz-4.4.0-116-generic ) points to /boot/vmlinuz-4.4.0-116-generic (/boot/vmlinuz-4.4.0-116-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-116-generic.postinst line 491. Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-116-generic /boot/vmlinuz-4.4.0-116-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-116-generic /boot/vmlinuz-4.4.0-116-generic update-initramfs: Generating /boot/initrd.img-4.4.0-116-generic W: mdadm: /etc/mdadm/mdadm.conf defines no arrays. gzip: stdout: No space left on device E: mkinitramfs failure cpio 141 gzip 1 update-initramfs: failed for /boot/initrd.img-4.4.0-116-generic with 1. run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1 Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-116-generic.postinst line 1052. dpkg: Fehler beim Bearbeiten des Paketes linux-image-4.4.0-116-generic (--configure): Unterprozess installiertes post-installation-Skript gab den Fehlerwert 2 zurück
Bei ubuntuusers gibt es einen nützlichen Artikel wie man am besten seine alten Kernel entsorgt.
Hier die Kurzfassung, weitere Details bitte direkt bei ubuntuusers nachlesen.
Aktuellen Kernel feststellen:
max@kronos01:~$ uname -a Linux kronos01 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Alte Kernel in Textdatei schreiben, prüfen ob aktueller Kernel dabei ist wenn ja entfernen:
max@kronos01:~$ dpkg -l 'linux-[ihs]*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\([-0-9]*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | tee zu_entfernende_Kernel
max@kronos01:~$ cat zu_entfernende_Kernel linux-headers-4.4.0-101 linux-headers-4.4.0-101-generic linux-headers-4.4.0-103 linux-headers-4.4.0-103-generic linux-headers-4.4.0-104 linux-headers-4.4.0-104-generic linux-headers-4.4.0-108 linux-headers-4.4.0-108-generic linux-headers-4.4.0-81 linux-headers-4.4.0-81-generic linux-headers-4.4.0-93 linux-headers-4.4.0-93-generic linux-headers-4.4.0-97 linux-headers-4.4.0-97-generic linux-headers-4.4.0-98 linux-headers-4.4.0-98-generic linux-image-4.4.0-101-generic linux-image-4.4.0-103-generic linux-image-4.4.0-104-generic linux-image-4.4.0-108-generic linux-image-4.4.0-81-generic linux-image-4.4.0-93-generic linux-image-4.4.0-97-generic linux-image-4.4.0-98-generic linux-image-extra-4.4.0-101-generic linux-image-extra-4.4.0-103-generic linux-image-extra-4.4.0-104-generic linux-image-extra-4.4.0-108-generic linux-image-extra-4.4.0-93-generic linux-image-extra-4.4.0-97-generic linux-image-extra-4.4.0-98-generic
Alte Kernel löschen und File löschen:
max@kronos01:~$ cat zu_entfernende_Kernel | xargs sudo apt-get -y purge max@kronos01:~$ rm zu_entfernende_Kernel
Quelle: https://wiki.ubuntuusers.de/Skripte/Alte_Kernel_entfernen/
I have found a nice help for fast rsync transfers over ssh. You can reach 40MB/s on gigabit network, depends on disk speed, SSD are faster.
rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>
More details & source: https://gist.github.com/KartikTalwar/4393116
First of all you’ll need to setup a .htpasswd file. To do so type in the following:
sudo htpasswd -c /etc/apache2/.htpasswd username
To use the authentication you can add the following to your VHost configuration:
<Location "/"> AuthType "Basic" AuthName "KeeWeb" AuthBasicProvider file AuthUserFile "/etc/apache2/.htpasswd" Require valid-user </Location>
WebDAV is needed to store your password database.
To enable WebDAV on Apache2 you need to type in the following command:
sudo a2enmod dav_fs
Now create a new directory for WebDAV and make it only accessable for the apache user:
mkdir /var/www/webdav chown www-data:www-data /var/www/webdav chmod 700 /var/www/webdav
Now we are going to create a new config for WebDAV in apache2:
Type in:
sudo vi /etc/apache2/sites-available/webdav.conf Add the following: Alias /webdav "/var/www/webdav" <Directory "/var/www/webdav"> DAV On Options Indexes </Directory> Save it and type in: sudo a2ensite webdav.conf && sudo service apache2 reload
Now test your settings by accessing yoururl.com/webdav.
You should see the Index of WebDAV.
First you need to install git:
sudo apt install git
After this you can clone the keeweb repository by typing in the following:
cd /var/www/ git clone -b gh-pages https://github.com/keeweb/keeweb.git
This will create a keeweb folder in your /var/www/ directory.
Now we need to create a keeweb config for our apache:
vi /etc/apache2/sites-available/keeweb.conf
Type in the following:
<VirtualHost *:443>
ServerName pass.yourdomain.com
ServerAlias www.pass.yourdomain.de
DocumentRoot /var/www/keeweb
</VirtualHost>
Save it and type in: sudo a2ensite keeweb.conf && sudo service apache2 reload
Now we add the authentication to our keeweb site.
To do so you can either create a .htaccess or add the authentication to your VHost like we setup above.
.htaccess:
vi /var/www/keeweb/.htaccess Add the following: AuthType Basic AuthName "KeeWeb" AuthUserFile /etc/apache2/.htpasswd Require valid-user Save it and quit.
If you enter the url pass.yourdomain.com you should see a login prompt and after this the keeweb index page.
YOU MADE IT!
Go to your keeweb index page and click on new. Now you created a temporary database. We want to save the database to our WebDAV to access it from everywhere. To do so, click on new in the bottom left corner.
Now you can change a few settings. I highly recommend to set a master password and a username! If you changed all your settings click on „Save as…“ and choose WebDAV.
Type in the path to your webdav and your username + password.
Example:
Aktuell habe ich auf einem meiner virtuellen Server Probleme mit der Performance, ein Blick in top offenbart das auf einer CPU 100% Waiting ist, Hintergrund war das der Provider einen ZFS scrub auf dem Host laufen ließ was die Disk Performance in den Keller gezogen hat. Ich nehme das zum Anlass ein paar Tools zu zeigen mit denen man sehen kann was in einem Linux System abgeht. Ich verzichte auf großartige Beschreibungen da die Tools und die MAN Pages für sich sprechen.
ATOP – AT Computing’s System & Process Monitor
atop lässt sich auf einem Ubuntu System mit apt install atop installieren.
In meinem Fall habe ich atop mit den Parametern -dD aufgerufen, so werden alle Prozesse angezeigt welche mit Festplatten Aktivität zu tun haben. avio zeigt die durchschnittlichen Millisekunden für eine Abfrage für Suche, Latenz und Datentransfer, der Wert mit 3035 ms ist viel zu hoch.
-d show disk-related process-info -D sort processes in order of disk-activity
TOP – display Linux processes
Das Standard Tool welches in der Regel auf jedem unixoiden Betriebsystem zur Verfügung steht.
Im Programm kann man noch Ansichten ändern etc. hier die aus meiner Sicht wichtigsten:
Dazu einfach die entsprechende Traste drücken:
IOSTAT – Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions.
iostat gibt die IO Statistiken aus. Wenn iostat mit dem Parameter 1 gestartet wird wird jede Sekunde eine Aktualisierung ausgegeben.
root@dev01:~# iostat 1
Linux 3.13.0-137-generic (dev01.m.test) 12/24/2017 _x86_64_ (4 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.26 0.02 0.09 7.43 0.00 92.20
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 3.18 44.11 34.80 3049729 2406628
avg-cpu: %user %nice %system %iowait %steal %idle
5.04 0.00 0.76 28.97 0.00 65.24
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 3.00 0.00 52.00 0 52
avg-cpu: %user %nice %system %iowait %steal %idle
5.28 0.00 0.50 24.87 0.00 69.35
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0.00 0.00 0.00 0 0
VMSTAT – Report virtual memory statistics
Gibt die Speichernutzung aus. Wenn vmstat mit dem Parameter 1 gestartet wird wird jede Sekunde eine Aktualisierung ausgegeben.
root@dev01:~# vmstat 1 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 10447220 272300 3105828 0 0 11 9 63 60 0 0 92 8 0 1 0 0 10446968 272300 3105956 0 0 128 24 2330 230 1 0 83 16 0 0 1 0 10446684 272300 3106100 0 0 0 4 2317 416 1 0 99 0 0 0 1 0 10446908 272300 3106100 0 0 0 96 2082 128 0 0 80 20 0 0 1 0 10446616 272304 3106224 0 0 128 76 2247 117 0 0 76 24 0 0 1 0 10446492 272304 3106356 0 0 128 0 2225 111 0 0 75 25 0 0 1 0 10446524 272304 3106356 0 0 0 0 2096 105 0 0 75 25 0 0 2 0 10446432 272304 3106484 0 0 128 0 2066 108 0 0 55 45 0 0 2 0 10446528 272304 3106484 0 0 128 12 2052 100 0 0 51 49 0 0 2 0 10446560 272304 3106484 0 0 0 0 1992 80 0 0 50 49 0 0 1 0 10446132 272308 3106864 0 0 256 12 2058 103 0 0 58 42 0 0 0 0 10445880 272308 3106868 0 0 128 0 2031 100 0 0 97 3 0 1 1 0 10445756 272308 3106996 0 0 128 0 2187 234 1 0 95 4 0 0 2 0 10445788 272308 3107124 0 0 0 28 2032 100 2 0 52 46 0 0 2 0 10445820 272308 3107124 0 0 0 0 1906 107 0 0 50 50 0 0 2 0 10445476 272308 3107124 0 0 256 16 2222 110 0 0 67 32 0
IOTOP – simple top-like I/O monitor
Zeigt alle Prozesse an und welchen I/O Load sie im System verursachen.
HTOP – interactive process viewer
htop ist ein grafisch gepimpter Prozess Viewer. Mit F5 lassen sich die Prozesse als Baum darstellen.
LSOF – list open files
Zeigt alle offenen Dateien in einem Linux System
root@dev1:~# lsof -u www-data COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME apache2 51385 www-data cwd DIR 252,0 4096 2 / apache2 51385 www-data rtd DIR 252,0 4096 2 / apache2 51385 www-data txt REG 252,0 662496 409596 /usr/sbin/apache2 apache2 51385 www-data mem REG 252,0 89696 653325 /lib/x86_64-linux-gnu/libgcc_s.so.1 apache2 51385 www-data mem REG 252,0 47600 653363 /lib/x86_64-linux-gnu/libnss_files-2.23.so apache2 51385 www-data mem REG 252,0 47648 653367 /lib/x86_64-linux-gnu/libnss_nis-2.23.so apache2 51385 www-data mem REG 252,0 93128 653357 /lib/x86_64-linux-gnu/libnsl-2.23.so apache2 51385 www-data mem REG 252,0 35688 653359 /lib/x86_64-linux-gnu/libnss_compat-2.23.so apache2 51385 www-data mem REG 252,0 22536 143878 /usr/lib/apache2/modules/mod_status.so apache2 51385 www-data mem REG 252,0 14344 143967 /usr/lib/apache2/modules/mod_setenvif.so apache2 51385 www-data mem REG 252,0 34832 143955 /usr/lib/apache2/modules/mod_negotiation.so apache2 51385 www-data mem REG 252,0 63504 143890 /usr/lib/apache2/modules/mod_mpm_event.so apache2 51385 www-data mem REG 252,0 18440 143969 /usr/lib/apache2/modules/mod_mime.so apache2 51385 www-data mem REG 252,0 18440 143942 /usr/lib/apache2/modules/mod_filter.so apache2 51385 www-data mem REG 252,0 10248 143913 /usr/lib/apache2/modules/mod_env.so apache2 51385 www-data mem REG 252,0 14344 143881 /usr/lib/apache2/modules/mod_dir.so apache2 51385 www-data mem REG 252,0 104864 653422 /lib/x86_64-linux-gnu/libz.so.1.2.8 apache2 51385 www-data mem REG 252,0 34824 143912 /usr/lib/apache2/modules/mod_deflate.so apache2 51385 www-data mem REG 252,0 38928 143951 /usr/lib/apache2/modules/mod_autoindex.so apache2 51385 www-data mem REG 252,0 10256 143964 /usr/lib/apache2/modules/mod_authz_user.so apache2 51385 www-data mem REG 252,0 10256 143950 /usr/lib/apache2/modules/mod_authz_host.so apache2 51385 www-data mem REG 252,0 22544 143873 /usr/lib/apache2/modules/mod_authz_core.so apache2 51385 www-data mem REG 252,0 10256 143954 /usr/lib/apache2/modules/mod_authn_file.so apache2 51385 www-data mem REG 252,0 10256 143947 /usr/lib/apache2/modules/mod_authn_core.so apache2 51385 www-data mem REG 252,0 14352 143960 /usr/lib/apache2/modules/mod_auth_basic.so apache2 51385 www-data mem REG 252,0 14344 143909 /usr/lib/apache2/modules/mod_alias.so apache2 51385 www-data mem REG 252,0 10256 143920 /usr/lib/apache2/modules/mod_access_compat.so apache2 51385 www-data mem REG 252,0 14608 653313 /lib/x86_64-linux-gnu/libdl-2.23.so apache2 51385 www-data mem REG 252,0 18976 653418 /lib/x86_64-linux-gnu/libuuid.so.1.3.0 apache2 51385 www-data mem REG 252,0 166032 653320 /lib/x86_64-linux-gnu/libexpat.so.1.6.0 apache2 51385 www-data mem REG 252,0 39224 653307 /lib/x86_64-linux-gnu/libcrypt-2.23.so apache2 51385 www-data mem REG 252,0 1868984 653299 /lib/x86_64-linux-gnu/libc-2.23.so apache2 51385 www-data mem REG 252,0 138696 653386 /lib/x86_64-linux-gnu/libpthread-2.23.so apache2 51385 www-data mem REG 252,0 204936 409578 /usr/lib/x86_64-linux-gnu/libapr-1.so.0.5.2 apache2 51385 www-data mem REG 252,0 159488 409580 /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0.5.4 apache2 51385 www-data mem REG 252,0 456632 653379 /lib/x86_64-linux-gnu/libpcre.so.3.13.2 ...
IFTOP – display bandwidth usage on an interface by host
Zeigt aktuelle Netzwerkverbindungen an und deren Traffic.
Folgende Pakete müssen auf einem Ubuntu 16.04 LTS installiert werden:
apt install nginx nginx-extras
Die Konfiguration wird abgelegt unter /etc/nginx/conf.d/exchange.conf
Folgende Dinge müssen angepasst werden:
server {
listen 80;
server_name mail.example.org autodiscover.example.org;
return 301 https://$host$request_uri;
}
server {
tcp_nodelay on;
listen 443;
ssl on;
ssl_certificate /etc/ssl/certs/star.example.org.pem;
ssl_certificate_key /etc/ssl/private/star.example.org.key;
ssl_session_timeout 5m;
server_name mail.example.org;
location / {
return 301 https://mail.example.org/owa;
}
proxy_http_version 1.1;
proxy_read_timeout 360;
proxy_pass_header Date;
proxy_pass_header Server;
proxy_pass_header Authorization;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_request_headers on;
more_set_input_headers 'Authorization: $http_authorization';
proxy_set_header Accept-Encoding "";
more_set_headers -s 401 'WWW-Authenticate: Basic realm="exchange-server.example.internal"';
# proxy_request_buffering off;
proxy_buffering off;
proxy_set_header Connection "Keep-Alive";
location ~* ^/owa { proxy_pass https://exchange-server.example.internal; }
location ~* ^/Microsoft-Server-ActiveSync { proxy_pass https://exchange-server.example.internal; }
location ~* ^/ecp { proxy_pass https://exchange-server.example.internal; }
location ~* ^/rpc { proxy_pass https://exchange-server.example.internal; }
location ~* ^/autodiscover { proxy_pass https://exchange-server.example.internal; }
location ~* ^/oab { proxy_pass https://exchange-server.example.internal; }
error_log /var/log/nginx/exchange-rproxy-ssl-error.log;
access_log /var/log/nginx/exchange-rproxy-ssl-access.log;
}
Ob alles einwandfrei Funktioniert lässt sich über einen Webservice von Microsoft testen:
https://testconnectivity.microsoft.com/
Quellen:
Viel Spaß 😉
Habe gerade auf ein frisch installiertes CentOS 7.4 Check_MK 1.4.0p19 installiert. Nach dem Start einer OMD Instanz kommt nur die Fehlermeldung:
OMD: Site Not Started You need to start this site in order to access the web interface.
Im Apache Log ist folgendes zu sehen:
[Mon Dec 04 08:50:48.097245 2017] [proxy_http:error] [pid 20887] [client x.x.x.x:31372] AH01114: HTTP: failed to make connection to backend: 127.0.0.1, referer: http://server.example.net/extern/ [Mon Dec 04 08:50:56.943253 2017] [proxy:error] [pid 20883] (13)Permission denied: AH00957: HTTP: attempt to connect to 127.0.0.1:5000 (127.0.0.1) failed [Mon Dec 04 08:50:56.943276 2017] [proxy:error] [pid 20883] AH00959: ap_proxy_connect_backend disabling worker for (127.0.0.1) for 0s [Mon Dec 04 08:50:56.943280 2017] [proxy_http:error] [pid 20883] [client x.x.x.x:31408] AH01114: HTTP: failed to make connection to backend: 127.0.0.1
netstat – tulpen zeigt aber das das Backend läuft:
# netstat -tulpen Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name ... tcp 0 0 127.0.0.1:5000 0.0.0.0:* LISTEN 997 71127 21004/httpd ...
Ein Blick in das Audit Log verrät das SELinux zuschägt:
#tail -f /var/log/audit/audit.log
...
type=AVC msg=audit(1512377448.096:3647): avc: denied { name_connect } for pid=20887 comm="httpd" dest=5000 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:commplex_main_port_t:s0 tclass=tcp_socket
type=SYSCALL msg=audit(1512377448.096:3647): arch=c000003e syscall=42 success=no exit=-13 a0=a a1=559b02b25650 a2=10 a3=7fffb621631c items=0 ppid=20882 pid=20887 auid=4294967295 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1512377508.204:3689): avc: denied { name_connect } for pid=21020 comm="httpd" dest=5000 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:commplex_main_port_t:s0 tclass=tcp_socket
type=SYSCALL msg=audit(1512377508.204:3689): arch=c000003e syscall=42 success=no exit=-13 a0=a a1=559b02b25650 a2=10 a3=7fffb621633c items=0 ppid=20882 pid=21020 auid=4294967295 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null)
...
Das Problem kann temporär zum testen wie folgt gelöst werden:
/usr/sbin/setsebool httpd_can_network_connect 1
um die Änderung permanent zu übernehmen:
/usr/sbin/setsebool -P httpd_can_network_connect 1
Wenn man bei Office365 oder Google Apps for Business einen eigenen Mailserver (Postfix) vorschalten möchte beim versenden/empfangen muss man die Mailserver von Microsoft/Google Whitelisten in den mynetworks bei Postfix.
Das Script löst alle SPF Record includes auf und generiert CIDR Maps die sich in Postfix einbinden lassen.
Beispiel:
max@dev1:~$ python get_subnets_of_spf_record_mynetwoks.py Working on job office365 Working on job google
Es werden 2 Files erzeugt:
max@dev1:~$ cat /etc/postfix/networks/google 64.18.0.0/20 OK 64.233.160.0/19 OK 66.102.0.0/20 OK 66.249.80.0/20 OK 72.14.192.0/18 OK 74.125.0.0/16 OK 108.177.8.0/21 OK 173.194.0.0/16 OK 207.126.144.0/20 OK 209.85.128.0/17 OK 216.58.192.0/19 OK 216.239.32.0/19 OK [2001:4860:4000::]/36 OK [2404:6800:4000::]/36 OK [2607:f8b0:4000::]/36 OK [2800:3f0:4000::]/36 OK [2a00:1450:4000::]/36 OK [2c0f:fb50:4000::]/36 OK 172.217.0.0/19 OK 108.177.96.0/19 OK max@dev1:~/test$ cat /etc/postfix/networks/office365 207.46.101.128/26 OK 207.46.100.0/24 OK 207.46.163.0/24 OK 65.55.169.0/24 OK 157.56.110.0/23 OK 157.55.234.0/24 OK 213.199.154.0/24 OK 213.199.180.0/24 OK 157.56.112.0/24 OK 207.46.51.64/26 OK 157.55.158.0/23 OK 64.4.22.64/26 OK 40.92.0.0/14 OK 40.107.0.0/17 OK 40.107.128.0/17 OK 134.170.140.0/24 OK [2a01:111:f400::]/48 OK 23.103.128.0/19 OK 23.103.198.0/23 OK 65.55.88.0/24 OK 104.47.0.0/17 OK 23.103.200.0/21 OK 23.103.208.0/21 OK 23.103.191.0/24 OK 216.32.180.0/23 OK 94.245.120.64/26 OK [2001:489a:2202::]/48 OK
In Posftix werden sie in der main.cf eingebunden:
# ----------------------------------------------------------------------
# My Networks
# ----------------------------------------------------------------------
mynetworks =
cidr:/etc/postfix/networks/local
cidr:/etc/postfix/networks/other
cidr:/etc/postfix/networks/google
cidr:/etc/postfix/networks/office365
Da sich zwischendurch die Records auch mal ändern können empfiehlt es sich einen Cronjob dafür einzurichten. Ich habe eine Variante mit diff die nur patcht wenn das Resultat nicht null ist.
Das Script lässt sich auch noch für andere Dienste / etc. anpassen:
lookup_spf = {
# Google Apps for Business
"google": {
"domain": "google.com",
"file" : "/etc/postfix/networks/google",
},
# Office365
"office365": {
"domain": "spf.protection.outlook.com",
"file" : "/etc/postfix/networks/office365",
},
# Example
"example": {
"domain": "example.com",
"file" : "/etc/postfix/networks/example",
},
}
Sourcecode:
#!/usr/bin/env python
#
# get_subnets_of_spf_record_mynetwoks.py
# Resolve all known ip addresses from spf record and generate cidr map for postfix
#
# Version 1.0
# Written by Maximilian Thoma (http://www.lanbugs.de)
#
# The generated files can be used in postfix config with for example mynetworks = cidr:/etc/postfix/<generated_file>
#
# This program is free software; you can redistribute it and/or modify it under the terms of the
# GNU General Public License as published by the Free Software Foundation;
# either version 2 of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with this program;
# if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
# MA 02110, USA
#
#
# Requirements:
# dnspython module -> pip install dnspython
#
import dns.resolver
from dns.exception import DNSException
import re
import sys
# Look for DNS Record at:
#
# "jobname": {
# "domain": "domainname",
# "file": "output_file",
# }
#
#
lookup_spf = {
# Google Apps for Business
"google": {
"domain": "google.com",
"file" : "/etc/postfix/networks/google",
},
# Office365
"office365": {
"domain": "spf.protection.outlook.com",
"file" : "/etc/postfix/networks/office365",
},
}
############################################################################################
def getspf(record, filehandler):
# Init Resolver
myResolver = dns.resolver.Resolver()
try:
# Try to lookup TXT record
myAnswer = myResolver.query(record,"TXT")
except DNSException:
sys.stderr.write("Failed to query record, SPF broken.")
return
results = []
for rdata in myAnswer:
# Get string out of records
for txt_string in rdata.strings:
# Append to SPF Records buffer if "spf" in string
if "spf" in txt_string:
results.append(txt_string)
# If results >=1
if len(results) >= 1:
# Work on records
for spf in results:
# Split parts
parts = spf.split(" ")
# Check parts
for part in parts:
s_include = re.match(r"^include:(?P<domain>.*)$", part)
s_ip4 = re.match(r"^ip4:(?P<ip4>.*)$", part)
s_ip6 = re.match(r"^ip6:(?P<ip6>.*)$", part)
# If in part "include" found, next round
if s_include:
getspf(s_include.group('domain'), filehandler)
# elif ip4 found
elif s_ip4:
filehandler.write(s_ip4.group('ip4') + " OK\n")
# elif ip6 found
elif s_ip6:
filehandler.write("[" + s_ip6.group('ip6').replace("/","]/") + " OK\n")
# else no valid record
else:
pass
# no results
else:
sys.stderr.write("No results")
pass
def main():
# Working on jobs
for jobname, config in lookup_spf.iteritems():
print "Working on job %s" % jobname
# open file
filehandler = open(config['file'], 'w')
# start query spf records
getspf(config['domain'], filehandler)
# close file
filehandler.close()
#getspf(lookup_spf)
if __name__ == "__main__":
main()
oder
https://gist.github.com/lanbugs/b223e16ae52e681bb5c9401c36fd5f1a
HP stellt für verschiedene Linux Distributionen seine eigenen Tools bereit. Unteranderem System Health Application and Command line Utilities, iLO Online Configuration Utilities und die Insight Management Agents.
HP SDR (Software Delivery Repository): http://downloads.linux.hpe.com/SDR/index.html
Repository für Debian 8 (jessie) hinzufügen
/etc/apt/sources.list.d/HP-mcp.list anlegen
deb http://downloads.linux.hpe.com/SDR/repo/mcp jessie/current non-free
Alternativ Repository für Ubuntu 16.04 LTS hinzufügen
/etc/apt/sources.list.d/HP-mcp.list anlegen
deb http://downloads.linux.hpe.com/SDR/repo/mcp xenial/current non-free
Repository PGP Keys importieren (curl wird benötigt)
curl http://downloads.linux.hpe.com/SDR/hpPublicKey1024.pub | apt-key add - curl http://downloads.linux.hpe.com/SDR/hpPublicKey2048.pub | apt-key add - curl http://downloads.linux.hpe.com/SDR/hpPublicKey2048_key1.pub | apt-key add - curl http://downloads.linux.hpe.com/SDR/hpePublicKey2048_key1.pub | apt-key add -
Paketquellen updaten
apt update
Pakete installieren (hp-health, hponcfg, hp-snmp-agents)
apt install hp-health hponcfg hp-snmp-agents
Das Kommando für die Health Tools ist hpasmcli.
root@rzm-srv01e13:~# hpasmcli
HPE management CLI for Linux (v2.0)
Copyright 2015 Hewlett Packard Enterprise Development LP.
--------------------------------------------------------------------------
NOTE: Some hpasmcli commands may not be supported on all Proliant servers.
Type 'help' to get a list of all top level commands.
--------------------------------------------------------------------------
hpasmcli>
Beispiele
show server
hpasmcli> show server System : ProLiant DL360 G7 Serial No. : XXXXXXXXXX ROM version : P68 08/16/2015 UEFI Support : No iLo present : Yes Embedded NICs : 4 NIC1 MAC: d8:9d:67:aa:aa:aa NIC2 MAC: d8:9d:67:bb:bb:bb NIC3 MAC: d8:9d:67:cc:cc:cc NIC4 MAC: d8:9d:67:dd:dd:dd Processor: 0 Name : Intel(R) Xeon(R) CPU E5640 @ 2.67GHz Stepping : 2 Speed : 2667 MHz Bus : 133 MHz Core : 4 Thread : 8 Socket : 1 Level1 Cache : 128 KBytes Level2 Cache : 1024 KBytes Level3 Cache : 12288 KBytes Status : Ok Processor: 1 Name : Intel(R) Xeon(R) CPU E5640 @ 2.67GHz Stepping : 2 Speed : 2667 MHz Bus : 133 MHz Core : 4 Thread : 8 Socket : 2 Level1 Cache : 128 KBytes Level2 Cache : 1024 KBytes Level3 Cache : 12288 KBytes Status : Ok Processor total : 2 Memory installed : 49152 MBytes ECC supported : Yes
show temp
hpasmcli> show temp Sensor Location Temp Threshold ------ -------- ---- --------- #1 AMBIENT 22C/71F 42C/107F #2 PROCESSOR_ZONE 40C/104F 82C/179F #3 PROCESSOR_ZONE 40C/104F 82C/179F #4 MEMORY_BD 35C/95F 87C/188F #5 MEMORY_BD 38C/100F 78C/172F #6 MEMORY_BD 35C/95F 87C/188F #7 MEMORY_BD 36C/96F 78C/172F #8 MEMORY_BD 38C/100F 87C/188F #9 MEMORY_BD 37C/98F 78C/172F #10 MEMORY_BD 37C/98F 87C/188F #11 MEMORY_BD 37C/98F 78C/172F #12 POWER_SUPPLY_BAY 39C/102F 59C/138F #13 POWER_SUPPLY_BAY 50C/122F 73C/163F #14 MEMORY_BD 32C/89F 72C/161F #15 PROCESSOR_ZONE 35C/95F 73C/163F #16 PROCESSOR_ZONE 34C/93F 64C/147F #17 MEMORY_BD 35C/95F 63C/145F #18 PROCESSOR_ZONE 43C/109F 69C/156F #19 SYSTEM_BD 39C/102F 69C/156F #20 SYSTEM_BD 43C/109F 71C/159F #21 SYSTEM_BD 50C/122F 65C/149F #22 SYSTEM_BD 52C/125F 71C/159F #23 SYSTEM_BD 45C/113F 69C/156F #24 SYSTEM_BD 50C/122F 69C/156F #25 SYSTEM_BD 39C/102F 63C/145F #26 SYSTEM_BD 49C/120F 66C/150F #27 SCSI_BACKPLANE_ZONE 50C/122F 60C/140F #28 SYSTEM_BD 72C/161F 110C/230F
show fan
hpasmcli> show fan Fan Location Present Speed of max Redundant Partner Hot-pluggable --- -------- ------- ----- ------ --------- ------- ------------- #1 SYSTEM Yes NORMAL 29% Yes 0 No #2 SYSTEM Yes NORMAL 29% Yes 0 No #3 SYSTEM Yes NORMAL 29% Yes 0 No #4 SYSTEM Yes NORMAL 29% Yes 0 No
show powersupply
hpasmcli> show powersupply Power supply #1 Present : Yes Redundant: No Condition: Ok Hotplug : Supported Power : 110 Watts Power supply #2 Present : Yes Redundant: No Condition: FAILED Hotplug : Supported
Kommandos lassen sich auch ohne die interaktive hpasmcli ausführen.
root@rzm-srv01e13:~# hpasmcli -s "show fan; show powersupply" Fan Location Present Speed of max Redundant Partner Hot-pluggable --- -------- ------- ----- ------ --------- ------- ------------- #1 SYSTEM Yes NORMAL 30% Yes 0 No #2 SYSTEM Yes NORMAL 30% Yes 0 No #3 SYSTEM Yes NORMAL 30% Yes 0 No #4 SYSTEM Yes NORMAL 30% Yes 0 No Power supply #1 Present : Yes Redundant: No Condition: Ok Hotplug : Supported Power : 115 Watts Power supply #2 Present : Yes Redundant: No Condition: FAILED Hotplug : Supported
Über hponcfg lässt sich das iLO ohne Systemneustart konfigurieren.
root@rzm-srv01e13:~# hponcfg
HP Lights-Out Online Configuration utility
Version 4.6.0 Date 09/28/2015 (c) Hewlett-Packard Company, 2015
Firmware Revision = 1.87 Device type = iLO 3 Driver name = hpilo
USAGE:
hponcfg -?
hponcfg -h
hponcfg -m minFw
hponcfg -r [-m minFw ]
hponcfg [-a] -w filename [-m minFw]
hponcfg -g [-m minFw]
hponcfg -f filename [-l filename] [-s namevaluepair] [-v] [-m minFw]
hponcfg -i [-l filename] [-s namevaluepair] [-v] [-m minFw]
-h, --help Display this message
-? Display this message
-r, --reset Reset the Management Processor to factory defaults
-b, --reboot Reboot Management Processor without changing any setting
-f, --file Get/Set Management Processor configuration from "filename"
-i, --input Get/Set Management Processor configuration from the XML input
received through the standard input stream.
-w, --writeconfig Write the Management Processor configuration to "filename"
-a, --all Capture complete Management Processor configuration to the file.
This should be used along with '-w' option
-l, --log Log replies to "filename"
-v, --xmlverbose Display all the responses from Management Processor
-s, --substitute Substitute variables present in input config file
with values specified in "namevaluepairs"
-g, --get_hostinfo Get the Host information
-m, --minfwlevel Minimum firmware level
HP liefert ein Konfigurationstool mit das Menügeführt einen durch das Grundsetup durchführt.
/sbin/hpsnmpconfig
Ich habe nur die Read/Write und ReadOnly Community geändert und alles andere auf Default lassen.
Anschließend habe ich noch Korrekturen durchgeführt in der /etc/snmp/snmpd.conf
# Anpassung der Communitys # Schreiben nur von Loopback aus rwcommunity supergeheim 127.0.0.1 # Lesezugriff fuer Monitoring rocommunity public 10.10.10.200 # Listener angepasst das er nur auf der Management NW Karte erreichbar ist agentAddress udp:10.10.10.1:161
Restart des SNMP Daemons nicht vergessen:
service snmpd restart
Sammlung aller SSH/SSL relevanten und meist genutzten Dienste und deren optimale SSH/SSL Konfiguration.
apache 2.4.18 | modern profile | OpenSSL 1.0.1e
Oldest compatible clients: Firefox 27, Chrome 30, IE 11 on Windows 7, Edge, Opera 17, Safari 9, Android 5.0, and Java 8
<VirtualHost *:443>
...
SSLEngine on
SSLCertificateFile /path/to/signed_certificate_followed_by_intermediate_certs
SSLCertificateKeyFile /path/to/private/key
# Uncomment the following directive when using client certificate authentication
#SSLCACertificateFile /path/to/ca_certs_for_client_authentication
# HSTS (mod_headers is required) (15768000 seconds = 6 months)
Header always set Strict-Transport-Security "max-age=15768000"
...
</VirtualHost>
# modern configuration, tweak to your needs
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
SSLHonorCipherOrder on
SSLCompression off
SSLSessionTickets off
# OCSP Stapling, only in httpd 2.3.3 and later
SSLUseStapling on
SSLStaplingResponderTimeout 5
SSLStaplingReturnResponderErrors off
SSLStaplingCache shmcb:/var/run/ocsp(128000)
nginx 1.10.1 | modern profile | OpenSSL 1.0.1e
Oldest compatible clients: Firefox 27, Chrome 30, IE 11 on Windows 7, Edge, Opera 17, Safari 9, Android 5.0, and Java 8
server {
listen 80 default_server;
listen [::]:80 default_server;
# Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
# certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
ssl_certificate /path/to/signed_cert_plus_intermediates;
ssl_certificate_key /path/to/private_key;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
# modern configuration. tweak to your needs.
ssl_protocols TLSv1.2;
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
ssl_prefer_server_ciphers on;
# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;
# OCSP Stapling ---
# fetch OCSP records from URL in ssl_certificate and cache them
ssl_stapling on;
ssl_stapling_verify on;
## verify chain of trust of OCSP response using Root CA and Intermediate certs
ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates;
resolver <IP DNS resolver>;
....
}
Quelle: https://mozilla.github.io/server-side-tls/ssl-config-generator/
SSL Webserver Test
Qualys hat eine tolle SSL Analyse Webseite, Link: https://www.ssllabs.com/ssltest/
Beispiel google.de:
Es zeigt einen schnellen Überblick plus sehr detailierte Informationen was nicht passt oder was man verbessern kann.
Alternative check with nmap
max@cmkdevel:~$ nmap --script ssl-cert,ssl-enum-ciphers -p 443 www.google.de Starting Nmap 7.01 ( https://nmap.org ) at 2017-08-01 09:08 CEST Nmap scan report for www.google.de (172.217.17.35) Host is up (0.031s latency). Other addresses for www.google.de (not scanned): 2a00:1450:400e:804::2003 rDNS record for 172.217.17.35: ams16s29-in-f3.1e100.net PORT STATE SERVICE 443/tcp open https | ssl-cert: Subject: commonName=www.google.de/organizationName=Google Inc/stateOrProvinceName=California/countryName=US | Issuer: commonName=Google Internet Authority G2/organizationName=Google Inc/countryName=US | Public Key type: rsa | Public Key bits: 2048 | Signature Algorithm: sha256WithRSAEncryption | Not valid before: 2017-07-19T11:48:30 | Not valid after: 2017-10-11T11:31:00 | MD5: f21e dd1d 579f 5ea3 1b3d 2e79 873d 616c |_SHA-1: f2af ac3f fb43 20be 9f6a b0fc 9356 808a f677 86fc | ssl-enum-ciphers: | TLSv1.0: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A | TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C | compressors: | NULL | cipher preference: server | TLSv1.1: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A | TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C | compressors: | NULL | cipher preference: server | TLSv1.2: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A | TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A | TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A | TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C | compressors: | NULL | cipher preference: server |_ least strength: C Nmap done: 1 IP address (1 host up) scanned in 3.31 seconds
/etc/postfix/main.cf
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3 smtpd_tls_mandatory_ciphers=high tls_high_cipherlist=EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CBC3-SHA, KRB5-DES, CBC3-SHA smtpd_tls_dh1024_param_file=/etc/ssl/dh2048.pem smtp_tls_mandatory_protocols = !SSLv2, !SSLv3 smtp_tls_protocols = !SSLv2, !SSLv3 lmtp_tls_mandatory_protocols = !SSLv2, !SSLv3 lmtp_tls_protocols = !SSLv2, !SSLv3 smtp_tls_security_level = may smtp_tls_loglevel = 1 smtpd_tls_auth_only = yes
Generate new Diffie Hellman Keys
openssl dhparam -out /etc/ssl/dh2048.pem 2048
Restart Postfix
service postfix restart
Mehr: http://www.postfix.org/TLS_README.html
Check Postfix SSL Configuration
Test mit openssl s_client
openssl s_client -starttls smtp -connect mailserver.example.org:25
Alternative mit nmap
nmap --script ssl-cert,ssl-enum-ciphers -p 993,995,25,587 -Pn mailserver.example.org
/etc/dovecot/conf.d/10-ssl.conf
ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 #only for dovecot >=2.2.6, enforce the server cipher preference ssl_prefer_server_ciphers = yes #disable SSLv2 and SSLv3 ssl_protocols = !SSLv2 !SSLv3
Restart dovecot
service dovecot restart
Überprüfen mit openssl s_connect
openssl s_client -connect mailserver.example.org:993 openssl s_client -connect mailserver.example.org:995
Alternative mit nmap
nmap --script ssl-cert,ssl-enum-ciphers -p 993,995 -Pn mailserver.example.org
/etc/ssh/sshd_config
# Remove dsa hostkeys HostKey /etc/ssh/ssh_host_ed25519_key HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256 Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes256-ctr Macs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256 AuthenticationMethods publickey LogLevel VERBOSE Subsystem sftp /usr/lib/ssh/sftp-server -f AUTHPRIV -l INFO PermitRootLogin No UsePrivilegeSeparation sandbox
Achtung ! DSA SSH Keys werden danach nicht mehr funktionieren.
Restart SSH
service ssh restart
/etc/openvpn/openvpn.conf
tls-version-min 1.2 tls-cipher TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384 cipher AES-256-CBC auth SHA512 reneg-sec 60
Quelle: https://community.openvpn.net/openvpn/wiki/Hardening / https://gist.github.com/pwnsdx/8fc14ee1e9f561a0a5b8
In der aktuellen Version von automysqlbackup kommt es auf Ubuntu 16.04 LTS wieder zu der Fehlermeldung:
mysqldump: [Warning] Using a password on the command line interface can be insecure.
Es gab vor einiger Zeit bereits einen Patch für das Problem allerdings hat sich der Code geändert dementsprechend waren anpassungen nötig. Orginalpatch zu finden unter: http://wiki.nkosi.org/Automysqlbackup / https://www.redeo.nl/2013/11/automysqlbackup-warning-using-password-command-line-interface-can-insecure/
Hier die neue angepasste Version für automysqlbackup v.2.5:
# Einfügen ab Zeile 644 vor if [ "$MAILCONTENT" = "files" ] # Patch error message since MySQL 5.6 if [ -s "$LOGERR" ] then XLOGERR=$BACKUPDIR/XERRORS_$DBHOST-`date +%N`.log grep -v "Using a password on the command line interface can be insecure." "$LOGERR" > $XLOGERR mv $XLOGERR $LOGERR fi
Ich habe auch ein Patchfile bereitgestellt, einfacher als im Code zu suchen.
max@cmkdevel:~$ cp /usr/sbin/automysqlbackup automysqlbackup.backup max@cmkdevel:~$ sudo patch /usr/sbin/automysqlbackup automysqlbackup.patch
Um Pakete ohne manuelle Eingaben zu Installieren müssen die Antworten für die Assistenten bereits vorliegen.
Als Beispiel eine unattended Installation von Postfix auf einem Ubuntu 16.04 LTS System.
Um die möglichen Parameter zu erfahren verwendet man das „debconf-show“ Tool, Postfix muss auf dem Testsystem bereits installiert sein.
max@cmkdevel:/opt$ sudo debconf-show postfix postfix/procmail: false postfix/chattr: false postfix/recipient_delim: + postfix/sqlite_warning: postfix/dynamicmaps_conversion_warning: postfix/mydomain_warning: postfix/relayhost: postfix/compat_conversion_warning: true postfix/not_configured: postfix/rfc1035_violation: false postfix/root_address: postfix/protocols: all postfix/destinations: $myhostname, cmkdevel.m.local, localhost.m.local, , localhost postfix/kernel_version_warning: postfix/relay_restrictions_warning: postfix/mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 postfix/mailbox_limit: 0 postfix/tlsmgr_upgrade_warning: * postfix/mailname: cmkdevel.m.local * postfix/main_mailer_type: Internet Site postfix/bad_recipient_delimiter: postfix/main_cf_conversion_warning: true postfix/retry_upgrade_warning:
Um für ein neues System jetzt die Antworten dem Assistenten zu übergeben verwendet man das Tool „debconf-set-selections“. Der Kommandostapel setzt alle Optionen und Installiert automatisch Postfix.
debconf-set-selections <<< "postfix postfix/main_cf_conversion_warning string true" debconf-set-selections <<< "postfix postfix/destinations string \$myhostname, $(hostname), localhost" debconf-set-selections <<< "postfix postfix/root_address string max@m.local" debconf-set-selections <<< "postfix postfix/bad_recipient_delimiter string ''" debconf-set-selections <<< "postfix postfix/mynetworks string 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128" debconf-set-selections <<< "postfix postfix/recipient_delim string +" debconf-set-selections <<< "postfix postfix/protocols string all" debconf-set-selections <<< "postfix postfix/relayhost string mail.example.org" debconf-set-selections <<< "postfix postfix/mailname string $(hostname)" debconf-set-selections <<< "postfix postfix/mailbox_limit string 0" debconf-set-selections <<< "postfix postfix/chattr string false" debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Satellite system'" apt-get install -y postfix
Viel Spaß 😉
Zum Python Code schreiben empfiehlt es sich auf Tabs und Spaces zu achten. PEP-8 schreibt für Python 2.x vor das Tabs in Spaces gewandelt werden sollten.
https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces
Hier eine passende .nanorc die man in sein Homeverzeichnis ablegt:
set tabsize 4 set nowrap set tabstospaces set autoindent set smooth