I have found a nice help for fast rsync transfers over ssh. You can reach 40MB/s on gigabit network, depends on disk speed, SSD are faster.
rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>
More details & source: https://gist.github.com/KartikTalwar/4393116
I had a huge problem at one of my clients. He has an Exchange 2016 CU8 with Windows 10 Clients and Outlook 2016 Build 8829 click-to-run. The customer has in total 20 mailboxes and all coworkers (5) in most cases have full access to each other. Some of the users have a big folder structure in the mailbox.
Outlook begins to stall while trying to access the mailboxes during startup of Outlook. If you look in the connection Details, you will see many connections to the different boxes and one or two lines where the VID/CID counts up really fast and changes the state between success and failure. Additionally, you see some garbage (random ASCII) in the mailbox name.
Big thanks to my collegue Wolle who is a geat Microsoft engineer. He found the issue which was causing the problem 🙂
The issue is caused because the user reaches the MAPI object and session limit of Exchange 2016. That is logged in „C:\Program Files\Microsoft\Exchange Server\V15\Logging\MapiHttp\Mailbox“ logs.
MoMTException:1246 (rop 1246) -> [TooManyObjectsOpenedException] Das Postfach /o=First Organization/ou=Exchange Administrative Group (XXXXXXXXXXXXXXXXXX)/cn=Recipients/cn=xxxxxxxxxxxxxxxxxxx-Customer kann nicht geöffnet werden. -> [MapiExceptionSessionLimit] Unable to open message store.
Additionally, the user runs into the MAPI limits due to the high access rate and the Exchange server begins to throttle down the connections.
These two things fixes the problems of my customer. Be aware that setting throttling to unlimited is typically not a good idea, but in this small environment it is okay.
In Exchange Powershell, create a new throttling policy and apply it to all affected mailboxes:
New-ThrottlingPolicy -Name "New Throttling" -RCAMAXConcurrency Unlimited get-mailbox -name | set-mailbox -ThrottlingPolicy "New Throttling"
A new registry key to raise the MAPI objects limit and the maximum allowed sessions per user:
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\MaxObjsPerMapiSession] "objtMessage"=dword:000005dc [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem] "Maximum Allowed Sessions Per User"=dword:00000080
After changing the Limit, you must restart the Exchange Information Store service.
Additional information:
First of all you’ll need to setup a .htpasswd file. To do so type in the following:
sudo htpasswd -c /etc/apache2/.htpasswd username
To use the authentication you can add the following to your VHost configuration:
<Location "/"> AuthType "Basic" AuthName "KeeWeb" AuthBasicProvider file AuthUserFile "/etc/apache2/.htpasswd" Require valid-user </Location>
WebDAV is needed to store your password database.
To enable WebDAV on Apache2 you need to type in the following command:
sudo a2enmod dav_fs
Now create a new directory for WebDAV and make it only accessable for the apache user:
mkdir /var/www/webdav chown www-data:www-data /var/www/webdav chmod 700 /var/www/webdav
Now we are going to create a new config for WebDAV in apache2:
Type in:
sudo vi /etc/apache2/sites-available/webdav.conf Add the following: Alias /webdav "/var/www/webdav" <Directory "/var/www/webdav"> DAV On Options Indexes </Directory> Save it and type in: sudo a2ensite webdav.conf && sudo service apache2 reload
Now test your settings by accessing yoururl.com/webdav.
You should see the Index of WebDAV.
First you need to install git:
sudo apt install git
After this you can clone the keeweb repository by typing in the following:
cd /var/www/ git clone -b gh-pages https://github.com/keeweb/keeweb.git
This will create a keeweb folder in your /var/www/ directory.
Now we need to create a keeweb config for our apache:
vi /etc/apache2/sites-available/keeweb.conf
Type in the following:
<VirtualHost *:443>
ServerName pass.yourdomain.com
ServerAlias www.pass.yourdomain.de
DocumentRoot /var/www/keeweb
</VirtualHost>
Save it and type in: sudo a2ensite keeweb.conf && sudo service apache2 reload
Now we add the authentication to our keeweb site.
To do so you can either create a .htaccess or add the authentication to your VHost like we setup above.
.htaccess:
vi /var/www/keeweb/.htaccess Add the following: AuthType Basic AuthName "KeeWeb" AuthUserFile /etc/apache2/.htpasswd Require valid-user Save it and quit.
If you enter the url pass.yourdomain.com you should see a login prompt and after this the keeweb index page.
YOU MADE IT!
Go to your keeweb index page and click on new. Now you created a temporary database. We want to save the database to our WebDAV to access it from everywhere. To do so, click on new in the bottom left corner.
Now you can change a few settings. I highly recommend to set a master password and a username! If you changed all your settings click on „Save as…“ and choose WebDAV.
Type in the path to your webdav and your username + password.
Example:
Ansible verwendet als Template Engine Jinja2. Als Beispiel wird für einen Switch die Radius Konfiguration erzeugt.
Man kann direkt vom Template aus die Konfiguration auf ein Gerät fallen lassen oder so wie in diesem Fall einfach als File speichern.
Eine ausführliche Anleitung was man alles mit der Jinja2 Template Engine machen kann ist unter http://jinja.pocoo.org/docs/2.10/ zu finden.
Template radius_demo.j2:
{% for rad in globals_demo.aaagroup %}
aaa group server radius {{ rad.name }}
{% for server in rad.server %}
server name {{ server }}
{% endfor %}
deadtime {{ rad.deadtime }}
{% endfor %}
{% for server in globals_demo.radiusserver %}
radius server {{ server.name }}
address ipv4 {{ server.address }} auth-port {{ server.authport }} acct-port {{ server.acctport }}
timeout {{ server.timeout }}
retransmit {{ server.retransmit }}
key {{ server.pskkey }}
{% endfor %}
globals_demo.yaml – Konfigdatei mit globalen Variablen welche in das Playbook geladen werden.
---
aaagroup:
- name: DOT1X
server:
- RADIUS1-DOT1X
- RADIUS2-DOT1X
- RADIUS3-DOT1X
- RADIUS4-DOT1X
deadtime: 1
radiusserver:
- name: RADIUS1-DOT1X
address: 192.168.1.101
authport: 1812
acctport: 1813
timeout: 4
retransmit: 3
pskkey: 7 02000B5409071D
- name: RADIUS2-DOT1X
address: 192.168.1.102
authport: 1812
acctport: 1813
timeout: 4
retransmit: 3
pskkey: 7 02000B5409071D
- name: RADIUS3-DOT1X
address: 192.168.1.103
authport: 1812
acctport: 1813
timeout: 4
retransmit: 3
pskkey: 7 02000B5409071D
- name: RADIUS4-DOT1X
address: 192.168.1.104
authport: 1812
acctport: 1813
timeout: 4
retransmit: 3
pskkey: 7 02000B5409071D
config_gen.yaml – Das Playbook welches die Konfiguration erzeugen soll
---
- hosts: all
connection: local
gather_facts: False
vars:
cli:
host: "{{ inventory_hostname }}"
username: "{{ ansible_user }}"
password: "{{ ansible_ssh_pass }}"
tasks:
- name: Load globals
include_vars:
file: globals_demo.yaml
name: globals_demo
- name: Generate config
template:
src: radius_demo.j2
dest: "/tmp/out_{{ inventory_hostname }}.cfg"
Testlauf config_gen.yaml Playbook
ansible-playbook -i demo.ini config_gen.yaml -v
PLAY [all] *****************************************************************
TASK [Load globals] ********************************************************
ok: [switch1] => {"ansible_facts": {"globals_demo": {"aaagroup": [{"deadtime": 1, "name": "DOT1X", "server": ["RADIUS1-DOT1X", "RADIUS2-DOT1X", "RADIUS3-DOT1X", "RADIUS4-DOT1X"]}], "radiusserver": [{"acctport": 1813, "address": "192.168.1.101", "authport": 1812, "name": "RADIUS1-DOT1X", "pskkey": "7 02000B5409071D", "retransmit": 3, "timeout": 4}, {"acctport": 1813, "address": "192.168.1.102", "authport": 1812, "name": "RADIUS2-DOT1X", "pskkey": "7 02000B5409071D", "retransmit": 3, "timeout": 4}, {"acctport": 1813, "address": "192.168.1.103", "authport": 1812, "name": "RADIUS3-DOT1X", "pskkey": "7 02000B5409071D", "retransmit": 3, "timeout": 4}, {"acctport": 1813, "address": "192.168.1.104", "authport": 1812, "name": "RADIUS4-DOT1X", "pskkey": "7 02000B5409071D", "retransmit": 3, "timeout": 4}]}}, "ansible_included_var_files": ["/home/ansible/_ansible_cisco_tests/globals_demo.yaml"], "changed": false}
TASK [Generate config] *****************************************************
ok: [switch1] => {"changed": false, "checksum": "b1cac79a0ba981ab398439a9102df687b7932ede", "gid": 1001, "group": "ansible", "mode": "0644", "owner": "ansible", "path": "/tmp/out_switch1.cfg", "size": 715, "state": "file", "uid": 1001}
PLAY RECAP *****************************************************************
switch1 : ok=2 changed=0 unreachable=0 failed=0
Ausgabe out_switch1.cfg
aaa group server radius DOT1X server name RADIUS1-DOT1X server name RADIUS2-DOT1X server name RADIUS3-DOT1X server name RADIUS4-DOT1X deadtime 1 radius server RADIUS1-DOT1X address ipv4 192.168.1.101 auth-port 1812 acct-port 1813 timeout 4 retransmit 3 key 7 02000B5409071D radius server RADIUS2-DOT1X address ipv4 192.168.1.102 auth-port 1812 acct-port 1813 timeout 4 retransmit 3 key 7 02000B5409071D radius server RADIUS3-DOT1X address ipv4 192.168.1.103 auth-port 1812 acct-port 1813 timeout 4 retransmit 3 key 7 02000B5409071D radius server RADIUS4-DOT1X address ipv4 192.168.1.104 auth-port 1812 acct-port 1813 timeout 4 retransmit 3 key 7 02000B5409071D
Möchte man die Konfig direkt auf ein Gerät ausrollen kann man das Playbook wie folgt erweitern:
- name: Apply RADIUS configuration to switch
ios_config:
src: radius_demo.j2
provider: "{{ cli }}"
match: line
Beispiel Playbook exec_command.yaml:
---
- hosts: switch1
connection: local
gather_facts: False
vars:
cli:
host: "{{ inventory_hostname }}"
username: "{{ ansible_user }}"
password: "{{ ansible_ssh_pass }}"
tasks:
- name: Call show inventory
ios_command:
commands: show inventory
provider: "{{ cli }}"
register: output
- name: Store output to file
copy:
content="{{ output.stdout[0] }}"
dest="out/inventory_{{ inventory_hostname }}.txt"
Mit „ios_command“ lassen sich Kommandos auf der CLI ausführen und die Ausgabe weiterverwenden. In dem Beispiel wird die Ausgabe in einer Datei gespeichert.
ansible-playbook -i hosts.yaml exec_command.yaml -v
PLAY [switch1] *********************************************************************************
TASK [Call show inventory] *********************************************************************
ok: [switch1] => {"changed": false, "stdout": ["NAME: \"1\", DESCR: \"WS-C2960-8TC-S\"\nPID: WS-C2960-8TC-S , VID: V01 , SN: FOCXXXXXXXX"], "stdout_lines": [["NAME: \"1\", DESCR: \"WS-C2960-8TC-S\"", "PID: WS-C2960-8TC-S , VID: V01 , SN: FOCXXXXXXXX"]]}
TASK [Store output to file] ********************************************************************
changed: [switch1] => {"changed": true, "checksum": "bb873c38a92c5fcf4f337c4730bf626ba36ac558", "dest": "out/inventory_switch1.txt", "gid": 0, "group": "root", "md5sum": "f7a2aca1b08cdaeb5564d1e368b6ff19", "mode": "0644", "owner": "root", "size": 87, "src": "/root/.ansible/tmp/ansible-tmp-1516308679.35-272245399379253/source", "state": "file", "uid": 0}
PLAY RECAP *************************************************************************************
switch1 : ok=2 changed=1 unreachable=0 failed=0
cat out/inventory_switch1.txt
NAME: "1", DESCR: "WS-C2960-8TC-S" PID: WS-C2960-8TC-S , VID: V01 , SN: FOCXXXXXXXX
Ich habe kürzlich Angefangen meine ersten Gehversuche mit Ansible zu machen. Ziel ist unteranderem meine Linux Maschinen zu managen als auch Cisco Geräte. Hier meine ersten Erfahrungen mit Ansible.
Was ist der Vorteil von Ansible?
Installation von Ansible auf einem Ubuntu System
Es ist empfehlenswert die letzte Version aus den PPA Repositories zu verwenden, die Version in 16.04 LTS ist 2.x, aktuell ist 2.4.2.0. Ich habe bei meinen Versuchen festgestellt das viele Plugins z.B. ansible-hardening (https://github.com/openstack/ansible-hardening) nicht funktionieren.
Kurzanleitung aktuelle Version auf Ubuntu
sudo apt update sudo apt install software-properties-common sudo apt-add-repository ppa:ansible/ansible sudo apt update sudo apt install ansible
Erste Gehversuche in Ansible
Ein Ansible Setup besteht aus 2 Bestandteilen, ein Inventory und YAML Files (Playbooks, etc.).
Beispiel meines Inventorys
[switches] switch1 [switches:vars] ansible_user=admin ansible_ssh_pass=admin ansible_ssh_common_args='-o StrictHostKeyChecking=no'
In den eckigen Klammern kann man Gruppen erzeugen, darunter sind die Mitglieder der Gruppe. Ein Host kann in mehreren Gruppen sein. Mit [gruppenname:vars] können gruppenspezifische Parameter mit übergeben werden. Der Host switch1 ist in meinem DNS auflösbar, sollte das nicht der Fall sein kann man hinter den Hostnamen die IP-Adresse statisch übergeben wie folgt:
[switches] switch1 ansible_host=192.168.1.44
Das Inventory lässt sich alternativ zum INI Format auch als YAML definieren.
---
all:
hosts:
switches:
hosts:
switch1:
ansible_host: 192.168.1.44
switch2:
switch3:
switch4:
vars:
ansible_user: admin
ansible_ssh_pass: admin
ansible_ssh_common_args: -o StrictHostKeyChecking=no
Erster Test: IOS_Facts vom Switch laden
test.yaml
---
- hosts: switch1
connection: local
gather_facts: False
vars:
cli:
host: "{{ inventory_hostname }}"
username: "{{ ansible_user }}"
password: "{{ ansible_ssh_pass }}"
tasks:
- name: Get IOS facts
ios_facts:
gather_subset: all
provider: "{{ cli }}"
register: iosfacts_out
tags: foobar
- name: print iosfacts_out
debug: var=iosfacts_out
Im YAML File sind folgende Dinge definiert:
z.B.
- hosts: - switch1 - switch2
Das Playbook ausführen
ansible-playbook -i hosts.yaml test.yaml
PLAY [switch1] **************************************************************************************
TASK [Get IOS facts] *******************************************************************************
ok: [switch1]
TASK [print iosfacts_out] **************************************************************************
ok: [switch1] => {
"iosfacts_out": {
"ansible_facts": {
"ansible_net_all_ipv4_addresses": [
"192.168.1.44"
],
... gekürzt ...
PLAY RECAP *****************************************************************************************
switch1 : ok=2 changed=0 unreachable=0 failed=0
Der erste Test war erfolgreich, alle weiteren Beispiele kommen in kleineren Beiträgen.
Viel Spaß 🙂