Generic

Backup MIAB (Mail in a Box) through Rsync towards Openmediavault Server

Mail in a box (MIAB) has a backup feature available. It stores full and incremential backups on the mailserver and it is possible to store the backup also on another device through RSYNC. In my situation I am saving the data to an OpenMediaVault NAS

Here I write my own: How I did it (quick and dirty cause I expect you to know things).

In short:
rsync over port 5678 to backup your data to the OpenMediaVault NAS

  1. Make sure you have a hostname available where rsync can be connected to, the hostname must point to the IP where the OMV (OpenMediaVault) is connected
  2. Rsync over SSH is being used.
  3. if you do not want to use port 22 with Rsync, you need to modify /root/mailinabox/management/backup.py line 19: change -p 22 to -p 5678
  4. Enable Rsync Server in the GUI (Grapical User Interface) of openmediavault.

Please note that it is not possible to use the ~/.ssh/config file where you can add the port as well. The reason is that the verification process needs in the backup.py a -p setting which is not overridden bij de config file.

  • SSH standard port 22, this we will change. (ie. port 22 is already in use towards another server)
  • In the router go to your portforwarding section and open port 5678 towards port 22 to your device (with OpenMediaVault).
  • MIAB and RSYNC needs to have the full path where to store the backup. In my situation: /media/a925efd7-ada5-48b5-80e6-383cc6274bcd/Backup (the folder must available and writable
  • Make sure that a user can login with SSH and can access OpenMediaVault
  • MIAB is providing a public key for auto-login needed for rsync. this key must be available in OpenMediavault. You can put the public key in: ~/.ssh/authorized_keys or in a folder in /var/lib/openmediavault/ssh/authorized_keys where you create a file with the name of the user
  • within MIAB you can use from /root/mailinabox/ the following: sudo management/backup.py –verif

to test if your public key is accepted: from MAIB ssh with the following command:  ssh -p 5678 -i /root/.ssh/id_rsa_miab user@domain.name

If this is giving you a direct login to your OpenMediaVault NAS you can use Rsync ;)

Missing something? Reply and ask

 

Kodi and texturecache

There is a nice tool for updating your db (mine is MySQL) with a tool called texturecache

this info is for my own purpose if useful use it

Crontab in place

0 */2 * * * sh /home/kodi/kodiupdate.sh > /dev/null 2>&1

#!/bin/bashNAME=texturecache
INIT_DIR=/etc/init.d
echo “Start Scanning Video Library vscan”
/home/kodi/texturecache.py vscan
sleep 5
echo “Start Cleaning Video Library vclean”
/home/kodi/texturecache.py vclean
sleep 5
echo “Start update with qax”
/home/kodi/texturecache.py qax
echo “Start scanning texturecache with function C”
/home/kodi/texturecache.py c
sleep 5
echo “Prune data missing on disk P”
/home/kodi/texturecache.py P
sleep 5
echo “Start update with Xd”
/home/kodi/texturecache.py Xd
sleep 3
echo “Start update with ./texturecache.py R”
/home/dennis/kodi/texturecache.py R
sleep 3
echo “end”

Make sure that there is a connection with your “Master Kodi”
create a samba link in /etc/fstab ie. example
//192.168.1.115/Userdata/ /media/kodi cifs guest,uid=1000,iocharset=utf8 0 0
now the thumbnails can be saved correctly

Performance issues HP Microserver gen8 and VMWARE EXSI 6.5

upgraded ESXI to 6.5 U1 which is version 6.5.0 build 5969303

You are running HPE Customized Image ESXi 6.5.0 version 650.10.1.0.47 released on July 2017 and based on ESXi 6.5.0 Vmkernel Release Build 5310538

unfortunately the HP Microserserver Gen8 is than running with: HPE_bootbank_scsi-hpvsa_5.5.0.102-1OEM.550.0.0.1331820

As I was (and still am but as of writing I am in maintenance mode) seeing spikes in my CPU usages. It might be caused by the bad performance

checking with:
cd /vmfs/volumes/datastore1
time dd if=/dev/zero of=tempfile bs=8k count=1000000

It took a very long time to see some output. (Very long time!)

than I tried to downgrade to the hpvsa-5.5.0-88.zip driver

guidelines:

  1. enter maintenance mode
  2. I do a reboot, but you can do it probably without
  3. copy the downloaded driver to /tmp/ and run the following command
  4. esxcli software vib install -d /tmp/hpvsa-5.5.0-88.zip
  5. the old driver will be removed and the 5.5.0-88 driver installed
  6. Now important: if you do reboot this way: you will not see your DataStores anymore, only your NFS datastores (in my situation) this is caused by VMWARE ESXI as it will be using vmw_ahci driver for the datastore.
  7. so disable the usage of this ‘default’ driver: esxcli system module set –enabled=false –module=vmw_ahci
  8. now you can reboot and your device will be using the 5.5.0-88 driver
  9. you can see this by using; cat /proc/driver/hpvsa/hpvsa0

update (taken from: https://homeservershow.com/forums/topic/9602-esxi-notes/)

When installing VIB you first need to un-install the old driver

How to downgrade the hpvsa driver in 5.5 and 6.0
The b120i driver from 5.5.0-90.0 to 5.5.0.98-1 has performance issues and requires a downgrade to 5.5.0-88.0. Newer versions may still be affected so check the forum or this thread for more details.

Download scsi-hpvsa-5.5.0-88OEM.550.0.0.1331820.x86_64.vib from the HP VIB depot and upload it to your datastore. It is the same driver for 5.5 and 6.0.

Enter maintenance mode and run the following command. You’ll need to reboot after changing the driver and then leave maintenance mode. Do NOT restart without reinstalling the driver after removing it.

important: remove the old driver !!

esxcli software vib remove -n Hewlett-Packard:scsi-hpvsa

install the NEW driver:

esxcli software vib install –viburl=/vmfs/volumes/[NAME OF DATASTORE]/scsi-hpvsa-5.5.0-88OEM.550.0.0.1331820.x86_64.vib

If you get an error saying bootbank cannot be live installed
The error will be similar to the following:

The transaction is not supported:
VIB VMware_bootbank_esx-base cannot be live installed.
VIB VMware_bootbank_esx-base cannot be removed live.
The bootbank is the system image that ESXi loads into a ramdisk and boots off.
It is only used during startup and during updates.

The error is generally due to two things.
First is corruption and second is if the boot device is not accessible for some reason.
On the Gen8 Microserver, doing an iLO update whilst the system is running may cause the micro-SD reader to temporarily stop working.
The solution is to shutdown the host, unplug the machine for 5 seconds, plug it back in and attempt the update again

the speed should have been returned.

 

Weatherstation and TE923 USB issues on a Raspberry Pi

I am using Weewx to read data from my weatherstation

But once in a while I have issues:

  • starting the driver / python app
  • after x days no data is being collected (and not being ftp-ed) and a restart does not function

I used the standard release of Weewx 3.5.0

Lucky there is a fanbase and group on google groups where  you can ask questions.

This weekend I upgraded my TE923 driver to  version: 0.18rc8. (meanwhile (22/7) this version is released as 0.18) This version can be found in the github repository and it seems to me that this version is solving some issues I had, but at time of writing I am still testing.

With this new driver I had also to make some changes to that driver and to my Raspberry Pi.

  1. I modified the TE923 Python driver: I commented this line out: 
     self.devh.reset()

    (somewhere around line 1530, this caused the message that the USB part was not being claimed by python. (in previous driver versions this was not being reported, so it has to be something in the driver causing this. By commenting this line out, it is making sure that there is no ‘fake reset’ of the usb part. (Note: I did not test what will happen if I am using this line again in combination with the 2nd modification / addition I made).

  2.  and I found a special line for the Raspberry Pi to init the USB for usage with the TE923 driver:

sudo nano -w /etc/udev/rules.d/99-te923.rules

add this line:

ATTRS{idVendor}=="1130", ATTRS{idProduct}=="6801", MODE="0660", GROUP="plugdev", RUN="/bin/sh -c 'echo -n $id:1.0 > /sys/bus/usb/drivers/usbhid/unbind'"

and save it
with

udevadm control --reload-rules
you reload the settings

now stop / start weewx again (/etc/init.d/weewx start / stop / start etc..

Update: 1 week after I wrote this the weatherstation stopped responding again ..
Cause: the SD card had no space left.
here a tip: if you use raspbian, make sure your whole SD card is being used. My Linux install was not extended with the option in the raspbian config. This options allows you to use the whole SD card size

use raspi-config and use the option to expand the filesystem

 

Update 2:

Server has more than 51 days of uptime… and still running smooth

Update 3:

Server is now up for 147 days. Due to some own power stuff and relocations I had to stop it for a while
but still happy that it is running smootly

 

PGOffline & MySQL

For many years I am using PGOffline, a tool to get all the content from yahoogroups.

PGOffline was using up to version 3 an Access Database but with the release of version  4 they went to an SQLite database.  Here came my problem.

For years I was inserting the data from the Access Database into an MySQL database so that with an PHP script I could search through more than 100.00 messages.

But after PGOffline went to SQLite I had an issue cause my ODBC connection was not working anymore. Due to some change in PC’s and notebooks I lost my access database with the ODBC and Queries as well (damn .. )

Altough the mailinglist is not that active anymore, there were some 6000 messages submitted in around 1,5 to 2 years. and I want to have them inserted in the MySQL database still ..

The issue I had was that the SQLite database of PGOffline contains a BLOB (Binary Object)  with the message and is not ‘simply’ readable.

for months I was trying and there was no solution for me to fix that. Until last weekend. And actually it was pretty easy. And I fixed it ..

1. Microsoft Access

2. Make sure you have access to your MySQL database and able to connect to it (do check my.cnf for the bind address (if you are on Linux) make sure you can connect to it from other machines) and make sure the firewall does not block its port (which I noticed after 15 minutes of trying or so)

3. install MySQL ODBC drivers in your environment

4. install SQLite ODBC drivers in your environment

In Access make sure you link to the database with linked tables.

the SQLite database is just the file. In my example its on the same computer as the Access database

The issue I had with the SQLite database and its linked tables was that the ODBC driver did not allow me to search with wildcards (select * from person  where person is like ‘a%’) . To make sure I have all subscribers I made an update query to a local table with all the persons, but in fact that is not needed.

Than I made sure I could read the messages

with this query: ‘StrConv([content],64)’ I was able to decode the BLOB.  and with some other InStr queries I removed the beginning and end of the start of the message as this is ‘garbage’ not needed, I added some HTML breaks.

Finally I created an update query inserting all the new messages from the SQL database not in the MySQL database, I even found that very old messages were not imported at all they were forgotten, now they were inserted as my Query statement request to insert all YahooMessageID’s not in the MySQL database ..

 

 

 

Munin help needed with renaming graph titles

[nerd mode] this article includes the update with solution ;-)

Does someone know how to configure Munin to change or rename graph titles of a smart switch with SNMP?

I have a smart Ethernet switch and I get my SNMP data, but the interface ports are called: interface port 1, interface port 2, interface port 3 etc.. But I want to have this renamed so that I can identify the various interface ports as: router, NAS, uplink, wireless, server1, server 2 etc.

Is it possible to configure this with munin?

I’ve looked in the docs, but cannot find anything about how to rename something.

Update, after long searching and looking through manuals/guides/how-to I have found what does the job [this was the entry telling me: hey that could be the solution]

in munin.conf select the config part where the device / node is configured

[highway1]
address 127.0.0.1
use_node_name no

Under use_node_name no you can put your manual ‘overrides’

Port one of my smart switch is called:

  • snmp_highway1_if_err_1 (for errors)
  • snmp_highway1_if_1.graph_title (for traffic)

So here I define the names of the interfaces:

snmp_highway1_if_err_1.graph_title [P1] Cable/Router Errors
snmp_highway1_if_1.graph_title [P1] Cable/Router Traffic
snmp_highway1_if_err_2.graph_title [P2] Smart Switch Boven Errors
snmp_highway1_if_2.graph_title [P2] Smart Switch Boven Traffic
snmp_highway1_if_err_3.graph_title [P3] AccessPoint/Switch Kamer Errors
snmp_highway1_if_3.graph_title [P3] AccessPoint/Switch Kamer Traffic

etc.

The magic is done with .graph_title, per interface you can define the graph title.

After 5 minutes of waiting my graphs are being updated and generated with the correct interface names so that you can remember what you have connected to which port of your smart switch (in my case).

networktraffic

Overlast bussen Apeldoorn de Maten (Zonnehoeve)

  • Effeweg.nl
  • Van nood (Oostenrijk)
  • Arke
  • Oad
  • Slangenreizen
  • Onafhankelijke busmaatschappijen die bijv. naar festivals gaan
  • en alle die ik vergeten ben

(hierbij uitgezonderd natuurlijk de buurtbus en de bussen die passagiers van treinen moet vervoeren als er treinen uitvallen).

 

  1. Vertrek: alle dagen rondom 7.30 (soms eerder)
  2. Aankomst: alle dagen mogelijk tussen 17.00 ~ in de nacht

Overlast:

  • Ronkende motoren van de bussen
  • Aircos van de bussen
  • Parkerende weg en ophalers
  • Plassende mensen in de bosjes, aanbellen om te mogen plassen
  • Parkeren op parkeerplekken van bewoners ipv bij het station
  • Parkeren in de berm,
  • Lichten in de woningen
  • Schreeuwende mensen
  • Toeterende autos
  • Versperren van de weg want: parkerende auto’s (terwijl het een doorgaande weg is en het dus gevaarlijk kan zijn

Buslus wordt illegaal gebruikt: er staat zelfs een stopverbod.

Op sommige dagen staat P&R de Maten geheel vol waardoor autos op de buslus parkeren.  In een jaar tijd totaal geen handhaving gezien.

Oorzaak is wel bekend: voor 2013 was er nog geen woonwijk maar alleen een toegangsweg met weiland naar Station de maten, nu in 2014 staan er woningen richting het station, echter houden de busmaatschappijen hier geen rekening mee

Hebben ze een vergunning om deze plek als opstap en uitstapplaats te gebruiken?  Er is vanuit de buurt verschillende malen via Sociale Media en E-mail gevraagd om actie / terugkoppeling van het gevraagde, maar de maatschappijen communiceren niet of willen niet communiceren.

Gemeente Apeldoorn, de project ontwikkelaar (want die wil fase 2 bouwen) en zie maar eens de woning verkocht te krijgen als nieuwe bewoners het bovenstaande weten ..

Oplossing simpel: verkassen naar de gigantische parkeerplaats bij het Omnisport Centrum (De Voorwaarts)

Iedereen blij. Makkelijk toch? Hier en daar wat websites aanpassen en wellicht wat folders, maar dat zal het dan wel zijn.

Wanneer? Liever gisteren dan morgen want ik zie de bussen voor de kerstmarkten al verschijnen ..

Update: we staan in de krant

 

13 years of Weblog: had to remove a 11 year old weblog post!

Last week I received an e-mail about a request to remove a weblog post from 2002 !

The post, about: ZigBee and Bluetooth was referring to Click Consult. Analysing it I saw that the PDF file it was linking to was receiving an Error 404 (it does not exist anymore).

So I removed it. One day later again the girl in the mail cause they found that I had it still on my MovableType export weblog. Damn I had to edit HTML now ..

So to have Click Consult to be in compliance with Google Webmaster guidelines I had to remove it .. uhmm Should I have charge them?

Plesk 11 issue with options FollowSymlinks or SymLinksIfOwnerMatch is off …..

I use a number of ‘open’ directories on my server for years, but I switched servers and updated Plesk to version 11.

Suddenly my open directories where not open anymore and no content was visible. In my error.log I find this error:

Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that RewriteRule directive is forbidden: /var/www/vhosts/domain.com/httpdocs/index.pl

I searched the internet and I could not find a good solution other than that I had to change my dir.conf where is stated: DirectoryIndex at_domains_index.html index.php index.html index.cgi index.pl index.xhtml index.htm index.shtml index.cfm

I even changed my vhost configuration what did not work either, it might helped finding the solution, so I will check if my changed vhost.conf is necessary.

in the end: changing dir.conf with the line about the indexes was the solution. I had to remove to remove at least the perl (.pl) one.

I now have:

DirectoryIndex at_domains_index.html index.php index.html index.htm

restarted apache with: service apache2 restart and issue was resolved