Posts on Blog

Android TV Cheat Sheet

May 20, 2022

Recently, I ventured further into the world of Android TV using Android 11 on a Xiaomi TV Stick 4K. In this post I’ll share some insights to get the most out of your Android TV.

Keep in mind that not all of these might work the same way or at all on any Android TV device! Most of them should, though. When in doubt, try and search for your device’s name and what you’re trying to do.

My main focus is to enable a customized, ad-free experience as well as to document quirks that I’ve encountered.

Note that some of the following things can be achieved by various means. For example, installing custom apps is also possible through certain third-party downloader apps. I’m focussing on achieving everything without third-party tools, notably the Android Debug Bridge (adb) which provides a command line interface for interacting with Android devices. It comes with an initial learning curve but enables many more possibilities in the long-run and is usable across devices without relying on third parties.

Managing the device with adb

Getting adb

Download the Android platform tools for your OS here. The zip file contains an executable adb among other things.

Having unzipped the archive, move into it’s content using your CLI - in my case Bash on Linux.

cd platform-tools
./adb version

You should see output stating the adb version you have now.

Activating debug access on Android TV

Now, you need to allow debug access on your Android TV. To access the corresponding setting, you need to enable Developer options, which are hidden behind a little trick in Android.

Open Settings -> Device preferences and move to a section called About [device] or similar. In here you’ll find various information like Android version, device model and also Android TV OS build. Tap/click on this a few times until a message appears on the bottom of the screen. It should tell you to press a few more times to activate Developer options.

Once Developer options are activated you should see them as a category when opening Settings -> Device preferences. Move into Developer options, scroll down to USB debugging and activate it.

You can connect with adb through USB but I recommend using the wireless network, because it’s much easier once you’ve gotten the hang of it. Some devices (especially devices in “Stick” format) don’t even have USB ports anymore or you need a special cable.

Connecting to Android TV with adb

Next, you need to find out the IP address of your Android TV device.

In Android TV the IP address can be found in Settings -> Device preferences -> About -> Status. Your router’s web interface should also have it somewhere.

Once you have the IP address of your device (in my case 192.168.178.10), run the following command to connect with adb. Make sure to use your device’s actual IP address when copy-pasting:

adb connect 192.168.178.10

adb might say something like Failed to authenticate [...]. If that happens, look at your TV screen. You should see a pop-up window, asking you to click OK to allow access. This is a security feature to make sure not anyone can access your Android TV through adb just because they’re in your network.

After having allowed access, run the following to check whether the connection was established:

adb devices

It should put out a List of devices attached with one element: Your device’s IP address.

Note, that it is considered a security risk to leave ADB debugging activated on Android permanently. It’s highly encouraged to deactivate it when you’re done and only re-activate when needed.

Ad-free YouTube

SmartTubeNext is an amazing third-party YouTube client for TVs. It’s ad-free and contains many more comfort features like SponsorBlock. It’s also highly customizable - I thoroughly recommend it.

You can find it’s code and get the APK file needed for installation on GitHub.

Scroll down to latest stable download and download the linked file.

Having connected to your device using adb, you can remotely install the APK file from your local file system as follows. You don’t need to transfer the apk file to your Android TV device - adb is capable of streamed installations of local apks to remote devices.

adb install smarttube_stable.apk

Make sure that the filename is correct and execute it.

Ad-free launcher

(Most) Android TV devices come with the standard Google Launcher as default. It’s looking slick but is unfortunately full of ads - not exactly what you might expect from your device, that you bought with your money.

Luckily there are many ad-free launchers out there that you can just install instead. A big name mentioned in many places would be “ATV Launcher”, which is well established and has many features.

ATV launcher’s design was a bit too crowded for my taste, some features require the paid version and it’s proprietary afaik.

If you don’t care about lots of widgets and are simply looking for a free, slick and open source launcher, I highly recommend FLauncher. It’s source code can be found on GitLab.

You can install it from the Play Store.

If you want to set a custom background image in FLauncher, check the chapter on uploading files onto Android TV. Once your desired image is in the file system, you can easily set it as background image in FLauncher’s settings.

Upload files onto Android TV

Having connected to your device using adb, you can push local files to Android TV as follows:

adb push my-local-image.png /sdcard/Pictures/my-remote-image.png

Note, that /sdcard represents the part of Android’s file system sometimes referred to as “internal SD card”. It’s basically the space where user data (Pictures, Music, Downloads, …) exists, which is why you usually want to put your files there.

Custom functions for physical remote buttons

Some Android TV devices come with physical remotes that exhibit buttons with weird functions - for instance my Xiaomi TV stick has a button that start’s a non-removable pre-installed app that doesn’t even work. With Button Mapper you can change what most of these buttons do (“remap” them).

This is also very helpful when using a custom launcher (see above to find out which one I recommend), because the home button (at least in my case) can be hard-wired to start the Google launcher and not another one which you might’ve set as default. In Button Mapper, just choose your home button and modify it to start an app - your desired launcher.

You can find Button Mapper in the Play Store and install it directly on your Android TV device.

Setting it up is self-explanatory so I won’t go into much detail here. Note, that some buttons might not be changeable - the premium version promises to allow for more buttons to be modified but I didn’t try it yet.

Volume too low or too high / Changing volume without remote

This was a funny one: Xiaomi’s TV Stick comes with a remote which automatically configures itself to control your TV as well. That’s very handy, because you only need one remote to turn your TV on/off, change the volume and navigate through Android TV at the same time.

Unfortunately, I’ve found the sound of any media playing through Android TV to be really quiet. I’ve had to put my TV on 100% volume to hear anything. For any other device 20% was more than enough.

The problem was that Android TV also has it’s own volume level - like any other Android device. The remote wasn’t changing that one though, because it was changing the TV’s volume instead.

So there are two volume levels to be adjusted but the remote can only change one. It is most likely possible to un-pair the Android TV remote and TV, turn Android’s volume up and pair the remote again but I find the following way to be much quicker.

With the following adb commands, we get a shell on Android TV and then simulate “volume up” button presses. I just did this a few times until the volume was on max and left it there. You should see a volume bar appear in Android TV, when you execute one of the input commands.

# Get a shell on your Android TV
adb shell
# Send a volume up key event
input keyevent 24
# Send a volume down key event
input keyevent 25

Thanks to ce4 on Android StackExchange for documenting this!

There is another, arguably more elegant way to do this using the media command in the adb shell. It allows you to directly set the volume to your desired value. I couldn’t find a media executable on my device though so I’m just documenting it for the sake of completeness. You can find a good example on StackOverflow.

Raspberry Pi: Configure WLAN/WiFi + SSH before first boot

May 15, 2022

Lots of Raspberry Pis are used for purposes that require a wireless network connection and remote access. To save time otherwise spent on connecting the Pi to a screen and keyboard or wired network to configure this, you can just configure network, SSH and an initial user before the first boot.

This post documents an interactive and a non-interactive approach.

Interactively with Raspberry Pi Imager

The Raspberry Pi Foundation provides an imaging tool, allowing you to not only flash the current Raspberry Pi OS to a given device but also to configure a few things in the same step.

You can find it here.

The tool is quite self-explanatory, so I won’t go into much detail here.

Select the source image and destination in the tool, click the “settings” button – the picture of a cogwheel – before clicking “Write”, and use the Advanced options menu to enter a username and password, along with any other preconfiguration you want.

Put the flashed micro SD card into your Pi, boot it up and enjoy!

Non-interactively with config files

If you don’t want to use the official imaging tool or you want to keep a set of config files to use often, here is another non-interactive approach.

To flash the image, you can use the official Imager, another interactive tool like Etcher or a CLI tool like dd.

Having flashed the Raspberry Pi OS image onto your Pi, use your file explorer or terminal to move to a newly created partition called boot on the micro SD card.

Initial user (mandatory)

Since 2022, Raspberry Pi OS doesn’t come with the well-known default user pi anymore. Instead, it asks you to create a user yourself on first boot interactively.

To create a user automatically on first boot, you have to provide a file userconf containing your desired user and password in the boot partition.

The file must contain your username and encrypted password like so:

username:encrypted-password

Make sure you are in the boot partition and use the following command, replacing myuser and mypassword, to create the file:

echo "myuser:$(echo 'mypassword' | openssl passwd -6 -stdin)" > userconf

For example, the combination myuser and mypassword should look like this:

$ cat userconf
myuser:$6$e1HRSQqoAr3UL0M8$nvsMz1SHlraA1R.2FZwbAq3CnM743E9.DZqBOYtnS317TaNbMoPM3OPskUmiTWUFwT.2y5k2FM7HrufRGi2Nr/

If you’re on Windows, make sure that userconf doesn’t have any other extension such as .txt.

WLAN

Create a file called wpa_supplicant.conf in the root of the boot partition and paste the contents corresponding to the WPA version you’re using.

If you’re on Windows, make sure that wpa_supplicant.conf doesn’t have any other extension such as .txt.

Change the country parameter as well as the ssid and psk accordingly and save the file:

WPA3 / WPA2 mixed network

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=US
network={
ssid="Your network name/SSID"
psk="Your WPA security key"
key_mgmt=WPA-PSK-SHA256
ieee80211w=2
}

WPA2-only network

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=US
network={
ssid="Your network name/SSID"
psk="Your WPA security key"
key_mgmt=WPA-PSK
}

SSH

Create an empty file simply called ssh in the boot partition. If you’re on Windows, make sure that ssh doesn’t have any other extension such as .txt.

First boot

Once all the files are saved, eject the SD card safely, put it in your Pi and boot it up. You should be able to connect via SSH using the user and password you’ve set before shortly after.

Windows Auto Dark Mode - Change multi-screen wallpaper

May 06, 2022

The tool

AutoDarkMode is a great open source tool to automatically switch between dark and light themes on Windows. You can set custom hours, switch manually or use sunset and sunrise from your location.

You can download it and contribute here on GitHub.

It can also change your wallpaper to a light or dark one - but only on one screen. Since I have a dual screen setup and a soft spot for ultra-wide wallpapers, there had to be a way!

Luckily, AutoDarkMode can call custom scripts on a light/dark switch event. With a simple PowerShell script, which I have adapted from a PowerShell function shared by Jose Espitia, we can set a given file as wallpaper in a certain mode (span in my case).

Download set-windows-wallpaper script

The script can be found on GitHub. You can clone the repo or download the file directly here (right-click -> save as).

Once downloaded, put the script into the location you want to keep it in permanently. This can be any folder you like really. I’ve put it into Documents.

The script can also set wallpapers in other modes - feel free to have a look into it, if you’re interested in that.

Change Powershell ExecutionPolicy

To run custom scripts, you need to change Powershell’s ExecutionPolicy. Start a Powershell as admin and run the following, answering [Y] when asked whether changes shall be made:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned

At this point you may test whether the script works. Run the following in Powershell. Make sure to adapt the path to the script and image to your setup.

C:\Users\val\Documents\set-windows-wallpaper.ps1 -File C:\Users\val\Pictures\Wallpaper\wallpaper-day.jpg -Style Span

Configure AutoDarkMode to run script

With the script in place and running successfully, you can configure AutoDarkMode to run it. Right-click on the AutoDarkMode icon in the system tray and click Open config directory.

In the config directory, open the file scripts.yaml and add the following to it. The file should already contain an example for a script. You can just delete that or comment it out for reference.

Make sure to replace the paths to the script and your desired light and dark images!

Enabled: true
Component:
 TimeoutMillis: 10000
 Scripts:
 - Name: Span wallpaper change
 Command: Powershell
 ArgsLight: [C:\Users\val\Documents\set-windows-wallpaper.ps1, -File, C:\Users\val\Pictures\Wallpaper\wallpaper-day.jpg, -Style, Span]
 ArgsDark: [C:\Users\val\Documents\set-windows-wallpaper.ps1, -File, C:\Users\val\Pictures\Wallpaper\wallpaper-night.jpg, -Style, Span]
 AllowedSources: [Any]

See the AutoDarkMode wiki for reference on adding custom scripts.

Test it

Now, you can test the script through AutoDarkMode by forcing a switch to the light and dark theme. Right-click on the AutoDarkMode icon in system tray and click the corresponding command. Your span wallpaper should change as configured.

If it doesn’t, make sure that the script works when run directly and that the config in scripts.yaml is correct. You can also consult the service.log file written by AutoDarkMode. It was very helpful in developing this script!

Workaround for broken DNS in WSL2

April 29, 2022

WSL2 uses the Windows host’s DNS - so if DNS is working on Windows, normally WSL2 should be fine as well! Unfortunately, DNS in WSL2 just randomly stopped working for me at some point. The /etc/resolv.conf file generated by WSL2 was still being correctly created but DNS just wouldn’t work.

Luckily there are plenty of ways to work around this, because we can just do DNS ourselves inside the Linux running in WSL2.

Prerequisite: Deactivate auto-generation of `/etc/resolv.conf`

For any of the following scenarios we need to stop WSL2 from writing to /etc/resolv.conf first, because we need to persistently modify it.

In WSL2, open a file /etc/wsl.conf, paste the following content and save:

[network]
generateResolvConf = false

If you also want to stop WSL2 from overwriting your /etc/hosts.conf, you need to add another line like so:

generateHosts = false

To have these changes take effect, you need to restart WSL2. Close your WSL2 window, start PowerShell as admin and run the following command to shutdown WSL2.

wsl.exe --shutdown

See here for reference, if you want to know in detail why this is necessary.

After the command has finished successfully, you can start WSL2 again.

Back in WSL, remove the broken DNS configuration file that WSL leaves behind.

rm /etc/resolv.conf

Next, we’ll set up DNS inside Linux. There are multiple ways of varying complexity depending on your use case.

One remote DNS server

If you just need to use one DNS server like a public DNS or a server provided by your organization that can resolve everything you need, this is the most simple setup.

In WSL2, open the file /etc/resolv.conf, paste the following content and save:

nameserver <your dns server>

If you wanted to use CloudFlare’s public DNS for example, it’d look like this:

nameserver 1.1.1.1

Multiple remote DNS servers

If you need to use multiple DNS servers, you’ll have to setup your own DNS server, which in turn uses the remote DNS servers you need. Don’t worry, it’s easy!

For example, in my WSL2 Ubuntu environment I need to use a specific DNS server for an internal domain and a public DNS for everything else.

Install dns server software dnsmasq

Firstly, install the dns server dnsmasq. You might need to use a different package manager depending on your Linux distro. This works on Ubuntu and other Debian-based distros:

sudo apt update && sudo apt install -y dnsmasq

Configure dnsmasq

After dnsmasq has been installed, open it’s config file /etc/dnsmasq.conf, add the following and save.

dnsmasq comes with a big config file full of deactivated settings as reference. If you want to keep all that, that’s no problem - I did it as well and just appended the following:

server=/<your internal domain>/<your internal DNS server>
server=<your external DNS server>
no-dhcp-interface=

<your internal DNS server> will only be used for domains under <your internal domain>. You can add as many of these as you want, if you have multiple internal domains for example.

<your external DNS server> is the public DNS server and will be used for everything else.

no-dhcp-interface= deactives the DHCP and TFTP capabilities of dnsmasq, because we only want it to provide DNS.

So if I have an internal domain called mycompany.internal, an internal DNS server 192.168.1.53 and the public CloudFlare DNS 1.1.1.1, it’d look like this:

server=/mycompany.internal/192.168.1.53
server=1.1.1.1
no-dhcp-interface=

Finally, start dnsmasq…

sudo service dnsmasq start

…and check whether it started successfully.

sudo service dnsmasq status

Get dnsmasq ip address

Now that we have installed, configured and started dnsmasq, we need to configure our Linux to use it as for DNS. We need to find the IP address that dnsmasq is listening on first.

This can be accomplished with netstat, which you can install with the package net-tools.

sudo apt install -y net-tools

This command gives us the IPs that any running instances of dnsmasq are listening on:

sudo netstat -tulpen | grep dnsmasq

If there is not output, make sure dnsmasq is up and running. If its not running, check for errors in the config file.

Set dnsmasq as DNS server to be used

Open /etc/resolv.conf and add the IP you’ve just found. In my case it was 127.0.0.53 but it can also be 0.0.0.0.

If it is 0.0.0.0 just put 127.0.0.1 as IP here.

nameserver 127.0.0.53

Save the file to start using dnsmasq as DNS server.

Start dnsmasq automatically at WSL start (hacky)

Since WSL2 is not a real Linux, we can’t use systemd or similar good ways to automatically start our newly configured DNS server for now (systemd is supposed to be added to Ubuntu WSL in the future).

I didn’t dig for a perfect solution here tbh, because I’m using Windows and WSL2 involuntarily.

That’s why I just wrote a tiny Shell script which checks whether dnsmasq is running and, if it doesn’t, starts it. This runs every time I open a shell in WSL2.

It’s not pretty and in a real Linux environment I’d never do it this way but for this specific use-case it’s totally fine imho.

Open your ~/.bashrc and append the following at the end of the file. If you’re using a different shell, such as ZSH, use the corresponding rc file like ~/.zshrc instead of ~/.bashrc.

service dnsmasq status > /dev/null
if [ $? -eq 3 ]; then echo "Dnsmasq not running, please provide sudo pw to start it..."; sudo service dnsmasq start; fi

Next time you start a shell, you’ll be prompted to enter your password to start dnsmasq, if it’s not yet running.

Fix silently failing Docker Desktop upgrade (Windows)

June 26, 2021

Docker Desktop has an internal update mechanism which will, as soon as an update is available, begin asking you to apply it on every start. Unfortunately this update mechanism is not working properly for me and other users. Once you allow Docker Desktop to run the update with admin privileges nothing seems to happen and Docker also doesn’t start properly until you restart it manually.

The cause seems to be some kind of privilege problem with the Docker service running in the background. Here is a workaround that works nicely for me.

Stop docker service

Start a command line such as PowerShell as admin and run the following command to stop the docker service (this will stop any running containers!)

net stop com.docker.service

Run installer

Download the most recent version of Docker Desktop from docker.com and run the installer, once the previous step was executed successfully.

The update should work now.

Sync system time automatically at Windows startup

June 19, 2021

Update: The Windows time sync service is not working reliably. It was working fine on my system for a few months but stopped syncing time properly eventually even though it is still being started automatically. Feel free to give it a try nonetheless but know that this seems to be another one of Windows’ great unreliable mysteries…

Unlike most other operating systems, Windows 10 doesn’t sync the time with an internet service at startup by default. This can become an issue, if your mainboard’s built-in clock provides a wrong time or uses a different time zone.

If the mainboard’s battery responsible for storing time dies, it can become an even more annoying problem, because Windows will always start with an ever-growing time skew, which can eventually result in web browsing become a problem due to Windows considering websites’ SSL certificates not yet valid - effectively rendering browsing the majority of web pages impossible!

This can be worked around by starting Windows’ time sync service automatically at startup. Below you’ll find instructions on doing this with the graphical Services console and PowerShell.

Services console

Press Windows Key + r at the same time to open the Run dialogue
Type services.msc and hit Run - the Services window opens
In the list, scroll to Windows Time (in german: Windows-Zeitgeber) and right-click on it, click Properties in the context menu
In the Properties window change Startup type to Automatic, click Start and click OK to save

PowerShell

Start PowerShell as admin
Run the following command:

Set-Service -Name W32Time -Status running -StartupType automatic

If PowerShell complains about not finding a service with the specified name, you can run the following command to get a list of all available services - Microsoft might have changed the time service’s name.

Get-Service

Let HomeAssistant notify you of updates

June 11, 2021

HomeAssistant releases new updates very regularly. Each month there is a new release which usually receives several patch releases as well. If you want to keep up with updates or even update automatically, it’s nice to get notified upon the release of an update.

Prerequisites

HomeAssistant has a built-in updater sensor which automatically checks for the updates and changes state, if it finds one. This can be used as trigger in an automation. The automation’s action can do all kinds of things; e.g. send a notification to a phone, which has the HomeAssistant app installed.

Having installed the HomeAssistant app on your phone, you can send notifications to it via HomeAssistant - including from an automation. The Mobile app integration must be enabled for this to work - it is by default.

If you want to be extra sure that the notification always arrives immediately, I’d suggest deactivating any battery optimization for the HomeAssistant app - at least on the recent versions of Android that goes for pretty much any app that sends notifications which you want to be sure to get without waking your phone from doze.

You can do it in the system settings; depending on the Android variant the path in the settings might vary but usually it’s here: Apps -> See all apps -> HomeAssistant -> Advanced -> Battery optimization -> Don’t optimize.

Automation

Using the updater sensor and app notification functionality, one can built an automation which sends a notification upon state change of binary_sensor.updater. You can find my automation snippet on GitHub.

The automation uses the most recent version’s number in it’s output text - this is accomplished by referencing an attribute of binary_sensor.updater called newest_version. You can change the notification’s content as you please by modifying the attributes below action -> data in the snippet.

Unless you have the exact same phone model as me, you must change the service in the automation’s action. In my case it’s set to notify.mobile_app_oneplus_nord_n105g - you must change it to your phone after having installed the HomeAssistant app and registered it with your instance.

The ID of your phone inside HomeAssistant can be found by opening the list of integrations <your homeassistant instance ip or hostname>/config/integrations, finding your phone there and clicking on the blue link ... entities. In the entity list, you’ll find a column called Entity ID and copy the phone’s name from there, replacing anything before the dot with notify as seen in the automation snippet.

The automation can be tested by setting the state of binary_sensor.updater from off to on in the HomeAssistant Developer Tools: <your homeassistant instance ip or hostname>/developer-tools/state

If the state was already on, you need to set it to off first.

The state change to on should trigger the automation and produce a notification like so:

Activation switch

Usually I also add a switch (“boolean input”) to my automations so I can quickly toggle functionalities from the HomeAssistant dashboard separate from the Automations config section. The code snippet for that can be found on GitHub as well.

It’s being referenced in the automation snippet, so you need to either add this switch as well as the automation or remove the relevant lines, if you don’t want the switch.

Having added it to a card called “Notifications”, it looks like this in the HomeAssistant dashboard:

Python cheat sheet for InfluxDB2 (influxdb_client)

March 02, 2021

Setup InfluxDB test instance with Docker

docker run -d -p 8086:8086 --name influxdb influxdb

This command pulls the latest InfluxDB image from Docker Hub, runs it with the name influxdb and makes the default port 8086 available on localhost. Once the container is up and running, we need to setup InfluxDB before we can use it.

Open a shell in the InfluxDB container:

docker exec -it influxdb bash

Start the interactive setup wizard by running influx setup. The wizard will ask you to provide initial admin credentials, your first organization, your first bucket and a retention period for it.

I have created a user val with a password (invisible in the wizard), an organization called my_org and a bucket my_bucket:

root@d8818fce8c18:/# influx setup
Welcome to InfluxDB 2.0!
Please type your primary username: val
Please type your password:
Please type your password again:
Please type your primary organization name: my_org
Please type your primary bucket name: my_bucket
Please type your retention period in hours.
Or press ENTER for infinite.:

Lastly, the wizard will ask you to confirm your inputs:

You have entered:
Username: val
Organization: my_org
Bucket: my_bucket
Retention Period: infinite
Confirm? (y/n): y
Config default has been stored in /root/.influxdbv2/configs.
User Organization Bucket
val my_org my_bucket

One advantage of performing the setup using the CLI is that it writes your initial admin user’s credentials to a config file, allowing you to manage your InfluxDB instance using the influx CLI directly with admin privileges.

We’ll use this to get our initial admin user’s token, which we’ll use later to set up the connection to InfluxDB in Python.

influx auth list gives us a list of existing users and tokens. For the sake of readability I’m using awk to limit output to just the token.

root@d8818fce8c18:/# influx auth list | awk 'FNR>1 {print $4}'
_U13AGG9YWojXjuh2OAUKqtsA20Fual8Vt_aVTvdJ1eWUJu0pH85ppkCEM3lL5hEpNS_8vPRh7nik8QkEvfZFA==

Save your token somewhere to have it at hand for the Connection setup step.

Now, we can exit the container shell:

root@d8818fce8c18:/# exit

Install influxdb_client with pip

Depending on your setup and linux distro you might want to do this with the distro’s package manager.

pip3 install influxdb-client # NOT 'influx-client' or 'influxdb'!

Connection setup (required for all steps below)

import influxdb_client

my_client = influxdb_client.InfluxDBClient(
 url=my_url,
 token=my_token,
 org='my_org'
)

To connect to the docker test instance, use the token obtained after the InfluxDB setup:

import influxdb_client

my_client = influxdb_client.InfluxDBClient(
 url='http://localhost:8086',
 token='_U13AGG9YWojXjuh2OAUKqtsA20Fual8Vt_aVTvdJ1eWUJu0pH85ppkCEM3lL5hEpNS_8vPRh7nik8QkEvfZFA==',
 org='my_org'
)

Test connection

my_client.ready()

{'started': datetime.datetime(2021, 2, 23, 16, 5, 29, 916723, tzinfo=tzlocal()),
 'status': 'ready',
 'up': '47m11.068245257s'}

Query data

Get query api

Firstly, obtain a query API instance:

influxdb_query_api = my_client.query_api()

Create a query

Check out the official documentation for the Flux query language to learn how to build queries.

This is a very basic example getting all data points of the measurement my_measurement from the last 100 minutes in my_bucket.

query = 'from(bucket:"my_bucket")\
|> range(start: -100m)\
|> filter(fn:(r) => r._measurement == "my_measurement")'

Run a query

result = influxdb_query_api.query(org='my_org', query=query)

See the official InfluxDB documentation for ways to use the result data including an example script.

Get dict from data point

For this example I have created a data point as shown here.

Assuming you have a influxdb_client.Point object, a dict representation of it’s contents can be obtained like so:

point.__dict__

The output will look something like this:

{'_tags': {'my_tag': 'my_value'}, '_fields': {'my_first_field': 'my second field value'}, '_name': 'my_measurement', '_time': datetime.datetime(2021, 3, 3, 13, 54, 7, 692337), '_write_precision': 'ms'}

Write data

Get write api

You need to specify which write option the client shall use when creating the instance. At the time of writing this post the possible write options are: synchronous, asynchronous, batching

There are shortcuts for synchronous and asynchronous.

influxdb_client.client.write_api.SYNCHRONOUS

influxdb_client.client.write_api.ASYNCHRONOUS

batching is accessible by creating an WriteOptions instance and passing it the WriteType of batching:

influxdb_client.client.write_api.WriteOptions(influxdb_client.client.write_api.WriteType.batching)

Using the write option, that you’ve decided for, you can get a write api instance:

influxdb_write_api = my_client.write_api(write_options=influxdb_client.client.write_api.SYNCHRONOUS)

Write a data point

A data point is created from an influxdb_client.Point object. To put our data in, we can either:

use various functions of the Point object itself
pass it a dict

I prefer using a dict, because I think it’s much more readable and easy to understand. This might be especially useful for people who know Python but are not perfectly familiar with how influxdb_client.Point objects work in detail.

my_time must be a timestamp, that InfluxDB understands.

point = influxdb_client.Point.from_dict(
 {
 'measurement': 'my_measurement',
 'tags': {'my_tag': 'my_value'},
 'fields': {
 'my_first_field': 'my first field value',
 'my_first_field': 'my second field value'
 },
 'time': 'my_time',
 },
 influxdb_client.WritePrecision.MS
)

influxdb_write_api.write('my_bucket', 'my_org', point)

For example, you can use the datetime module:

import datetime

point = influxdb_client.Point.from_dict(
 {
 'measurement': 'my_measurement',
 'tags': {'my_tag': 'my_value'},
 'fields': {
 'my_first_field': 'my first field value',
 'my_first_field': 'my second field value'
 },
 'time': datetime.datetime.now(),
 },
 influxdb_client.WritePrecision.MS
)

influxdb_write_api.write('my_bucket', 'my_org', point)

Buckets

Firstly, obtain a buckets API instance:

influxdb_buckets_api = my_client.buckets_api()

Get all buckets in current org

bucket_objects = influxdb_buckets_api.find_buckets()

Get all bucket names in current org

bucket_objects = influxdb_buckets_api.find_buckets()
bucket_names = [bucket.name for bucket in bucket_objects.buckets]

Get bucket object by bucket name

my_bucket = influxdb_buckets_api.find_bucket_by_name('my_bucket')

Create bucket

See this section for getting my_org_id

new_bucket = influxdb_client.domain.bucket.Bucket(
 name='my_new_bucket',
 retention_rules=[],
 org_id=my_org_id
)
influxdb_buckets_api.create_bucket(new_bucket)

Delete bucket

bucket = influxdb_buckets_api.find_bucket_by_name('my_bucket')
influxdb_buckets_api.delete_bucket(bucket.id)

Organizations

Firstly, obtain a organizations API instance:

influxdb_org_api = influxdb_client.OrganizationsApi(my_client)

Get all organizations

orgs = influxdb_org_api.find_organizations()

Get organization id

Look for your organization in the list of all organizations orgs, which you obtained in the previous step.

for org in orgs:
 if org.name == 'my_org':
 my_org_id = org.id
 print(my_org_id)

Create organization

my_new_org will be the new organization’s name.

influxdb_org_api.create_organization('my_new_org')

Delete organization

To delete an organization, you need it’s ID. See Get organization id.

influxdb_org_api.delete_organization(my_org_id)

Talk: How SUSE migrated 200k user accounts to UCS

February 16, 2021

During my time as a Consultant & Engineer with Univention, I supported Daniel Schmidt and his colleagues at SUSE in their migration of 200k user accounts from a legacy identity management to Univention Corporate Server (UCS).

It was a live migration over the course of weekend and I provided technical consulting as well as live patches and enhancements to UCS for issues and areas of improvements which were found during the migration.

During this year’s virtual Univention Summit, Daniel presented that story in a talk which I’d like to share with you. The talk is in German and - at this point - has no English subtitles.

Thanks a lot for the kudos @18:11 to you, Daniel! :)

I’ve had a great time working with you and your team. Best wishes from Bremen!

Python script for Cloudflare DNS record updates (DynDNS)

February 06, 2021

Note: This is an update to a post from 2019 and features a rewrite in Python with various new features including IPv6 support, logging, argument parser, …

Why

I’m using Cloudflare as CDN and DNS provider as well as domain registrar. For a system with an external IPv4 address that changes daily I needed a way to dynamically update a DNS record at Cloudflare with the system’s current external IP address.

There are numerous ways to do DynDNS but I wanted to get into Cloudflare’s API anyway and it turns out that this is, at least in my opinion, much easier to set up than some generic DynDNS package.

How

Getting the script

I’ve written a Python script, based on a bash script by benkulbertis.

The Python script does not require a general API token. So you can set up API tokens specifically authorized for what the script needs to do.

You can get the script here.

wget https://raw.githubusercontent.com/va1entin/tools/master/cloudflare_update_record/cloudflare_update_record.py

Setting up API tokens

The script needs two tokens:

one to read DNS records and settings
one to actually edit a DNS zone

To set the tokens up, log in to your Cloudflare account and go to this page.

Note, that you can also limit the tokens to a specific domain! This is of course important to know if you have multiple domains in your Cloudflare account and the script shall only read/edit the settings of one of these.

Edit token

On the API token page, click Create Token and use the template Edit zone DNS.

Alternatively, click Get started next to Create Custom Token and configure the token as follows:

Type	Resource	Permission
Zone	DNS	Edit

Click here for a screenshot.

Read token

On the API token page, click Create Token and click Get started next to Create Custom Token.

Type	Resource	Permission
Zone	DNS	Read

Click here for a screenshot.

Create config file

The script uses a YAML-format config file and assumes it at ./cloudflare_update_record_config.yaml. You can give a different path using the -c | --config parameter.

read_token: "<YOUR READ TOKEN>"
edit_token: "<YOUR EDIT TOKEN>"
zone_name: "<YOUR ZONE NAME>"
record_name: "<YOUR RECORD NAME>"

So if I wanted to change a DNS record called hello.example.com using a read token foo and an edit token bar, the config would look like this:

read_token: "foo"
edit_token: "bar"
zone_name: "example.com"
record_name: "hello"

To change the root record of your domain - example.com itself - use @ as record_name, just like in the Cloudflare dashboard:

read_token: "foo"
edit_token: "bar"
zone_name: "example.com"
record_name: "@"

Run script

Lastly run the script with your desired parameters. I recommend reading the brief usage info at least once:

./cloudflare_update_record.py -h

Logging

The scripts logs to cloudflare_update_record.log with log level info by default. You can change the log file and log level, see -h.

IP provider

It gets the external IP address from a provider, by default: icanhazip.com

You can specify a different provider, see -h. The provider must return just the IP address as plain text on a HTTP GET request - no additional HTML or anything else. Assuming an external IPv4 address 1.1.1.1, it should look like this, when tested with curl:

# curl https://ipv4.icanhazip.com
1.1.1.1

Update a DNS A (IPv4) record

cloudflare_update_record.py -4

Update a DNS AAAA (IPv6) record

cloudflare_update_record.py -6

Check the log for a message INFO: DNS A record update succeeded, IP changed to: <YOUR IP ADDRESS> and remember that it might take up to 24 hours for your DNS update to be propagated around the world.

Simple CLI audio metadata editor: tag_dat.py

December 05, 2020

Audio metadata is often not particularly easy to manage in a simple and quick manner. There’s lots of tools providing this feature but many of them have a ton of other additional functionality or are full-on music library managers. I wanted a very simple command-line tool to quickly change the title, artist and album tags of audio metadata - the result is “tag_dat.py” and available in my tools repository.

tag_dat.py iterates over a file or all files in a given path (default: current path), sets the filename (without file ending) as title and given arguments album and artist as audio tags. The script uses the amazing “mutagen” library under the hood.

Here are a few usage examples:

# Set MyArtist and MyAlbum for all files in current path
./tag_dat.py -ar MyArtist -al MyAlbum

# Set MyArtist and MyAlbum for my_file.mp3
./tag_dat.py -ar MyArtist -al MyAlbum -f my_file.mp3

# Set MyArtist and MyAlbum for all files in path music/MyAlbum/
./tag_dat.py -ar MyArtist -al MyAlbum -p music/MyAlbum/

Get Ledger Live working on NixOS

November 14, 2019

The Ledger Nano series are hardware wallets for crypto currencies. They’re managed through an application called Ledger Live. Ledger Live runs on linux but might - depending on your distro - require some adjustments in order to correctly identify the device via USB. I couldn’t test whether this config works with all Ledger devices so far. Feel free to drop me a tweet if you have anything to add to this post and I’ll gladly update it.

On NixOS I needed the following additional config snippets to use my Ledger fully:

Define a group called “plugdev”

groups.plugdev = {};

Add user(s) to “plugdev”

Note that you your user will most likely already be in various other groups such as networkmanager or wheel. Just append plugdev to the extraGroups list. Replace myUser with your user name.

users.users = {
...
 myUser = {
 extraGroups = [ "plugdev" ];
...
 };
};

Add extra rules for udev

Ledger provides udev rules which need to be added kind of hidden (linux -> Troubleshooting -> Option 3) on their website. You can check out my NixOS config containing these rules on GitHub. I’ve decided to not statically post them here and instead refer to the GitHub mirror of my NixOS config, because they might change with Ledger firmware updates. You must change “val” to your username.

I’m not entirely sure whether it’s really necessary but I reload the rules after I’ve changed them and rebuilt NixOS just to make sure they are being applied:

udevadm trigger
udevadm control --reload-rules

Providing access to thousands of external storages using Nextcloud

October 28, 2019

A few weeks ago I’ve held a talk involving a solution I’ve built @univention for a customer using Univention Corporate Server and Nextcloud at this year’s Nextcloud conference.

The talk is available on YouTube.

The documentation for the cool solution I’m referring to in the talk can be found in the Univention Knowledge Base.

Thanks a lot to the folks at Nextcloud and especially Nina and Jos for the invitation and great discussions!

Nextcloud for UCS now comes with SSO out of the box

September 29, 2019

A few weeks ago I’ve held a talk involving a solution I built @univention for a customer using Univention Corporate Server and Nextcloud at this year’s Nextcloud conference. The video of that will probably be available in the next few weeks and I’ll share it here once it’s available.

UPDATE: See here for the video of the talk.

Another interesting thing regarding Nextcloud and UCS happened in the meantime though: The Nextcloud app for UCS now comes with pre-configured Single Sign-On (SSO) using SAML. This is very cool because lots of our customers want to make use of SSO but the configuration is not always trivial. Using a blog post I wrote for the Univention blog in February, Nextcloud, namely Arthur Schiwon aka @blizzz, were able to bake the configuration into the app and thus make it really easy for people to get started with UCS, Nextcloud and SSO! If you’d like to find out more about SSO with SAML, feel free to check out my blog post about it.

Simple file encryption module for Python projects

September 08, 2018

tl;dr

Plasm focuses on a very specific use case: encrypting and decrypting files using public key encryption. If plasm doesn’t fit the scope of what you want to do, feel free to have a look at pynacl’s documentation and use it directly.

Why

I have experimented a lot with camera software for the Raspberry Pi. One project that particularly stuck with me is motionEye, a web frontend for the motion daemon, which allows you to easily set up an open source motion triggered camera for about 50€.

MotionEye also offers a cloud upload functionality, which I like a lot. Unfortunately it doesn’t allow the user to encrypt their media files before uploading them. Since I wanted to use Google Drive as my data grave but at the same time not upload unencrypted media files into the cloud, I decided to build a simple python module to easily hack file encryption into motionEye.

What it is

The result of this is plasm, which stands for PynacL AbStraction Module. Plasm is based on pynacl, which in turn binds to libsodium.

Plasm uses public key encryption and is supposed to make the creation of public key pairs as well as encryption and decryption of files very simple.

One key requirement for me was that plasm had to be able to encrypt using only a public key. The private key, which is required to decrypt, shouldn’t be required so that I could store it in a safe place on another device. To accomplish this, I chose to use the sealedBox construct, because it only requires a public key. The main difference of the sealedBox compared to another construct simply called Box is that it doesn’t offer cryptographic proof of the sender’s authorship, which is out of scope for plasm anyway.

Additionally, I wanted to protect the private key by encrypting it as well using secret key encryption. That’s why plasm requires a password for creating a key pair and decrypting files. The password is used to encrypt and decrypt the key so that an attacker needs both the private key file and the password to be able to decrypt files.

Installation and how it works

Please have a look at the README to find out how to use plasm.

Plasm is currently not available via PyPi, but you can install it directly from Github as follows:

wget https://github.com/va1entin/plasm/archive/refs/heads/master.zip -O plasm-master.zip
pip3 install plasm-master.zip

Integration with motionEye

The integration with motionEye currently requires a little patch to be applied to it’s uploadservices.py, which is located at /usr/local/lib/python2.7/dist-packages/motioneye/uploadservices.py on Raspbian. You can download it directly here (right click -> save as).

Make sure to change the path to your public key accordingly and add remove_input_file=True as shown in the README, if you want plasm to remove the unencrypted input file once encryption is finished.

Plasm also logs what it’s doing using the logging module by default. Most logging happens at debug log level currently. Have a look at the motionEye wiki to find out how to enable debug logging.

Conclusion

Plasm is licensed under the Apache 2.0 License. Feel free to report issues or create merge requests at Github. Please note that I don’t plan to extend plasm’s scope on other use cases at the moment to keep it as simple as possible. If plasm doesn’t fit the scope of what you want to do, feel free to have a look at pynacl’s documentation and use it directly.

Setting up unattended upgrades on Debian

August 25, 2018

Why

I have a Raspberry Pi running on Raspbian in my home network. It mainly provides a DNS server based on Pi Hole and runs some cronjobs for me. Of course you don’t want to login to such a system regularly and install package upgrades. That’s where unattended upgrades come into play. I’ll tell you how to set unattended upgrades up on Debian and walk you through some config parameters that I find very useful.

How

Firstly, you need to install the unattended-upgrades pkg:

apt install unattended-upgrades

unattended-upgrades is run automatically by the cronjob /etc/cron.daily/apt-compat.

The config file

Now let’s have a look at the config file:

vim /etc/apt/apt.conf.d/50unattended-upgrades

The config file might look a bit messy at first due to a LOT of comments and the (in my opinion) unaesthetic apt config syntax. The most interesting part to begin with is the “Origins-Pattern”:

Unattended-Upgrade::Origins-Pattern {
"o=Debian,a=stable";
"o=Debian,a=stable-updates";
"o=Debian,a=proposed-updates";
"origin=Debian,codename=${distro_codename},label=Debian-Security";
};

Defining package origins

The Origins-Pattern defines from where unattended-upgrades will install upgrades. If an origin is missing here, package upgrades from there will be ignored. Which origins to configure here is totally up to you of course. Maybe you have installed some exotic PPA, which you always want to upgrade manually? To find out how to configure the origins you want, you can have a look at apt’s lists:

val@pi:~ $ ls /var/lib/apt/lists/ | grep InRelease
archive.raspberrypi.org_debian_dists_stretch_InRelease
raspbian.raspberrypi.org_raspbian_dists_stretch_InRelease

These InRelease files provide the information you need. To find out their origin and suite (o= and a=), use the grep command:

val@pi:~ $ grep -E '(Origin|Suite)' /var/lib/apt/lists/archive.raspberrypi.org_debian_dists_stretch_InRelease
Origin: Raspberry Pi Foundation
Suite: stable

So the origin of this particular apt list is “Raspberry Pi Foundation” and it’s suite is “stable”. To configure this for unattended-upgrades, you’d adapt the config to look as follows:

Unattended-Upgrade::Origins-Pattern {
"o=Raspberry Pi Foundation,a=stable";
"o=Raspbian,a=stable";
"o=Debian,a=stable";
"o=Debian,a=stable-updates";
"o=Debian,a=proposed-updates";
"origin=Debian,codename=${distro_codename},label=Debian-Security";
};

Defining package blacklist

If you don’t want to upgrade certain packages from a defined origin automatically, you can put them in the blacklist, that should already be in the config file:

Unattended-Upgrade::Package-Blacklist {
// "vim";
// "libc6";
// "libc6-dev";
// "libc6-i686";
};

Automatically remove unused dependencies (autoremove)

To automatically remove unused dependencies, set the following config parameter to true:

Unattended-Upgrade::Remove-Unused-Dependencies "false";

Automatically reboot

Some packages require a reboot. You probably want to control when that happens. In my case the pi reboots at 2 AM, when everyone’s usually asleep, if needed.

Unattended-Upgrade::Automatic-Reboot-Time "02:00";

Debugging

unattended-upgrades offers a debug parameter, that makes debugging very easy. With –apt-debug and –verbose you can get even more debug output. To debug previous runs of unattended-upgrades, check the log files: /var/log/unattended-upgrades/unattended-upgrades.log and /var/log/unattended-upgrades/unattended-upgrades-dpkg.log.

val@pi:~ $ sudo unattended-upgrades --dry-run --debug
sudo: unable to resolve host pi
Initial blacklisted packages:
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: ['o=Raspberry Pi Foundation,a=stable', 'o=Raspbian,a=stable', 'o=Debian,a=stable', 'o=Debian,a=stable-updates', 'o=Debian,a=proposed-updates', 'origin=Debian,codename=stretch,label=Debian-Security']
...

Userscript to hide YouTube comments

March 17, 2018

Why

Most comments below YouTube videos are, well, irrelevant, to say the least. Since they’re pretty much in the middle of the page, they still regularly caught my eyes and distracted me when I was checking the recommended videos and scrolled down the site. That’s why I built a little userscript to hide the comments by default and make them visible with the click of a button.

Installing the script

Chrome

Just download the script by right-clicking and selecting Save link as. Open chrome://extensions and drag-and-drop the file into Chrome. Chrome should ask, if you want to install it. You can also disable or remove it here.

Alternatively, you can also use a dedicated userscript manager from the Chrome web store. Read the paragraph on Firefox, to learn how to install the script in such a manager.

Firefox and others

Firefox requires you to install a userscript manager add-on. One of the most famous ones is Greasemonkey. I’m using a relatively new add-on called Tampermonkey, though, because it is available across different browsers and also offers a nice configuration interface with a built-in editor. Feel free to use whatever you want. I’ll describe how to install the script in Tampermonkey here.

After Tampermonkey is installed, click on the following link. Tampermonkey should ask, if you want to install the script: Click to install

Using the script

With the script enabled, comments will be hidden by default.

You can make them visible by clicking the button Show and scrolling a little.

You can also easily modify the texts on the buttons. Just open the script and change the two values of the two variables at the very top as you wish.

Source code

The source is available on GitHub.

How to play YouTube in background on mobile

March 05, 2018

One of the most requested features for mobile devices in, well, forever, is playing YouTube videos without having the app in foreground. YouTube introduced this feature with YouTube Red for paying customers. So if you’re lucky enough to live in one of the countries, where you can actually subscribe to YouTube Red, you can pay a few bucks per month to get this.

If you’re not so lucky (like me) or just don’t want to pay, you could still use a mobile browser. Until recently at least, when YouTube blocked playing videos without having the according tab open on mobile devices. This is accomplished by utilizing the so called page visibility and fullscreen API. This allows a webpage to check, if you’re actually viewing the page.

Luckily, Timothy Chien built an add-on for Firefox that blocks these APIs and allows you to listen to YouTube videos with another tab open or Firefox in background entirely on mobile.

Just download Firefox from your favourite app store, if you haven’t already, and visit this page to install the add-on.

This way you don’t need to access YouTube in desktop mode, change your user-agent or do anything else.

Also, the code for the add-on is available on GitHub. So feel free to contribute!

Fix NoScript 10 (Quantum) default settings

January 09, 2018

The release of Firefox 57 was a major disruptive event in the Mozilla community. One of the major changes was the migration of add-ons away from XUL/XPCOM to WebExtensions. While this change makes it easier for developers to integrate their existing (Chrome) add-ons with Firefox, it also forced Firefox add-on developers to rewrite their entire add-ons from scratch. Some devs (like the very popular DownThemAll) were very frustrated and abandoned decades of code and tens of thousands of users.

Giorgio Maone, the developer of NoScript, one of the most famous and important add-ons, decided to not let his users down and migrated it to WebExtensions. The result is an entirely new UI and a whole new workflow. Being a NoScript user for about 10 years now, it took me quite some time to get used to everything but I managed to get used to the new NoScript.

But there’s one thing I didn’t get used to: the new default settings.

The new NoScript comes with quite a lot of websites, that are fully trusted by default. Some of these might be totally rational to most users (google.com, youtube.com, etc…), others not so much (wlxrs.com, a parked domain, as far as I can tell). Nonetheless, I don’t want NoScript to decide who I, as it’s user, trust. Another weird default setting are the “default” permissions for websites. There are 3 primary states: Default, Trusted and Untrusted.

Default = frames, fetch, “other”

Trusted = Anything

Untrusted = Nothing

Since every site, that is not explicitly trusted or untrusted, gets assigned the “default” state, it is allowed to do some things and it’s not even entirely clear what these are (“other”).

So I modified the default config, to not allow anything by default and also not trust or untrust any website. You can import this config either by downloading this file and using the import button on the NoScript options page to upload it or by copy pasting the following JSON into the text field that appears at the bottom of the page when the “Debug” checkbox is activated:

{
 "DEFAULT": {
 "capabilities": [],
 "temp": false
 },
 "TRUSTED": {
 "capabilities": [
 "script",
 "object",
 "media",
 "frame",
 "font",
 "webgl",
 "fetch",
 "other"
 ],
 "temp": false
 },
 "UNTRUSTED": {
 "capabilities": [],
 "temp": false
 },
 "sites": {
 "trusted": [],
 "untrusted": [],
 "custom": {},
 "temp": []
 },
 "enforced": true,
 "autoAllowTop": false
}

Generating CPU load on Windows

December 19, 2017

I recently wanted to check, if there was a problem with my CPU under heavy load, when running on Windows. For such purposes CPUSTRES.exe from the Sysinternals suite is very helpful.

CPUSTRES is a handy little tool to check your process for faults on heavy load. It comes with a simple GUI and allows you to instantly put heavy CPU load on your system.

CPUSTRES comes as a simple standalone program with a GUI and 4 threads to activate. You can also assign different intesities per thread. Task manager helps checking how much load is actually put on the CPU.