Home Uniquely NZ Travel Howto Pauline Small Firms Search
Upgrading Corinna's Electrical System
With Remote Access and Management via a Raspberry Pi

The initial document on the Upgrade to Corinna's Electrical System has got far too long so I have decided to cover this addition of a Gateway to allow remote monitoring using a Raspberry Pi on new pages. There are currently two new parts: the straightforward addition of The Raspberry Pi using the Venus OS and The Extension to Venus OS Large with Node-RED to allow the addition of a dedicated Dashboard and additional smart management.

Introduction

The Victron Smart devices I am using on Corinna (and some of their earlier devices) offer multiple ways to control, configure and view their data. Initially I just used the Victron SmartConnect App on an Android Phone and the in-built Bluetooth in my Smart devices - that is very effective but the combination has some shortfalls, namely the bluetooth is short range (<10m), only one device can be viewed at a time and the App can not run in the background. Victron however offer several more ways to connect their devices for different purposes which I am going to explore here. The earlier devices and most of the new devices can connect directly via Victron's VE Direct which is basically a serial connection using a small standard connector ( JST PH 2.0 4pin terminal connectors) on the devices and connections, generally in a star configuration to a 'host'. I am only going to use such connections in this article but for completeness I should mention some other alternatives.

Some Victron devices also have VE.Can bus connections which are mainly used for interchanging real time information between devices to keep them synchronised - a battery may need to control the charge rate from a charger and several inverters may need to be synchronised to give a 3 phase output. They are generally daisy-chained together. To quote Wikipedia: "A Controller Area Network (CAN bus) is a robust vehicle bus standard designed to allow micro controllers and devices to communicate with each other's applications without a host computer. It is a message-based protocol, designed originally for multiplex electrical wiring within automobiles to save on copper, but it can also be used in many other contexts. For each device, the data in a frame is transmitted sequentially but in such a way that if more than one device transmits at the same time, the highest priority device can continue while the others back off." NMEA-2000 which is used for integrating marine instruments and other marine systems in a one example of CAN bus. There is yet another daisy-chained connection used by Victron Devices called the VE.Bus. The Smart devices that I need to connect all have VE.Direct and Bluetooth connectivity and that is all this article will consider.

Victron make a series of what they call GX devices which in their words "Operate as a communication centre of an installation. All the other system-components - such as inverter/chargers, solar chargers, and batteries - are connected to it. Monitoring can be carried out locally and remotely - via the free-to-use Victron Remote Management portal (VRM). The GX-device can also provides Remote firmware updates and allows inverter/charger settings to be changed remotely." I have not yet found what the GX stands for but Global or Gateway Xconnection seems to sum it up! There are at least 6 Victron GX devices with varying processing power, number of connections, types of connections, sensors monitored and local display options. They are all Linux based and all run the same customised Linux distribution called Venus. It all seems very well thought out and compatible and one of the most important features is that they all act as a Gateway to the Victron VRM portal which you can log into via any browser or their apps anywhere in the world and see all the current and historic data. GX devices can store data locally for long periods if their internet connect is broken and then upload when connectivity is restored. Once you have a GX device live you can use the existing SmartConnect Apps through the VRM and GX device to monitor and configure just the same as locally via bluetooth or a VE.Direct cable. The VRM portal gives very powerful display options and allows you to monitor for any errors and set alert levels on key parameters - you can then have emails sent to warn you of problems. The VRM portal is a free service for registered Victron users.

The GX devices are however not cheap, the lowest prices are for the Venus GX and its recent replacement the Cerbo GX are ~£287 with an extra £215 if you have a local display on the Cerbo and cables are an extra ~£15 per device. However Victron has committed to make certain of its projects open source including the Venus OS, the Linux distribution used to run all the GX devices. In fact Victron note that while being a very successful project in its standard configuration, the Venus OS, and compatible hardware platforms can also be the perfect platform for Open Source contributions. They give examples such as adding drivers for extra products and, if interesting enough, they offer to add transmission and readout of that data to the VRM Portal. In particular, adding readouts from generator, start/stop, and temperature sensors would be very interesting to them as would be add logic on a product, for example to control the relay, or do other things.

Most important for me is that they offer full information about Venus OS; and compatible hardware including how to install on a Raspberry Pi at https://github.com/victronenergy/venus/wiki/raspberrypi-install-venus-image which contains several other links worth following. The story behind this can be found in their blog at https://www.victronenergy.com/blog/2017/09/06/raspberry-pi-running-victrons-venus-firmware/ . To cut a long story short I found this to be sufficiently encouraging that within a couple of days of finding the paper above I had ordered various bits including a Raspberry Pi 3B+ , taken delivery and amazingly commissioned the complete system running it locally and via the portal. I will however go into more detail for those who wish to follow in my footsteps but before doing so consider the complexity of your system - the Victron Cerbo GX can handle dozens of devices with many different types of connection without converters, whilst the RPI 3B+ just has 4 USB ports and even the VE.Direct inputs need converters rather than cables. It is a very cost effective and fun solution for a small system like mine but less so for a complex system.

The Full(er) Raspberry Pi Setting-up Story

The Raspberry Pi is a small single board computer designed to have many accessible outputs and inputs to allow easy interfacing to hardware and as teaching tool for coding. It has proved very popular and over 40 millions have been sold making it the best-selling British computer. Most Pis are made in Wales. There have been a number of versions and the hardware now includes good Wifi and Bluetooth as well as USB ports, an ethernet input and an HDMI output for a monitor. The Raspberry Pi uses a SoC (System on a Chip) as an almost fully contained microcomputer and the 3B+ version has a custom Broadcom 1.4 GHz 64-bit quad core ARM Cortex-A53 processor with 1 GB RAM. This SoC does not contain any kind of data storage, which is common for a microprocessor SoC, but there is a socket for a microSD on the board.

The latest version of the Raspberry Pi is version 4 but the Venus OS currently works best on version 3B+ . The Raspberry Pi 3B+ is still in production but is currently on long delivery in many places including Amazon. So I was precipitated into ordering before I had done as much homework as usual when I found The Pihut had the Pi 3B+ in stock at the standard £34. Rather than mess around I also ordered the official 2.5 Amp mains power supply for development and a rather nice and very flexible official case for £6 to protect it as well as an extra 16 Gbyte Sandisk Edge microSD card (class 10) at £6. It all came by courier in under 24 hours leaving me time to get it working before ordering the expensive VE.Direct to USB cables to be delivered. They came from OnBoardEnergy by courier the following day from enabling me to have the end to end system running in well under 48 hours from locating a source of a Pi 3B+.

I am always very suspicious when people say anything involving computers is quick and easy but setting up the Pi was. I followed the procedures at https://https://github.com/victronenergy/venus/wiki/raspberrypi-install-venus-image fairly closely and I recommend you do the same so I will not go into a lot of detail. I downloaded the latest Venus OS which at the time was 2.73 and unzipped it (double click then drag the image out to the same download folder or anywhere you can remember). I used BalenaEtcher https://www.balena.io/etcher/ to burn it onto the microSD card rather than my standard Mint software on my home machines. BalenaEtcher is one of those pieces of software which just seems to work including verifying the resulting image on the card - it comes as an Appimage so it is self-contained and just needs a double click to run under Linux or most operating systems. Once flashed the MicroSD card slips into a socket under the board which was still accessible after the case was assembled. I followed the recommended route and started with a network cable plugged into board and connected power and opened my browser and went to http://venus.local and it was all running just like that. It was about 30 mins from unpacking and examining the bits to having everything running via a network cable!

The only problem was that the WiFi was not working. It quickly became obvious the WiFi was not enabled on the board by default so I added a HDMI cable to a monitor and plugged in a keyboard/pad giving me command line access to see what was going on. I tried the connmanctl command line utility as it was already installed. See https://wiki.archlinux.org/title/ConnMan#Wi-Fi . I confirmed that wifi was disabled by running connmanctl in a terminal:

connmanctl technologies

and checked the line that says Powered: True/False to confirm it was disabled.

To power the wifi on you run 

connmanctl enable wifi

I checked again giving:

$ connmanctl technologies
...
...
/net/connman/technology/wifi
Name = WiFi
Type = wifi
Powered = True
Connected = True
Tethering = False
$

After that I could return to browse venus.local and used the Settings menu to find and connect to my WiFi. I initially used the house WiFi for testing. While I was using a terminal I also set up SSH so I had Remote terminal access and also set a root password. I will cover that in the next section. Once it was all checked I unplugged the monitor and keyboard and went to the Corinna where I changed the Wifi to my old tethered phone which provides my internet access on the boat. This was about two hours after DHL rang the doorbell with my box from the Pihut.

This was the point I ordered my VE.Direct to USB cable. They are expensive at £26.99 and instructions exist on the Internet on how to make your own but the proper ones have an isolated connection which none of the home made ones have. Victron insist that they need to be isolated to avoid interference and earth loops and to protect one's expensive equipment - it has been reported that it can be zapped by lightening strikes which is a risk on sailing boats but less so on canals. I spent a while exploring the use of the VRM portal and various ways of connecting, monitoring and configuring. I was surprised how easy and flexible it all was. The only thing to note is that the the VRM portal is very powerful and flexible for monitoring and setting alerts etc but the SmartConnect App is the best way to configure ones devices, either locally using Bluetooth or remotely using the VRM tab in the App.

Setting Up SSH Access (Optional but useful to explore within the system)

If you have to ask what SSH is you should ignore this section. If you know what it is you probably realise that it is nice to have Shell (Console or Terminal) Access over your local network for simple jobs especially on a machine with no keyboard or display (headless in the jargon) and I wanted to be able to explore the system. The SS stands for Secure Shell as it is an encrypted connection, the H goes back to an earlier less politically correct definition.

The procedures are different to what I am used to in Debian based distros such as Ubuntu or Mint which is what I normally use - the Venus OS does not seem to use sudo or su for superuser access but uses an actual login as root which can be extremely dangerous so take care. This is sort of covered in https://https://github.com/victronenergy/venus/wiki/raspberrypi-install-venus-image which I followed at the time.

You first need to set the root password and to do so you must first set your access level to Superuser. A Superuser is Godlike and can do anything including trashing the whole system but I suppose that does not matter if it only takes a few minutes to flash a new SSD.

Go to Settings -> General and set the Access Level to User and Installer. In my case it was already User and Installer, if not the password is, I understand, ZZZ which is at least less common than many defaults.

Now you have access to the Superuser features and can create a root password

Go to Settings -> General -> Set root password and create a root password.

Note that, the root password will be reset by a firmware update. The reason is that the passwd file is on the rootfs, which is fully replaced by an update.

Before you can login via ssh, you must enable Remote Support (Settings -> General). Besides enabling the reverse tunnel it also enables the sshd daemon - all very impressive sounding but you do not need to understand the details.

To login, enter the ip address of the GX device in a ssh client. Most users will simply do this from the command line in a Linux system by:

ssh root@venus.local

or use a ssh client app if you are using android or windoz. The first time it will require extra conformation to add you to a list of known hosts as below - ignore the gobbledygook and just answer yes (not Y).

:~$ ssh root@venus.local
The authenticity of host 'venus.local (192.168.1.92)' can't be established.
ECDSA key fingerprint is SHA256:rSpOAXHFgBKNBzuhB0SHxxxndjdCSpyQas46e0SrVzY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'venus.local,192.168.1.92' (ECDSA) to the list of known hosts.
root@venus.local's password: ************
Last login: Tue Nov 9 10:59:22 2021
root@raspberrypi2:~#

I usually login in as a normal user rather than root but in this case you need to to be the Superuser (ie logged in as root)

The advice is to create a root password. But use it to login only the first time, and then install a public ssh key(s). Thereafter login with the keys. Again ignore this if you do not understand public and private keys and settle for the procedure above as I am doing. Using a ssh key for authentication, instead of a root password, has the advantage that it isn't lost during a firmware update as the keys are stored on the /data partition. The Victron article has details.

SFTP: An additional advantage of enabling SSH access is that you are actually doing more than that as other access routes also become available through the SSH daemon including SFTP (Secure File Transport Protocol) which means that you can use an FTP program such as Filezilla on your machine which enables you to view a remote file system as easily as a local one, upload and download files and carry out simple actions on the files such as rename, delete and setting permissions. SCP (Secure Copy Protocol) is also available for command line transfers which can be faster but less robust.

Android SSH Apps: I want to be able to access from my phones and pads over SSH. I already had JuicySSH on some of them and I added an extra Keyboard overlay as the standard touch keyboard is missing many keys needed for programming or terminal use. In particular the standard editor available in the Venus OS seems to be vi which makes use of Escape to change modes. I am using the popular Android 'Hackers Keyboard' App which has everything I need. I have used the AndFTP App to provide FTP access.

More about The Venus OS (for power users and the curious!)

The following is mostly notes from my early reading and has been augmented when I have done some more exploring:

The Venus OS is a customized Linux. It uses a dual file system boot mechanism: two bootable partitions that swap between active and backup with each software update, plus a data partition that survives a software update. Anything extra one installs will generally need to be reinstalled after a Venus OS update. I believe that by version 2.80 the root filesystem will be read only

The underlying Linux distribution is openembedded-core (currently Yocto Dunfell in v2.80). As the name implies it is optimised for embedded systems, normally headless with no keyboard or monitor, with minimal utilities and users like me who have used desktop distributions such as Debian and its derivatives Ubuntu and Mint will find it very different.

It uses BusyBox so some common commands may have reduced functionality.

The apt package manager is not installed. One needs to use opkg to install things from the Victron repository which is comparatively restricted in the packages it provides. wget is however available in Busybox for downloads.

Adding extra functionality to Venus should be possible as long as there are no conflicts between the hardware used by Venus and the apps you install and provided you have enough processing power.

You may need to tweak partition sizes in order to fit any added functionality.

Raspberry Pi versions 2B and 3B are currently fully supported. Raspberry Pi version 4 support is still being developed with initial support as part of Venus v2.70 but still, I understand, flaky as of v2.73 which was the latest version when I bought my Pi.

More on opkg: Most Linux distributions (or mobile device operating systems like say Android or iOS) allow the functionality of the system can be upgraded rather significantly by downloading and installing pre-made packages from package repositories (local or on the Internet). The opkg utility is a lightweight package manager is designed to add software to stock firmware of embedded devices and serves as a full package manager for the root file system, including kernel modules and drivers. opkg attempts to resolve dependencies with packages in the repositories - if this fails, it will report an error and abort the installation of that package. Missing dependencies with third-party packages are probably available from the source of the package. See https://openwrt.org/docs/guide-user/additional-software/opkg for more information. I understand the Victron Venus Repository is restricted in the applications it offers. opkg list-installed lists all installed packages

More on Busybox: BusyBox is a software suite that provides many Unix utilities in a single executable file. It runs in a variety of environments such as Linux and Android although many of the tools it provides are designed to work with interfaces provided by the Linux kernel. It was specifically created for embedded operating systems with very limited resources. The authors dubbed it "The Swiss Army knife of Embedded Linux", as the single executable replaces basic functions of more than 300 common commands. In fact, looking at the latest version as used in the Venus OS, there are over 370 commands crammed into 2.1 Mbytes. The command busybox --listall provides a useful list of the commands currently implemented in a form you can pipe into grep.

BusyBox initially aimed to put a complete bootable system on a single floppy disk that would serve both as a rescue disk and as an installer for the Debian distribution. Since that time, it has been extended to become the de facto standard core user space toolset for embedded Linux devices and Linux distribution installers so it is no surprise to find it as part of the Venus OS.

More on the Linux Kernel. I have been logging into the Venus OS using SSH from my pad using the JuicySSH and Hackers Keyboard Apps. The Hackers KeyBoard App provides a full set of keys including Escape and cursor up and down. and this is what I found for the Kernel Version and folder structure

Last login: Wed Nov 17 13:15:20 2021 from 192.168.43.98
root@raspberrypi2:~# uname -a
Linux raspberrypi2 4.19.81-v7 #1 SMP Mon Aug 30 18:57:52 UTC 2021 armv7l GNU/Linux

The choice of kernel is interesting, it is relatively old and very specific in the v7, the only other OS which I have found which used it was Tiny Core Linux v11 . However this may shortly be outdated as there is an upgrade in the pipeline to bump the kernel to 5.10.81 for some RPI versions.

More on the partitioning and folder structure: The remainder of the output serves as an example of the use of ssh access (and busybox) and shows the partition and file structure implemented in the Venus OS v 2.73. This was done for my own interest and is not necessary to follow or understand any of this for normal use of the Venus OS and setting up a Raspberry Pi system.

root@raspberrypi2:~# ls / -l
drwxr-xr-x 2 root root 1024 Jan 1 1970 Settings
drwxr-xr-x 2 root root 3072 Aug 30 19:04 bin
drwxr-xr-x 2 root root 1024 Aug 30 19:00 boot
drwxr-xr-x 13 root root 4096 Nov 9 11:42 data
drwxr-xr-x 17 root root 3500 Nov 9 10:59 dev
drwxr-xr-x 50 root root 4096 Nov 9 13:31 etc
drwxr-xr-x 3 root root 1024 Aug 30 19:03 home
drwxr-xr-x 7 root root 4096 Aug 30 19:04 lib
drwx------ 2 root root 12288 Aug 30 19:04 lost+found
lrwxrwxrwx 1 root root 10 Aug 30 18:57 media -> /run/media
drwxr-xr-x 2 root root 1024 Aug 30 18:57 mnt
drwxr-xr-x 3 root root 1024 Aug 30 19:02 opt
dr-xr-xr-x 251 root root 0 Jan 1 1970 proc
drwxr-xr-x 14 root root 820 Nov 17 13:20 run
drwxr-xr-x 2 root root 4096 Aug 30 19:04 sbin
drwxr-xr-x 3 root root 1024 Nov 9 10:59 service
dr-xr-xr-x 12 root root 0 Jan 1 1970 sys
lrwxrwxrwx 1 root root 8 Jan 1 1970 tmp -> /var/tmp
drwxr-xr-x 3 root root 16384 Jan 1 1970 u-boot
drwxr-xr-x 11 root root 1024 Aug 30 19:03 usr
drwxr-xr-x 9 root root 1024 Jan 1 1970 var

root@raspberrypi2:~# df -h -T
Filesystem Type Size Used Available Use% Mounted on
/dev/root ext4 4.8G 288.2M 4.3G 6% /
devtmpfs devtmpfs 458.5M 4.0K 458.5M 0% /dev
tmpfs tmpfs 463.0M 252.0K 462.8M 0% /run
tmpfs tmpfs 463.0M 428.0K 462.6M 0% /var/volatile
/dev/mmcblk0p1 vfat 43.7M 23.7M 20.0M 54% /u-boot
/dev/mmcblk0p4 ext4 4.9G 22.2M 4.6G 0% /data

root@raspberrypi2:~# cat /etc/fstab
/dev/root / auto defaults 1 1
proc /proc proc defaults 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0
tmpfs /run tmpfs mode=0755,nodev,nosuid,strictatime 0 0
tmpfs /var/volatile tmpfs defaults 0 0
/dev/mmcblk0p1 /u-boot vfat defaults 0 0
/dev/mmcblk0p4 /data ext4 noatime 0 2

root@raspberrypi2:~# fdisk -l
Disk /dev/mmcblk0: 15 GB, 15931539456 bytes, 31116288 sectors
486192 cylinders, 4 heads, 16 sectors/track
Units: sectors of 1 * 512 = 512 bytes
Device Boot StartCHS EndCHS StartLBA EndLBA Sectors Size Id Type
/dev/mmcblk0p1 * 64,0,1 764,2,20 8192 97875 89684 43.7M c Win95 FAT32 (LBA)
/dev/mmcblk0p2 768,0,1 1023,63,32 98304 10268375 10170072 4965M 83 Linux
/dev/mmcblk0p3 1023,63,32 1023,63,32 10268672 20537343 10268672 5014M 83 Linux
/dev/mmcblk0p4 1023,63,32 1023,63,32 20537344 31115263 10577920 5165M 83 Linux

As you can see, the tools in BusyBox are not ideal - utilities I would normally use like blkid and lsblk are missing or lack options like df. fdisk shows the permanent partitioning in a less than friendly way and I was tempted to edit out most of the columns as below!

Device         Boot Size  Id Type
/dev/mmcblk0p1 *    43.7M c  Win95 FAT32 (LBA)
/dev/mmcblk0p2      4965M 83 Linux
/dev/mmcblk0p3      5014M 83 Linux
/dev/mmcblk0p4      5165M 83 Linux

So it seems we have a basic LBA partitioning scheme with 4 partitions partition holding a small initial FAT32 boot partition and the remainder of the micro SD is divided into 3 approximately equal Linux ext4 partitions, one holding the /data partition (/dev/mmcblk0p4) which survives updates and two partitions used in turn to give a backup during and after updating which is a very robust way to proceed. I often do a fresh install into an extra partition when doing major version upgrades ~ every two years on my home systems. There seems to be extensive use of temporary file systems which makes the outputs above difficult to interpret.

What else can we learn: The df output shows disk is much bigger than we need and even a 4 Gbyte SD would be more than adequate.

If you want to learn more about ways to examine partitioning on Linux systems in general I found the article at https://www.binarytides.com/linux-command-check-disk-partitions/ clear and useful, the comments contain some additional suggestions. An alternative is to remove the card and plug it into a another machine with partitioning software such as gparted.

Safe ways to turn off the system.

When you want to turn off your Raspberry Pi, simply pulling the power cord is effective but not ideal. This is because the Raspberry Pi may be writing data to the SD card, in which case simply powering down may result in data loss or, even worse, a corrupted SD card. I should stress this is extremely rare but even so some users keep a spare SD card flashed with a Venus OS.

The problem occurs because the Venus OS does not offer a shutdown option for the Pi or their own units. It does offer a reboot option and some users suggest using that and switching off immediately after the power light flickers off in the belief that is a safe time as no writing is taking place.

I favour using ssh which enables me to use a proper shutdown command namely:

shutdown -h now

the now can be replaced by a time or delay if required - the following command will shut down the Raspberry Pi in 20 minutes time:

shutdown -h 20

or halt at 1730:

shutdown -h 17:30

The -h (for halt) can be replaced by -r (for restart) if required. This can be useful if there has been an anomaly and avoids pulling the plug. Option -c (for cancel) will cancel any 'timers' and operations.

Introduction to the Venus OS Large and Node-RED

There is an extended build of Venus OS under development called Venus OS Large which adds Node-RED and an optional Signal K server. To quote Victron Energy:

Node-RED is a tool for connecting hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows. With it, one can for example program a functionality such as activating a relay based on a temperature measurement. Or make far more complex algorithms, tying relays, measurements, or other data available from Venus OS or elsewhere together. All without having to write real source code, as Node-RED calls Low-code programming for event-driven applications.

Node-RED features a fully customisable dashboard, viewable in a web browser - both locally but also remotely, via the VRM Servers.

The Signal K server is aimed for yachts, and multiplexes data from NMEA0183, NMEA 2000, Signal K and other sensor inputs.

Venus OS Large should run with Node-Red on my Raspberry Pi 3B +. It is still very much 'Work in Progress' but looks very promising for the future for those with some programming knowledge who wish to tinker or customise their systems and have a sufficiently powerful processor in their Victron controller or Raspberry Pi. Images are available as beta versions for both. It was reported they work well, several users were using this without any issues. The idea of an improved dashboard and extra control from Node-Red was very attractive to me and I tried it out once I had got the basic system up and running.

To cut the story short for this introduction: The Raspberry Pi 3B+ has adequate power for what I have tried so far and a new page on the use of the Venus OS Large which includes Node-RED describes my journey of exploration. In general the use and testing of the Node-RED extension has not affected the basic operation of the Venus OS, data gathering of the Pi and transfer to the Victron Portal and once it was installed I have been able to mostly work remotely through the VRM Portal rather than on a very cold Corinna over the winter.

Conclusions - How is it all working in practice?

It was still relatively early days when I initially wrote this but even then I could tell it is all working on the Raspberry Pi as it would with a Victron Venus GX (which has no display), in fact it is better in many ways at a much reduced cost. The major cost has been the VE.Direct to USB isolated cables, which are required, and are more expensive (£27) than the VE.Direct cables to Victron's own Gateways. The Raspberry Pi 'gateway' also has Bluetooth so it can be accessed directly by the SmartConnect App for setting up, a feature which most of the Victron GX devices lack.

So we now have several routes to control and monitor the system and can do it locally and remotely. As far as I can tell there is no difference in functionality between using the SmartConnect App via Bluetooth or the VRM portal, except perhaps in smoothness as there are always some lags through the internet. I find the WiFi gives better coverage through the boat than Bluetooth despite being initially provided by an elderly tethered Samsung Galaxy S3 mini (3g) which lives on a bracket in a window. So we have everything we had before with more local range even before we started to use the portal for world wide coverage. I am yet to check the data use fully but I do not need to worry too much in the UK and EU with a GiffGaff Goodybag providing 15 Gbytes of 4g data a month for £10. Initial impression is that it may be ~1Gbyte/month.

I have not finished exploring the Portal (and it is frequently getting updated and improved) but I find it very easy to watch the Solar and Battery status real time, daily and historic data from the comfort of the house. The views can be fully customised and the configuration is saved on the Portal. There is an App for Android devices as well as access through a browser. It does not seem to be possible to simply control my devices to say switch the inverter to and from Eco mode or configure any parameters directly from the Portal but that can all be done by the SmartConnect App and both can be open at the same time. I can see the logic for separating monitoring from configuration but a few simple switches would be convenient, in particular inverter mode.

I have checked that the data continues to be stored in the Raspberry Pi if the internet connection is broken and then uploads to the portal as soon as it is reconnected. However, if the Pi is rebooted local data not already uploaded does seem to be lost. Even if the internet connection is temporarily broken one can still access the basic dashboard via any machine on the local wifi network (or via a network cable for ultimate reliability onboard) at venus.local.

Overall I am delighted with progress so far and I am gradually building up a comprehensive set of data which is stored on the VRM Portal for at least 6 months and can also be downloaded for additional processing in addition to that gathered earlier via Bluetooth.

The use of The Venus OS Large which includes Node-RED covered in the next page has added an extra dimension and is showing great promise. Node-RED allows much greater flexibility including 'intelligent' control of the devices via your own configurable dashboards. Node-RED can be accessed and programmed via the VRM portal and I have done all the Node-RED programming and 99% of testing remotely. I implemented remote control of the inverter including an option to automatically return the Inverter to Eco (Search) mode from Continuously On after an hour to save power if one forgets. I have also added an extra 'Tab' to the dashboard to monitor the Pi CPU loads and processor temperature and shut down and restart the Pi and Node-RED itself. And this is early days!

Appendix A - Simple cloning of the microSD card as a backup

The following method provides as perfect a clone as one can get between identical microSD cards and leaves you with a compressed backup on a hard drive. This is not be done on the Pi - the card is removed and cloned on a separate Linux based computer. The only requirement is a matching SD card socket with or without an adapter. It uses a command line utility dd. See https://opensource.com/article/18/7/how-use-dd-linux

dd is a simple and powerful image-copying tool that's been around since the early days of unix and nothing's come along that does the job of cloning better. Using dd, you can make perfect byte-for-byte images of almost anything digital. But there's some truth to that old Unix joke: "dd stands for disk destroyer." If you type even one wrong character in a dd command, there is a possibility you can instantly and permanently wipe out an entire drive of valuable data including your system drive. Check and check again before writing to a disk with dd.

The magic incantations below use dd to read an entire SD card bit by bit and compress it and save in an archive as backup by piping the output of dd into gzip. You need to be very sure where the microSD drive is mounted hence the initial fdisk -l and note you need to enter the drive not the individual partitions. In the case of a 16 Gbyte card with a Victron Venus OS the final archive was only 136 Mbytes. The Venus OS large gives a larger archive but still only 450 Mbytes including all the Node-RED flows.

The archive is unzipped and written byte by byte back to preferably an identical card. If the card is bigger you will have unallocated space after the partitions, if it is smaller it have at least one unreadable partition. The following is the important bits of a clone to a larger card of the same type. You will see all the sizes of the partitions are exactly the same and all the extra space is unallocated. Once you have ejected using a system facility so it is shut down properly you can increase the partition sizes using gparted if you need to use the extra space. Always make sure you are writing to the correct devices as dd destroys disks.

peter@gemini:~$ sudo fdisk -l
Disk /dev/mmcblk2: 14.86 GiB, 15931539456 bytes, 31116288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x51c5b8f7Device Boot Start End Sectors Size Id Type
/dev/mmcblk2p1 * 8192 97875 89684 43.8M c W95 FAT32 (LBA)
/dev/mmcblk2p2 98304 10268375 10170072 4.9G 83 Linux
/dev/mmcblk2p3 10268672 20537343 10268672 4.9G 83 Linux
/dev/mmcblk2p4 20537344 31115263 10577920 5G 83 Linux

peter@gemini:~$ sudo dd bs=4M if=/dev/mmcblk2 | gzip > /media/DATA/image`date +%d%m%y`.gz
[sudo] password for peter:
3798+1 records in
3798+1 records out
15931539456 bytes (16 GB, 15 GiB) copied, 819.868 s, 19.4 MB/s

peter@gemini:~$ ls -lh /media/DATA | grep image
-rw-rw-r-- 1 peter peter 136M Jan 2 11:06 image020122.gz

// Remove SD card and replace with new destination card - everything will be destroyed on it

peter@gemini:~$ sudo gzip -dc /media/DATA/image020122.gz | sudo dd bs=4M of=/dev/mmcblk2
[sudo] password for peter:
0+415559 records in
0+415559 records out
15931539456 bytes (16 GB, 15 GiB) copied, 541.94 s, 29.4 MB/s

peter@gemini:~$ sudo fdisk -l
Disk /dev/mmcblk2: 59.49 GiB, 63864569856 bytes, 124735488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x51c5b8f7Device Boot Start End Sectors Size Id Type
/dev/mmcblk2p1 * 8192 97875 89684 43.8M c W95 FAT32 (LBA)
/dev/mmcblk2p2 98304 10268375 10170072 4.9G 83 Linux
/dev/mmcblk2p3 10268672 20537343 10268672 4.9G 83 Linux
/dev/mmcblk2p4 20537344 31115263 10577920 5G 83 Linux

Appendix B - Updating the Venus OS including to the Large version

The updating of Venus OS has turned out to be easier than I expected. I have not actually updated the ordinary Venus OS as I switched at an early stage to the Venus OS Large but the procedure is the same. The update of OS large has to use the off-line procedure at the present where you download the appropriate update file (.swu file) from here to the top level of a USB stick, it does not matter if other files are present. You open the remote console (via VRM or preferably via the local network at local.venus) and go to menu -> Settings -> Firmware -> Install from SD/USB -> Check for firmware from SD/USB -> Select/Install and that should be the update complete barring re-entering the root password which seems to be the only parameter that does not persist through the update (on security grounds??) . This time it is Menu -> Settings -> General and type the password into the set root password box. Everything else like WiFi seems to be persistent.

Changing from the normal Venus OS to Venus OS Large is exactly the same procedure as any other off-line install and you have to start with a working normal version prior to installing Venus OS Large. I installed using the procedure in Venus OS Large. The initial system was version 2.73 image and the Large version I upgraded to and describe in most of his document was Venus OS Large v2.80~21-large-23 dated 25 September 2021 downloaded for the Raspberry Pi from here. I used a new MicroSD card and bought two identical ones so I could make exact clones for backup. In theory the cards only need to be the same size when cloning but in practice it has been reported that anomalies can occur due to detailed differences in the internal implementation of the cards.

There is now a new version which has enough differences to make me consider an update but first I needed to know exactly what changes I have made and how to reproduce them in case it went wrong. I have listed the only changes/settings I believe I have made here. Many (in brackets) can be made in Remote Console once you are connected. The ones I know are not persistent are in bold. Includes Node-RED for Venus OS Large versions

  1. Pair Bluetooth on phones with Victron Venus/Pi device. (Must be paired in SmartConnect not in system of the device you are pairing)
  2. Provide Password for Wifi (Settings -> Wifi)
    1. Find and note ip address.
  3. Set access level to Superuser (Settings -> General)
  4. Create a root password. Needs to be re-created after an update to enable SSH and SFTP login.
  5. Enable Remote Support to enable sshd and reverse tunnel on VRM and LAN in Remote Console
  6. Turn Off auto-update (Settings -> Firmware)
  7. Turn on Node-RED (Venus OS Large Feature)
  8. Make changes to provide persistence in Node-RED in settings.js (Survived update but possible that settings.js is changed in future updates overwriting the change I have made)
  9. Add extra nodes to Node-RED palette. (Manage Palette within Node-RED Menu)
    1. node-red-dashboard
    2. node-red-contrib-victron
    3. node-red-contrib-cpu
    4. node-red-contrib-stoptimer
    5. node-red-contrib-timeswitch
    6. node-red-contrib-os
    7. node-red-contrib-simpletime
    8. node-red-contrib-ui-artless-gauge

So to recap, Installing Node-RED is a two stage process. First you install Venus OS which is the first 7 steps then you download the Venus OS Large update onto a USB stick and update just as you would the Venus OS. so far only 4. had to be redone after the update. Node-RED is covered in much more detail in a separate page The Extension to Venus OS Large with Node-RED

NOTE: The Appendix covering Partitioning on the Venus OS has moved to the The Extension to Venus OS Large with Node-RED page.

Appendix C. Powering the Raspberry Pi and power usage estimates.

A disadvantage of using a Raspberry Pi (or any other Venus controller) is that it uses some of ones valuable power itself and also needs a Wifi connection to the internet if one is going to use the Victron VRM portal for remote monitoring and/or a mobile device round the boat.

We currently use a dedicated tethered Android mobile phone for our local network and internet connection (Samsung S3 Mini for 3G or Samsung M32 for 4G) which currently runs off the 12 volt supply via a standard cigar lighter converter to 5V whilst cruising or a 240 converter whilst on the shore hookup whilst stationary. Tethering is a major power drain on a mobile phone and the S3 Mini runs noticeably warm.

The Raspberry Pi 3B+ was run over the winter from an official mains supply providing up to 2.5 Amps at 5V, the Pi 4 official supply is 3 Amps. Whilst the peak demands could be this high when there are many peripherals also connected the average which concerns us more should be much lower but is still sufficient for the Pi to run slightly warm and with the processor ~ 25 to 30 degrees above ambient with a typical CPU load of 4%.

A quick check using a current measurement using a digital meter in a 12.5 V supply line to a 12 to 5 volt power supply indicted a lower than expected minimum reading of ~ 180 ma but with many upward variations (~ 2.5 watts). The resolution on the load output of the SmartSolar MPPT 100|20 is only .1 Amp but the reading of .1 A at 26.5 V is consistent with that. The Orion Tr has a no load current of of <40ma (~ 1 watt) and efficiency of ~80% so overall this gives an a best of estimate of ~60 Wh per day for the Pi 3B+ system.

There are currently 2 relays on the Raspberry Pi system which take an additional 70ma at 5V when activated ~ 350mwatts which is ~0.5 watts when active and two levels of inefficient conversion are taken into account. With current usage of 5 hours a day this adds 2.5 Wh/day

The Samsung S3 Mini is even more of a difficulty to measure and estimate but say 40 Whr day based on ~6 hrs tethering from a 7.2 Whr battery of 3.6 V with three levels of losses from 27V.

So monitoring has an overall penalty of up to 102 Whr/day or 4 Ah/day if solely taken from solar through the 24 V Lithium battery load output, the Orion TR 24|12 to the 12 V Lead Acid Domestic battery and then further 12 to 5 volt convertion. This figure can be compaired to the cumulative load figures on my dashboard which also have loses of various forms in the domestic battery and they are in the same ballpark.

An alternative is to power via a converter from the 24V system and I have a simple converter board from Amazon but am not currently using it.

 

Before you leave

I would be very pleased if visitors could spare a little time to give us some feedback - it is the only way we know who has visited the site, if it is useful and how we should develop it's content and the techniques used. I would be delighted if you could send comments or just let me know you have visited by sending a quick Message.

Link to W3C HTML5 Validator Copyright © Peter & Pauline Curtis
Content revised: 13th May, 2022