Docker Volume Control

The Raspberry Pi web.py rest service currently saves files to the folder /home/pi/data on the Docker container. This however is only available for the lifetime of the container. In practice this doesn't matter too much because the service is very stable but when I run updates for example I'd like the current list to persist.

Quick aside: Stopping a running container
(useful if I've left it running for 6 months)

First list the containers with docker ps

pi@raspberrypi:~/Server $ docker ps
CONTAINER ID        IMAGE          ...     NAMES
65cb45e19d62        rest-server    ...     loving_mirzakhani 

We are interested in the Names column. In this case run  
docker stop loving_mirzahkani

As per the documentation, volumes are the preferred way to persist data used by containers. So to set this up for the rest server I took the following steps.

Comment out the line to populate the image folder. We don't need to copy anything now so we can remove this line:

COPY data /home/pi/data/

(The data folder originally container start-up data for the webservice)

Then it's simply a case of rebuilding the image and running with this command, using the --mount syntax

docker run -d -p 8080:8080 --mount source=smktdata,target=/home/pi/data rest-server

The first time this was run it automatically created a volume called smktdata

The volume details can be shown with the command

docker volume inspect smktdata

pi@raspberrypi:~/Server $ docker volume inspect smktdata
[
    {
        "CreatedAt": "2019-07-05T18:14:50Z",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/smktdata/_data",
        "Name": "smktdata",
        "Options": null,
        "Scope": "local"
    }
] 

The Mountpoint is the location of the actual data. Files can be viewed with
sudo ls /var/lib/docker/volumns/smktdata/_data

I also have a NAS which is mounted automatically at startup. To use a folder on the NAS, I used this start-up command

docker run -d -p 8080:8080 -v /home/pi/NAS/data:/home/pi/data rest-server

Note that in this case I used the -v (--volume) syntax, this is equivalent to --mount. It essentially just combines all the options into a single field

Windows Screensaver Update

My Windows "Random" photos screensaver code hasn't really been significantly touched for many years now, but as it's working really well on a daily basis on my home PCs, I thought I'd add it to GitHub.

You can use this URL to clone: https://github.com/jscott7/PhotosScreensaver.git
(A wiki has also been started: https://github.com/jscott7/PhotosScreensaver/wiki)

As part of the process I performed a bit of a clean up. It's OK, this is a very small project!

First, the settings have an option to change the delay between photos updating, unfortunately while you could change this it wasn't being used.

The settings for the root folder for photos and delay are saved in the registry, I'm using the key HKCU\SOFTWARE\JscoPhotoScreenSaver

public static void SaveSetting(string name, object value)
{
   var key = Registry.CurrentUser.CreateSubKey("SOFTWARE\\JscoPhotoScreenSaver");
   if (value != null)
   {
       key.SetValue(name, value);
   }
}

Then save the photos path to this key with:

SettingsUtilities.SaveSetting("photopath", filePathBox.Text);

To load back use:

object rootPath = SettingsUtilities.LoadSetting("photopath");

There was an inefficiency with the loading of photo files. I was duplicating this for each window. Not a problem for single monitors but it's unnecessary for multiple monitor setups. Fortunately this was easy to fix by moving the logic to the App.xaml.cs file.

Next, to install, the exe built by the project needs to be renamed to PhotosScreensaver.scr. You then right-click on it and select Install. I added a post-build step to the project to automatically perform this rename.

I also made a few tweaks to follow the MS Naming guidelines

Finally, I made use of string interpolation which is new from C# 6. Previously I used a StringBuilder to generate a debug log. This was:

log.Append("Show").Append(window.Width).Append("-").Append(window.Height).Append("-").Append(window.Left).Append("-").AppendLine(window.Top.ToString()); 

And now, with string interpolation it's much cleaner:

log.Append($"Show {window.Width}-{window.Height}-{window.Left}-{window.Top}");

Raspberry Pi Dashboard with Dakboard

A while ago I bought a cheap Raspberry Pi touchscreen. This like so many impulse buys spent many months gathering dust in a cupboard but given the progress of my webserver, I thought I'd try setting it up as a display.

I was inspired by the excellent Scott Handelman blog to setup a dashboard  using Dakboard and my Raspberry Pi display.

Back when I first got the display, I'd set the screen to portrait mode, but with the stand I have I needed to reset it back to landscape.  It took a while to remember how to do this.

First in  /boot/config.txt I'd added a line display_rotate=1 which rotates by 90 degrees. I removed this and rebooted.

Unfortunately, the response to the touch screen was still in portrait mode.

To change this I had to edit /etc/X11/xorg.conf.d/99-calibration.conf

And change the SwapAxes option back from "1" to "0".

Section "InputClass"
        Identifier      "calibration"
        MatchProduct    "ADS7846 Touchscreen"
        Option  "Calibration"   "145 3995 290 3945"
        Option "SwapAxes"    "0"
EndSection

(I also needed to recalibrate, this is available from Preferences -> Calibrate TouchScreen on my device)

I then created a Dakboard account and setup a simple view; Date, RSS feed and local weather. You are then given a private URL which can be used to display this on any device.

In order to startup the Raspberry Pi displaying this page full screen with no mouse pointre,  I followed the instructions on Scott Hanselman's blog.

Edit (take a copy first):  ~/.config/lxsession/LXDE-pi/autostart

Replace contents with:

@xset s off
@xset -dpms
@xset s noblank
@chromium-browser --noerrdialogs --incognito --kiosk https://dakboard.com/app?p=
private-url

(There are also good instructions here)

Unfortunately, while it does look great, Dakboard doesn't quite work for me.
The ability to use a my own webservice as data input is only supported on the paid-for Premium plan. At $5.95 per month this is too much for me so I'm going to look into creating my own webpage. After all, isn't creating something all the fun!

As an aside. With a 750mA power supply the Pi shows the lightening bolt in the top right hand corner, which indicates it's underpowered. Not surprising given we're also powering the screen.
 

I have an 850mA supply, which is slightly better but I would need more if I wanted to avoid the computer crashing.

Loading old Physics Data Files - Part 2

Previously, my attempt at running a FORTRAN program to load my old ZEUS data ntuples failed because CERNLIB isn't compatible with a 64-bit Linux OS:

 Test loading ntuple files
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
LOCB/LOCF: address 0x562cca6e2c80 exceeds the 32 bit address space
or is not in the data segments
This may result in program crash or incorrect results
Therefore we will stop here
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 

Unfortunately, the recommendation in  /usr/share/doc/libpacklib1-dev/README.64-bit to statically link didn't work for me:

Static linking is the default behavior if you use the "cernlib" script when
linking, like this:

 # Should work everywhere:
 gfortran -o myprogram myprogram.F `cernlib -G Motif pawlib` 

Fortunately, Linux Mint still provides a 32-bit version of the latest OS. So, at this point rather than trying to fix it in 64-bit I thought the best approach would be to setup a Virtual Machine.

For the VM host, I decided to use VirtualBox. This is easy to install on Ubuntu from the Software Center.

After downloading the 32-bit Linux Mint ISO I setup a new VM with 4MB of Memory and Dynamically Allocated Storage with 70GB Virtual Size.

After installing Mint, I reinstalled the CERNLIB packages from the Mint Software Manager and loaded up the ntuple in PAW as per my last post

Now, given this was working I could try and load the file using my FORTRAN program:

      program main

      implicit none
      integer istat, NEvents, idnt
      real hmem
      common/pawc/hmem(2000000)

      print *, "Test loading ntuple files"
      call hlimit(2000000)  
     
      call hropen(80,'ntuple','mc97_1.rz','',4096,istat)
 
      if (istat.ne.0) then
          print *, "Failed to open input file"
         stop
      endif

      print *, "Loaded mc96_1.rz"
      call hrin(10, 9999999, 0) 
      call hnoent(10,NEvents)
      print *, NEvents
c      print *, idnt    

      call hrend('ntuple')
      close(80)  

      end program main

And this time, success!

 Test loading ntuple files
 Loaded mc96_1.rz
       51781

There's a few points to bear in mind:

• common/pawc/hmem(2000000) is required to reserve locations to a common /PAWC/, for the HBOOK working space (an array)
• call hlimit(2000000) informs HBOOK of the storage limit
• call hropen opens the direct access rz file
• call hrin(10, 9999999, 0) reads a histogram from the current directory of the direct access file into the current directory in memory. The 9999999 to to read the highest cycle
• call hnoent Gets the number of events in the in-memory identifier

Loading old Physics Data Files - Part 1

It's been a while since I built the CERNLIB libraries on my laptop. Since then the old laptop has finally failed and I've installed CERNLIB from the Ubuntu repositories, so I thought it was time to try and load my old data files again.

The data are saved as binary ntuples packaged as .rz files. So, to begin I tested opening them in PAW (Physics Analysis Workstation).

To load, simply run paw, take the default workstation type and enter:
hi/file 1 mc96_2.rz

jonathan@jonathan-Inspiron-5759:~/Development/Physics$ paw
 ******************************************************
 *                                                    *
 *            W E L C O M E    to   P A W             *
 *                                                    *
 *       Version 2.14/04      12 January 2004         *
 *                                                    *
 ******************************************************
 Workstation type (?=HELP) <CR>=1 : 
 Version 1.29/04 of HIGZ started
PAW > hi/file 1 mc96_2.rz 4096
PAW > 

The arguments are:

• '1' is the 'logical unit' of the file
• The filename, 'mc96_2.rz'
• Record length in words (I saved the data back in the day with 4096) It's important this is included or a segmentation error is returned:

PAW > hi/file 1 mc96_2.rz

 *** Break *** Segmentation violation
 Traceq lun = 0, level = 99 

 TRACEQ.  In-line trace-back still not available.
 Longjump 
PAW > 

The structure of the ntuple currently loaded in memory can be seen with nt/print 10
(10 is the ntuple identifier. Again, I originally saved it with this value)

PAW > nt/print 10


 ******************************************************************
 * Ntuple ID = 10     Entries = 5882      ntuple
 ******************************************************************
 * Var numb * Type * Packing *    Range     *  Block   *  Name    *
 ******************************************************************
 *      1   * R*4  *         *              * FLT      * TrigDat(15)
 *      1   * I*4  *         *              * TLT      * TLT(15)
 *      1   * R*4  *         *              * TRK      * VCT_XVC
 *      2   * R*4  *         *              * TRK      * VCT_YVC
 *      3   * R*4  *         *              * TRK      * VCT_ZVC
 *      4   * I*4  *         *              * TRK      * NVTRKC
 *      5   * I*4  *         *              * TRK      * NTRKC
 *      6   * R*4  *         *              * TRK      * CHVCC 

...

 ******************************************************************
 *  Block   *  Entries  * Unpacked * Packed *   Packing Factor    *
 ******************************************************************
 * FLT      *  5882     * 60       * 60     *       1.000         *
 * TLT      *  5882     * 60       * 60     *       1.000         *
 * TRK      *  5882     * 40       * 40     *       1.000         *
 * CAL      *  5882     * 60       * 60     *       1.000         *
 * ELEC     *  5882     * 144      * 144    *       1.000         *
 * ZUFOS1   *  5882     * 16       * 16     *       1.000         *
 * ZUFOS2   *  5882     * 4        * 4      *       1.000         *
 * ZUFOS3   *  5882     * 12       * 12     *       1.000         *
 * ZUFOS4   *  5882     * 96       * 96     *       1.000         *
 * ZUFOS5   *  5882     * 16       * 16     *       1.000         *
 * TEMP     *  5882     * 8        * 8      *       1.000         *
 * TAG1     *  5882     * 4        * 4      *       1.000         *
 * TAG2     *  5882     * 4        * 4      *       1.000         *
 * LUMI1    *  5882     * 12       * 12     *       1.000         *
 * LUMI2    *  5882     * 12       * 12     *       1.000         *
 * GEN      *  5882     * 32       * 29     *       1.103         *
 * MCTRUE   *  5882     * 88       * 88     *       1.000         *
 * BGDTUP   *  5882     * 28       * 28     *       1.000         *
 * Total    *    ---    * 696      * 693    *       1.004         *
 ******************************************************************
 * Blocks = 18           Variables = 126          Columns = 174   *
 ******************************************************************

And to page through the data use nt/scan 10

PAW > nt/scan 10
/NTUPLE/SCAN: Only showing first 30 of expressions
+-------+--------------+-------------+--------------+--------------+--------------+-------------+-------------+--------------+-----
| Event |   TrigDat    |   TLT       |   VCT_XVC    |   VCT_YVC    |   VCT_ZVC    |   NVTRKC    |   NTRKC     |   CHVCC      |   FC
+-------+--------------+-------------+--------------+--------------+--------------+-------------+-------------+--------------+-----
|     1 |              |             | -1.43087     |  1.24444     |  18.4673     |  2          |  3          |  3.27624     |  0. 
| *   1 |  0.          |  0          |              |              |              |             |             |              |     
| *   2 |  0.          |  0          |              |              |              |             |             |              |     
| *   3 |  0.          |  0          |              |              |              |             |             |              |     

I've got 174 columns in this particular ntuple. To restrict the selection use:
nt/scan 10 varlis=[Comma separated list of columns]

PAW > nt/scan 10 varlis=ENE44M:ZufoPz
+-------+-------------+--------------+--------------+--------------+
| Event |   ENE44M    |   TEMPLUME   |   TEMPLUMG   |   ZufoPz     |
+-------+-------------+--------------+--------------+--------------+
|     1 |  1000       | -500.        | -500.        |  2.55895     |
|     2 |  1000       | -500.        | -500.        |  11.4667     |
|     3 |  1000       | -500.        | -500.        | -1.63046     |
|     4 |  1000       | -500.        | -500.        |  1.37581     |
|     5 |  1000       | -500.        | -500.        | -2.54238     |
|     6 |  1000       | -500.        | -500.        | -1.91473     |
|     7 |  1000       | -500.        | -500.        |  1.51911     |
|     8 |  1000       | -500.        | -500.        |  0.          |
|     9 |  1000       | -500.        | -500.        |  12.3712     |
|    10 |  1000       | -500.        | -500.        | -4.4222      |
|    11 |  1000       | -500.        | -500.        |  0.0533053   |
|    12 |  1000       | -500.        | -500.        | -0.0746093   |
|    13 |  1000       | -500.        | -500.        |  33.6832     |
|    14 |  1000       | -500.        | -500.        |  0.223857    |
|    15 |  1000       | -500.        | -500.        |  8.72115     |
|    16 |  1000       | -500.        | -500.        |  10.5479     |
|    17 |  1000       | -500.        | -500.        |  2.4528      |
|    18 |  1000       | -500.        | -500.        | -1.17858     |
|    19 |  1000       | -500.        | -500.        |  1.03199     |
+-------+-------------+--------------+--------------+--------------+

After this, I knew I could load the data but I really wanted to run the files through my old analysis code in FORTRAN.

Rather than try to compile the old project, I created a simple FORTRAN program to try and load the ntuple:

jonathan@jonathan-Inspiron-5759:~/Development/Physics/New Stuff/Development$ more Zeus1.f
      program main

      print *, "Test loading ntuple files"
     
      call hlimit(2000000)      
      call hropen(80,'ntuple','nommc_1033.rz','',4096,istat)
 
      if (istat.ne.0) then
          print *, "Failed to open input file"
         stop
      endif

      print *, "Loaded nommc_1033.rz"
      end program main

(I'll talk about these commands in the next blog)

Compilation is easy with the official repository, we just need to link against packlib:

gfotrtran Zeus1.f  -lpacklib

And with no problems it created the default executable a.out (that takes me back!)

So with huge anticipation I ran it and, oh...

jonathan@jonathan-Inspiron-5759:~/Development/Physics/New Stuff/Development$ ./a.out
 Test loading ntuple files
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
LOCB/LOCF: address 0x7f57832df7a0 exceeds the 32 bit address space
or is not in the data segments
This may result in program crash or incorrect results
Therefore we will stop here
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

The reason was to be found in the readme file in the same location as the libraries:
/usr/share/doc/libpacklib1-dev/README.64-bit

CERNLIB was never designed to run on machines where the size of a pointer
is 64 bits.  The code implicitly assumes that
 sizeof(void *) == sizeof(int) == 4
in many different places.  The biggest culprits are the ZEBRA and COMIS sets
of routines in the packlib and pawlib libraries, respectively.  This would be
difficult to fix without rewriting megabytes of FORTRAN source code.

My old 2002 laptop was of course 32bit so this wasn't a problem before. My next blog will describe how I got this working.

Germinating Seeds

Here's a quick an simple way to view a seed germinating.

All it took was kitchen towel, a straight glass, two sunflower seeds and some water.

Take 4 or 5 sheets of kitchen towel, slightly damp and roll into a cylinder. We're going to put this into the glass.

On the side of the kitchen towel, where it would be about 3/4 of the way up the glass, put the seed. We put two in, on opposite sides.

Now, place the kitchen towel into the glass and fill the bottom with water up to a depth of about 2cm.

At this point the glass was put on a shelf and left alone apart from topping up the water once the kitchen towel fully absorbed it.

After 4 days a root appeared and headed down very quickly, over 2cm a day.
We had placed one seed point up and the other point down. This had no discernible effect as the root appeared more from the side.

Then after another 2 days, the leaf came out.

Here it is, just before I planted out into a pot

The seed itself required water, warmth and oxygen to germinate. By keeping it damp, but not submerged, the seed took in a large amount of water to allow it to soften and swell the coating as well as trigger the internal metabolism.

Distance Sensor

Another online purchase saw me get a set of HC-SR04 Ultrasonic sensors. These sensors contain an ultrasonic transmitter and receivers. Sound waves sent from the transmitter reflect off a distant surface and are sensed by the receiver. The time taken for the round trip can be used to calculate the distance. Details how it works are on the datasheet.
The company that sold the sensor, www.elegoo.com provided an download for Arduino, this consisted of a library and sample sketch.
The .cpp and .h files needed to be setup as an Arduino Library. To do this i needed to copy the files to ~/Arduino/Libraries/SR04/  The SR04 folder must match the library name.
The simple sketch below could then be created:
#include "SR04.h"
#define ECHO_PIN 11
#define TRIG_PIN 12

SR04 sr04 = SR04(ECHO_PIN, TRIG_PIN);
long distance;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  delay(1000);
}

void loop() {
  // put your main code here, to run repeatedly:
  distance = sr04.Distance();
  Serial.print(distance);
  Serial.println("cm");
  delay(1000);
}
Unfortunately, when I tried to upload to the Arduino I got this error:

/home/jonathan/Development/Arduino/arduino-1.6.7/hardware/tools/avr/bin/avrdude: error while loading shared libraries: libreadline.so.6: cannot open shared object file: No such file or directory

Looking in the /lib/x86_64-linux-gnu folder I discovered I only had libreadline.so.7.
An upgrade to Ubuntu 18.04 meant the installed version of the Arduino IDE, 1.6.7 was no longer compatible. However, simply installing the latest version 1.8.7 resolved the problem and I was able to upload the sketch.
Finally I added a piezoelectric buzzer configured to beep more rapidly the closer something got to the sensor.


The speaker was plugged into pin 8 and ground, with the code changed as below:
#define SPEAKER 8
void loop() {
  // put your main code here, to run repeatedly:
  distance = sr04.Distance();
  Serial.print(distance);
  Serial.println("cm");

  tone(SPEAKER, 550, 100);
  int delayTime = distance * 10;
  delay(delayTime);
}

This is what the setup looked like:

In practice the maximum distance reported by the sensor was around 1.25m.