Setting up a new Laptop (2021 Edition)

It's been 6 years since I last setup a dual booting laptop. That particular machine is starting to show its age now, particularly with a spinning disk hard-drive.

Before doing anything I created a Recovery USB. Search Recovery Drive on the task bar to open Recovery Media Creator. I followed the instructions to setup a recovery drive on a 32GB USB drive.

The new machine has a 1TB SSD and I wanted to really keep as much as possible for Windows. So, I checked how much space I used on my existing Linux install. The command df . -BG is useful for this. giving space used in GB. It turned out I had only used 75 GB. 

I downloaded the Ubuntu 21.10 ISO and used Startup Disk Creator to flash it to an old 4GB USB stick

To free up space for the Linux installation I needed to shrink the Windows disk partition. I used the Windows tool for this. Search for "Create and format hard disk partitions". This opened the Disk Management Tool:

I right-clicked on the main Windows Partition and selected Shrink... I then shrank by 150GB.

Once everything was in place. I inserted the Ubuntu USB and rebooted. For this machine I needed to explicitly tell it to boot from the USB by hitting F12 during startup to be able to change the boot order.

A welcome screen should pop up with options to Try Ubuntu or Install Ubuntu. Select the latter and follow the next screens, selecting language, keyboard and Wi-Fi when asked. 

• When you get to Installation Type select "Something else, you can create or resize partitions yourself"

This will open a window listing the partitions. The empty 150GB partition created earlier is visible here. Select it and click on + to create a new partition. Do this twice:

1. Create a Swap Partition. Select Logical Partition, beginning of this space and choose a size (I used 4GB) and under Mount point choose Swap
2. Create a Root Partition. Select Primary Partition, beginning of this space and let it take the remaining space. Under Mount point choose /

Previously I created a home partition but decided I didn't really need it this time.

After going through the remaining screens the installation proceeded. 

Jittery screen

The Ubuntu installation was a success but the screen did have jitters, especially duing start up. I found the solution to this by editing /etc/default/grub and changing the GRUB_CMDLINE_LINUX_DEFAULT line to:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_psr=0"

After editing enter sudo update-grub and reboot

PSR is a Power saving setting (Panel Self Refresh) used to optimise power consumed by buffering some frames to the display when the view is static.

Bitlocker

After rebooting I now got the Grub menu to be able to select Ubuntu or Windows. When I selected Windows however I got a warning screen popup

"The system boot information has changed since BitLocker was enabled. You must supply a BitLocker recovery password to start this system."

I was able to log onto my Microsoft Account to obtain the recovery key: aka.ms/myrecoverykey

The machine can now boot easily to Windows or Ubuntu.

Python Range Bar Plots

I have some data in a json file that contains a list of timings for different tests. Each test has a name, start timestamp and duration in milliseconds. It looks something like this:

	{
	"timings": [
	{
	"testname": "Test A",
	"start": 123456,
	"duration": 121
	},
	{
	"testname": "Control",
	"start": 123599,
	"duration": 25
	},
	{
	"testname": "Test B",
	"start": 123400,
	"duration": 220
	}
	]
	}

view raw BarPlotData.json hosted with ❤ by GitHub

Now, I love the Matplotlib Python library. I used it a lot when learning Machine Learning during the 2020 lockdowns. It is an incredibly rich and powerful tool  for creating professional data visualizations. So, as I'm new to it I thought I'd see if I could create a Range Bar plot of these data. 

In essence I want to show each test as a individual bar, the size of which corresponds to the duration and each bar's left most edge corresponding to the start time.

Here is the script:

	# I have a json file with a list of tests. Each test has a name, start time and duration
	# I want to display these in a bar plot
	import numpy as np
	import matplotlib.pyplot as plt
	import json
	# Read the json file
	f = open('data.json')
	data = json.load(f)
	test_names = []
	start_times = []
	durations = []
	colors = []
	for timing in data['timings']:
	test_names.append(timing['testname'])
	start_times.append(timing['start'])
	durations.append(timing['duration'])
	# Change the color of the longest duration bar
	max_duration = max(durations)
	for duration in durations:
	if duration == maxduration:
	colors.append("red")
	else:
	colors.append("blue")
	# Now create the plot
	plt.title('Test metrics')
	plt.xlabel('Start time/duration (ms)')
	plt.barh(test_names, durations, left=start_times, color=colors)
	plt.show()

view raw BarPlot.py hosted with ❤ by GitHub

It's very simple. Most of the script is concerned with reading the data from the file and generating the data structures. I thought it would be nice to highlight the longest duration test in red. This would be useful for seeing if the times for different tests were close.

 

And the output:

Very simple. There are of course many options to polish this. I'll update the blog as I refine them.

.NET Versions and Transport Layer Security (TLS)

This little blog came about from looking into a strange problem where a query to a rest service was failing with :

System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.

System.IO.Exception: Unable to read data from the transport connection: An Existing connection was forcibly closed by the remote host

The cause of this is from an inconsistent security protocol being used by the client and server. In this case the server is using TLSv1.2 and the client SSLv3.

You can get or set the SecurityProtocol in code:

	// This needs to be called before the process creates its first connection to the server
	ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

view raw SecurityProtocol.cs hosted with ❤ by GitHub

I'm running this on .NET framework 4.8. So I should be using the SystemDefault protocol for the OS ( Windows 10) which is TLSv1.2. But when I printed out the output from the SecurityProtocol above I got SSLv3 | TLSv1.1 So what's going on?

Supported Runtime

In my app.config file I had the following startup entry:

	<configuration>
	<startup useLegacyV2RuntimeActivationPolicy="true">
	<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.2" />
	<supportedRuntime version="v2.0.50727" />
	<supportedRuntime version="v4.0" />
	</startup>
	</configuration>

view raw supportedRuntime.app.config hosted with ❤ by GitHub

Documentation here The versions are checked in order with the first one that matches a version installed on your being taken. If that version is present on your computer that will be used.

So with 4.5.2 the default version of SSL3 TLS1 and TLS 1.1 are used. 

There's a few fixes or workarounds for this. First build the code and retarget .NET 4.7.2 or later. Unfortunately this wasn't an option because I didn't own the executable code. This also ruled out explicitly setting the SecurityProtocol in code although this isn't recommended as it makes future changes difficult. You can also edit registry keys (see the link below) to change the default behaviour but I thought this would be too intrusive.

The solution was to add the following to the app.config:

	<runtime>
	<AppContextSwitchOverrides value="Switch.System.Net.DontEnableSchUseStrongCrypto=false"/>
	</runtime>

view raw DontEnableSchUseStrongCrypto.app.config hosted with ❤ by GitHub

For 4.5.2 this also requires the latest patches and registry keys . In an ideal word you would of course update. But software development rarely works in an ideal world.

Publishing .NETCore Application and running on Linux

Over the last year or so I've been writing a simple tool in .NET Core to compare two CSV files. It can be found in on GitHub here. I won't go into details about the tool in this post but being a .NET Core application I've been able to develop it on Windows and also deploy and run on Linux. This post describes how I published and deployed.

I've been using Visual Studio. When I want to create a new deployment I right-click on the Project and select Publish...

I have setup a profile, called "FolderProfile" (Very imaginative!)

• TargetLocation : Folder name
• Configuration : Release | Any CPU (from my solution)
• Deployment mode : Framework-dependent
• TargetFramework : netcoreapp3.0
• TargetRuntime : Portable

(I used Framework-dependent deployment for cross-platform binaries. This creates a dll which can be run with dotnet <filename.dll> on any platform. Of course this requires the .NET runtime to be installed on the Linux machine.)

 

Then click on Publish.

 

This created a folder under Release/netcoreapp3.0/publish

 

I copied these files onto my Linux machine. Here are the important ones:

total 68
drwxr-xr-x  2 jonathan jonathan  4096 Mar 12 19:27 .
drwxr-xr-x 64 jonathan jonathan  4096 Mar 12 19:35 ..
-rwx------  1 jonathan jonathan   431 Jul  2  2020 CSVComparison.deps.json
-rwx------  1 jonathan jonathan 14336 Jul  2  2020 CSVComparison.dll
-rwx------  1 jonathan jonathan  4860 Jul  2  2020 CSVComparison.pdb
-rwx------  1 jonathan jonathan   154 Jul  2  2020 CSVComparison.runtimeconfig.json 

I've previously installed the .NET Core runtime on Ubuntu. As of the time of writing the installed version (dotnet --version) is 3.1.404.

The IMDB/movie_data.csv file is a 50000 row CSV I generated from the IMDB movie dataset. I had originally set this up for a machine learning exercise but it is also a good sized dataset to checkout the tool. I made two copies and edited one of the comments on the candidate file. Then, to run the CSV Comparison, I navigated to the folder containing the binaries and ran with this command-line:

dotnet ./CSVComparison.dll ../IMDB/movie_data2.csv ../IMDB/movie_data2.csv ../IMDB/movie_config.xml ./output

The output:

Reference: ./movie_data.csv
Candidate: ./movie_data2.csv
Saving results to output/ComparisonResults.csv
Finished. Comparison took 10517ms

And the result:

?xml version="1.0" encoding="utf-8"?>
<ComparisonDefinition xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/20
01/XMLSchema">
  <Delimiter>,</Delimiter>
  <KeyColumns>
    <Column>row</Column>
  </KeyColumns>
  <HeaderRowIndex>0</HeaderRowIndex>
  <ToleranceValue>0.1</ToleranceValue>
  <IgnoreInvalidRows>false</IgnoreInvalidRows>
  <ToleranceType>Relative</ToleranceType>
  <ExcludedColumns />
</ComparisonDefinition>

Date run: 22/03/2021 18:40:12
Reference: ./movie_data.csv
Candidate: ./movie_data2.csv
Number of Reference rows: 50001
Number of Candidate rows: 50001
Comparison took 10517ms
Number of breaks 1

Break Type,Key,Column Name,Reference Row, Reference Value, Candidate Row, Candidate Value
ValueMismatch,7230,review,96,"Exceptional movie that handles a theme of

Time for some Maths

While tidying up my old University Maths notes the other day I came across this problem. (I didn't get it right first time round!)

It's quite interesting and I thought I'd also use the opportunity to setup nice looking maths on the blog. So, the question:

Find the first two terms of the series for tanx in powers of x.

Now, 

$$tanx=\frac{sinx}{cosx}$$

and the power series expansions of sinx and cosx begin as:

$$sinx=x-\frac{x^3}{3!}+\frac{x^5}{5!}+...$$

$$cosx=1-\frac{x^2}{2!}+\frac{x^4}{4!}+...$$

Put this together:

$$tanx=\frac{x-\frac{x^3}{6}+...}{1-\frac{x^2}{2}+...}$$

Multiply by $\frac{1+\frac{x^2}{2}}{1+\frac{x^2}{2}}$ (This is just multiplying by 1 and chosen so we can deal with the $x^2$ term in the denominator), so:

$$tanx=\frac{(1+\frac{x^2}{2})(x-\frac{x^3}{6}+...)}{(1+\frac{x^2}{2})(1-\frac{x^2}{2}+...)}$$

If we now just multiply out the first term we get

$$tanx=\frac{x+\frac{x^3}{2}-\frac{x^3}{6}+...}{1+\frac{x^2}{2}-\frac{x^2}{2}+...}$$

$$tanx=\frac{x+\frac{x^3}{3}+...}{1+...}$$

The other terms in $x^4$ and $x^5$ can be ignored, giving:

$$tanx=x+\frac{x^3}{3}+...$$

To display the formulae I've added MathJax to the blog. 

Go to Theme then from the Customise dropdown select Edit HTML. Add the following just below the <head> element:

	<script src='http://cdn.mathjax.org/mathjax/latest/MathJax.js' type='text/javascript'>
	MathJax.Hub.Config({
	extensions: [&quot;tex2jax.js&quot;,&quot;TeX/AMSmath.js&quot;,&quot;TeX/AMSsymbols.js&quot;],
	jax: [&quot;input/TeX&quot;, &quot;output/HTML-CSS&quot;],
	tex2jax: {
	inlineMath: [ [&#39;$&#39;,&#39;$&#39;], [&quot;\\(&quot;,&quot;\\)&quot;] ],
	displayMath: [ [&#39; $$&#39;,&#39;$$ &#39;], [&quot;\\[&quot;,&quot;\\]&quot;] ],
	},
	&quot;HTML-CSS&quot;: { availableFonts: [&quot;TeX&quot;] }
	});
	</script>

view raw MathJax.js hosted with ❤ by GitHub

Finally, here's a useful link: MathJax basic tutorial and quick reference - Mathematics Meta Stack Exchange

 

Setting up a Minecraft Mod project using Eclipse

Here's some instructions how to setup a new Mod project for Minecraft Java using Eclipse. It's really so I can remember how to do this in future!

First, I built spigot. The instructions are here: https://www.spigotmc.org/wiki/buildtools/ (I built spigot_1.16.5.jar Windows)

There's a whole world of exploring to do around setting up Minecraft servers, let alone playing the game and I can't do it justice here. Needless to say, I'm making the assumption you already know how to run a Minecraft server locally.

Now, to create the project follow these steps:

1. Open Eclipse
2. Select File -> New -> Java Project
3. In the Dialog, enter the name of the project, MinecraftMod, say, and click Next 
   
4. Select the Libraries tab on the next Dialog then select Add External JARS... Add the spigot jar built earlier
   
   
   
   
   
5. Click on Finish
6. Select File -> New -> Class
7. In the Dialog, enter com.javaminecraft into the Package field, the Name of the Mod into the Name field and make sure the main method stub is unchecked
   
   
8. Click Finish
   
9. Enter the Java code for the Mod (I won't tell you what to write here!)
10. We now need to create the plugin.yml. Select File -> New -> File
11. Call the file plugin.yml
12. Make sure the Parent Folder is the name of the project, MinecraftMod in this case
13. Enter the yaml required for the Mod (again, not for this blog)
14. To be able to add to the Minecraft server we need to export as a JAR. Right-click on the Project name and click on Export...
15. Select Java -> JAR File
    
    
    
16. The export destination will be the Minecraft plugins folder for the server
17. Restart the Minecraft server
    

There. Hopefully this is a simple recipe for setting up from scratch.