Setting up a new Python environment

Over many years of working in software development, managing dependencies to you code, whether in binary or source code format has always been a pain, to be honest. Take your pick from:

• Cut and paste code
  
• Linking against common libraries
  
• Coping .zip files (with suitably convoluted folder structure)
• Installing files to a common location
• dlls
• .NET GAC
  
• Maven
• NuGet
• And so on..
  

And now into the mix comes Python with its 2.x (OK not really now) and 3.x versions, pip and conda..

A discussion of pip and conda can be found here

Anyway, having installed a new Ubuntu partition I wanted to get back the development environment I'd been using for the last couple of years. This is what I did...

Install Visual Studio Code

It's my tool of choice for Python. On Ubuntu I followed these instructions

I then installed the Python and Pylance extensions for VS Code

To check the Python environment being used, Open the Command Palette with Ctrl-Shift-P and then Python : Select Interpreter

Install Anaconda

It has its quirks but I have been using Anaconda. I installed from scratch using the instructions here.

The default installation included all the necessary libraries for simple machine learning code. 

Installing Cirq

I've been looking at Quantum Computing over the last year using Cirq. I couldn't get this installed using the instructions here.

However python -m pip install Cirq worked and it appeared in my Anaconda base (root) environment in VS Code. Success!

Installing Pygame

And finally, for some playing around with game programming I installed Pygame  with python -m pip install pygame.

So my environment is back up and running on a new OS installation.

Arrgh! Segmentation fault

My Physics analysis code runs in 3 modes. For 1996 data, for 1997 data and for both year's data. The 3rd mode also runs over different values of cuts. This is so as to build a set of results that can be used to estimate systematic errors in the analysis. For example, changing the value of the cut on Calorimeter energy can provide an indication of how well is is modeled.

To run all these checks a number of "Steering card" files were created. Each one listing out the cuts to be made for a single batch run. The analysis would then run over the entire set of '96/'97 data for each batch in turn and build a list of result ntuples.

Now, while running '96 and '97 individually was successful, when running both, on the second iteration the process crashed with the following.

	ANALYSE: File contains 14614 Events
	Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
	Backtrace for this error:
	#0 0xb5aa8e63
	#1 0xb5aa7f8e
	#2 0xb7fbabbf
	#3 0xb6f7fa9f
	#4 0x4a0c95
	#5 0x4b13c7
	#6 0x4ba4bf
	#7 0x4ad0c4
	#8 0x4ad5ba
	#9 0xb57d0f20
	#10 0x494ec0
	Segmentation fault (core dumped)

view raw SegmentationFaultRaw.txt hosted with ❤ by GitHub

The dreaded Fortran segmentation fault! 

In years gone by, I would have taken the approach of writing loads of print(*,*) statements around the "ANALYSE: File contains..." log and eventually homed in on the offending code but surely I can do better now!

It turns out I can. I can add a DEBUG flag: -fcheck=all -g

	../data/mc96/mc96_1.rz
	ANALYSE: File contains 14614 Events
	Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
	Backtrace for this error:
	#0 0xb5a8de27 in ???
	#1 0xb5a8cf8e in ???
	#2 0xb7f9fbbf in ???
	#3 0xb6f64a9f in ???
	#4 0x44634d in isr_cuts96_
	at ./src/isr_cuts96.fpp:408
	#5 0x457df8 in anevent96_
	at ./src/anevent96.fpp:147
	#6 0x46282a in do_all_
	at ./src/do_all.fpp:131
	#7 0x453a0a in MAIN__
	at ./src/analysis.fpp:44
	#8 0x453f2a in main
	at ./src/analysis.fpp:66
	Segmentation fault (core dumped)

view raw SegmentationFaultWithInfo.txt hosted with ❤ by GitHub

This is much more useful

(fcheck=all adds runtime checks for array bounds amongst other checks, -g adds extra information for the gdb debugger)

The line in isr_cuts96.fpp is:

call hf1(510,temp,1.)

This is filling a histogram  with data. The fact it's failing on the second iteration of the batch suggests the histogram isn't being correctly cleared down after the first run. 

The module I have to do the clearing down is called terminate.fpp

A review of the HBOOK User Guide led me to needing this call

CALL HDELET ID
Action: Deletes a histogram and releases the corresponding storage space.
Input parameter description:
ID identifier of a histogram. ID deletes all existing histograms.

Adding this to terminate.fpp fixed the problem and I am now able to run multiple batches in the same process

call hdelet(0)

Why this ran OK back in 2000 is probably a combination of the older CernLib library and Fortran compiler.