MaPSA Probe Testing

Contact jennetd@fnalNOSPAMPLEASE.gov with questions

Hardware

Components

From Fermilab:

• MaPSA Probe Card, with SHV Cables
• Interface Board, with 3x twisted pair and power cables
• MaPSA Chuck, with vacuum line and teflon pins. FOR CHUCK WITH SILICON BACKING vacuum should not exceed 22 mmHg
• FC7, attached to L21 Mezzanine and Imperial Board, with 12V power adapter, ethernet cable, and 68-Pin LVD SCSI Cable. Power adapter should be this one
• MaPSAs to test

From testing site:

• Semi-Automated Probe Station. Summit 12k working for Fermilab, but R61 manual probe station is not adequate
• Testing PC, Linux (CentOS7)
• RS-232 to USB Cable
• Additional Ethernet Card
• HV Power Supply capable of 1000V Bias. Keithley 2410 Source Meter works well at Fermilab
  • Communication settings: BAUD = 9600, BITS = 8, PARITY = None, TERMINATOR = CR, FLOW-CTRL = None
• Triple Output Power Supply
• Micro SD card
• Global Specialties 1368 Triple Programmable DC Power Supply. Requirements:

MPA version 1, room temp:

MPA version 2, room temp:

Cold testing

Make sure the cold cover is secured in place.

Turn on dry air flow in the probe station (at Fermilab, use level 2).

To start using the espec chiller, turn the main power on (switch on the left). Press power. If nothing happens, try reset then power. When the chiller is powered on, a light will turn on in the lower unit that says "standby." To set the temperature, choose "set constant" and enter the desired temperature. Then press the oper/stop button. Under constant mode, press start. The chiller may not start making noise right away, but the "standby" light should be replaced with "run." The monitor temp screen is accessible from the main window and will show the temperature as you cool down. The velox program also displays the temperature.

Software for MPA Version 2 (Latest)

Install testing code on Linux PC

The operating system used at Fermilab is CentOS7, kernel 3.10.0.1160.62.1. Instructions will need to be adjusted if you are using a different OS.

As root, do:

yum update
yum install boost-devel python3-devel
yum install epel-release
yum install root
yum install root-net-http root-net-httpsniff root-graf3d-gl root-physics root-montecarlo-eg root-graf3d-eve root-geom libusb-devel xorg-x11-xauth.x86_64
yum install git cmake gcc-c++ gcc binutils libX11-devel libXpm-devel libXft-devel libXext-devel
yum install make rpm-build git-core erlang 
yum install python3-tkinter

Install the NI visa libraries by following these instructions. At Padua we used the rpm file ni-rhel7centos7-drivers-2022Q2.rpm

The full list of available packages is here

sudo yum install ni-rhel7centos7-drivers-2022Q2.rpm
sudo yum install ni-visa ni-visa-devel
sudo yum install ni-daqmx ni-daqmx-devel
sudo dkms autoinstall

Install wireshark and rarpd, useful for setting up the FC7

yum install wireshark wireshark-gnome
wget https://archives.fedoraproject.org/pub/archive/epel/6/x86_64/Packages/r/rarpd-ss981107-42.el6.x86_64.rpm
yum localinstall rarpd-ss981107-42.el6.x86_64.rpm

Install pugixml:

yum install pugixml pugixml-devel pugixml-doc

Install ipbus:

yum groups mark remove uhal
sudo yum groupremove uhal
sudo curl https://ipbus.web.cern.ch/doc/user/html/_downloads/ipbus-sw.centos7.repo -o /etc/yum.repos.d/ipbus-sw.repo
sudo yum clean all
sudo yum groupinstall uhal controlhub

If you need help, try this page.

Load the firmware image to SD card

If you already have an SD card formatted with the golden image, skip to the next section.

Formatting a new SD card with firmware will require an SD card reader. Instructions for formatting the SD card with the Golden Image are available here. NB: if you are using a Mac, use

diskutil list

instead of

fdisk -l

The disk must be unmounted (not ejected!) in order to run the dd command. You can do this in the Disk Utility App.

Communicate with the FC7

Get the MAC address of your FC7 from this page. Edit /etc/ethers so that it reads

08:00:30:00:29:61 192.168.0.8 -> fc7.board.001

For example MAC address 08:00:30:00:29:61 and example FC7 IP address 192.168.0.8 (you can choose this).

Run the following command:

rarpd -a

Try to ping the FC7 by doing, for example,

ping 192.168.0.8

Troubleshoot using wireshark. Some useful hints can be found here.

Download Ph2_ACF from here. Edit settings/MaPSAstation.xml so that line 4 lists the IP address of your FC7. Note that you need to clone recursively to get the submodules!

Flash the firmware

To see the firmware that is available on your SD card, do:

cd /home/fnaltest/Ph2_ACF
source setup.sh
mkdir build; cd build; cmake ..
make -j8
fpgaconfig -c settings/MaPSAstation.xml -l

To check what firmware is available on your SD card: Install Ph2_ACF and view the firmware that is available on your SD card.

For MPA v1: use firmware image d19c_mpa_none_28042019.bin For MPA v2: use firmware image uDTC_MPA_SEU_Dev_PM.bin

If the firmware image you need is not already loaded onto your SD card, copy the relevant file from lxplus here: /afs/cern.ch/user/j/jdickins/public/firmware Load it onto the SD card by doing

fpgaconfig -c settings/MaPSAstation.xml -f $firmwarefile -i $firmwarefile

Check that it is now listed when you run

fpgaconfig -c settings/MaPSAstation.xml -l

Now load the firmware image to the FC7 by doing

fpgaconfig -c settings/MaPSAstation.xml -i $firmwarefile

While this command runs, you will see the lights change under the SCSI cable. Once the firmware is properly loaded, you will see a solid green light and blue light flashing at 1 Hz.

Install python packages (MPA v2)

Note that the version 2 testing code uses python3, and does not depend on the d19cScripts repository.

No longer as root, install python 3.6 and related packages

curl https://bootstrap.pypa.io/pip/3.6/get-pip.py -o get-pip.py
python3 get-pip.py
python3 -m pip install --user numpy scipy matplotlib ipython jupyter pandas seaborn
python3 -m pip install --user pyvisa pyvisa-py pyusb

Install Gpib

wget https://sourceforge.net/projects/linux-gpib/files/latest/download
tar -zxvf download
cd linux-gpib-4.3.4
tar -zxvf linux-gpib-user-4.3.4.tar.gz
cd linux-gpib-user-4.3.4
./configure
make
sudo make install
cd language/python/
python3 ./setup.py  install

Run a test (MPA v2)

Install MPA testing code:

cd MaPSA_Testing
git clone https://gitlab.cern.ch/jdickins/MPA2_Test.git
cd MPA2_Test
git checkout dev_mpa2

Make sure the text file utilities/ipaddr.dat contains the IP address of your FC7, and that your ethernet port and IP address are included in utilities/tbconfig.py. Make sure there is no newline character at the end of this file! It has been known to mess things up.

To launch from the command line:

sudo ./start_mpa.sh

Commands

pon()

and

poff()

can be used to power the MaPSA on/off.

Or, to launch the gui:

sudo ./start_gui_mpa.sh

The initial output will tell you whether your FC7 is available and appropriately configured:

IP=192.168.0.8
______________________________________________________
             Starting MPA2 Test System                 
                                                      

PING 192.168.0.8 (192.168.0.8) 56(84) bytes of data.
64 bytes from 192.168.0.8: icmp_seq=1 ttl=64 time=0.058 ms

--- 192.168.0.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.058/0.058/0.058/0.000 ms
=>  MPA Testbench correctly found at address 192.168.0.8
=>  Loading scripts
->  Board Selected MAC > 08:00:30:00:29:61 IP > 192.168.0.8
->  Loaded configuration for MPA

The initial GUI display looks like this:

If power on is successful (good contact, reasonable current draw), you will see something like this printed in the terminal (not GUI) window:

->  SSA enabled 
->  MPA enabled 
---> Enabling the MPA SSA Board I2C master
---> Enabling the MPA SSA Board I2C master
->     Sent Hard-Reset pulse 
->  P_dig:  74.130mW  [V=  0.983V - I= 75.450mA]
->  P_ana:  71.609mW  [V=  1.222V - I= 58.600mA]
->  P_pad:   9.840mW  [V=  1.230V - I=  8.000mA]
->  Total: 155.579mW  [             I=142.050mA]
... more stuff ...
Line Status: 
   Tuning done/applied: 1
   Line ID: 5,   Idelay: 5,   Bitslip: 6,   WA FSM State: 14,   PA FSM State: 14
Line Status: 
   Tuning done/applied: 1
   Line ID: 5,   Idelay: 5,   Bitslip: 6,   WA FSM State: 14,   PA FSM State: 14
->      Initialised SLVS pads and sampling edges
->     Sampling phases tuned
->     Activated normal readout mode

The GUI button that used to be green and say "Power O" will now be red and say "Power Off" - the MPA is powered.

If contact is not good between the probe needles and probe pads, you will see something like this printed in the terminal (not GUI) window:

->  SSA enabled 
->  MPA enabled 
---> Enabling the MPA SSA Board I2C master
---> Enabling the MPA SSA Board I2C master
->     Sent Hard-Reset pulse 
->  P_dig:  70.282mW  [V=  0.981V - I= 71.625mA]
->  P_ana:  71.548mW  [V=  1.222V - I= 58.550mA]
->  P_pad:   6.514mW  [V=  1.229V - I=  5.300mA]
->  Total: 148.344mW  [             I=135.475mA]
... more stuff ...
Line Status: 
   Tuning done/applied: 0
   Line ID: 5,   Idelay: 0,   Bitslip: 0,   WA FSM State: 0,   PA FSM State: 12
Failed tuning line 5
->      Initialised SLVS pads and sampling edges
->     Sampling phases tuned
->     Activated normal readout mode
X>  I2C Periphery read  - Adr=[0x881a], Value=[NOVALUE] - ERROR

The MPA is not powered. Likely you will need to adjust the contact of your probe needles.

Select using the check-boxes which MPA tests you would like to run. The beginning of each test prints out the EFUSE information that uniquely identifies the chip. For example:

Reading EFuse
00111000   00100000   00100001   10100101
Lot N: 1 ; Wafer N: 1 ; Position:  165 ; Status:  2 ; Process bin:  0 ; ADC reference:  7 ;
(165, 1, 1, 2, 0, 7)

After testing an MPA, it is useful to look at the results. Navigate to the "Plotting" tab to see drawing options:

To overlay the S-curves from each pixel on the chip, navigate to the "Debug S-Curves" tab:

For both of these plotting features, the most recent test result corresponding to the MPA you have specified will be shown.

Analysis (MPA v2)

Clone the analysis repository into the same directory as the testing and results directories and check out the branch for the round 3 prototypes

cd ..
git clone https://github.com/jennetd/OT.git Analysis
cd Analysis
git checkout round3

Create directories for the pickle files and plots that we will generate:

mkdir pickles plots

Create a text file (in this example, round3.txt) containing the names of the MaPSAs you want to analyze. The text file should have two items per row:

NameOfMaPSAResultsDir NameOfMaPSAToDisplayOnPlots

Create the pickle file and summary plots by doing

python CollectMaPSAs.py -n round3 -f round3.txt

Instructions for generating multi-mapsa summaries coming soon.

Keithley Configuration from Operation with VISA Drivers

Baud Rate: 9600
Data Bits: 8
Parity: None
Terminator: CR
Flow Control: None

Software for MPA Version 1 (OLD)

Testing instruction slides (originally MPA 1for Micross): https://indico.cern.ch/event/1093262/contributions/4597663/attachments/2338379/3987033/MaPSAInstructions.pdf

Software installation is the same as for MPA v2 up until the section installing python packages:

Install python packages (MPA v1)

(No longer need to be root). Install python 2.7 and related packages

curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
python2.7 get-pip.py
python2.7 -m pip install --user numpy scipy matplotlib pandas seaborn
python2.7 -m pip install --user pyvisa pyvisa-py pyusb

Install Gpib

wget https://sourceforge.net/projects/linux-gpib/files/latest/download
tar -zxvf download
cd linux-gpib-4.3.4
tar -zxvf linux-gpib-user-4.3.4.tar.gz
cd linux-gpib-user-4.3.4
./configure
make
sudo make install
cd language/python/
python ./setup.py  install

Install d19c software (will be replaced for MPA 2):

cd MaPSA_Testing
git clone https://gitlab.cern.ch/cms_tk_ph2/d19c-firmware.git FC7
cd FC7/sw/fc7

Before compiling edit two makefiles. In FC7/sw/fc7/fc7/Makefile, the CPP_FLAGS should be

CPP_FLAGS = -std=c++11 -g -O0 -rdynamic -Wall -MMD -MP -fPIC ${INCLUDE_PATH}

In FC7/sw/tests/Makefile, the CPP_FLAGS should be

CPP_FLAGS = -std=c++11 -g -Wall -O0 -rdynamic -MMD -MP -fPIC ${INCLUDE_PATH} -DBIG_ENDIAN_HACK

Now compile:

source setup.sh
make

Run a test (MPA v1)

Install MPA testing code:

cd MaPSA_Testing
git clone https://gitlab.cern.ch/jdickins/MPA_Test.git
cd MPA_Test

Contact jennetd@fnalNOSPAMPLEASE.gov with access problems.

Make sure the text files d19cScripts/ipaddr.dat and myScripts/ipaddr_my_mp.dat have the IP address of your FC7.

sudo ./minimalstart_mpa_27.sh

To start, run

pon()

If pon is successful (good contact, reasonable current draw), you will see something like

>>> pon()
->     Sent Hard-Reset pulse
->     P_dig: 133.749 mW [V= 0.990 V - I= 135.100 mA]
->     P_ana: 66.686 mW [V= 1.201 V - I= 55.525 mA]
->     P_pad: 16.557 mW [V= 1.189 V - I= 13.925 mA]
->     Total: 216.992 mW [I= 204.550 mA]
->     Initialised SLVS pads and sampling edges
->     Sampling phases tuned
->     Activated normal readout mode
True
True

If contact is not good between the probe needles and probe pads, you will see something like

>>> pon()
->     Sent Hard-Reset pulse
->     P_dig: 128.632 mW [V= 0.991 V - I= 129.800 mA]
->     P_ana: 66.235 mW [V= 1.201 V - I= 55.150 mA]
->     P_pad: 16.632 mW [V= 1.188 V - I= 14.000 mA]
->     Total: 211.499 mW [I= 198.950 mA]
Failed tuning line 0ng phases..
->     Initialised SLVS pads and sampling edges
->     Sampling phases tuned
->     Activated normal readout mode
False
False

Once good contact is achieved and pon() is run successfully, can test an MPA by doing

mpa_test(basepath="../Results_MPATesting/", mapsaid="HPK47_2", chipid="Chip7")

Here, the first argument is the output directory, the second argument is the MaPSA ID, and the third is the chip number.

Also do an IV scan (once per MaPSA):

IVScan(“HPK38_1")

This scans the MaPSA between 0-800V in step of 10V. The time between current measurements can be adjusted with the delay argument. By default the delay is 0.5 seconds.

When you have finished testing a whole MaPSA, you can draw summary plots by doing

python MakeModulePlots.py $mapsaname

This will read directly from the results in Results_MPATesting/$mapsaname

JennetElizabethDickinson - 2023-01-31