quadEM Driver

Contents

• Overview
• Databases
  • quadEM.template
  • TetrAMM acquisition modes
  • Plugins: NDPluginStats, NDPluginStdArrays, NDFileNetCDF, NDPluginTimeSeries, NDPluginFFT
  • fast_pid_control.db
• Performance
• Common Setup
• TetrAMM Setup
• AH401 and AH501 series Setup
• NSLS Electrometer Setup
• APS Electrometer Setup

Overview

quadEM supports quad electrometers/picoammeters, typically used for photodiode-based x-ray beam position monitors, or split ion chambers. They can also be used for any low-current measurement that requires high speed digital input. There is support for several models:

1. The AH401 series (AH401B, AH401D) and AH501 series (AH501, AH501C, AH501D) picoammeters originally designed by Synchrotron Trieste (elettra). They are now sold commerically by CAENels. These devices communicate using TCP or UDP over 100 Mbit/s Ethernet or high-speed serial. They provide 4-channel current measurements at up to 6510 Hz (AH501 series) or 1000 Hz (AH401 series).
2. The TetrAMM picoammeter sold by CAENels. This device communicates using TCP/IP over 1 Gbit/s Ethernet. It provides 4-channel current measurements at up to 20 kHz.
3. The NSLS Quad Electrometer (called NSLS_EM in this document). This device consists of a 4-channel digital electrometer unit with Ethernet communication. The device provides 4-channel current measurements at up to 2500 Hz. The Sydor SI-EP-B4 is based on this design and is software comptabible.
4. The NSLS2 Quad Electrometer (called NSLS2_EM in this document). This device consists of a 4-channel digital electrometer unit with Ethernet communication. The device provides 4-channel current measurements at up to 10,000 Hz. This design is based on a transconductance amplifier and can measure currents into the mA range, unlike the NSLS_EM which is based on a current integration chip and is limited to currents less than about 1 micro-amp. This unit is packaged with a Zynq processing and run this EPICS IOC software internally.
5. The Quad Electrometer built by Steve Ross from the APS (called APS_EM in this document). This device consists of a 4-channel digital electrometer unit and 2 VME boards. The device provides 2 readings per diode at up to 813 Hz. This device appears to no longer be in use on any APS beamlines, so the support is now deprecated and will be removed in a future release.
6. The PCR4 picoammeter from SenSiC. This device communicates using TCP/IP over 1 Gbit/s Ethernet. It provides 4-channel current measurements at up to 53,000 Hz.

The AH401 series, NSLS_EM, and APS_EM are based on the same principle of an op-amp run as a current amplifier with a large feedback capacitor, and a high resolution ADC. The AH501 series, TetrAMM, and NSLS2_EM are based on a transimpedance input stage for current sensing, combined with analog signal conditioning and filtering stages. The AH501C, AH501D, and TetrAMM have an integrated programmable bias supply.

The quadEM software includes asyn drivers that provide support for the following:

• Analog input records using the NDPluginStats plugin from the synApps areaDetector module. This provides digitally averaged readings of the current, sum, difference and position at speeds that are determined by the AveragingTime record described below. The quadEM drivers do the callbacks on the asynGenericPointer interface for NDArray objects required to use the areaDetector plugins. The NDPluginStats module is used to compute the averaged readings. It also provides additional statistics including standard deviation, min, max, and a histogram of values. This averaging is termed the "primary averaging".
• Analog input records using the asynFloat64Average device support from asyn. These records can either process at the standard EPICS scan rates, normally 10Hz to 0.1 Hz. Beginning with asyn R4-34 they can also use SCAN=I/O Intr, which allows them average and process at any integer divisor of the device sampling rate. This averaging is termed "fast averaging", though it does not necessarily run faster than the primary averaging described above.
• The NDPluginStats provide time-series of the statistics. These can be used to for on-the-fly data acquisition applications, where the meter is triggered by an external pulse source, such as one derived from a motor motion.
• One could also use the standard asynFloat64 device support with ai records and scan=I/O Intr, in which case the record would update with every reading from the device. This is likely to overwhelm the EPICS IOC if the electrometer is run at very fast sampling times.
• Streaming data to disk at the full rate from the device. This is done using the file plugins from areaDetector.
• Time series data (like a digital scope) of the current, sum, difference and position at speeds up to 20000 Hz (TetrAMM), 6510 Hz (AH501 series), 1000 Hz (AH401 series) 2500 Hz (NSLS_EM), or 813 Hz (APS_EM). The data is available in standard EPICS waveform records, using the NDPluginTimeSeries plugin from areaDetector/ADCore. The time per point can be greater, in which case it does averaging.
• FFTs of the time series data, providing the power spectrum of each signal as another EPICS waveform record. This uses the NDPluginFFT plugin from areaDetector/ADCore.
• epid record support.

  This can provide "fast feedback" via an asyn D/A converter (e.g. dac128V), also at speeds up to 20000 Hz, 6510 Hz, 1000Hz, 2500 Hz, or 813Hz. If it is run slower it does signal averaging. This support is provided in the synApps std module. The quadEM drivers do the callbacks on the asynFloat64 interface required to use the epid fast feedback device support. This fast feedback is limited to controlling devices in the same IOC as the quadEM driver, because it uses asyn links.

  The epid record can also be used in "slow feedback" mode using the averaging provided by the primary averaging or fast averaging records described above. In this mode it can use Channel Access links, and so it not constrained to controlling devices in the same IOC.

The following manuals provide detailed information on these devices:

• APS electrometer: Electrometer Users Guide
• AH401B electrometer: AH401B Users Manual
• AH401D electrometer: AH401D Users Manual
• AH501C electrometer: AH501C Users Manual
• AH501D electrometer: AH501D Users Manual
• TetrAMM electrometer: TetrAMM Users Manual

The support is based on asynNDArray driver from areaDetector, which in turn is based on asynPortDriver. It consists of a base class (drvQuadEM.cpp) which is device-independent. There are device-dependent classes for the TetrAMM (drvTetrAMM.cpp), AH401 and AH501 series (drvAHxxx.cpp) NSLS electromater (drvNSLS_EM.cpp), and the APS electrometer (drvAPS_EM.cpp).

The quadEM driver works as follows:

• The data from the device-dependent drivers are first placed into a ring buffer whose size is defined in the constructor and configuration function. The driver provides 4 current readings. 7 additional values are computed from these 4 values: SumX, SumY, SumAll, DiffX, DiffY, PositionX, PositionY. The meanings of X and Y are discussed in the geometry section below.
• The PV called AveragingTime determines the time period over which to average the readings. The AveragingTime divided by the SampleTime determines the number of samples to average, NumAverage_RBV. When this number of samples have been accumulated in the ring buffer a separate thread copies them to a set of NDArrays and calls any registered plugins.
• There is a separate NDPluginStats plugin loaded for each of the 11 data values. This plugin receives an array of dimensions [NumAverage_RBV]. This plugin computes not only the mean, but also the standard deviation, histogram of values, etc.
• One of the NDArrays contains all of the data values, and has dimensions [11,NumAverage_RBV]. This array can be passed to any of the file writing plugins, which can thus stream all of the data to disk for arbitrarily long time periods.
• The [11,NumAverage_RBV] NDArray is also passed to the NDPluginTimeSeries plugin which is used to generate time-series arrays.
• The time-series plugin output is used by the NDPluginFFT plugin to compute FFTs, producing arrays containing the frequency power-spectrum of the time series data.
• An NDStdArrays plugin is also loaded. This can be used to pass all of the data [11,NumAverage_RBV] or any of the individual data arrays to any channel access client.
• The computationally intensive work of calculating the statistics is done in plugins, so can be done in different threads (if CallbacksBlock is set to No), each potentially running in a separate core on modern CPUs.
• In addition to placing the data from each time point into the ring buffer, the driver does callbacks on the asynFloat64 interface for each data value. This is used for two things:
  1. Fast feedback support with the epid record.
  2. Fast averaging support with ai records. These records can be processed periodically with SCAN=1 second, 0.1 second, etc., or they can have SCAN=I/O Intr. In this case the ai record device support sums the callback readings until NumAverage readings have been received, at which point the average is computed and the record is processed. The quadEM database computes NumAverage from the FastAveragingTime record and writes NumAverage to the .SVAL field in each ai record.

Note: This version of the driver requires a minimum firmware version on some models

• AH501D Firmware version 2.0.
• TetrAMM Firmware version 2.9.11

Prior to R5-0 the quadEM driver assumed the following geometry for the 4 current inputs:

             4
             
          1     2
             
             3
  

The PV for the computed quantities were called Sum12, Sum34, Sum1234, Diff12, Diff34, Position12, and Position34. The differences, and hence positions, were computed as 2-1 and 4-3, which would correspond to Position12 being positive to the right and Position34 being positive up in the above diagram.

For R5-0 and later two geometries are supported, and the names of the Sum, Diff and Position PVs were changed. The computed quantities are called SumX, SumY, SumAll, DiffX, DiffY, PositionX, and PositionY.

The first geometry is the same as that illustrated above, and is called Diamond. For this geometry only 2 diodes are used for the position calculation in each direction. SumX=(1+2), SumY=(3+4), DiffX=(2-1), DiffY=(4-3). This geometry is identical to the geometry assumed prior to R5-0. The X diodes are 1&2 rather than 1&3 so that it is possible to use just the first 2 inputs on the AH501 series to increase readout speed in cases where all 4 diodes are not used. This would not be possible if diodes 1 and 3 were used for the X calculations.

The second geometry is:

          1     2
       
             
          4     3
  

This geometry is called Square. For this geometry all 4 diodes are used for the position calculation in each direction. SumX=SumY=(1+2+3+4), DiffX=(2+3)-(1+4), DiffY=(1+2)-(3+4).

R9-5 added a variant of the square geometry, SquareCC:

          1     4


          2     3
  

It is similar to Square, but the 4 diodes are arranged counterclockwise. SumX=SumY=(1+2+3+4), DiffX=(3+4)-(1+2), DiffY=(1+4)-(2+3).

For all geometries SumAll=(1+2+3+4), PositionX=DiffX/SumX, and PositionY=DiffY/SumY. X positive is to the right and Y positive is up for both geometries.

Databases

quadEM.template

The quadEM.template database provides control of the electrometer using the standard asyn device support. NDStats.template from areaDetector provides digitally averaged readings of the current, sum, difference and position with user-defined averaging time. It also provides the standard deviation, minimum, maximum, and other statistics, including a histogram of array values.

TetrAMM Acquisition Modes

The TetrAMM can be used in 8 different acquisition modes, depending on the values of the TriggerMode and AcquireMode records. The following table describes these modes. Note that AcquireMode=Single is completely equivalent to AcquireMode=Multiple and NumAcquire=1, so the table does not list AcquireMode=Single.

This is the medm screen to control the quadEM with the records in quadEM.template.

quadEM.adl

Plugins

The example IOCs provided with quadEM load a file called commonPlugins.cmd, which loads the following set of plugins from the areaDetector module. For more information see the documentation in the links in the table below. Other plugins can also be loaded, for example the TIFF, HDF5 or Nexus file writing plugins, etc.

Note that the first time the IOC is started all of the plugins will have EnableCallbacks=Disable. It is necessary to enable each of the plugins that will be used. The plugins will also initially start with CallbacksBlock=No. Setting CallbacksBlock=Yes can reduce CPU load on slow processors like the MVME2100 (see the performance tables below). The values of EnableCallbacks and CallbacksBlock are saved by autosave, and will be restored the next time the IOC is started.

This is the medm screen for all of the plugins defined in commonPlugins.cmd.

QECommonPlugins.adl

This is the medm screen for the Current1: NDPluginStats plugin loaded by commonPlugins.cmd.

NDStats.adl

This is the medm screen for the netCDF1: NDFileNetCDF plugin loaded by commonPlugins.cmd.

NDFileNetCDF.adl

This is the medm screen to control the NDPluginTimeSeries plugin. In this example the time per point from the TetrAMM is 50 microseconds, and averaging time for the time series plugin is 1 millisecond, or 20 points. The plugin has 2000 time points, and is operating in circular buffer mode.

NDPluginTimeSeries.adl

This is a plot of the time series for Current 1. It shows the last 2 seconds of data in circular buffer mode.

NDTimeSeriesPlot.adl

This is a plot of the time series for the horizontal currents, sum, and position. It shows the last 2 seconds of data in circular buffer mode.

quadEM_plogAll.adl

This is a plot of the FFT for Current 1. DC offset suppression is enabled and 10 FFTs are being averaged to improve the signal/noise ratio.

NDFFTFreqSpectrumPlot.adl

This is an medm screen for control of all 11 FFT plugins.

QEFFTPlugins.adl

This is an medm screen that displays the FFTs of the Current, Sum, and Position for the time-series data above. 100 FFTs are being averaged to improve the signal/noise ratio.

quadEM_plotAll.adl

fast_pid_control.db

The quadEM can be used to do fast feedback with the EPID record from the synApps "std" module. The EPID record will process at up to the SampleTime discussed above. The DT field of the EPID record controls the time between feedback calculations, and this is constrained to be an integer multiple N of the quadEM SampleTime. If N>1 then N samples are averaged for each feedback operation.

This is the medm screen that controls the fast feedback of the pitch of the monochromator on APS beamline 13-ID. The readback and control PVs cannot be changed after iocInit. The update rate (SCAN rate of EPID record) only controls the rate at which the EPID record displays "snapshots" of the values of the input, output, and error. It does not affect the rate at which the feedback is actually being done, which is controlled by the DT field in the EPID record, can be much faster than the SCAN value.

pid_control.adl

This is the medm screen that controls the PID parameters. These include the feedback coefficients KP, KI, and KD. The DT (delta time) field controls the rate at which the feedback is actually been run. In this case DT=68.0 ms, which is 55 times longer than the sampling time (1.23 ms), so 55 position readings are being averaged each time the feedback is run.

pid_parameters.adl

This is an medm screen that displays the setpoint of the pitch of the monochromator (in red), and the actual readback from the quadEM (in blue). Note that the readback here is only the snapshot values from the EPID record. For full-speed readings of the readback the waveform records from the quadEM_TimeSeries.template database would be used.

pid_plot_readback.adl

This is pid_fast.template, an example substitutions file. This example uses a quadEM asyn port driver named "APS_EM". ICHAN is set to read from addr=9 and 10, which are the two position values of the quadEM driver. The INPUT_DATA and INPUT_INTERVAL strings are the drvInfo strings for these parameters in the driver. The output is sent to an asyn port driver named DAC1, which is a dac128V 12-bit A/D converter in this data. OCHAN is set to write to addr=1 and 2, which are the second and third DAC channels on that card. The OUTPUT_DATA string is the drvInfo string for the double precision data in the dac128V.

file "$(STD)/stdApp/Db/fast_pid_control.db"
{
pattern
{P,                 PID,  INPUT,     INPUT_DATA,   INPUT_INTERVAL, ICHAN, OUTPUT, OCHAN,  OUTPUT_DATA,    LOPR,     HOPR,  DRVL,  DRVH,  PREC,   KP,    KI,  KD,   DT,        SCAN}
{quadEMTest:, MonoPitch, APS_EM, QE_DOUBLE_DATA,   QE_SAMPLE_TIME,     9,   DAC1,     1,  DOUBLE_DATA,  -32767,    32767,     0,  4095,     3,  .02,  100.,  0., .001, ".1 second"}
{quadEMTest:,  MonoRoll, APS_EM, QE_DOUBLE_DATA,   QE_SAMPLE_TIME,    10,   DAC1,     2,  DOUBLE_DATA,  -32767,    32767,     0,  4095,     3,  .02,  100.,  0., .001, ".1 second"}
}

Performance

AH401B

The following table shows the CPU utilization of a Linux machine (Xeon E5630 2.53GHz, 8 cores), and vxWorks MVME5100, with the AH401B electometer. The CPU load was measured under the following conditions. This is the maximum data rate of the AH401B and the time series waveform records and FFTs are processing at 1 Hz.

• ValuesPerRead=1
• IntegrationTime=0.001
• PingPoing=Yes
• AveragingTime=0.1
• NumChannels=4
• ReadFormat=Binary4
• Time series plugin
  • Plugin enabled
  • TSNumPoints=2048
  • TSAveragingTime=0.001
  • TSRead.SCAN=1 second
  • TSAcquireMode=Circular Buffer
• FFT plugins enabled

It can be seen that the load on the Linux machine is only 6% of a single core, while the load on the MVME5100 is 15%.

TetrAMM

The following table shows the CPU utilization of the Linux machine and vxWorks MVME5100, with the TetrAMM electometer. The CPU load was measured under the following conditions. ValuesPerRead=5 is the maximum data rate of the AH401B and the time series waveform records and FFTs are processing at 1 Hz.

• ValuesPerRead=5, 10, 20, 50, 100
• AveragingTime=0.1
• NumChannels=4
• ReadFormat=Binary4
• Time series plugin
  • Plugin enabled
  • TSNumPoints=2048
  • TSAveragingTime=0.001
  • TSRead.SCAN=1 second
  • TSAcquireMode=Circular Buffer
• FFT plugins enabled

It can be seen that the worst-case load on the Linux machine is only 14% of a single core, while the load on the MVME5100 is 100% when ValuesPerRead=5. Using ValuesPerRead=20 or greater uses less than 22% of the CPU on the MVME5100, which is probably reasonable in practice. That value still produces 5 kHz updates for time-series and fast feedback.

Common Setup

All of the device-dependent startup scripts invoke quadEM/iocBoot/commonPlugins.cmd. This file can be edited to add additional plugins, such as the TIFF, HDF5, or Nexus file writers, etc.

Each iocBoot/iocXXX directory, e.g. iocBoot/iocTetrAMM, contains an example startup script for Linux and Windows (st.cmd) and another for vxWorks (st.cmd.vxWorks). Each of these startup scripts invokes the device-dependent startup script(s), e.g. TetrAMM.cmd, NSLS_EM.cmd, etc.

TetrAMM Setup

These meters communicate via IP, so they must be configured with an IP address reachable from the host IOC machine. The CAEN ELS Device Manager software must be used to configure the device IP address and port number.

An example startup script is provided in
quadEM/iocBoot/iocTetrAMM/TetraAMM.cmd

This will need to be edited to set the correct IP address of the meter to be used.

AH401 and AH501 Series Setup

These meters communicate via IP, so they must be configured with an IP address reachable from the host IOC machine. The Lantronix module in the meters can be configured to use either UDP or TCP. UDP can reduce CPU load, however it is more susceptible to synchronization problems since it does not retransmit dropped packets. The startup script must include the UDP qualifier on the drvAsynIPPortConfigure command if UDP is selected.

Example startup scripts are provided in
quadEM/iocBoot/iocAH401B/AH401B.cmd and
quadEM/iocBoot/iocAH501/AH501.cmd

These will need to be edited to set the correct IP address of the meters to be used.

These scripts each invoke a generic script:
quadEM/iocBoot/AHxxx.cmd.

NSLS_EM Setup

These meters communicate via IP. They only support DHCP, they cannot be given a static IP address. Each module on the subnet must have a unique Module ID, which is set by a 16-position rotary switch on the device front panel. The driver sends a broadcast message on the subnet to determine the Module ID and current IP address of all NSLS_EM modules. If the specified module ID is found then it configures communication to that module by calling drvAsynIPPortConfigure internally.

An example startup script is provided in
quadEM/iocBoot/iocNSLS_EM/NSLS_EM.cmd

This will need to be edited to set the broadcast address of the network the device is connected to, and the Module ID of the device.

NSLS2_EM Setup

These meters communicate via IP. They run the EPICS IOC software internally on the Zynq processor.

An example startup script is provided in
quadEM/iocBoot/iocNSLS2_EM/NSLS2_EM.cmd

This will need to be edited to set the Module ID of the device.

APS Electrometer Setup

The APS_EM VME card cannot generate interrupts, but it can output a TTL pulse each time new data is available, at up to 815 Hz. If this pulse is input to an Ip-Unidig (or other asyn digital I/O device with interrupt and callback capabilities), then the ipUnidig interrupt routine will call the APS_EM driver each time new data is available. The Ip-Unidig channel where the APS_EM pulse is connected is specified in the unidigChan argument to drvAPS_EMConfigure command in the startup script. If an Ip-Unidig or other interrupt source is not being used then the APS_EM driver will poll for new data at the system clock rate, typically 60Hz.

An example startup script is provided in
quadEM/iocBoot/iocAPS_EM/APS_EM.cmd.
and
quadEM/iocBoot/iocAPS_EM/st.cmd.vxWorks.