pygama.dsp.processors package#

Contains a list of DSP processors, implemented using Numba’s numba.guvectorize() to implement NumPy’s numpy.ufunc interface. In other words, all of the functions are void functions whose outputs are given as parameters. The ufunc interface provides additional information about the function signatures that enables broadcasting the arrays and SIMD processing. Thanks to the ufunc interface, they can also be called to return a NumPy array, but if this is done, memory will be allocated for this array, slowing things down.

The pygama processors use the ufunc framework, which is designed to encourage highly performant python practices. These functions have several advantages:

  1. They work with numpy.array. NumPy arrays are arrays of same-typed objects that are stored adjacently in memory (equivalent to a dynamically allocated C-array or a C++ vector). Compared to standard Python lists, they perform computations much faster. They are ideal for representing waveforms and transformed waveforms.

  2. They perform vectorized operations. Vectorization causes commands to perform the same operation on all components of an array. For example, the ufunc np.add(a, b, out=c) (equivalently c=a+b) is equivalent to:

    for i in range(len(c)): c[i] = a[i] + b[i]

    Loops are slow in python since it is an interpreted language; vectorized commands remove the loop and only call the Python interpreter once.

    Furthermore, ufuncs are capable of broadcasting their dimensions. This involves a safety check to ensure the dimensions of a and b are compatible sizes. It also will automatically replicate a size-1 dimension over another one, enabling the addition of a scalar to a vector quantity. This is useful, as it allows us to process multiple waveforms at once.

    One of the biggest advantages of vectorized ufuncs is that many of them will take advantage of SIMD (same input-multiple data) vectorization on a vector-CPU. Modern CPUs typically have 256- or 512-bit wide processing units, which can accommodate multiple 32- or 64-bit numbers. Programming with these, however, is quite difficult and requires specialized commands to be called. Luckily for us, many NumPy ufuncs will automatically use these for us, speeding up our code!

  3. ufuncs are capable of calculating their output in place, meaning they can place calculated values in pre-allocated memory rather than allocating and returning new values. This is important because memory allocation is one of the slowest processes computers can perform, and should be avoided. With ufuncs, this can be done using the out keyword in arguments (ex numpy.add(a, b, out=c), or more succinctly, numpy.add(a, b, c)). While this may seem counterintuitive at first, the alternative (c = np.add(a,b) or c = a+b) causes an entirely new array to be allocated, with c pointing at that. These array allocations can add up very quickly: e = a*b + c*d, for example, would allocate 3 different arrays: one for a*b, one for c*d, and one for the sum of those two. As we write ufuncs, it is important that we try to use functions that operate in place as much as possible!

Submodules#

pygama.dsp.processors.bl_subtract module#

pygama.dsp.processors.convolutions module#

pygama.dsp.processors.convolutions.cusp_filter(length: int, sigma: float, flat: int, decay: int) → Callable#

Apply a CUSP filter to the waveform.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Parameters:

  • length (int) – the length of the filter to be convolved.

  • sigma (float) – the curvature of the rising and falling part of the kernel.

  • flat (int) – the length of the flat section.

  • decay (int) – the decay constant of the exponential to be convolved.

Return type:

Callable

JSON Configuration Example

"wf_cusp": {
    "function": "cusp_filter",
    "module": "pygama.dsp.processors",
    "args": ["wf_bl", "wf_cusp(101,f)"],
    "unit": "ADC",
    "init_args": ["len(wf_bl)-100", "40*us", "3*us", "45*us"]
}
pygama.dsp.processors.convolutions.t0_filter(rise: int, fall: int) → Callable#

Apply a modified, asymmetric trapezoidal filter to the waveform.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Parameters:

  • rise (int) – the length of the rise section. This is the linearly increasing section of the filter that performs a weighted average.

  • fall (int) – the length of the fall section. This is the simple averaging part of the filter.

Return type:

Callable

JSON Configuration Example

"wf_t0_filter": {
    "function": "t0_filter",
    "module": "pygama.dsp.processors",
    "args": ["wf_pz", "wf_t0_filter(3748,f)"],
    "unit": "ADC",
    "init_args": ["128*ns", "2*us"]
}
pygama.dsp.processors.convolutions.zac_filter(length: int, sigma: float, flat: int, decay: int) → Callable#

Apply a ZAC (Zero Area CUSP) filter to the waveform.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Parameters:

  • length (int) – the length of the filter to be convolved.

  • sigma (float) – the curvature of the rising and falling part of the kernel.

  • flat (int) – the length of the flat section.

  • decay (int) – the decay constant of the exponential to be convolved.

Return type:

Callable

JSON Configuration Example

"wf_zac": {
    "function": "zac_filter",
    "module": "pygama.dsp.processors",
    "args": ["wf_bl", "wf_zac(101,f)"],
    "unit": "ADC",
    "init_args": ["len(wf_bl)-100", "40*us", "3*us", "45*us"],
}

pygama.dsp.processors.dwt module#

pygama.dsp.processors.dwt.discrete_wavelet_transform(wave_type: str, level: int) → Callable#

Apply a discrete wavelet transform to the waveform and return only the approximate coefficients.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args. The output waveform dimension must be specified.

Parameters:

  • wave_type (str) – The wavelet type for discrete convolution ('haar', 'db1', ...)

  • level (int) – The level of decompositions to be performed (1, 2, ...)

Return type:

Callable

JSON Configuration Example

"dwt":{
    "function": "discrete_wavelet_transform",
    "module": "pygama.dsp.processors",
    "args": ["wf_blsub", "dwt(250)"],
    "unit": "ADC",
    "prereqs": ["wf_blsub"],
    "init_args": ["'haar'", "3",]
}

pygama.dsp.processors.fftw module#

pygama.dsp.processors.fftw.dft(buf_in: ndarray, buf_out: ndarray) → Callable#

Perform discrete Fourier transforms using the FFTW library.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Note

FFTW optimizes the FFT algorithm based on the size of the arrays, with SIMD parallelized commands. This optimization requires initialization, so this is a factory function that returns a Numba gufunc that performs the FFT. FFTW works on fixed memory buffers, so you must tell it what memory to use ahead of time. When using this with ProcessingChain, to ensure the correct buffers are used, call get_variable() to give it the internal memory buffer directly. With build_dsp(), you can just give it the name, and it will automatically happen. The possible dtypes for the input/outputs are:

numpy.dtype

Size

numpy.dtype

Size

float32/float

\(n\)

complex64

\(n/2+1\)

float64/double

\(n\)

complex128

\(n/2+1\)

float128/longdouble

\(n\)

complex256/clongdouble

\(n/2+1\)

complex64

\(n\)

complex64

\(n\)

complex128

\(n\)

complex128

\(n\)

complex256/clongdouble

\(n\)

complex256/clongdouble

\(n\)
Return type:

Callable

pygama.dsp.processors.fftw.inv_dft(buf_in: ndarray, buf_out: ndarray) → Callable#

Perform inverse discrete Fourier transforms using the FFTW library.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Note

FFTW optimizes the FFT algorithm based on the size of the arrays, with SIMD parallelized commands. This optimization requires initialization, so this is a factory function that returns a Numba gufunc that performs the FFT. FFTW works on fixed memory buffers, so you must tell it what memory to use ahead of time. When using this with ProcessingChain, to ensure the correct buffers are used, call get_variable() to give it the internal memory buffer directly. With build_dsp(), you can just give it the name, and it will automatically happen. The possible dtypes for the input/outputs are:

numpy.dtype

Size

numpy.dtype

Size

complex64

\(n/2+1\)

float32/float

\(n\)

complex128

\(n/2+1\)

float64/double

\(n\)

complex256/clongdouble

\(n/2+1\)

float128/longdouble

\(n\)

complex64

\(n\)

complex64

\(n\)

complex128

\(n\)

complex128

\(n\)

complex256/clongdouble

\(n\)

complex256/clongdouble

\(n\)
Return type:

Callable

pygama.dsp.processors.fftw.psd(buf_in: ndarray, buf_out: ndarray) → Callable#

Perform discrete Fourier transforms using the FFTW library, and use it to get the power spectral density.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Note

FFTW optimizes the FFT algorithm based on the size of the arrays, with SIMD parallelized commands. This optimization requires initialization, so this is a factory function that returns a Numba gufunc that performs the FFT. FFTW works on fixed memory buffers, so you must tell it what memory to use ahead of time. When using this with ProcessingChain, to ensure the correct buffers are used, call get_variable() to give it the internal memory buffer directly. With build_dsp(), you can just give it the name, and it will automatically happen. The possible dtypes for the input/outputs are:

numpy.dtype

Size

numpy.dtype

Size

complex64

\(n\)

float32/float

\(n\)

complex128

\(n\)

float64/double

\(n\)

complex256/clongdouble

\(n\)

float128/longdouble

\(n\)

float32/float

\(n\)

float32/float

\(n/2+1\)

float64/double

\(n\)

float64/double

\(n/2+1\)

float128/longdouble

\(n\)

float128/longdouble

\(n/2+1\)
Return type:

Callable

pygama.dsp.processors.fixed_time_pickoff module#

pygama.dsp.processors.gaussian_filter1d module#

pygama.dsp.processors.gaussian_filter1d.gaussian_filter1d(sigma: int, truncate: float = 4.0) → ndarray#

1-D Gaussian filter.

Note

This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args.

Parameters:

  • sigma (int) – standard deviation for Gaussian kernel

  • truncate (float) – truncate the filter at this many standard deviations.

Return type:

ndarray

pygama.dsp.processors.get_multi_local_extrema module#

pygama.dsp.processors.histogram module#

pygama.dsp.processors.linear_slope_fit module#

pygama.dsp.processors.log_check module#

pygama.dsp.processors.min_max module#

pygama.dsp.processors.moving_windows module#

pygama.dsp.processors.multi_a_filter module#

pygama.dsp.processors.multi_t_filter module#

pygama.dsp.processors.optimize module#

class pygama.dsp.processors.optimize.Model(func: Callable, w_in: ndarray, baseline: float, beg: int, end: int)#

Bases: object

A class representing the function to minimize.

errordef = 1.0#

pygama.dsp.processors.param_lookup module#

pygama.dsp.processors.param_lookup.param_lookup(param_dict: dict[int, float], default_val: float, dtype: str | numpy.dtype) → ufunc#

Generate the numpy.ufunc lookup(channel, val), which returns a NumPy array of values corresponding to various channels that are looked up in the provided param_dict. If there is no key, use default_val instead.

Return type:

ufunc

pygama.dsp.processors.peak_snr_threshold module#

pygama.dsp.processors.pole_zero module#

pygama.dsp.processors.presum module#

pygama.dsp.processors.pulse_injector module#

pygama.dsp.processors.saturation module#

pygama.dsp.processors.soft_pileup_corr module#

pygama.dsp.processors.time_over_threshold module#

pygama.dsp.processors.time_point_thresh module#

pygama.dsp.processors.trap_filters module#

pygama.dsp.processors.upsampler module#

pygama.dsp.processors.wiener_filter module#

pygama.dsp.processors.wiener_filter.wiener_filter(file_name_array: list[str]) → ndarray#

Apply a Wiener filter to the waveform.

Note

The convolution is performed in the frequency domain. This processor is composed of a factory function that is called using the init_args argument. The input and output waveforms are passed using args. The input must be the Fourier transform of the waveform. The output is the filtered waveform in the frequency domain.

Parameters:

file_name_array (list[str]) – Array with path to an HDF5 file containing the time domain version of the superpulse in one column and noise waveform in another, the superpulse HDF5 group must be titled spms/processed/superpulse and the noise waveform must be called spms/processed/noise_wf.

Return type:

ndarray

JSON Configuration Example

"wf_wiener": {
    "function": "wiener_filter",
    "module": "pygama.dsp.processors",
    "args": ["wf_bl_fft", "wf_wiener(2000,f)"],
    "unit": "dB",
    "init_args": ["/path/to/file/wiener.lh5"]
}

pygama.dsp.processors.windower module#