Interpret the output of the statistical_checks module

This notebook is not meant as a tutorial on how to run the statistical_checks module, as this was already covered in the tutorial about the pygwb_stats script here. In this tutorial, we go over the plots generated by the module, and provide a brief description of each of the plots to help the user interpret the results.

For more information on the module, we refer the user to the module API page here. For more general information about the module, we refer the user to the pygwb paper.

Plots

Plotting running quantities

The two plots below show the running point estimate and the standard deviation as the analysis run evolves, i.e., as more data is accumulated. This is particularly useful to determine whether the point estimate evolves towards a non-zero value, meaning it is detecting something, or not. In addition, the standard deviation should follow a \(1/\sqrt{\rm time}\) behavior, which can be easily verified in the plot. More information about the above quantities can be found in the pygwb paper.

Output of plot_running_point_estimate()

Output of plot_running_sigma()

Plotting the IIFT of the point estimate integrand

Output of plot_IFFT_point_estimate_integrand()

The plot below shows the inverse Fourier transform of the point estimate integrand. In the case of detection, the plot should display a peak around 0, corresponding to a signal that is present only when the two data-streams are not time-shifted, whereas the signal disappears for other non-zero values. Combining this plot with the previous two plots provides additional certainty in the detection of a signal.

Plotting spectra

The plots below show the signal-to-noise ratio (SNR) as a function of frequency, together with the standard deviation. Both the areal and imaginary part of the spectrum are shown, as the real part of the spectrum contains information about the signal, whereas the imaginary part contains information about the noise. The information of the standard deviation plot allows us to tell how noisy some of the frequency bins were, and therefore, together with the information of the SNR plot, how much a frequency bin contributed to the analysis.

Output of plot_SNR_spectrum()

Output of plot_cumulative_SNR_spectrum()

Output of plot_real_SNR_spectrum()

Output of plot_imag_SNR_spectrum()

Output of plot_sigma_spectrum()

Output of plot_cumulative_sensitivity()

Plotting the evolution as a function of time (i.e. analysis segment)

The plot below shows the value of the point estimate, and its standard deviation per analysis segment, as a function of time. Deviations from the mean of the whole analysis run are shown as well. The quantities are shown before and after the delta-sigma cut, allowing to visualize its effect. Outliers in the standard deviation plot should be removed after the delta-sigma cut. If not, these should be flagged for follow-up. Any other irregularities in these plots are easily picked up and can then be flagged for further investigation.

Output of plot_omega_sigma_in_time()

Plotting the effect of the delta-sigma cut

The few plots below illustrate the effect of the delta-sigma cut on the quantities of interest (see here for information about the delta-sigma cut in pygwb). A clear cut should be seen in the value of the delta-sigma itself when comparing before and after cut values (the cut should be at the value specified in the parameter file for the delta-sigma cut). Note that any outliers should in principle be removed by the delta-sigma cut, as this removes any abnormally loud segments. Any remaining outliers should be flagged for follow-up.

Output of plot_hist_sigma_dsc()

Output of plot_scatter_sigma_dsc()

Output of plot_scatter_omega_sigma_dsc()

Output of plot_hist_omega_pre_post_dsc()

Testing Gaussianity

The Kolmogorov-Smirnov test can be used to verify the consistency of the data with some assumed distribution. For stochastic searches, we assume Gaussianity of our data. Hence, this test is good to verify this assumption. The plot shows the cumulative distribution function, together with the one of the data. The maximum deviation from the assumed distribution is displayed as a test statistic, together with the p-value. More information about the KS test can be found here.

Output of plot_KS_test()

Visualizing the variation in variance

The plot below illustrates the variation in the variance. The histogram should be centered around 1, as this would correspond to values of the variance centered around the mean of the variance. Large fluctuations or outlier bins should be investigated.

Output of plot_hist_sigma_squared()

Fitting as a function of time

The two plots below plot the values of the point estimate and the standard deviation as a function of time and allow for a linear fit to be performed. Any trend in these quantities can easily be visualized in this plot, together with the numerical values of the fit parameters.

Output of plot_omega_time_fit()

Output of plot_sigma_time_fit()

Effect of gating

If gating was performed during the analysis run, the duration of the gates can be represented visually as a function of the analysis segment. Gates lasting longer than a few seconds should be flagged for follow-up. More information on the gating procedure can be found in the pygwb paper or in the implementation of gating in the pygwbmodule here.

Output of plot_gates_in_time()