-----------------------
Data Quality Dashboards
-----------------------
An MEG signal is a measurement of a very small magnetic field, in the order of
100 fT, where femtotesla :math:`1fT = 10^{-15} T` and picotesla
:math:`1pT = 10^{-12} T`. EEG scalp signals are about 50 to 100
:math:`\mu\text{V}`. Because the signals of interest are this small, the noise
level of the empty magnetically shielded room is the baseline that decides how
well brain activity can be measured. These dashboards track that baseline over
time.
.. note::
The empty-room recordings are processed automatically every day at 09:00 UAE
time by the ``Empty-room Data Quality Dashboard`` GitHub Action; the website
rebuilds one hour later. If a plot looks stale, check that workflow's last
run in the repository's Actions tab.
.. _data_quality_metrics:
Data Quality metrics
====================
The metrics, defined in the table below, serve as the basis to assess the
quality of empty-room data acquired from either the MEG-KIT or MEG-OPM system.
The SNR (Signal to Noise Ratio) of an experiment can be qualitatively
evaluated from these metrics: poor values can point to a new unidentified
recurrent noise source, a defect in the equipment, or a one-off event. Poor
noise conditions translate into experiments needing more trials, or heavier
artifact removal during analysis.
.. csv-table:: Noise Metrics Table
:file: ../../data/data-quality-dashboards/noise_metrics.csv
:header-rows: 1
KIT Data Quality Dashboard
==========================
This dashboard monitors the quality of the data generated from the KIT-MEG
system. Empty room data is recorded from the KIT system regularly. The dataset
on NYU Box is BIDS valid, so each recording lives under
``sub-emptyroom/ses-YYYYMMDD/meg/`` and is identified by the BIDS ``acq-kit``
entity. For every recording the metrics described above are computed per
sensor and summarised across sensors.
Latest recordings
-----------------
**How to read this table:** one row per recording, most recent first.
The table shows the latest recordings; expand the dropdown below it for the
complete history.
- **Status** compares the recording against the provisional thresholds: it is
green (within thresholds) when the median sensor RMS is at or below 1000 fT
**and** the median noise floor is at or below 30 fT per sqrt(Hz); it is red
(above thresholds) otherwise. A red row means the room was noisier than the
acceptance level at that time, so check the topography and spectrum below to
find out why.
- **RMS Median (fT)** is the median over sensors of each sensor's root mean
square amplitude in the analysis window: the overall noise magnitude.
- **Noise Floor Median (fT/sqrtHz)** is the median over sensors of the power
spectral density in the 5 to 7 Hz band: the broadband floor, away from mains
and drift.
- **Line Noise Median (fT)** is the median amplitude at 50 Hz, the UAE mains
frequency: how much electrical interference leaks into the room.
- **Run** distinguishes multiple recordings made in the same session.
- **Details** is a free text field the team can fill to explain a result (for
example, construction work that day).
.. csv-table:: Most recent KIT recordings
:file: ../../data/data-quality-dashboards/kit-empty-room-metrics-recent.csv
:header-rows: 1
.. dropdown:: Full recording history (most recent first)
.. csv-table:: KIT empty-room metrics per recording
:file: ../../data/data-quality-dashboards/kit-empty-room-metrics-display.csv
:header-rows: 1
Sensor noise topography
-----------------------
**How to read these maps:** each disc shows the per sensor RMS noise of one
recording, projected onto the KIT sensor layout viewed from above (nose at the
top). A localised bright spot points to a noisy or faulty individual sensor; a
whole map shifting to brighter colours points to a noisy session.
.. warning::
Each map has its **own colour scale**. Always read the colourbar maximum
before comparing two maps: a map that looks uniformly dark but whose scale
tops at 30000 fT is far noisier than a bright-looking map whose scale tops
at 1500 fT.
.. image:: ../../_static/2-data-quality-dashboards/kit_topomap_recent.png
:width: 100%
:alt: KIT per sensor RMS noise topography for recent sessions
The map below is the interactive version of the same recent recordings. Pick a
recording in the dropdown, then hover any sensor dot to read the sensor name
(for example ``MEG 041``) together with its RMS, noise floor and 50 Hz line
noise. Use it to identify exactly which sensor causes a hotspot seen in the
static maps above.
.. raw:: html
Noise per sensor across sessions
--------------------------------
**How to read this heatmap:** rows are sensors, columns are sessions, colour is
RMS noise. A bright **horizontal** band identifies a sensor that is
consistently noisy across sessions (a sensor problem: it should be flagged to
the MEG team). A bright **vertical** band identifies a single noisy session
(an environment problem that day). Hover any cell for the exact sensor,
session and value.
.. raw:: html
Amplitude spectrum
------------------
**How to read the spectrum:** the median amplitude across sensors of the most
recent recording, per frequency. Expect a smooth floor with a rise at low
frequencies (slow drifts, breathing of the building) and a sharp peak at 50 Hz
with harmonics at 100 and 150 Hz (mains interference). New peaks at other
frequencies usually identify a specific device switched on near the MSR;
noting the frequency is the fastest way to track down the culprit.
.. dropdown:: KIT amplitude spectrum (most recent session)
.. raw:: html
Metrics over time
-----------------
**How to read the time series:** each point is one recording, so these plots
show the trend of the room's noise across months. A single outlier usually has
a one-off explanation (check the Details column of the table above); a
sustained upward drift is the signal to investigate the room, the electronics
or the helium system before experiments degrade.
.. dropdown:: KIT median sensor RMS over time
.. raw:: html
.. dropdown:: KIT median noise floor over time
.. raw:: html
.. dropdown:: KIT line noise (50 Hz) over time
.. raw:: html
.. dropdown:: KIT average, variance and maximum over time (legacy metrics)
These three raw statistics of the signal are kept for continuity with the
original dashboards; the RMS, noise floor and line noise metrics above are
the ones used for the acceptance thresholds.
.. raw:: html
Perspectives on Data Quality dashboards
=======================================
Implemented:
- Per sensor metrics with sensor topography maps and a sensor by session heatmap, so faulty sensors and channel based issues are now visible.
- Incremental tracking: a recording already present in the metrics CSV is skipped, so files are not redownloaded or recomputed unless a force flag is set.
- The metrics CSV is updated per recording as each file is processed, not only at the end.
- The analysis window is cropped to bound memory use.
- Generation runs in a dedicated GitHub Action that commits the artifacts, rather than during the docs build.
Still open:
- Check the lab manual to refine the metrics and thresholds (the current thresholds are provisional starting points).
- Whiten the data to compute a noise covariance and a whitened noise figure.
- Trigger the dashboard refresh on dataset changes (a file added on Box) rather than only on a daily schedule.
- Populate the OPM data quality dashboard (a single OPM empty-room recording exists on Box so far).
- The OPM interactive topomap appears automatically once the next OPM recording is processed (it needs the stored sensor positions file; KIT already works today through its builtin layout).