=============================================
Stage 5: build MNE events from the triggers
=============================================

Author: Hadi Zaatiti <hadi.zaatiti@nyu.edu>

What & why
----------

Stage 5 turns the trigger information of each recording into an MNE **events**
array (``N x 3``: sample, previous value, event id) saved as a ``.eve`` text
file, plus an **audit TSV** with one row per detected event. Stage 6 reads these
files to build epochs and evoked responses.

Three variants exist; pick the one that matches how the triggers were coded
during acquisition:

.. list-table::
   :header-rows: 1
   :widths: 34 22 44

   * - Script
     - Input
     - Use when
   * - ``5-kit_compute_events_single_channel.py``
     - raw ``.con``
     - **Single-channel mode**: each event type on its own analog line
       (KIT 224 to 231). The event id is the KIT channel number.
   * - ``5-kit_compute_events_multi_channel.py``
     - raw ``.con``
     - **Binary-coded mode**: several lines rise together to encode one code.
       Channels rising within ``events.multi_channel.max_onset_skew_ms`` are
       combined into one event.
   * - ``5-a-kit_compute_events_STI014.py``
     - stage-2 ``.fif``
     - The digital ``STI 014`` stim channel already carries the event codes
       (the FIF was created with the ``kit2fiff`` stim settings). Uses
       ``mne.find_events``. **This is the KidLang variant.**

All three write into their own ``derivatives/<script-name>/`` folder, name the
outputs with the recording's BIDS entities plus ``desc-<label>``, and register
what they produced in an index CSV. Stage 6 reads ``events.desc`` and
``events.source`` from the YAML to find the right files.

What you need to provide
------------------------

- **Analog variants** (single and multi channel): the raw BIDS recordings
  ``sub-<id>/[ses-<id>/]meg/*_meg.con`` **and** a paired events JSON sidecar
  declaring the trigger mode (``"TriggerMode": "single_channel"`` or a
  multi-channel/binary-coded label). The JSON is located with the same
  fallback hierarchy as the stage-1 sanity check (exact scope, then
  subject + task, subject, dataset root).
- **STI 014 variant**: the FIFs written by stage 2 and listed in its
  ``kit2fiff_summary.csv``; no events table or JSON is needed.

Configuration file
------------------

Shared YAML, given with ``-c`` / ``--config``:

.. code-block:: yaml

   project:
     name: "kidlang"
     root_env: "MEG_DATA"

   subjects:
     include: []
     exclude: []

   bids_selection:
     tasks: []
     runs: []
     processings: []

   kit2fiff:
     coreg:
       mode: manual            # tells 5-a which stage-2 folder to read

   events:
     desc: sti014              # label written by stage 5; defaults: sti014 (5-a), autopulses (others)
     source: sti014            # branch read by stage 6: single_channel | multi_channel | sti014 | legacy | null
     find_events:              # 5-a only: keyword arguments of mne.find_events
       stim_channel: "STI 014"
       shortest_event: 1
       min_duration: 0.002
     multi_channel:            # multi-channel variant (and stage 1 multi) only
       max_onset_skew_ms: 2.0

   plots:                      # analog variants: stim-channel plots
     save_stim_plots: true
     format: pdf               # pdf | png | svg
     duration_s: 10.0
     misc_scaling: 1.0
     channels_mne: ["MISC 001", "MISC 002"]   # default: all eight trigger lines
     subdir: figures

How to run
----------

From the repository root with ``neurowaves`` activated:

.. code-block:: bash

   MODULE=pipeline.mne_pipelines.kit_general_pipelines
   CONFIG=pipeline/mne_pipelines/kit_general_pipelines/pipeline_config_files/kidlang_config.yml

   # KidLang: STI 014 codes from the stage-2 FIFs
   python -m $MODULE.5-a-kit_compute_events_STI014 -c $CONFIG

   # single-channel analog triggers
   python -m $MODULE.5-kit_compute_events_single_channel -c <config.yml>

   # binary-coded analog triggers
   python -m $MODULE.5-kit_compute_events_multi_channel -c <config.yml>

Options (all variants):

- ``--desc <label>``: override ``events.desc`` for the output names. If you use
  it, set the same label in the YAML before running stage 6.
- ``--overwrite``: replace existing ``.eve`` / ``_detail.tsv`` pairs. Without
  it an existing pair is reported as ``skipped_existing_output`` (or
  ``incomplete_existing_output`` when one of the two files is missing or
  empty).

The VS Code entries **(Kidlang) Find events** and **Compute events single
channel** run the STI 014 and single-channel variants.

What the scripts do
-------------------

Single-channel variant
^^^^^^^^^^^^^^^^^^^^^^

1. Resolve the BIDS root, subjects and recordings (``bids_selection``).
2. Locate the events JSON; skip the run if it is missing or does not declare
   ``single_channel``.
3. Open the ``.con`` with ``mne.io.read_raw_kit`` (no preload).
4. For each trigger line ``MISC 001`` to ``MISC 008``, detect pulses with
   ``detect_pulses_on_channel`` (same thresholds as stage 1) and map the line
   to its KIT channel number, which becomes the event id (224 to 231).
5. Optionally save a stim-channel plot (``plots`` block) under
   ``sub-<id>/figures/`` with ``desc-stimplot``.
6. Write ``<base>.eve`` and ``<base>_detail.tsv``; append to
   ``auto_events_index.csv``.

Multi-channel variant
^^^^^^^^^^^^^^^^^^^^^

Same discovery and detection, then rising edges across lines that occur within
``max_onset_skew_ms`` of each other are combined into one binary code. A later
rise starts a new event even if another line is still high. Overlaps,
non-coactive codes and falling-edge-only changes are flagged in the detail TSV
and the index as needing review; stage 6 can still read the file.

STI 014 variant
^^^^^^^^^^^^^^^

1. Resolve the stage-2 folder from ``kit2fiff.coreg.mode``
   (``2-kit_con_to_fif`` or ``2-kit_con_to_fif_manual``) and keep only the
   FIFs listed as current in ``kit2fiff_summary.csv``.
2. For each FIF, call ``mne.find_events(raw, **events.find_events)``.
3. Replace the ``desc-rawkit`` label and ``_raw`` suffix of the FIF name with
   ``desc-<label>_events``; write the ``.eve`` and a detail TSV with
   ``sample``, ``prev``, ``event_id`` and ``onset``.
4. Append to ``events_summary.csv``.

File naming
-----------

.. code-block:: text

   sub-<id>[_ses-<id>][_task-<task>][_acq-<l>][_run-<n>][_split-<n>][_rec-<l>][_space-<l>][_proc-<l>]_desc-<label>_events.eve
   sub-<id>..._desc-<label>_events_detail.tsv

Examples:

- ``sub-test1_task-400events_desc-autopulses_events.eve``
- ``sub-Y0312_task-dialogue_desc-sti014_events.eve``

Outputs
-------

.. code-block:: text

   <bids_root>/derivatives/5-kit_compute_events_single_channel/     # or ..._multi_channel/
     auto_events_index.csv
     configs/config_<timestamp>.yml
     sub-<id>/[ses-<id>/]
       sub-<id>..._desc-autopulses_events.eve
       sub-<id>..._desc-autopulses_events_detail.tsv
       figures/sub-<id>..._desc-stimplot.pdf

   <bids_root>/derivatives/5-a-kit_compute_events_STI014/
     events_summary.csv
     configs/config_<timestamp>.yml
     sub-<id>/[ses-<id>/]
       sub-<id>..._desc-sti014_events.eve
       sub-<id>..._desc-sti014_events_detail.tsv

Index columns:

- ``auto_events_index.csv`` (analog variants): ``subject``, ``file``,
  ``status``, ``events_eve``, ``detail_tsv``, ``n_events``, ``stim_plot``,
  ``comments``, ``mne_version``.
- ``events_summary.csv`` (STI 014): ``subject``, ``input_file``,
  ``events_found``, ``output_eve``, ``status``, ``mne_version``.

Detail TSV columns (analog variants): ``sample``, ``onset_s``,
``channel_mne``, ``channel_kit``, ``event_id``, ``width_ms``, ``amp_max``,
``amp_mean``.

Edge cases & safeguards
-----------------------

- **Missing or wrong-mode events JSON** (analog variants): run skipped and
  recorded in the index (``skipped_missing_events_json``,
  ``skipped_wrong_trigger_mode``).
- **No events found**: nothing is written for that recording; the index says
  ``no_events``.
- **MNE split files** (``..._raw-1.fif``): the STI 014 variant processes the
  base file only.
- **Existing outputs**: preserved unless ``--overwrite``.
- **Stage 6 ambiguity**: with ``events.source: null`` stage 6 searches every
  stage-5 branch and skips a recording if more than one event file matches.
  Pin ``source`` when more than one variant has been run.

Troubleshooting
---------------

- **No outputs (analog)**: check that the events JSON exists at a valid scope
  and that ``TriggerMode`` matches the script you chose.
- **No outputs (STI 014)**: stage 2 must have run with the same
  ``kit2fiff.coreg.mode``; the FIFs must appear in ``kit2fiff_summary.csv``.
  Check that ``kit2fiff.stim`` / ``slope`` / ``stimthresh`` produced a
  non-empty ``STI 014`` channel.
- **Unexpected counts**: compare with the stage-1 sanity check and open the
  ``_detail.tsv``; for ``find_events`` tune ``shortest_event`` /
  ``min_duration`` in ``events.find_events``.
- **Stage 6 finds no events**: ``events.desc`` in the YAML must equal the label
  used when stage 5 ran (including any ``--desc`` override).

Summary
-------

Stage 5 converts the trigger stream into canonical MNE event files with
entity-rich names and an audit trail. Choose the variant by trigger coding
(single channel, binary coded, or ``STI 014``), keep ``events.desc`` and
``events.source`` in the YAML consistent with it, and stage 6 will epoch
directly from these files.