Skip to content

Core: cicada_tools.core

from cicada_analysis.cicada_tools.core.<module> import <function>


core.array_utils (Array and string manipulation tools)

  • find_nearest

find_nearest(array, value, is_sorted=True) -> int

Return the index of the element in array closest to value.

Parameter Type Description
array 1-D np.ndarray The array to search
value float The target value
is_sorted bool Uses binary search when True (faster); set False for unsorted arrays

Returns: int — Index of the closest element

  • validate_indices_in_string_format

validate_indices_in_string_format(str_indices) -> bool

Validate index string format; return True if valid, else False with a printed error message.

Parameter Type Description
str_indices str The index string to validate

Returns: bool — True if valid, False otherwise

  • extract_indices_from_string

extract_indices_from_string(str_indices) -> list[int]

Parse a string like "0-4 7 10-12" into [0, 1, 2, 3, 4, 7, 10, 11, 12].

Parameter Type Description
str_indices str Index string with ranges and individual values

Returns: list[int] — Expanded list of indices

  • alpha_numeric_comparator

alpha_numeric_comparator(data) -> int

Convert a string label to an integer key for natural numeric sorting (e.g., 'p7' -> 7).

Parameter Type Description
data str The string label to convert

Returns: int — Numeric key for sorting

  • flatten

flatten(lst) -> list

Recursively flatten a nested iterable of any depth.

Parameter Type Description
lst iterable The nested iterable to flatten

Returns: list — Flattened list

print_info_dict(my_dict) -> None

Pretty-print a nested dict (session metadata, config, etc.).

Parameter Type Description
my_dict dict The dictionary to print

Returns: None

  • take_items

take_items(data, indices) -> np.ndarray | list

The items of data at indices: the rows of an array (any shape, first axis = items), or the items of a list / object array of variable-length arrays such as spike trains.

Parameter Type Description
data np.ndarray | list (n_items, ...) array, or list / object array of arrays
indices list[int] Indices of the items

Returns: np.ndarray | list — a new array data[indices] for a non-object array, else a list of the items

  • n_distinct_values_sampled

n_distinct_values_sampled(values, max_samples=200000) -> int

The number of distinct values in values, counted on every size // max_samples-th element (all of them when the array has fewer than 2 × max_samples elements), without a full copy or sort of large arrays. Typical use: telling binary / discrete data (rasters) from continuous data (traces).

Parameter Type Default Description
values np.ndarray Array of any shape
max_samples int 200000 Sets the stride; up to about 2 × max_samples elements are read

Returns: int — number of distinct values in the sample

  • decimate_min_max

decimate_min_max(values, times, factor) -> tuple

Downsample the rows of values for display, keeping the minimum and maximum of each bin so that brief events (transients, spikes) stay visible: the number of points is divided by factor (bins of 2 × factor samples, the last one may be shorter, each drawn as 2 points, min then max, at the bin's first and last time). NaNs are ignored. For drawing only: analyses use the full-resolution data.

Parameter Type Description
values np.ndarray (n_rows, n_samples)
times np.ndarray (n_samples,), time of each sample
factor int Division of the number of points (1 = unchanged)

Returns: tuple[np.ndarray, np.ndarray] — (decimated_values, decimated_times), shapes (n_rows, 2 * n_bins) and (2 * n_bins,) with n_bins = ceil(n_samples / (2 * factor)); the inputs unchanged if factor <= 1 or n_samples <= 2 * factor


core.path_utils (Foundational filesystem path utilities)

  • get_subfiles

get_subfiles(current_path: str, relative_path: bool = False, depth: int = 1) -> list[str]

Return file names found by walking current_path.

Parameter Type Default Description
current_path str The directory to search
relative_path bool False Return relative paths
depth int 1 Search depth

Returns: list[str] — List of file names

  • get_subdirs

get_subdirs(current_path: str, depth: int = 1) -> list[str]

Return subdirectory names found by walking current_path.

Parameter Type Default Description
current_path str The directory to search
depth int 1 Search depth

Returns: list[str] — List of subdirectory names

  • path_leaf

path_leaf(path: str) -> str

Return the final component of a path (file name or last directory name).

Parameter Type Description
path str The input path

Returns: str — Final path component

  • fit_file_name_to_path_limit

fit_file_name_to_path_limit(directory, base_name, suffix, max_path=259) -> str

base_name shortened so that directory/base_name + suffix fits the Windows path length limit (no change on other OS, or when it already fits): the end of base_name is cut, at least 8 characters kept and trailing spaces, dots and _ stripped, with a printed message. Deep results folders combined with long session identifiers otherwise make savefig fail on Windows.

Parameter Type Default Description
directory str Folder the file is written to
base_name str File name without suffix
suffix str Fixed end of the file name, kept intact (timestamp, extension)
max_path int 259 Maximum full path length

Returns: str — base_name, possibly shortened (without suffix)

  • safe_file_name

safe_file_name(name, max_length=150) -> str

name usable as a file name on any OS: forbidden characters (< > : " / \ | ? * and control characters) become _, runs of _ are collapsed, leading and trailing spaces, dots and _ are removed, and the result is cut to max_length. Session identifiers can be paths or contain :.

Parameter Type Default Description
name str Proposed file name (without directory); converted with str()
max_length int 150 Maximum length of the result

Returns: str — the sanitized name, 'unnamed' if nothing is left


core.table_utils (Saving the data of the results and figures; trial / epoch table helpers)

  • save_tables

save_tables(tables, folder)

Save each table as <folder>/<name>.csv, without the index (names made safe for file names, shortened to the Windows path limit).

Parameter Type Default Description
tables dict[str, DataFrame] Tables by file name, without extension; None is skipped
folder str Folder to write to (created if missing)

Returns: list[str] — paths written

  • save_samples

save_samples(name, folder, rows=None, **arrays)

Save series drawn sample by sample: each array as <name>_<key>.npy, and the description of their rows (order, ids, groups) as <name>_rows.csv.

Parameter Type Default Description
name str Common start of the file names (e.g. '<session>_raster')
folder str Folder to write to (created if missing)
rows DataFrame None One row per row of the arrays, in their order
**arrays Arrays by key (e.g. data=, times=); None is skipped

Returns: list[str] — paths written

  • flat_columns

flat_columns(table)

A grouped table as one row of plain columns per group: the index levels become columns and multi-level column names are joined with _ (e.g. ('tau', 'mean') becomes tau_mean), so that the .csv has one header row.

Parameter Type Default Description
table DataFrame e.g. the result of groupby(...).agg([...])

Returns: DataFrame — a new table

  • ragged_to_arrays

ragged_to_arrays(trains)

Rows of different lengths (e.g. the spike times of each unit) as two flat arrays, for save_samples.

Parameter Type Default Description
trains list[array-like] One array per row

Returns: tuple[np.ndarray, np.ndarray] — the row index of each value, and the values

  • intervals_table

intervals_table(intervals_by_group, names)

Intervals drawn per group (e.g. the event, epoch and stimulation bands over a display) as one table.

Parameter Type Default Description
intervals_by_group list[list[tuple]] Per group, its (start, stop) intervals
names list[str] Name of each group

Returns: DataFrame — columns group, start, stop; one row per interval

  • filter_table_with_dict

filter_table_with_dict(table, filtering_dict) -> pd.DataFrame

Filter a DataFrame by multiple column conditions specified as a dict ({column: [allowed values]}); for a column holding several values per row (e.g. tags), any of them. The dict is not modified.

Parameter Type Description
table pd.DataFrame Table to filter
filtering_dict dict Filter conditions

Returns: pd.DataFrame — Filtered DataFrame

  • table_time_columns

table_time_columns(table) -> list

Columns of a trial / epoch table holding times (name containing time, one or several numbers per row), start_time first: the times a PSTH can be aligned on.

Parameter Type Description
table pd.DataFrame Trial/epoch table

Returns: list — List of time column names

  • table_column_times

table_column_times(table, column) -> np.ndarray

One time per row of a time column (see table_time_columns): the first value for a row holding several (e.g. Allen lick_times), NaN for a row without a time.

Parameter Type Description
table pd.DataFrame Trial/epoch table
column str Name of a time column

Returns: np.ndarray — 1D float array, one time per row


core.period_utils (Foundational frame-period and timestamp utilities)

  • get_continous_time_periods

get_continous_time_periods(binary_array) -> list[tuple[int, int]]

Convert a binary 0/1 array to a list of (start, stop) frame index pairs.

Parameter Type Description
binary_array 1-D np.ndarray Array of 0s and 1s

Returns: list[tuple[int, int]] — List of (start, stop) frame index pairs

  • from_timestamps_to_frame_epochs

from_timestamps_to_frame_epochs(time_stamps_array, frames_timestamps, as_list) -> list | np.ndarray

Convert a 2-D timestamp array to frame-index epoch pairs.

Parameter Type Description
time_stamps_array (2, n_epochs) array Timestamp array
frames_timestamps 1-D array Frame timestamps in seconds
as_list bool If True return list[tuple[int, int]]; if False return (2, n_epochs) array

Returns: list | np.ndarray — Frame-index epoch pairs

  • from_timestamps_array_to_list

from_timestamps_array_to_list(time_stamps_array: np.ndarray, frames_timestamps: np.ndarray) -> list[tuple[float, float]]

Filter a 2-D timestamp array to only epochs within the recording range.

Parameter Type Description
time_stamps_array np.ndarray 2-D timestamp array
frames_timestamps np.ndarray Frame timestamps

Returns: list[tuple[float, float]] — Filtered epoch list

  • get_complementary_frames

get_complementary_frames(total_frames, active_periods) -> tuple[np.ndarray, np.ndarray]

Return the frames not covered by active_periods (list of (start_frame, end_frame), both inclusive) among total_frames frames: (bool_complementary, complementary_frames), a boolean mask and the frame indices.

Parameter Type Description
total_frames int Total number of frames
active_periods list[tuple[int, int]] List of (start_frame, end_frame) tuples

Returns: tuple[np.ndarray, np.ndarray] — (boolean mask, complementary frame indices)

  • fill_gaps_in_continuous_periods

fill_gaps_in_continuous_periods(boolean_serie: np.ndarray, max_gap_size: int) -> np.ndarray

Fill small gaps between continuous True periods in a boolean array like [True, True, True, False, True, True] into [True, True, True, True, True, True] with max_gap_size >= 2.

Parameter Type Description
boolean_serie np.ndarray Boolean array with gaps
max_gap_size int Maximum gap size to fill

Returns: np.ndarray — Boolean array with gaps filled

  • extend_invalid_times

extend_invalid_times(invalid_times: np.ndarray, padding: float | tuple[float, float]) -> np.ndarray

Pad an invalid-times array: the same margin on both sides, or, with (pre, post), each start moved back by post and each end forward by pre, so that an event whose window [T - pre, T + post] touches an invalid interval falls in the padded one. Starts are not moved below 0.

Parameter Type Description
invalid_times np.ndarray (2, n_invalid): row 0 starts, row 1 ends (s)
padding float | tuple Margin in seconds, or the (pre, post) window of the events

Returns: np.ndarray — (2, n_invalid) padded copy

  • filter_epoch_timestamps

filter_epoch_timestamps(epochs_timestamps: np.ndarray, extended_invalid_times: np.ndarray | None, epochs_are_sorted: bool, invalid_times_are_sorted: bool, filter_consecutive: bool = False, use_main_epoch: bool = False, main_epoch_timestamps: np.ndarray | None = None, main_epoch_name: str = "", baseline_length: float = 0.0, verbose: bool = False, epoch_name: str = "") -> np.ndarray

Filter a (2, n_epochs) timestamp array using three optional criteria: invalid-time overlap, main-epoch containment, and consecutive-movement distance.

Parameter Type Default Description
epochs_timestamps np.ndarray (2, n_epochs) timestamp array
extended_invalid_times np.ndarray None Extended invalid time periods
epochs_are_sorted bool Whether epochs are sorted
invalid_times_are_sorted bool Whether invalid times are sorted
filter_consecutive bool False Filter consecutive events
use_main_epoch bool False Use main epoch filtering
main_epoch_timestamps np.ndarray None Main epoch timestamps
main_epoch_name str "" Main epoch name
baseline_length float 0.0 Baseline length in seconds
verbose bool False Verbose output
epoch_name str "" Epoch name

Returns: np.ndarray — Filtered epoch timestamps

  • filter_events_based_on_epochs

filter_events_based_on_epochs(events_ts: np.ndarray, epochs: np.ndarray) -> np.ndarray

Filter event timestamps, keeping only those that fall inside at least one epoch window.

Parameter Type Description
events_ts np.ndarray Event timestamps
epochs np.ndarray Epoch timestamps

Returns: np.ndarray — Filtered event timestamps

  • split_events_by_epoch_groups

split_events_by_epoch_groups(events_ts, epochs_by_group) -> dict

Split event timestamps by epoch group: an event belongs to every group with an epoch containing it (filter_events_based_on_epochs: strictly inside, an event exactly on an epoch edge is left out).

Parameter Type Description
events_ts np.ndarray 1-D event timestamps
epochs_by_group dict {group: (2, n_epochs) array} (row 0 starts, row 1 ends, same unit as the events); a None value keeps every event in that group

Returns: dict — {group: 1-D array of its events}, groups in the input order (possibly empty arrays)

  • valid_intervals

valid_intervals(epochs, start, end, invalid_times=None) -> numpy.ndarray

Time left to count events in: the epochs cut to [start, end] (the recording), overlaps merged, minus the invalid times. Events kept with filter_events_based_on_epochs(events, intervals) and the duration (intervals[1] - intervals[0]).sum() then cover the same time.

Parameter Type Default Description
epochs np.ndarray | None (2, n) starts / ends; None for the whole [start, end]
start float Start of the recording
end float End of the recording (same unit)
invalid_times np.ndarray None (2, n) starts / ends to leave out

Returns: numpy.ndarray — (2, n) sorted, disjoint intervals of positive length; (2, 0) if none

  • get_epoch_frames

get_epoch_frames(epoch_times, reference_ts, max_idx) -> np.ndarray

Frames of the epochs: for each epoch, the frames of reference_ts from the frame nearest its start to the frame nearest its stop (excluded), epoch after epoch (a frame in two epochs is listed twice); frames above max_idx are left out.

Parameter Type Description
epoch_times np.ndarray (2, n_epochs) start and stop times (same unit as reference_ts)
reference_ts np.ndarray Timestamps of the frames, sorted
max_idx int Highest frame index kept

Returns: np.ndarray — 1-D frame indices, empty if none

  • get_epoch_transition_list

get_epoch_transition_list(epoch_times) -> np.ndarray

Starts of the epochs that are transitions: every start, except that of a first epoch starting at 0 (nothing before it). Needs at least one epoch.

Parameter Type Description
epoch_times np.ndarray (2, n_epochs) start and stop times

Returns: np.ndarray — the transition times


core.event_alignment (Event alignment)

  • align_array_to_events

align_array_to_events(data, event_frames, window_frames, subtract_baseline=False) -> np.ndarray

Core event-alignment — extracts a fixed-length window around each event.

Parameter Type Default Description
data np.ndarray (n_cells, n_frames) or (n_frames, H, W)
event_frames list[int] List of event frame indices
window_frames tuple[int, int] (n_pre, n_post) — output length = n_pre + n_post + 1
subtract_baseline bool False Subtract mean of full pre-event window per event

Returns: np.ndarray — (n_cells, n_events, n_t) or (n_events, n_t, H, W). Events outside recording bounds are NaN-filled (not dropped).

  • align_array_to_timestamps

align_array_to_timestamps(data, event_timestamps, frame_timestamps, window_s, subtract_baseline=False, time_axis=1, drop_outside=False) -> np.ndarray

Timestamp wrapper around align_array_to_events. Computes frame window from median sampling rate. The single alignment function on event times (replaces align_array_on_timestamps_list and align_array_from_table_stim_time).

Parameter Type Default Description
data np.ndarray Data array to align
event_timestamps np.ndarray Event timestamps in seconds
frame_timestamps np.ndarray Frame timestamps in seconds
window_s tuple[float, float] (pre_s, post_s) in seconds — all timestamps must be in seconds; the event is at the frame nearest to its time
subtract_baseline bool False Subtract baseline
time_axis int 1 Time axis: 0 for time-first data (n_frames,) or (n_frames, n_features) → (n_events, n_t) or (n_events, n_t, n_features)
drop_outside bool False Drop events whose window leaves the recording (default: NaN-filled)

Returns: np.ndarray — Event-aligned array. Same return shape as align_array_to_events. Window in frames: window_to_frames

  • window_to_frames

window_to_frames(sampling_rate, window_s) -> tuple

Convert a time window to frame counts. Returns (n_pre, n_post) frames of a (pre_s, post_s) window in seconds. The rule of every PSTH: pre rounded up (at least pre_s before the event), post rounded down (not past post_s after it). A product within 1e-9 frame of a whole number counts as that number, so float error does not add or drop a frame (e.g., 100 Hz × 0.29 s = 28.999999999999996 → 29).

Parameter Type Description
sampling_rate float Sampling rate in Hz
window_s tuple[float, float] Time window as (pre_s, post_s) in seconds

Returns: tuple[int, int] — (n_pre, n_post) frame counts

  • psth_grid_edges

psth_grid_edges(sampling_rate, window_s) -> np.ndarray

Edges of the common time grid of the population PSTHs (imaging, behavior): one bin per sample of a series at sampling_rate (the lowest rate of the sessions), each centred on its sample k / sampling_rate (the samples of window_to_frames). Each sample is half a bin from the edges, so rounding cannot put it in the next bin; series at higher rates are averaged within each bin.

Parameter Type Description
sampling_rate float Samples per second of the grid (the lowest rate)
window_s tuple[float, float] Time window as (pre_s, post_s) in seconds

Returns: np.ndarray — n_pre + n_post + 2 edges in seconds, from (-n_pre - 0.5) / rate to (n_post + 0.5) / rate

  • epoch_reference

epoch_reference(starts, stops, ref_in_epoch="start") -> np.ndarray

The time or frame each epoch is aligned on: its start, end or middle. Raises ValueError for another ref_in_epoch.

Parameter Type Default Description
starts array-like Start of each epoch (times or frames)
stops array-like End of each epoch, same length
ref_in_epoch str | None "start" "start" (also for None), "end" or "middle"

Returns: np.ndarray — One reference per epoch; the middle is (start + stop) / 2, a float also for frames

  • event_aligned_array_to_long_format_df

event_aligned_array_to_long_format_df(array, time_range, time_step) -> pd.DataFrame

Convert an event-aligned array to a long-format DataFrame.

Parameter Type Description
array np.ndarray (n_events, n_times) or (n_events, n_times, n_samples)
time_range tuple[float, float] (t_start, t_end) in seconds
time_step float Nominal time step between frames in seconds

Returns: pd.DataFrame — Long-format DataFrame with columns: data, event, time, sample


core.parameter_utils (Reading plugin parameter values as given by the GUI and the YAML configs)

  • is_selected

is_selected(value) -> bool

Whether a data-source parameter (e.g. ROI response series keys ['ophys', 'Fluorescence', 'dff'], or a spike-times column name) holds a selection. "Nothing selected" can appear as '', None, [], {}, 'none' or ['none', 'none', 'none'] depending on the widget or on a hand-written YAML.

Parameter Type Description
value any The parameter value

Returns: bool — False for None, a blank string, 'none', a list or tuple starting with 'none', and empty containers; True otherwise

  • normalize_groups

normalize_groups(groups) -> list[tuple]

The value of a group selector parameter as a list: the widget gives {name: [items, color]} (color None when the parameter has no colour), a hand-written YAML may give {name: items}.

Parameter Type Description
groups dict | None The parameter value

Returns: list[tuple] — (name, items as a list, color or None) per group, in the parameter's order

  • select_cells

select_cells(items, cell_indices_by_cell_type, n_cells, ids=None, keep=None) -> np.ndarray

Cells of a cell group: each item is a cell type (key of cell_indices_by_cell_type, which takes precedence) or cell indices typed as '1-4 6 15-17', kept in the typed order; indices outside 0 .. n_cells - 1 are left out, then the cells False in keep. A text that is neither gives no cell.

Parameter Type Default Description
items list | None Cell types and / or index strings; None for every cell
cell_indices_by_cell_type dict {cell type: indices} (calcium_imaging.get_cell_indices_by_cell_type)
n_cells int Number of cells of the series
ids list None Identifier of each cell (e.g. unit ids): typed numbers are these identifiers (ids not found left out) instead of positions
keep np.ndarray None Boolean mask of the cells that can be selected, by position (e.g. CicadaAnalysis.good_cells)

Returns: np.ndarray — int cell positions, in the order of the items (an index in two items is listed twice)


core.system_utils (Information about the computer running the analysis)

  • total_ram_gb

total_ram_gb() -> float

The total physical memory of this computer, in GB. Total rather than available memory: the available figure reported by some OS (notably Windows) is unreliable for deciding whether a computation will fit.

Returns: float — Total RAM in GB