Core: cicada_tools.core¶
from cicada_analysis.cicada_tools.core.<module> import <function>
core.array_utils (Array and string manipulation tools)¶
-
find_nearest¶
find_nearest(array, value, is_sorted=True) -> int
Return the index of the element in array closest to value.
| Parameter | Type | Description |
|---|---|---|
array |
1-D np.ndarray |
The array to search |
value |
float | The target value |
is_sorted |
bool | Uses binary search when True (faster); set False for unsorted arrays |
Returns: int — Index of the closest element
-
validate_indices_in_string_format¶
validate_indices_in_string_format(str_indices) -> bool
Validate index string format; return True if valid, else False with a printed error message.
| Parameter | Type | Description |
|---|---|---|
str_indices |
str | The index string to validate |
Returns: bool — True if valid, False otherwise
-
extract_indices_from_string¶
extract_indices_from_string(str_indices) -> list[int]
Parse a string like "0-4 7 10-12" into [0, 1, 2, 3, 4, 7, 10, 11, 12].
| Parameter | Type | Description |
|---|---|---|
str_indices |
str | Index string with ranges and individual values |
Returns: list[int] — Expanded list of indices
-
alpha_numeric_comparator¶
alpha_numeric_comparator(data) -> int
Convert a string label to an integer key for natural numeric sorting (e.g., 'p7' -> 7).
| Parameter | Type | Description |
|---|---|---|
data |
str | The string label to convert |
Returns: int — Numeric key for sorting
-
flatten¶
flatten(lst) -> list
Recursively flatten a nested iterable of any depth.
| Parameter | Type | Description |
|---|---|---|
lst |
iterable | The nested iterable to flatten |
Returns: list — Flattened list
-
print_info_dict¶
print_info_dict(my_dict) -> None
Pretty-print a nested dict (session metadata, config, etc.).
| Parameter | Type | Description |
|---|---|---|
my_dict |
dict | The dictionary to print |
Returns: None
-
take_items¶
take_items(data, indices) -> np.ndarray | list
The items of data at indices: the rows of an array (any shape, first axis = items), or the items of a list / object array of variable-length arrays such as spike trains.
| Parameter | Type | Description |
|---|---|---|
data |
np.ndarray | list | (n_items, ...) array, or list / object array of arrays |
indices |
list[int] | Indices of the items |
Returns: np.ndarray | list — a new array data[indices] for a non-object array, else a list of the items
-
n_distinct_values_sampled¶
n_distinct_values_sampled(values, max_samples=200000) -> int
The number of distinct values in values, counted on every size // max_samples-th element (all of them when the array has fewer than 2 × max_samples elements), without a full copy or sort of large arrays. Typical use: telling binary / discrete data (rasters) from continuous data (traces).
| Parameter | Type | Default | Description |
|---|---|---|---|
values |
np.ndarray | Array of any shape | |
max_samples |
int | 200000 | Sets the stride; up to about 2 × max_samples elements are read |
Returns: int — number of distinct values in the sample
-
decimate_min_max¶
decimate_min_max(values, times, factor) -> tuple
Downsample the rows of values for display, keeping the minimum and maximum of each bin so that brief events (transients, spikes) stay visible: the number of points is divided by factor (bins of 2 × factor samples, the last one may be shorter, each drawn as 2 points, min then max, at the bin's first and last time). NaNs are ignored. For drawing only: analyses use the full-resolution data.
| Parameter | Type | Description |
|---|---|---|
values |
np.ndarray | (n_rows, n_samples) |
times |
np.ndarray | (n_samples,), time of each sample |
factor |
int | Division of the number of points (1 = unchanged) |
Returns: tuple[np.ndarray, np.ndarray] — (decimated_values, decimated_times), shapes (n_rows, 2 * n_bins) and (2 * n_bins,) with n_bins = ceil(n_samples / (2 * factor)); the inputs unchanged if factor <= 1 or n_samples <= 2 * factor
core.path_utils (Foundational filesystem path utilities)¶
-
get_subfiles¶
get_subfiles(current_path: str, relative_path: bool = False, depth: int = 1) -> list[str]
Return file names found by walking current_path.
| Parameter | Type | Default | Description |
|---|---|---|---|
current_path |
str | The directory to search | |
relative_path |
bool | False | Return relative paths |
depth |
int | 1 | Search depth |
Returns: list[str] — List of file names
-
get_subdirs¶
get_subdirs(current_path: str, depth: int = 1) -> list[str]
Return subdirectory names found by walking current_path.
| Parameter | Type | Default | Description |
|---|---|---|---|
current_path |
str | The directory to search | |
depth |
int | 1 | Search depth |
Returns: list[str] — List of subdirectory names
-
path_leaf¶
path_leaf(path: str) -> str
Return the final component of a path (file name or last directory name).
| Parameter | Type | Description |
|---|---|---|
path |
str | The input path |
Returns: str — Final path component
-
fit_file_name_to_path_limit¶
fit_file_name_to_path_limit(directory, base_name, suffix, max_path=259) -> str
base_name shortened so that directory/base_name + suffix fits the Windows path length limit (no change on other OS, or when it already fits): the end of base_name is cut, at least 8 characters kept and trailing spaces, dots and _ stripped, with a printed message. Deep results folders combined with long session identifiers otherwise make savefig fail on Windows.
| Parameter | Type | Default | Description |
|---|---|---|---|
directory |
str | Folder the file is written to | |
base_name |
str | File name without suffix |
|
suffix |
str | Fixed end of the file name, kept intact (timestamp, extension) | |
max_path |
int | 259 | Maximum full path length |
Returns: str — base_name, possibly shortened (without suffix)
-
safe_file_name¶
safe_file_name(name, max_length=150) -> str
name usable as a file name on any OS: forbidden characters (< > : " / \ | ? * and control characters) become _, runs of _ are collapsed, leading and trailing spaces, dots and _ are removed, and the result is cut to max_length. Session identifiers can be paths or contain :.
| Parameter | Type | Default | Description |
|---|---|---|---|
name |
str | Proposed file name (without directory); converted with str() |
|
max_length |
int | 150 | Maximum length of the result |
Returns: str — the sanitized name, 'unnamed' if nothing is left
core.table_utils (Saving the data of the results and figures; trial / epoch table helpers)¶
-
save_tables¶
save_tables(tables, folder)
Save each table as <folder>/<name>.csv, without the index (names made safe for file names, shortened to the Windows path limit).
| Parameter | Type | Default | Description |
|---|---|---|---|
tables |
dict[str, DataFrame] | Tables by file name, without extension; None is skipped | |
folder |
str | Folder to write to (created if missing) |
Returns: list[str] — paths written
-
save_samples¶
save_samples(name, folder, rows=None, **arrays)
Save series drawn sample by sample: each array as <name>_<key>.npy, and the description of their rows (order, ids, groups) as <name>_rows.csv.
| Parameter | Type | Default | Description |
|---|---|---|---|
name |
str | Common start of the file names (e.g. '<session>_raster') |
|
folder |
str | Folder to write to (created if missing) | |
rows |
DataFrame | None | One row per row of the arrays, in their order |
**arrays |
Arrays by key (e.g. data=, times=); None is skipped |
Returns: list[str] — paths written
-
flat_columns¶
flat_columns(table)
A grouped table as one row of plain columns per group: the index levels become columns and multi-level column names are joined with _ (e.g. ('tau', 'mean') becomes tau_mean), so that the .csv has one header row.
| Parameter | Type | Default | Description |
|---|---|---|---|
table |
DataFrame | e.g. the result of groupby(...).agg([...]) |
Returns: DataFrame — a new table
-
ragged_to_arrays¶
ragged_to_arrays(trains)
Rows of different lengths (e.g. the spike times of each unit) as two flat arrays, for save_samples.
| Parameter | Type | Default | Description |
|---|---|---|---|
trains |
list[array-like] | One array per row |
Returns: tuple[np.ndarray, np.ndarray] — the row index of each value, and the values
-
intervals_table¶
intervals_table(intervals_by_group, names)
Intervals drawn per group (e.g. the event, epoch and stimulation bands over a display) as one table.
| Parameter | Type | Default | Description |
|---|---|---|---|
intervals_by_group |
list[list[tuple]] | Per group, its (start, stop) intervals |
|
names |
list[str] | Name of each group |
Returns: DataFrame — columns group, start, stop; one row per interval
-
filter_table_with_dict¶
filter_table_with_dict(table, filtering_dict) -> pd.DataFrame
Filter a DataFrame by multiple column conditions specified as a dict ({column: [allowed values]}); for a column holding several values per row (e.g. tags), any of them. The dict is not modified.
| Parameter | Type | Description |
|---|---|---|
table |
pd.DataFrame | Table to filter |
filtering_dict |
dict | Filter conditions |
Returns: pd.DataFrame — Filtered DataFrame
-
table_time_columns¶
table_time_columns(table) -> list
Columns of a trial / epoch table holding times (name containing time, one or several numbers per row), start_time first: the times a PSTH can be aligned on.
| Parameter | Type | Description |
|---|---|---|
table |
pd.DataFrame | Trial/epoch table |
Returns: list — List of time column names
-
table_column_times¶
table_column_times(table, column) -> np.ndarray
One time per row of a time column (see table_time_columns): the first value for a row holding several (e.g. Allen lick_times), NaN for a row without a time.
| Parameter | Type | Description |
|---|---|---|
table |
pd.DataFrame | Trial/epoch table |
column |
str | Name of a time column |
Returns: np.ndarray — 1D float array, one time per row
core.period_utils (Foundational frame-period and timestamp utilities)¶
-
get_continous_time_periods¶
get_continous_time_periods(binary_array) -> list[tuple[int, int]]
Convert a binary 0/1 array to a list of (start, stop) frame index pairs.
| Parameter | Type | Description |
|---|---|---|
binary_array |
1-D np.ndarray |
Array of 0s and 1s |
Returns: list[tuple[int, int]] — List of (start, stop) frame index pairs
-
from_timestamps_to_frame_epochs¶
from_timestamps_to_frame_epochs(time_stamps_array, frames_timestamps, as_list) -> list | np.ndarray
Convert a 2-D timestamp array to frame-index epoch pairs.
| Parameter | Type | Description |
|---|---|---|
time_stamps_array |
(2, n_epochs) array |
Timestamp array |
frames_timestamps |
1-D array | Frame timestamps in seconds |
as_list |
bool | If True return list[tuple[int, int]]; if False return (2, n_epochs) array |
Returns: list | np.ndarray — Frame-index epoch pairs
-
from_timestamps_array_to_list¶
from_timestamps_array_to_list(time_stamps_array: np.ndarray, frames_timestamps: np.ndarray) -> list[tuple[float, float]]
Filter a 2-D timestamp array to only epochs within the recording range.
| Parameter | Type | Description |
|---|---|---|
time_stamps_array |
np.ndarray | 2-D timestamp array |
frames_timestamps |
np.ndarray | Frame timestamps |
Returns: list[tuple[float, float]] — Filtered epoch list
-
get_complementary_frames¶
get_complementary_frames(total_frames, active_periods) -> tuple[np.ndarray, np.ndarray]
Return the frames not covered by active_periods (list of (start_frame, end_frame), both inclusive) among total_frames frames: (bool_complementary, complementary_frames), a boolean mask and the frame indices.
| Parameter | Type | Description |
|---|---|---|
total_frames |
int | Total number of frames |
active_periods |
list[tuple[int, int]] | List of (start_frame, end_frame) tuples |
Returns: tuple[np.ndarray, np.ndarray] — (boolean mask, complementary frame indices)
-
fill_gaps_in_continuous_periods¶
fill_gaps_in_continuous_periods(boolean_serie: np.ndarray, max_gap_size: int) -> np.ndarray
Fill small gaps between continuous True periods in a boolean array like [True, True, True, False, True, True] into [True, True, True, True, True, True] with max_gap_size >= 2.
| Parameter | Type | Description |
|---|---|---|
boolean_serie |
np.ndarray | Boolean array with gaps |
max_gap_size |
int | Maximum gap size to fill |
Returns: np.ndarray — Boolean array with gaps filled
-
extend_invalid_times¶
extend_invalid_times(invalid_times: np.ndarray, padding: float | tuple[float, float]) -> np.ndarray
Pad an invalid-times array: the same margin on both sides, or, with (pre, post), each start moved back by post and each end forward by pre, so that an event whose window [T - pre, T + post] touches an invalid interval falls in the padded one. Starts are not moved below 0.
| Parameter | Type | Description |
|---|---|---|
invalid_times |
np.ndarray | (2, n_invalid): row 0 starts, row 1 ends (s) |
padding |
float | tuple | Margin in seconds, or the (pre, post) window of the events |
Returns: np.ndarray — (2, n_invalid) padded copy
-
filter_epoch_timestamps¶
filter_epoch_timestamps(epochs_timestamps: np.ndarray, extended_invalid_times: np.ndarray | None, epochs_are_sorted: bool, invalid_times_are_sorted: bool, filter_consecutive: bool = False, use_main_epoch: bool = False, main_epoch_timestamps: np.ndarray | None = None, main_epoch_name: str = "", baseline_length: float = 0.0, verbose: bool = False, epoch_name: str = "") -> np.ndarray
Filter a (2, n_epochs) timestamp array using three optional criteria: invalid-time overlap, main-epoch containment, and consecutive-movement distance.
| Parameter | Type | Default | Description |
|---|---|---|---|
epochs_timestamps |
np.ndarray | (2, n_epochs) timestamp array | |
extended_invalid_times |
np.ndarray | None | Extended invalid time periods |
epochs_are_sorted |
bool | Whether epochs are sorted | |
invalid_times_are_sorted |
bool | Whether invalid times are sorted | |
filter_consecutive |
bool | False | Filter consecutive events |
use_main_epoch |
bool | False | Use main epoch filtering |
main_epoch_timestamps |
np.ndarray | None | Main epoch timestamps |
main_epoch_name |
str | "" | Main epoch name |
baseline_length |
float | 0.0 | Baseline length in seconds |
verbose |
bool | False | Verbose output |
epoch_name |
str | "" | Epoch name |
Returns: np.ndarray — Filtered epoch timestamps
-
filter_events_based_on_epochs¶
filter_events_based_on_epochs(events_ts: np.ndarray, epochs: np.ndarray) -> np.ndarray
Filter event timestamps, keeping only those that fall inside at least one epoch window.
| Parameter | Type | Description |
|---|---|---|
events_ts |
np.ndarray | Event timestamps |
epochs |
np.ndarray | Epoch timestamps |
Returns: np.ndarray — Filtered event timestamps
-
split_events_by_epoch_groups¶
split_events_by_epoch_groups(events_ts, epochs_by_group) -> dict
Split event timestamps by epoch group: an event belongs to every group with an epoch containing it (filter_events_based_on_epochs: strictly inside, an event exactly on an epoch edge is left out).
| Parameter | Type | Description |
|---|---|---|
events_ts |
np.ndarray | 1-D event timestamps |
epochs_by_group |
dict | {group: (2, n_epochs) array} (row 0 starts, row 1 ends, same unit as the events); a None value keeps every event in that group |
Returns: dict — {group: 1-D array of its events}, groups in the input order (possibly empty arrays)
-
valid_intervals¶
valid_intervals(epochs, start, end, invalid_times=None) -> numpy.ndarray
Time left to count events in: the epochs cut to [start, end] (the recording), overlaps merged, minus the invalid times. Events kept with filter_events_based_on_epochs(events, intervals) and the duration (intervals[1] - intervals[0]).sum() then cover the same time.
| Parameter | Type | Default | Description |
|---|---|---|---|
epochs |
np.ndarray | None | (2, n) starts / ends; None for the whole [start, end] |
|
start |
float | Start of the recording | |
end |
float | End of the recording (same unit) | |
invalid_times |
np.ndarray | None | (2, n) starts / ends to leave out |
Returns: numpy.ndarray — (2, n) sorted, disjoint intervals of positive length; (2, 0) if none
-
get_epoch_frames¶
get_epoch_frames(epoch_times, reference_ts, max_idx) -> np.ndarray
Frames of the epochs: for each epoch, the frames of reference_ts from the frame nearest its start to the frame nearest its stop (excluded), epoch after epoch (a frame in two epochs is listed twice); frames above max_idx are left out.
| Parameter | Type | Description |
|---|---|---|
epoch_times |
np.ndarray | (2, n_epochs) start and stop times (same unit as reference_ts) |
reference_ts |
np.ndarray | Timestamps of the frames, sorted |
max_idx |
int | Highest frame index kept |
Returns: np.ndarray — 1-D frame indices, empty if none
-
get_epoch_transition_list¶
get_epoch_transition_list(epoch_times) -> np.ndarray
Starts of the epochs that are transitions: every start, except that of a first epoch starting at 0 (nothing before it). Needs at least one epoch.
| Parameter | Type | Description |
|---|---|---|
epoch_times |
np.ndarray | (2, n_epochs) start and stop times |
Returns: np.ndarray — the transition times
core.event_alignment (Event alignment)¶
-
align_array_to_events¶
align_array_to_events(data, event_frames, window_frames, subtract_baseline=False) -> np.ndarray
Core event-alignment — extracts a fixed-length window around each event.
| Parameter | Type | Default | Description |
|---|---|---|---|
data |
np.ndarray | (n_cells, n_frames) or (n_frames, H, W) |
|
event_frames |
list[int] | List of event frame indices | |
window_frames |
tuple[int, int] | (n_pre, n_post) — output length = n_pre + n_post + 1 |
|
subtract_baseline |
bool | False | Subtract mean of full pre-event window per event |
Returns: np.ndarray — (n_cells, n_events, n_t) or (n_events, n_t, H, W). Events outside recording bounds are NaN-filled (not dropped).
-
align_array_to_timestamps¶
align_array_to_timestamps(data, event_timestamps, frame_timestamps, window_s, subtract_baseline=False, time_axis=1, drop_outside=False) -> np.ndarray
Timestamp wrapper around align_array_to_events. Computes frame window from median sampling rate. The single alignment function on event times (replaces align_array_on_timestamps_list and align_array_from_table_stim_time).
| Parameter | Type | Default | Description |
|---|---|---|---|
data |
np.ndarray | Data array to align | |
event_timestamps |
np.ndarray | Event timestamps in seconds | |
frame_timestamps |
np.ndarray | Frame timestamps in seconds | |
window_s |
tuple[float, float] | (pre_s, post_s) in seconds — all timestamps must be in seconds; the event is at the frame nearest to its time |
|
subtract_baseline |
bool | False | Subtract baseline |
time_axis |
int | 1 | Time axis: 0 for time-first data (n_frames,) or (n_frames, n_features) → (n_events, n_t) or (n_events, n_t, n_features) |
drop_outside |
bool | False | Drop events whose window leaves the recording (default: NaN-filled) |
Returns: np.ndarray — Event-aligned array. Same return shape as align_array_to_events. Window in frames: window_to_frames
-
window_to_frames¶
window_to_frames(sampling_rate, window_s) -> tuple
Convert a time window to frame counts. Returns (n_pre, n_post) frames of a (pre_s, post_s) window in seconds. The rule of every PSTH: pre rounded up (at least pre_s before the event), post rounded down (not past post_s after it). A product within 1e-9 frame of a whole number counts as that number, so float error does not add or drop a frame (e.g., 100 Hz × 0.29 s = 28.999999999999996 → 29).
| Parameter | Type | Description |
|---|---|---|
sampling_rate |
float | Sampling rate in Hz |
window_s |
tuple[float, float] | Time window as (pre_s, post_s) in seconds |
Returns: tuple[int, int] — (n_pre, n_post) frame counts
-
psth_grid_edges¶
psth_grid_edges(sampling_rate, window_s) -> np.ndarray
Edges of the common time grid of the population PSTHs (imaging, behavior): one bin per sample of a series at sampling_rate (the lowest rate of the sessions), each centred on its sample k / sampling_rate (the samples of window_to_frames). Each sample is half a bin from the edges, so rounding cannot put it in the next bin; series at higher rates are averaged within each bin.
| Parameter | Type | Description |
|---|---|---|
sampling_rate |
float | Samples per second of the grid (the lowest rate) |
window_s |
tuple[float, float] | Time window as (pre_s, post_s) in seconds |
Returns: np.ndarray — n_pre + n_post + 2 edges in seconds, from (-n_pre - 0.5) / rate to (n_post + 0.5) / rate
-
epoch_reference¶
epoch_reference(starts, stops, ref_in_epoch="start") -> np.ndarray
The time or frame each epoch is aligned on: its start, end or middle. Raises ValueError for another ref_in_epoch.
| Parameter | Type | Default | Description |
|---|---|---|---|
starts |
array-like | Start of each epoch (times or frames) | |
stops |
array-like | End of each epoch, same length | |
ref_in_epoch |
str | None | "start" | "start" (also for None), "end" or "middle" |
Returns: np.ndarray — One reference per epoch; the middle is (start + stop) / 2, a float also for frames
-
event_aligned_array_to_long_format_df¶
event_aligned_array_to_long_format_df(array, time_range, time_step) -> pd.DataFrame
Convert an event-aligned array to a long-format DataFrame.
| Parameter | Type | Description |
|---|---|---|
array |
np.ndarray | (n_events, n_times) or (n_events, n_times, n_samples) |
time_range |
tuple[float, float] | (t_start, t_end) in seconds |
time_step |
float | Nominal time step between frames in seconds |
Returns: pd.DataFrame — Long-format DataFrame with columns: data, event, time, sample
core.parameter_utils (Reading plugin parameter values as given by the GUI and the YAML configs)¶
-
is_selected¶
is_selected(value) -> bool
Whether a data-source parameter (e.g. ROI response series keys ['ophys', 'Fluorescence', 'dff'], or a spike-times column name) holds a selection. "Nothing selected" can appear as '', None, [], {}, 'none' or ['none', 'none', 'none'] depending on the widget or on a hand-written YAML.
| Parameter | Type | Description |
|---|---|---|
value |
any | The parameter value |
Returns: bool — False for None, a blank string, 'none', a list or tuple starting with 'none', and empty containers; True otherwise
-
normalize_groups¶
normalize_groups(groups) -> list[tuple]
The value of a group selector parameter as a list: the widget gives {name: [items, color]} (color None when the parameter has no colour), a hand-written YAML may give {name: items}.
| Parameter | Type | Description |
|---|---|---|
groups |
dict | None | The parameter value |
Returns: list[tuple] — (name, items as a list, color or None) per group, in the parameter's order
-
select_cells¶
select_cells(items, cell_indices_by_cell_type, n_cells, ids=None, keep=None) -> np.ndarray
Cells of a cell group: each item is a cell type (key of cell_indices_by_cell_type, which takes precedence) or cell indices typed as '1-4 6 15-17', kept in the typed order; indices outside 0 .. n_cells - 1 are left out, then the cells False in keep. A text that is neither gives no cell.
| Parameter | Type | Default | Description |
|---|---|---|---|
items |
list | None | Cell types and / or index strings; None for every cell | |
cell_indices_by_cell_type |
dict | {cell type: indices} (calcium_imaging.get_cell_indices_by_cell_type) |
|
n_cells |
int | Number of cells of the series | |
ids |
list | None | Identifier of each cell (e.g. unit ids): typed numbers are these identifiers (ids not found left out) instead of positions |
keep |
np.ndarray | None | Boolean mask of the cells that can be selected, by position (e.g. CicadaAnalysis.good_cells) |
Returns: np.ndarray — int cell positions, in the order of the items (an index in two items is listed twice)
core.system_utils (Information about the computer running the analysis)¶
-
total_ram_gb¶
total_ram_gb() -> float
The total physical memory of this computer, in GB. Total rather than available memory: the available figure reported by some OS (notably Windows) is unreliable for deciding whether a computation will fit.
Returns: float — Total RAM in GB