Data sources¶
CICADA analyses NWB files. They can be on your disk or on the
DANDI archive. Wherever a file comes from, it is opened as
an NWBSession and the analyses see the same data.
Local files¶
| Where | How |
|---|---|
| GUI | File > Open new dataset and choose a folder: every .nwb file in it is listed. File > Load group opens the files listed in a sessions YAML. |
| Headless | List the files in a sessions YAML (identifier + path), see the headless workflow. |
| Python | NWBSession("/path/to/file.nwb") (cicada-nwb). |
A sessions YAML looks like this:
sessions:
- identifier: session_001
path: /absolute/path/to/session_001.nwb
- identifier: session_002
path: dandi://000219@0.260826.1155/sub-180201-180208/sub-180201-180208_ses-180208-a000_behavior+ophys.nwb
Files on DANDI¶
A DANDI address can be used anywhere a file path is accepted: in a sessions YAML, in the
GUI and in NWBSession(...). Nothing needs to be downloaded beforehand.
DANDI support comes with cicada-nwb (it installs remfile, requests and platformdirs).
Note
DANDI access is in the development version and comes with the next release.
In the GUI¶
File > Open new dataset from DANDI... (or Add data from DANDI...) takes a dandiset
(000728, a DANDI link or DOI), a folder of it or a single file. It lists the files with
their sizes without opening them; tick the ones to analyse and click Open.
Streaming or download¶
| Mode | What happens |
|---|---|
stream (default) |
Only the parts of the file that are read are fetched, and kept in a disk cache |
download |
The file is downloaded once (size and checksum checked) and reused at the next open. Falls back to streaming if the file is larger than the free disk space or than 10 GB |
auto |
Download when the file is 1 GB or smaller, else stream |
Choose the mode in the GUI dialog, in Python with dandi_mode="download" (NWBSession,
run_from_config, generate_config), or on the command line with cicada-run --dandi-mode download.
Downloaded files go to the folder you choose: the Download to row of the GUI dialog
(remembered for the next time), dandi_download_dir=... in Python (NWBSession,
run_from_config, generate_config), or --dandi-download-dir on the command line. Without a
choice they go to the cicada folder of your Downloads folder (e.g. C:\Users\<you>\Downloads\cicada),
which the GUI row also shows at first. Files are stored as <folder>/<dandiset>/<version>/<path>,
so files of different dandisets never collide and a file already there is reused.
Streamed pieces go to a hidden cache folder (the environment variable CICADA_DANDI_CHUNKS
changes it); the GUI clears them when it closes, cicada-dandi cache clear from the command line.
Streaming speed
Streaming is convenient for a few sessions or a first look. Analyses that read whole recordings are faster on downloaded files.
Addresses¶
| Address | Example |
|---|---|
| DANDI URI (version optional) | dandi://000728@0.240827.1809/sub-…/sub-…_ophys.nwb |
| DANDI web link | https://dandiarchive.org/dandiset/000728/0.240827.1809/files?location=sub-… |
| DANDI API asset link | https://api.dandiarchive.org/api/assets/<asset_id>/download/ |
Any .nwb URL |
https://…/file.nwb (always streamed) |
Without a version, the latest published version is used (the draft, with a warning, when
nothing is published). Once the file is opened, CICADA records the version-pinned
address (dandi://<dandiset>@<version>/<path>) in the sessions and combined YAMLs, so a
replay reads the same data.
What an NWB file must contain¶
CICADA reads standard NWB files, as checked by DANDI and
nwbinspector. It does not work around files that
do not follow the standard; for example, a file without a Subject can fail to open.
Check your files with nwbinspector before analysing them.
A detailed list of what each analysis needs (series, tables, timestamps) is coming.