How to prepare data for CellExplorer - Neuropixels 2.0 recorded from SpikeGLX

[Virginia9733]

Hi Peters,
Thank you very much for developing such a wonderful tool.

I tried to prepare the raw data for CellExplorer, following previous discussion as posted here: #114

I have converted the raw .bin file to .dat file, and concatenated multiple sessions of the same day recording into one .dat file.
What puzzles me is that I am not sure how to convert the .meta file generated by SpikeGLX into the .xml file.
And the channel map was a bit unclear to me as well, since I have a relatively complicated channelmap, I wonder how should I sort them by groups? Groups means different shanks?
I would really appreciate your advice. Many thanks.

Also, the TTL information was stored in the t0.nidq.bin, and t0.nidq.meta file, I wonder how to convert them to be compatible with CellExplorer.

[petersenpeter]

Hi Muhang Li,

Although we currently do not have scripts to import metadata from SpikeGLX, we do have code to import metadata from OpenEphys, which could likely be adapted. The binary data format is the same, so it’s just a matter of reading their metadata.

Below is the script we use for Neuropixels 1 and 2 data acquired with Open Ephys. It imports the relevant metadata from the json file called structure.oebin.

loadOpenEphysSettingsFile: reads and imports the structure.oebin (metadata file) from Open Ephys. This file contains the layout, which allows the projection of the active channels onto the probe layout, sample rates, and other relevant metadata.

preprocessOpenEphysData: detects the recording structure from Open Ephys. This is less relevant but will assist with the necessary preprocessing steps. However, it is built for Open Ephys.

If you prefer to build the session struct (metadata file for CellExplorer) yourself, you can find the definition here: https://cellexplorer.org/datastructure/data-structure-and-format/#session-metadata.

If you share one of the metadata files, I could also take a quick look and try to make sense of it.

[Virginia9733]

Thank you so much for your prompt reply!

I tried to upload one of the meta file, but failed, as the file type not supported.
Would you mind open them via the shared folder?
https://drive.google.com/drive/folders/1-1hA6x7-nPIkdCKHPGhZi9MuprcNbx4U?usp=sharing

I have two probes for each session, three recording sessions in the same day.
In the folder I uploaded the following files:
tr02_HC_g0_t0.imec0.ap.meta - meta file for probe 1
tr02_HC_g0_t0.imec1.ap.meta - meta file for probe 1
tr02_HC_g0_t0.nidq.meta - meta file for TTL

Thank you very much.

A second question, should I concatenate raw .bin files for each probe across all the trials first, then change the extension from .bin to .dat, or should I convert the extension first, then concatenate all the .dat.
After concatenation, how can I deal with the meta file. Or how can I generate the new meta file for the concatenated .bin or .dat?

Sorry for asking naive questions. Thank you very much!

[Virginia9733]

I tried to build a xml file that can load SpikrGLX output. but when I load the session, I got the error:

cell_metrics = ProcessCellMetrics('session', session);
[05-03-2026 23:51:48: tr05_OF_probe0] Getting spikes
spikes.mat structure not up to date. Reloading spikes.
loadSpikes: Loading Phy data
Importing 45/345 clusters from phy
Saved variable 'noiseLevel' to E:\Neuropixels_Data\Chronic_Recording\Neuropixels2.0\Original_sbj_241105\Powerpixels_Testing\tr05_OF\CellExplorer_Convert\tr05_OF_probe0\tr05_OF_probe0.noiseLevel.channelInfo.mat
Getting waveforms from dat file (nPull=600). Bandpass filter applied: 500 - 8000 Hz. Bad channels detected: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
Unable to perform assignment because the left and right sides have a different number of elements.

Error in [getWaveformsFromDat](matlab:matlab.lang.internal.introspective.errorDocCallback('getWaveformsFromDat', 'E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\calc_CellMetrics\getWaveformsFromDat.m', 164)) ([line 164](matlab: opentoline('E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\calc_CellMetrics\getWaveformsFromDat.m',164,0)))
spikes.maxWaveformCh1(ii) = goodChannels(maxWaveformCh1);
^^^^^^^^^^^^^^^^^^^^^^^^^
Error in [loadSpikes](matlab:matlab.lang.internal.introspective.errorDocCallback('loadSpikes', 'E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\calc_CellMetrics\loadSpikes.m', 925)) ([line 925](matlab: opentoline('E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\calc_CellMetrics\loadSpikes.m',925,0)))
spikes = getWaveformsFromDat(spikes,session,'showWaveforms',parameters.showWaveforms,'saveMat', parameters.saveMat);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in [ProcessCellMetrics](matlab:matlab.lang.internal.introspective.errorDocCallback('ProcessCellMetrics', 'E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\ProcessCellMetrics.m', 278)) ([line 278](matlab: opentoline('E:\Neuropixels_Data_Analysis\CellExplorer\CellExplorer-main\ProcessCellMetrics.m',278,0)))
spikes{1} = loadSpikes('session',session,'labelsToRead',preferences.loadSpikes.labelsToRead,'getWaveformsFromDat',parameters.getWaveformsFromDat,'showWaveforms',parameters.showWaveforms);

tr05_OF_probe0.xml