Support for multi component datafields (XY-dataset)

[jgoedeke]

Hi,
I tried to parse a file with a single channel containing a XY-dataset. This means it contains a component group with two components one for the xdata (here a timestamp in seconds) and one for the ydata (here without unit).

It looks like the library does not parse multiple components properly, meaning it can only open timestamped ASCI and normal datasets. When having multiple components in one group the xdata is still filled via the xstepwidth from the CD key which is not applicable in this case.

IMCtermite/lib/imc_channel.hpp

Lines 478 to 482 in befda07

	// fill xdata_
	for ( unsigned long int i = 0; i < num_values; i++ )
	{
	xdata_.push_back(xoffset_+(double)i*xstepwidth_);
	}

Steps to reproduce:

>>> from IMCtermite import imctermite
>>> imc = imctermite('XY_dataset_example.dat'.encode())
>>> channel = imc.get_channels(True)[0]
>>> channel['xdata'][-5:]
[13089, 13090, 13091, 13092, 13093]
>>> channel['ydata'][-5:]
[3169029, 2796202, 2796202, 2796202, 2982616]

XY_dataset_example.zip

I am using version 2.0.20 with python 3.11 on linux.

I am willing to create a PR but I am wondering how we can determine which blocks belong together. Because for parsing the xdata in the same way as the ydata for XY-datasets we have to make sure we have to corresponding CC, CP, CD, NT, CR, CN and Cb keys. Does this require a change in channel parsing? Maybe by adding component group and component vectors to represent the right structure?

Best regards,
Jan

[mario-fink]

Hi Jan,

thanks for using the package and creating the issue!

You're welcome to provide a PR to support XY data. To do this you have to collect all CC keys/blocks associated to the preceding CG block. For instance, in your example XY_dataset_example.dat the |CG,1,5,2,2,2; block features 2 CC blocks (first 2) and the field type 2 (second 2) corresponding to XY data with multiple y-datasets with a common x-timedataset. Listing all blocks associated to a group is done here and ff.

IMCtermite/lib/imc_raw.hpp

Lines 206 to 220 in befda07

	void generate_channel_env()
	{
	channels_.clear();
	// declare single channel environment
	imc::channel_env chnenv;
	chnenv.reset();
	// collect affiliate blocks for every channel WITH CHANNEL and AFFILIATE
	// BLOCK CORRESPONDENCE GOVERNED BY BLOCK ORDER IN BUFFER!!
	for ( imc::block blk: rawblocks_ )
	{
	if ( blk.get_key().name_ == "CN" ) chnenv.CNuuid_ = blk.get_uuid();
	else if ( blk.get_key().name_ == "CD" ) chnenv.CDuuid_ = blk.get_uuid();
	else if ( blk.get_key().name_ == "CT" ) chnenv.CTuuid_ = blk.get_uuid();

Best regards,
Mario