Data manager

fyt.core.data_manager

DataManager

Class for managing dataset loading and preprocessing.

Parameters:

Name	Type	Description	Default
config	DataManagerConfig	Data manager configuration.	required
task	TaskType	Learning task type; classification splits are stratified on the target, regression splits are not.	CLASSIFICATION

Attributes:

Name	Type	Description
SUPPORTED_FORMATS	tuple[str, ...]	Supported dataset file extensions.

categorical_columns property

The list of categorical column names.

Note: If categorical columns are not specified in the config, they will be automatically detected based on data types.

Returns:

Type	Description
list[str]	list[str]: List of categorical column names.

numerical_columns property

The list of numerical column names.

Note: If numerical columns are not specified in the config, they will be automatically detected based on data types.

Returns:

Type	Description
list[str]	list[str]: List of numerical column names.

k_fold_split(n_splits=5, random_state=42, shuffle=True, data=None)

Generate K-Fold splits of the dataset.

Splits the data into K folds for cross-validation.

Parameters:

Name	Type	Description	Default
n_splits	int	Number of folds.	5
random_state	int	Random seed for shuffling.	42
shuffle	bool	Whether to shuffle the data before splitting into folds.	True
data	DataFrame | None	Optional DataFrame to split. If None, uses the internal dataset. The idea is that, you can first split the data into train and test using split_data, and then apply K-Fold only on the training set.	None

Yields:

Name	Type	Description
DataSplit	DataSplit	A DataSplit object for each fold.

split_data(test_size=0.2, random_state=42)

Split the dataset according to the config.

Parameters:

Name	Type	Description	Default
test_size	float	Proportion of the dataset to include in the test split.	0.2
random_state	int	Random seed for reproducibility.	42

Returns:

Name	Type	Description
DataSplit	DataSplit	A DataSplit object containing training and testing data.

use_test_data()

Check if test data path is provided in the config.

Returns:

Name	Type	Description
bool	bool	True if test data path is provided, False otherwise.