Architecture

This document describes the technical architecture of fyt — a configuration-driven ML pipeline framework for tabular data (classification and regression, selected via the top-level task: config field).

Table of Contents

• System Overview
• Layered Architecture
• Component Responsibilities
• Data Flow
• Configuration System
• Design Patterns
• Extending fyt
• Cross-Cutting Concerns
• External Integrations
• Testing Architecture

---

System Overview

fyt transforms a YAML configuration file into a complete ML experiment. Users describe what to do (which model, which imputation strategy, which features to select) and the framework assembles, executes, and tracks the pipeline automatically.

┌─────────────────────────────────────────────────────┐
│                  User                                │
│           YAML configuration file                    │
└───────────────────────┬─────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────┐
│               CLI Entry Point                        │
│     python -m fyt run_pipeline --config <file>       │
└───────────────────────┬─────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────┐
│            Training Pipeline (Orchestrator)          │
│  ┌──────────┐  ┌──────────────┐  ┌───────────────┐  │
│  │   Data   │→ │ Preprocessing│→ │   Feature     │  │
│  │ Manager  │  │  + Encoding  │  │   Selection   │  │
│  └──────────┘  └──────────────┘  └───────┬───────┘  │
│                                          │           │
│                                          ▼           │
│                                   ┌───────────────┐  │
│                                   │    Trainer    │  │
│                                   │ (train + eval)│  │
│                                   └───────┬───────┘  │
└───────────────────────────────────────────┼──────────┘
                                            │
                                            ▼
                                  ┌──────────────────┐
                                  │  MLflow Tracker  │
                                  │ metrics/artifacts│
                                  └──────────────────┘

---

Layered Architecture

The codebase is organized into five horizontal layers, each with a distinct responsibility. Dependencies only flow downward.

┌─────────────────────────────────────────────────┐
│  1. Interface Layer                              │
│     fyt/__main__.py  fyt/commands/               │
│     Parses CLI args, loads config, invokes cmd   │
└─────────────────────────┬───────────────────────┘
                          │
┌─────────────────────────▼───────────────────────┐
│  2. Configuration Layer                          │
│     fyt/configs/models.py  fyt/configs/enums.py  │
│     Pydantic schemas — validates YAML on load    │
└─────────────────────────┬───────────────────────┘
                          │
┌─────────────────────────▼───────────────────────┐
│  3. Core Pipeline Layer                          │
│     fyt/core/                                    │
│     Orchestration + all pipeline stages          │
└─────────────────────────┬───────────────────────┘
                          │
┌─────────────────────────▼───────────────────────┐
│  4. Registry Layer                               │
│     fyt/registries/                              │
│     Constructs strategy objects from config      │
│     Open for extension via @register decorator   │
└─────────────────────────┬───────────────────────┘
                          │
┌─────────────────────────▼───────────────────────┐
│  5. Implementation Layer                         │
│     fyt/wrappers/  fyt/imputations/              │
│     Concrete algorithms and third-party adapters │
└─────────────────────────────────────────────────┘

Supporting every layer are two horizontal concerns:

• fyt/utils/ — logging, config loading, experiment tracking, seed management, plotting
• fyt/metrics.py — metric registry shared by the optimizer and evaluator (including the sklearn scorer mapping that keeps the tuned objective identical to the reported metric)

---

Component Responsibilities

Interface Layer

File	Responsibility
fyt/__main__.py	Discovers and loads commands from fyt/commands/ via pkgutil.iter_modules. Enables adding new commands without touching the main entry point.
fyt/commands/run_pipeline.py	Wires all components together via DI, iterates over seeds, and delegates to TrainingPipeline.

The @command decorator (in utils/commands.py) handles config deserialization: it inspects the function signature, infers the Pydantic config type, loads the YAML file, and passes the typed config object to the function.

Core Pipeline Layer

Component	File	Responsibility
TrainingPipeline	core/training_pipeline.py	Sequences each stage, propagates random_state, returns MetricResults.
DataManager	core/data_manager.py	Loads CSV, auto-detects categorical vs. numerical columns, produces train/test splits or K-Fold iterators.
TargetProcessor	core/processing/target_processor.py	Fits a label/ordinal encoder on training labels; transforms both splits consistently.
PreProcessor	core/processing/pre_processor.py	Builds a ColumnTransformer+Pipeline for imputation, scaling, encoding, batch correction, and custom transformations. Fit only on train data to prevent leakage.
FeatureSelector	core/feature_selection/feature_selector.py	Applies a selection strategy (Boruta, RFE, SelectKBest, SelectFromModel, VarianceThreshold) then runs correlation filtering.
Trainer	core/trainer.py	Fits the model and makes predictions. Nothing else.
HyperparameterOptimizer	core/hyperparameter_optimizer.py	Owns the Optuna study. In leakage-safe mode (default) each trial evaluates a full sklearn Pipeline of freshly cloned preprocessing + selection + model, so CV folds see no leaked statistics.
MetricsEvaluator	core/metrics_evaluator.py	Computes requested metrics on the holdout set with a unified averaging policy.
ExperimentReporter	core/experiment_reporter.py	Logs params, metrics, plots, and artifacts through the experiment logger.
InferencePipeline	core/inference_pipeline.py	Persistable bundle of the fitted preprocessor, selector, target encoder, and model; supports save/load/predict/predict_proba on raw data.
AggregatedMetricsManager	core/aggregated_metrics_manager.py	Collects MetricResults across seeds and computes mean, median, percentiles, and IQR.

Registry Layer

All registries extend ComponentRegistry[T] (in fyt/registries/base.py) — a generic, decorator-based registry. Each subclass owns its own _registry dict. New strategies are registered with @MyRegistry.register("key") at module load time and are immediately available via MyRegistry.create("key", **kwargs).

Registry	Produces	Built-in keys
ModelRegistry	sklearn / XGBoost estimator, VotingClassifier, StackingClassifier	logistic_regression, random_forest, xgboost, svm, knn, gradient_boosting, decision_tree, voting, stacking
NumericalImputationRegistry	SimpleImputer, KNNImputer, MissForestWrapped, QRILCImputer	mean, median, most_frequent, constant, knn, miss_forest, qrilc
CategoricalImputationRegistry	SimpleImputer	most_frequent, constant
NumericalTransformerRegistry	Scalers, LogTransformer, CombatWrapped, MeanBinarizer	standard, min_max, robust, log, combat, mean_binarizer
CategoricalTransformerRegistry	OrdinalEncoder, OneHotEncoder	ordinal, onehot
ScalingRegistry	StandardScaler, MinMaxScaler, RobustScaler	standard, min_max, robust
FeatureSelectionRegistry	Boruta wrapper, RFE, SelectKBest, SelectFromModel, VarianceThreshold	select_k_best, select_percentile, rfe, select_from_model, variance_threshold, boruta
TargetEncoderRegistry	LabelEncoder, OrdinalEncoder	label_encoder, ordinal_encoder

MetricsRegistry (in fyt/metrics.py) follows the same pattern for evaluation metrics and cross-validation scorers.

Implementation Layer

Module	Purpose
wrappers/boruta_wrapper.py	sklearn-compatible adapter around the boruta library
wrappers/missforest_wrapper.py	sklearn-compatible adapter around missforest
wrappers/combat_wrapper.py	sklearn-compatible adapter around ComBat batch correction
imputations/qrilc.py	Custom QRILC imputer for left-censored proteomics/metabolomics data

All wrappers implement fit, transform, and fit_transform so they slot into sklearn pipelines without modification.

---

Data Flow

A single-seed run follows these steps in order:

1. Load & Split
   CSV / Parquet file
     → DataManager (validates target column, infers column types)
     → DataManager.split_data()     [stratified for classification, plain for regression]
     → DataSplit(X_train, X_test, y_train, y_test)

2. Target Encoding
   y_train, y_test
     → TargetProcessor.fit_transform(y_train), .transform(y_test)
     → classification: label/ordinal encoding; regression: passthrough + numeric check

3. [Optional] Hyperparameter Optimization — on RAW features
   X_train_raw, y_train_enc
     → HyperparameterOptimizer.optimize()
     → leakage-safe (default): each trial CV-evaluates
       Pipeline([clone(PreProcessor), clone(FeatureSelector), model])
       so imputation/scaling/selection are re-fit inside every fold
     → best_params

4. Feature Preprocessing
   X_train, X_test
     → PreProcessor.fit(X_train)    [zero-imputation → impute → scale/encode → transform]
     → PreProcessor.transform(X_train / X_test)   (fit only on train)

5. Feature Selection
   X_train_proc, y_train_enc
     → FeatureSelector.fit / transform

6. Training & Evaluation
     → Trainer.train (best_params merged, seed injected when supported)
     → MetricsEvaluator.compute → MetricResults

7. Reporting & Persistence
     → ExperimentReporter (params, metrics, plots, artifacts) when a logger is configured
     → model_output_dir: TrainingPipeline.to_inference_pipeline().save(...)

For multi-seed runs, the steps repeat per seed with per-seed fault isolation (one failing seed logs and continues; the run fails only if every seed fails). After all seeds complete, AggregatedMetricsManager logs statistical summaries.

---

Configuration System

YAML → Pydantic → Component

configs/commands/<model>/run_pipeline_command.yaml
          │
          │   loaded by @command decorator
          ▼
RunPipelineCommandConfig (Pydantic)
   ├── task: TaskType (classification | regression)
   ├── seeds: list[int]
   ├── model_output_dir: Path | None
   ├── experiment_logger: BaseExperimentLoggerConfig | None   (None ⇒ no tracking)
   ├── data_manager: DataManagerConfig
   ├── target_processor: TargetProcessorConfig
   ├── pre_processor: PreProcessorConfig
   │     ├── numerical_imputations: list[NumericalImputationComponentConfig]
   │     ├── numerical_transformations: list[NumericalTransformationComponentConfig]
   │     ├── categorical_imputations: list[CategoricalImputationComponentConfig]
   │     └── categorical_transformations: list[CategoricalTransformationComponentConfig]
   ├── feature_selector: FeatureSelectorConfig
   │     ├── feature_selection: FeatureSelectionConfig
   │     └── correlation_filter: CorrelationFilterConfig
   └── trainer: TrainerConfig
         ├── model_type: ModelType
         ├── model_params: dict
         ├── metrics / metric_averaging
         ├── optuna_config: OptunaConfig (incl. leakage_safe_cv)
         ├── cross_validation_config: CrossValidationConfig
         └── feature_importance_config: FeatureImportanceConfig

Validation happens at load time. Unknown or misspelled keys are rejected (extra="forbid"), enum values are checked, and cross-field consistency is enforced (model vs task, metrics vs task, loss metrics vs optimization direction).

Config Inheritance

All config models extend YamlBaseSettings (a thin wrapper around pydantic-settings), which allows individual component configs to be loaded from component-specific YAML files. Command-level configs include or override these defaults.

Adding a New Configuration Option

1. Add the field to the relevant Pydantic model in fyt/configs/models.py.
2. If it introduces a new strategy, add a value to the relevant enum in fyt/configs/enums.py.
3. Register the new strategy in the corresponding registry (see Extending fyt).

---

Design Patterns

Dependency Injection

No component instantiates its own dependencies. All objects are constructed in run_pipeline.py and passed via constructors. This makes every component independently testable.

# run_pipeline.py — construction site
data_manager = DataManager(config=config.data_manager)
pre_processor = PreProcessor(config=config.pre_processor)
trainer = Trainer(config=config.trainer, logger=mlflow_logger)
pipeline = TrainingPipeline(
    data_manager=data_manager,
    pre_processor=pre_processor,
    trainer=trainer,
    ...
)

Registry Pattern

The core extensibility mechanism. ComponentRegistry[T] (in fyt/registries/base.py) is a generic base class where each subclass maintains its own _registry dict mapping string keys to factory callables.

ComponentRegistry[T]  (fyt/registries/base.py)
├── _registry: ClassVar[dict[str, Callable]]   ← per-subclass, enforced by __init_subclass__
├── register(key) → decorator                  ← adds entries at import time
├── create(key, **kwargs) → T | None           ← looks up and invokes the factory
├── get_available() → list[str]                ← introspection
└── normalize_key(key) → str                  ← enum.value or str.lower()
    │
    ├── ModelRegistry
    ├── NumericalImputationRegistry
    ├── CategoricalImputationRegistry
    ├── NumericalTransformerRegistry
    ├── CategoricalTransformerRegistry
    ├── ScalingRegistry
    ├── FeatureSelectionRegistry
    └── TargetEncoderRegistry

Key behaviours:

• "passthrough" key — create("passthrough") always returns None, signalling the pipeline to skip that step.
• Class decoration — decorating a class (not a function) automatically wraps it so create(key, **kwargs) calls TheClass(**kwargs).
• Enum key support — enum values are automatically normalized to lowercase strings, so YAML strings and enum values are interchangeable.
• Subclass isolation — __init_subclass__ guarantees every registry subclass gets its own _registry; entries never leak between registries.

Strategy Pattern

The enum + registry combo implements the strategy pattern. Components depend on the abstract interface (sklearn's TransformerMixin or BaseEstimator), not on concrete classes. Strategies are swapped by changing a single YAML value.

Pipeline Pattern (sklearn)

PreProcessor assembles a ColumnTransformer that applies different transformation chains to numerical and categorical columns in parallel, then optionally feeds the result into further steps. This ensures fit is called only on training data and transform reuses learned parameters on test data.

---

Extending fyt

The registry pattern is designed so that no existing file needs to be modified to add a new strategy.

Adding a new imputer, transformer, or model

# my_custom_strategies.py  (anywhere on the Python path)
from sklearn.base import BaseEstimator, TransformerMixin
from fyt.registries.numerical_imputation import NumericalImputationRegistry

class MyImputer(BaseEstimator, TransformerMixin):
    def fit(self, X, y=None): ...
    def transform(self, X): ...

@NumericalImputationRegistry.register("my_imputer")
def _create_my_imputer(**kwargs):
    return MyImputer(**kwargs)

After importing this module (e.g. in a startup script or plugin entry point), "my_imputer" is a valid YAML value for any numerical imputation step.

Adding a new metric

from fyt.metrics import MetricsRegistry
from sklearn.metrics import matthews_corrcoef

@MetricsRegistry.register("mcc")
def mcc(y_true, y_pred):
    return matthews_corrcoef(y_true, y_pred)

"mcc" is now available in trainer.metrics and as a cross-validation scorer.

Adding a new CLI command

Create fyt/commands/my_command.py with a main() function decorated with @command. fyt/__main__.py discovers it automatically via pkgutil.iter_modules — no other file changes required.

Introspection

Every registry exposes get_available() to list registered keys at runtime:

from fyt.registries.model import ModelRegistry
print(ModelRegistry.get_available())
# ['decision_tree', 'gradient_boosting', 'knn', 'logistic_regression',
#  'random_forest', 'stacking', 'svm', 'voting', 'xgboost']

---

Cross-Cutting Concerns

Logging

Logging uses the standard logging module with a rich console handler, configured by setup_logger() in fyt/utils/logger.py (called by the CLI entry point). Modules create their logger at import:

logger = logging.getLogger(__name__)

Log level is controlled via the LOGLEVEL environment variable (default INFO).

Seed Management

fyt/utils/seed_manager.py exposes set_global_seed(seed) which seeds random and numpy; the command layer calls it before each seed iteration. TrainingPipeline additionally propagates the seed to every SeedAware component (trainer, selector, preprocessor, optimizer), and ensemble base estimators and MissForest receive it explicitly.

Error Handling

Custom exceptions live in fyt/exceptions/. Commands catch and log pipeline errors with enough context to identify which seed or stage failed without crashing the entire multi-seed run.

Experiment Metadata

MlflowLogger automatically attaches git metadata (branch, commit hash) and the package version to every run. This makes it possible to reproduce any logged experiment exactly.

---

External Integrations

MLflow

utils/experiment_logger.py defines BaseExperimentLogger (abstract) and MlflowLogger (concrete). The trainer and command layer depend on BaseExperimentLogger, so MLflow can be swapped for another tracker without touching pipeline code.

MLflow is run locally via Docker Compose (SQLite backend, port 5001). Artifact paths and tracking URI are set in configs/mlflow_logger.yaml.

Optuna

Hyperparameter tuning lives in HyperparameterOptimizer. The orchestrator hands it the raw training split (target already encoded) plus a factory of fresh preprocessing clones; in leakage-safe mode each trial cross-validates a complete sklearn Pipeline, so fold scores are honest. The rest of the pipeline only sees the best-parameters dict that comes out of it.

scikit-learn

All preprocessing, feature selection, and most models expose the standard sklearn interface (fit, transform, predict, predict_proba). Third-party libraries (Boruta, MissForest, ComBat) are wrapped in adapters in fyt/wrappers/ to conform to this interface.

---

Testing Architecture

Tests mirror the fyt/ package structure under tests/ (tests/core/, tests/core/processing/, tests/core/feature_selection/, tests/commands/, tests/registries/, tests/wrappers/, tests/metrics/, tests/imputations/, tests/utils/), plus tests/integration/ with end-to-end runs on synthetic data (classification, regression, Optuna with leakage-safe CV, train→save→ load→predict round-trips) and tests/helpers.py with shared dataset builders.

Principles:

• Components are tested in isolation with injected fakes; the integration suite exercises the real wiring end-to-end without external services.
• Tests use small, synthetic DataFrames — no dependency on real datasets.
• Coverage is reported via --cov=fyt (no enforced threshold currently).
• Tests run in parallel with pytest-xdist (-n auto).

make test       # uv run pytest -n auto --verbose --cov=fyt tests
make validate   # format + lint + type check + tests + wheel build