Introduction to Datasets

Learn more about utilizing Datasets within Vectice.

Datasets Overview

Datasets in Vectice reflect the datasets used for data analysis, cleaning, model training, testing, and validation during development. Datasets can be registered in the UI from various source systems, where the dataset's origin, version, and resource metadata are stored. Vectice will automatically version the metadata of datasets to enable lineage.

The Vectice API enables you to register all datasets used during development to the Vectice UI, including your origin, cleaned, and modeling datasets.

Dataset type	Description
Origin datasets	Origin datasets refer to your datasets containing raw data.
Cleaned datasets	Cleaned datasets refer to your datasets that have been cleaned and prepared for data modeling or data analysis.
Modeling datasets	Modeling datasets combine training, testing, and validating data in a single dataset.

Things you should know

• Data scientists can capture basic statistics when registering their datasets' metadata to Vectice. These statistics include the mean, median, variance, quartiles, and more. Learn more by viewing our Capturing Dataset Statistics section.

Datasets Best Practices

• Registered datasets should have the dataset types Origin, Cleaned, and Modeling appended to the end of the corresponding dataset name for easy identification in the UI.

• Create a phase with the primary objective of origin datasets registration. This is usually included in the Data Preparation phase of a CRISP-DM project.

• Project phases that aim to clean and process raw data should have a step that registers the cleaned datasets. This is typically completed in the Data Preparation phase of a CRISP-DM project.

• Mark your most valuable iterations and assets in the UI by selecting the star next to the corresponding iteration and assets before beginning the phase review. This will make it easier for stakeholders and subject matter experts to identify the iterations and assets in review.

Select the star next to the valuable iterations before completing a phase, even without review. This allows you to identify the most valuable iterations for that phase.

How to register datasets to Vectice?

Registering datasets is your first step to knowledge capture in Vectice. You can register all datasets used during development, including your origin datasets (raw datasets), cleaned datasets, and modeling datasets.

View our How to register datasets guide for more information on how to register datasets during iterative development.