Overview

We are now in the final stage of the mlr3hf project, and almost everything planned for this week has been completed.

The complete implementation is available in the mlr3hf repository.

During the final phase, I focused mainly on improving the user experience, documentation, and integration with the mlr3 ecosystem.

Some of the important additions and improvements include:

tutorial.Rmd and quick-revision.Rmd in the vignettes/ directory
sugar.R containing user-friendly helper functions
zzz.R containing package imports and related package-level definitions
Active bindings in HFData.R
Direct conversion of Hugging Face datasets into mlr3 tasks
Support for predefined train, test, and validation splits

The goal throughout the project has been to make working with Hugging Face datasets as simple as possible for an R user.

Documentation and Package Structure

The package now contains two important vignettes:

tutorial.Rmd
quick-revision.Rmd

The tutorial provides a detailed introduction to using mlr3hf, while the quick revision vignette gives users a shorter overview of the package and its main functionality.

I also added sugar.R and zzz.R.

The sugar.R file contains user-friendly helper functions such as hfdata(), while zzz.R is used for package-level definitions such as @importFrom statements.

This keeps the package structure clean and makes the user-facing functions easier to discover.

User-Friendly Access with `HFData`

One of the major parts of the package is the HFData class.

The class provides several active bindings that allow users to access useful information about a Hugging Face dataset without having to manually interact with the Hugging Face API.

The available active bindings are:

Binding	Description
`nrow`	Number of rows in the dataset
`ncol`	Number of columns in the dataset
`repo_link`	URL of the dataset repository on Hugging Face
`desc`	Repository description, including siblings, configs, and tags
`storage`	Storage information for the repository
`configs`	All configurations available in the repository
`siblings`	All filenames contained in the repository
`data`	Loaded dataset
`colnames`	Names of all columns in the dataset
`splits`	Available dataset splits as row ID lists
`coltypes`	Data type of each column
`feature_names`	Column names excluding the target column

For example, we can create an HFData object using:

hf_data <- hfdata(
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The active bindings can then be accessed directly:

hf_data$repo_link
hf_data$configs
hf_data$siblings

For example, repo_link directly returns the URL of the dataset repository on Hugging Face. This URL can then be opened in a browser to inspect the dataset repository.

Similarly, configs returns all configurations available for the dataset, while siblings returns the filenames available in the repository.

This makes it possible to explore a Hugging Face dataset directly from R without manually navigating through the Hugging Face API.

Creating an `mlr3` Task Directly

One of the main goals of mlr3hf is to make Hugging Face datasets immediately usable within the mlr3 ecosystem.

With the package, a user can create an mlr3 task directly:

hf_data <- task(
  "hf",
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The resulting object is an mlr3 task and can be directly used with the rest of the mlr3 ecosystem.

For example, it can be passed to learners, resampling strategies, prediction workflows, and performance measures.

This removes several manual steps that would otherwise be required to download a dataset, load it into R, identify the target column, and construct an mlr3 task.

`hfdata()`

We also provide a user-friendly helper function, hfdata().

hf_data <- hfdata(
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The hfdata() function provides direct access to the HFData object and its active bindings.

For example:

hf_data$repo_link
hf_data$configs
hf_data$siblings
hf_data$data
hf_data$colnames
hf_data$coltypes
hf_data$feature_names

Some of these bindings can even be used when only the repo_id is provided.

For example, information such as the repository URL, configurations, and repository files does not require the user to specify a target column.

This makes HFData useful not only for model training, but also for exploring and understanding a Hugging Face dataset before using it.

Working with Predefined Dataset Splits

Another useful feature of mlr3hf is support for predefined dataset splits.

Many Hugging Face datasets already provide predefined train, test, and sometimes validation splits.

Instead of creating a new resampling split, mlr3hf exposes these predefined splits through the $splits active binding.

The splits are represented as vectors of row IDs, which means they can be directly reused in mlr3 workflows.

For example:

library(mlr3learners)

hf_data <- hfdata(
  repo_id = "a4n9i/scikit_iris",
  config = "default",
  target = "Species"
)

task <- as_task(hf_data)

train_row_id <- hf_data$splits$train
test_row_id  <- hf_data$splits$test

We can then pass these row IDs directly to ResamplingCustom.

resampling <- rsmp("custom")

resampling$instantiate(
  task,
  train_sets = list(train_row_id),
  test_sets = list(test_row_id)
)

resampling

This produces:

## ──  : Custom Splits ──────────────────────────────────────────
## • Iterations: 1
## • Instantiated: TRUE
## • Parameters: list()

The resulting resampling object can be used like any other mlr3 resampling strategy:

rr <- resample(
  task,
  lrn("classif.rpart"),
  resampling
)

rr$aggregate()

For example:

## classif.ce
## 0.6333333

The important point here is that we are reusing the original train/test split provided by the dataset, rather than generating a new random split.

This is especially useful when a dataset creator has already defined meaningful training and evaluation splits.

Using a Predefined Validation Split

Some Hugging Face datasets also provide a predefined validation split.

This can be particularly useful for learners that support internal validation.

For example, classif.xgboost can use an internal validation task for early stopping and monitoring validation performance.

First, we extract the predefined splits:

train_ids <- hf_data$splits$train
valid_ids <- hf_data$splits$validation
test_ids  <- hf_data$splits$test

We then create a training task containing only the training rows:

tsk_train <- task$clone(deep = TRUE)
tsk_train$filter(train_ids)

Similarly, we create a separate validation task:

tsk_valid <- task$clone(deep = TRUE)
tsk_valid$filter(valid_ids)

The validation task can then be attached to the training task:

tsk_train$internal_valid_task <- tsk_valid

Now we create an XGBoost learner:

lrn <- lrn("classif.xgboost")

We tell the learner to use the predefined validation task:

lrn$validate <- "predefined"

We can then configure early stopping and the evaluation metric:

lrn$param_set$set_values(
  nrounds = 500,
  early_stopping_rounds = 10,
  eval_metric = "mlogloss"
)

Finally, we train the learner:

lrn$train(tsk_train)

The validation score can be inspected using:

lrn$internal_valid_scores

For example:

## $mlogloss
## [1] 13.82791

After training, we can evaluate the model on the predefined test split:

tsk_test <- task$clone(deep = TRUE)
tsk_test$filter(test_ids)

pred <- lrn$predict(tsk_test)

pred$score(msr("classif.ce"))

For example:

## classif.ce
## 0.6333333

A learner such as classif.rpart does not use internal_valid_task, whereas learners such as classif.xgboost can use it for internal validation and early stopping.

Keeping the User Perspective in Mind

Throughout the development of mlr3hf, one of the most important things we tried to keep in mind was the user perspective.

The goal was not simply to build another interface for downloading Hugging Face datasets.

Instead, we wanted to answer a more practical question:

How can an R user take a dataset from Hugging Face and start using it in the mlr3 ecosystem with as little effort as possible?

This led to features such as:

Direct creation of mlr3 tasks
hfdata() for convenient dataset access
Active bindings for dataset exploration
Support for predefined train/test/validation splits
Reuse of existing dataset splits instead of unnecessarily creating new ones

The intention is to make the workflow feel natural for an R user.

Acknowledgements

I would like to sincerely thank my mentors, Toby Dylan Hocking and Sebastian Fischer, for their guidance and support throughout this project.

Their feedback and discussions helped me understand the project from both a technical and a user-oriented perspective.

Whenever we discussed a feature or design decision, we tried to keep one question in mind:

How will this help the user?

That mindset had a significant influence on the final design of mlr3hf.

I am very grateful for the opportunity to work on this project and learn from both of my mentors.

We have now reached the final stage of the project, and I hope that mlr3hf will be useful to R users and researchers who want to work with datasets available on Hugging Face.

It has been a challenging but rewarding journey, and I am happy with what we have built together.

We built it with love for the R and mlr3 community. ❤️

Overview

Documentation and Package Structure

User-Friendly Access with HFData

Creating an mlr3 Task Directly

hfdata()

Working with Predefined Dataset Splits

Using a Predefined Validation Split

Keeping the User Perspective in Mind

Acknowledgements

User-Friendly Access with `HFData`

Creating an `mlr3` Task Directly

`hfdata()`