What Is the Project?

Before introducing my project, I would like to briefly explain what Hugging Face is and why datasets from Hugging Face are useful.

Hugging Face is a platform that provides a large collection of datasets, models, and other resources that are widely used in machine learning and deep learning. In my project, I mainly focused on the Hugging Face Hub for datasets. You can visit the Hugging Face website to explore these resources.

So, why do we need these datasets?

Researchers, students, and developers often need datasets to experiment with, develop, and evaluate machine learning models. However, simply downloading a dataset is not always enough. In the R ecosystem, users of mlr3 work with datasets through a specific abstraction called a Task.

An mlr3 Task contains the dataset together with information about its features, target variable, and other relevant metadata. This allows the dataset to be directly used with the different components of the mlr3 ecosystem.

This is where mlr3hf comes in.

The goal of my project was to develop an R package that allows users to access datasets from the Hugging Face Hub and convert them into objects that can be used within the mlr3 ecosystem.

My Contributions

Since the project was developed largely from scratch, I worked on several different parts of the package, including dataset downloading, caching, metadata handling, Parquet access, testing, and conversion into mlr3 objects.

During GSoC, I opened more than 20 pull requests and issues, covering different aspects of the implementation, testing, documentation, CI, and project infrastructure.

A complete list of my pull requests and issues is provided below.

Pull Requests and Issues

I opened more than 20 pull requests and issues throughout the GSoC period.

Every pull request that I submitted was eventually merged, and there are currently no pending changes from my GSoC work upstream.

Challenges I Faced

The biggest challenge for me was not writing individual functions. It was making design decisions.

When developing a package from scratch, there are often several ways to solve the same problem. Choosing one approach requires thinking about future users, maintainability, performance, edge cases, and how the different components of the package will interact.

The caching design was a particularly challenging part.

I also had to learn how real open-source projects work:

• understanding an existing codebase,
• reading documentation and source code,
• discussing design decisions,
• writing tests,
• responding to code review,
• debugging unexpected behavior,
• and thinking about how future contributors will understand the code.

At several points, I was unsure whether my approach was the right one. Instead of simply giving me the answer, my mentors encouraged me to think through the problem and explain my reasoning.

That process helped me become more comfortable with uncertainty.

I learned that not knowing the answer immediately is normal in software engineering. What matters is developing the ability to investigate, reason, experiment, discuss, and eventually arrive at a good solution.

What I Learned Beyond Technical Skills

GSoC taught me several things that I don’t think I could have learned as effectively from a course or textbook.

Current State of the Package

At the end of GSoC, the major planned components of the project have been implemented.

The package currently provides functionality for downloading and accessing datasets from Hugging Face, caching them locally, working with different revisions, and converting the data into objects usable by the mlr3 ecosystem.

However, I do not consider the project “finished,” in the sense that no software project is ever completely finished.

A few concrete areas remain open for future work:

• support for additional dataset formats beyond Parquet, csv and tsv,
• broader test coverage for edge cases in the caching layer,
• and continued documentation updates as the mlr3 ecosystem evolves.

If you find a bug, or see an area that can be improved, raise an issue, contributions are welcome.

Acknowledgements

I sincerely acknowledge my mentors, Toby Dylan Hocking and Sebastian Fischer, for their guidance and support throughout this journey. I am also grateful to The R Project for Statistical Computing for providing me with the opportunity to be part of this experience.

However, I want to acknowledge my mentors for something beyond simply helping me complete the project.I would also like to acknowledge the Hugging Face documentation, and the other dependencies used by this package, which were valuable resources throughout the development of mlr3hf.

Documentation and Package Structure

The package now contains two important vignettes:

• tutorial.Rmd
• quick-revision.Rmd

The tutorial provides a detailed introduction to using mlr3hf, while the quick revision vignette gives users a shorter overview of the package and its main functionality.

I also added sugar.R and zzz.R.

The sugar.R file contains user-friendly helper functions such as hfdata(), while zzz.R is used for package-level definitions such as @importFrom statements.

This keeps the package structure clean and makes the user-facing functions easier to discover.

User-Friendly Access with HFData

One of the major parts of the package is the HFData class.

The class provides several active bindings that allow users to access useful information about a Hugging Face dataset without having to manually interact with the Hugging Face API.

The available active bindings are:

For example, we can create an HFData object using:

hf_data <- hfdata(
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The active bindings can then be accessed directly:

hf_data$repo_link
hf_data$configs
hf_data$siblings

For example, repo_link directly returns the URL of the dataset repository on Hugging Face. This URL can then be opened in a browser to inspect the dataset repository.

Similarly, configs returns all configurations available for the dataset, while siblings returns the filenames available in the repository.

This makes it possible to explore a Hugging Face dataset directly from R without manually navigating through the Hugging Face API.

Creating an mlr3 Task Directly

One of the main goals of mlr3hf is to make Hugging Face datasets immediately usable within the mlr3 ecosystem.

With the package, a user can create an mlr3 task directly:

hf_data <- task(
  "hf",
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The resulting object is an mlr3 task and can be directly used with the rest of the mlr3 ecosystem.

For example, it can be passed to learners, resampling strategies, prediction workflows, and performance measures.

This removes several manual steps that would otherwise be required to download a dataset, load it into R, identify the target column, and construct an mlr3 task.

hfdata()

We also provide a user-friendly helper function, hfdata().

hf_data <- hfdata(
  repo_id = "scikit-learn/iris",
  config = "default",
  target = "Species"
)

The hfdata() function provides direct access to the HFData object and its active bindings.

For example:

hf_data$repo_link
hf_data$configs
hf_data$siblings
hf_data$data
hf_data$colnames
hf_data$coltypes
hf_data$feature_names

Some of these bindings can even be used when only the repo_id is provided.

For example, information such as the repository URL, configurations, and repository files does not require the user to specify a target column.

This makes HFData useful not only for model training, but also for exploring and understanding a Hugging Face dataset before using it.

Working with Predefined Dataset Splits

Another useful feature of mlr3hf is support for predefined dataset splits.

Many Hugging Face datasets already provide predefined train, test, and sometimes validation splits.

Instead of creating a new resampling split, mlr3hf exposes these predefined splits through the $splits active binding.

The splits are represented as vectors of row IDs, which means they can be directly reused in mlr3 workflows.

For example:

library(mlr3learners)

hf_data <- hfdata(
  repo_id = "a4n9i/scikit_iris",
  config = "default",
  target = "Species"
)

task <- as_task(hf_data)

train_row_id <- hf_data$splits$train
test_row_id  <- hf_data$splits$test

We can then pass these row IDs directly to ResamplingCustom.

resampling <- rsmp("custom")

resampling$instantiate(
  task,
  train_sets = list(train_row_id),
  test_sets = list(test_row_id)
)

resampling

This produces:

## ──  : Custom Splits ──────────────────────────────────────────
## • Iterations: 1
## • Instantiated: TRUE
## • Parameters: list()

The resulting resampling object can be used like any other mlr3 resampling strategy:

rr <- resample(
  task,
  lrn("classif.rpart"),
  resampling
)

rr$aggregate()

For example:

## classif.ce
## 0.6333333

The important point here is that we are reusing the original train/test split provided by the dataset, rather than generating a new random split.

This is especially useful when a dataset creator has already defined meaningful training and evaluation splits.

Using a Predefined Validation Split

Some Hugging Face datasets also provide a predefined validation split.

This can be particularly useful for learners that support internal validation.

For example, classif.xgboost can use an internal validation task for early stopping and monitoring validation performance.

First, we extract the predefined splits:

train_ids <- hf_data$splits$train
valid_ids <- hf_data$splits$validation
test_ids  <- hf_data$splits$test

We then create a training task containing only the training rows:

tsk_train <- task$clone(deep = TRUE)
tsk_train$filter(train_ids)

Similarly, we create a separate validation task:

tsk_valid <- task$clone(deep = TRUE)
tsk_valid$filter(valid_ids)

The validation task can then be attached to the training task:

tsk_train$internal_valid_task <- tsk_valid

Now we create an XGBoost learner:

lrn <- lrn("classif.xgboost")

We tell the learner to use the predefined validation task:

lrn$validate <- "predefined"

We can then configure early stopping and the evaluation metric:

lrn$param_set$set_values(
  nrounds = 500,
  early_stopping_rounds = 10,
  eval_metric = "mlogloss"
)

Finally, we train the learner:

lrn$train(tsk_train)

The validation score can be inspected using:

lrn$internal_valid_scores

For example:

## $mlogloss
## [1] 13.82791

After training, we can evaluate the model on the predefined test split:

tsk_test <- task$clone(deep = TRUE)
tsk_test$filter(test_ids)

pred <- lrn$predict(tsk_test)

pred$score(msr("classif.ce"))

For example:

## classif.ce
## 0.6333333

A learner such as classif.rpart does not use internal_valid_task, whereas learners such as classif.xgboost can use it for internal validation and early stopping.

Keeping the User Perspective in Mind

Throughout the development of mlr3hf, one of the most important things we tried to keep in mind was the user perspective.

The goal was not simply to build another interface for downloading Hugging Face datasets.

Instead, we wanted to answer a more practical question:

How can an R user take a dataset from Hugging Face and start using it in the mlr3 ecosystem with as little effort as possible?

This led to features such as:

• Direct creation of mlr3 tasks
• hfdata() for convenient dataset access
• Active bindings for dataset exploration
• Support for predefined train/test/validation splits
• Reuse of existing dataset splits instead of unnecessarily creating new ones

The intention is to make the workflow feel natural for an R user.

Acknowledgements

I would like to sincerely thank my mentors, Toby Dylan Hocking and Sebastian Fischer, for their guidance and support throughout this project.

Their feedback and discussions helped me understand the project from both a technical and a user-oriented perspective.

Whenever we discussed a feature or design decision, we tried to keep one question in mind:

How will this help the user?

That mindset had a significant influence on the final design of mlr3hf.

I am very grateful for the opportunity to work on this project and learn from both of my mentors.

We have now reached the final stage of the project, and I hope that mlr3hf will be useful to R users and researchers who want to work with datasets available on Hugging Face.

It has been a challenging but rewarding journey, and I am happy with what we have built together.

We built it with love for the R and mlr3 community. ❤️

1. Implemented a Hugging Face-style Caching System

I initially started by implementing cache.R, but after several design discussions with mentors, I redesigned the caching mechanism to closely follow the architecture used by the Hugging Face Hub. This provides a more robust and maintainable solution while remaining familiar to users already working with Hugging Face.

The cache is organized under ~/.cache/huggingface/ and contains two separate caches.

~/.cache/huggingface/
└── hub/
    └── <repo_name>/
        ├── blobs/
        │   └── <etag>                  # actual downloaded file
        │
        ├── snapshots/
        │   └── <commit_hash>/
        │       └── <file_name>         # symlink/copy → ../../blobs/<etag>
        │
        └── refs/
            └── <branch_or_tag>         # contains commit hash

dataset <- cache_hfhub(
  repo_id = "scikit-learn/iris",
  file_name = "Iris.csv"
)

~/.cache/huggingface/
└── datasets/
    └── <repo_folder>/
        └── <config>/
            ├── train/
            │   ├── blobs/
            │   │   └── <etag>
            │   └── snapshots/
            │       └── <commit_hash>/
            │           └── <filename>.parquet
            │
            ├── test/
            │   ├── blobs/
            │   └── snapshots/
            │
            └── validation/
                ├── blobs/
                └── snapshots/

dataset <- cache_parquet(
  repo_id = "scikit-learn/iris",
  config = "default"
)

2. Implemented list_datasets()

I implemented list_datasets(), allowing users to search datasets directly from the Hugging Face Hub.

Example:

list_datasets(search = "machine-learning")

The function supports keyword-based dataset discovery and returns matching repositories, making it easier for users to explore datasets directly from R.

3. Implemented download_desc()

I implemented download_desc.R, which retrieves repository metadata before downloading any files.

The function provides useful information such as:

• Available files in the repository
• File names
• Total storage

This metadata will also be integrated into the HFData R6 class so that users can inspect repository details during object initialization.

4. Developed the Initial Version of HFData

I have completed a working implementation of HFData.R.

Before cleaning and refactoring the code, I will test it extensively on multiple Hugging Face datasets to identify implementation issues and edge cases. This testing phase helped improve the overall reliability of the backend before finalizing the public API.

5. Testing

A significant amount of work has gone into writing unit tests.

6. Development Infrastructure

In addition to implementing the package functionality, I also implemented overall development workflow by:

• Resolving R CMD check issues.
• Configuring automated documentation generation.
• Setting up pkgdown.
• Deploying the documentation using GitHub Pages and Netlify.
• Improving the package structure and development workflow.

Current Status

The core backend infrastructure is now functional.

The following components have been implemented:

• cache_parquet()
• cache_hfhub()
• list_datasets()
• download_desc()
• Unit testing framework
• Documentation infrastructure
• CI setup and package website deployment

The project has now reached the stage where the backend infrastructure is largely complete, allowing future work to focus on usability, benchmarking, and polishing the user experience.

Goals for the Next Phase

The next milestones are:

• HFData.R implementation.
• Increase unit test coverage to 90%+.
• Benchmark overall code implementations and evaluate their performance.
• Update and correct the documentation to reflect the latest API changes.
• Add more examples and improve the overall user documentation before the final release.

cache_parquet.R

Many datasets on Hugging Face are available through the refs/convert/parquet branch, which contains automatically generated Parquet files. Since Parquet is an efficient storage format and integrates well with analytical backends, mlr3hf downloads datasets from this branch whenever possible.

The cache_parquet() function downloads Parquet datasets and stores them in the local cache. It currently exposes the following arguments:

• repo_id
• config
• revision
• split

If revision is not specified, the latest version of the dataset is downloaded. To reproduce an older version of a dataset, simply provide the corresponding commit hash through the revision argument.

dataset.latest <- cache_parquet(
  repo_id = "a4n9i/iris-tabular",
  config = "default"
)

dataset.latest

$train
[1] "~/.cache/huggingface/datasets/datasets--a4n9i--iris-tabular/default/train/snapshots/f408ebffc0876b0f284fa4c90453e52e43c67677/0000.parquet"

dataset.old <- cache_parquet(
  repo_id = "a4n9i/iris-tabular",
  config = "default",
  revision = "d8ba9e5adfa0070361298ae66161ca0b26e78806"
)

dataset.old

$train
[1] "~/.cache/huggingface/datasets/datasets--a4n9i--iris-tabular/default/train/snapshots/d8ba9e5adfa0070361298ae66161ca0b26e78806/0000.parquet"

cache_hfhub.R

Not every dataset on Hugging Face is available as Parquet. Some repositories contain datasets in their original formats, such as CSV or JSON.

The internal function cache_hfhub() downloads these canonical dataset files directly from the Hugging Face Hub. Although this is currently an internal helper function, it provides the foundation for supporting repositories that do not contain Parquet files.

dataset <- cache_hfhub(
  repo_id = "a4n9i/iris-tabular",
  file_name = "iris.csv"
)

dataset

Expected output

~/.cache/huggingface/hub/datasets--a4n9i--iris-tabular/snapshots/e97efa4eb2e5a170a564aa2d05597c7fc8ac8a4c/iris.csv

Like cache_parquet(), this function also supports downloading datasets from different branches or specific commit revisions.

list_datasets.R

Finding an appropriate dataset is the first step before downloading it.

The list_datasets() function retrieves datasets from the Hugging Face Datasets API and returns a summary containing useful information, including:

• Dataset ID
• Whether the dataset is gated
• Number of downloads

dataset_list <- list_datasets(5)

dataset_list

Expected output

                                    id  gated downloads
                                <char> <char>     <int>
1:       Glint-Research/Fable-5-traces  FALSE     19185
2: armand0e/claude-fable-5-claude-code  FALSE      7978
3:      lazarus19/Vibe-Coding-Instruct  FALSE      1709
4:                       XDOF/ABC-130k   auto     87103
5:                  LocalLaws/LOCUS-v1  FALSE      1130

In future versions of mlr3hf, I also plan to expose additional metadata such as the number of rows and columns whenever that information is available.

download_desc.R

Another internal helper function is download_desc(), which retrieves repository metadata from Hugging Face.

This information is useful before downloading a dataset because it allows mlr3hf to inspect repository properties such as storage size and modification time.

download_desc("scikit-learn/iris")

Example output

$id
[1] "scikit-learn/iris"

$author
[1] "scikit-learn"

$sha
[1] "0bda0ce801be0fa2f464ff845a9d5ceae99aad7d"

$lastModified
[1] "2022-06-20T14:17:01.000Z"

$usedStorage
[1] 5309549

...

Internally, this metadata is used to make smarter download decisions. For example, the repository size can be used to prevent users from accidentally downloading extremely large datasets that may exceed their available storage.

The metadata retrieved by download_desc() will also be useful in higher-level classes such as HFData, where fields like $lastModified can be exposed through active bindings.

Current Progress

The following functionality has been implemented so far:

• Downloading datasets from the refs/convert/parquet branch.
• Supporting dataset revisions through commit hashes.
• Downloading canonical dataset files from the Hugging Face Hub.
• Caching downloaded datasets locally.
• Listing datasets available on Hugging Face.
• Retrieving repository metadata for smarter download management.

Future Work

The next major milestones for mlr3hf are:

• Implementing the mlr3 backend for Hugging Face datasets.
• Creating mlr3 Tasks directly from the downloaded datasets.
• Exposing richer dataset metadata, including the number of rows and columns.
• Continuing to improve the caching and download infrastructure.

Acknowledgements

I would like to thank my mentors, Toby Hocking and Sebastian Fischer, for their continuous guidance throughout the development of mlr3hf.

Many of the features described in this post began as rough or incomplete ideas. Before implementing them, I regularly discussed the design, API, and overall approach with my mentors. Their feedback helped me better understand the problem, refine the implementation, and make design decisions that aligned with the goals of the project.

I am grateful for their patience, insightful discussions, and constructive suggestions, which have been invaluable in shaping the package. Looking forward to work more with the mentor.

Completed Work

The following components have been implemented and are currently functional:

• cache_parquet.R
• cache_hfhub.R
• defaults.R
• download_desc.R
• list_datasets.R

These functions provide the foundation for efficiently accessing and managing datasets from Hugging Face Hub.

Acknowledgements

I would like to thank Sebastian Fischer and Toby Hocking for their continuous support, guidance, and valuable feedback throughout the project.And I look forward to continuing this collaboration during the remainder of GSoC.

Hugging Face Inspired Caching System

One of the major milestones achieved so far is the implementation of a caching system inspired by the Hugging Face Hub architecture.

The cache follows a structure similar to Hugging Face’s caching mechanism:

• Blobs store the actual file contents.
• Snapshots represent a specific dataset revision.
• Symlink connect snapshots and blob without duplicating data.

This design allows multiple revisions of the same dataset to coexist locally while minimizing storage requirements.

Currently, users can:

• Download datasets directly from Hugging Face Hub.
• Access datasets from specific revisions or commit hashes.
• Work with public, gated, and private datasets (with appropriate authentication).
• Reuse previously downloaded files through the cache, avoiding unnecessary downloads.

Future Work: Task Creation and Data Backends

With the caching infrastructure in place, the next phase of development will focus on creating mlr3 tasks from downloaded datasets.

Upcoming Milestones

The major goals for the coming weeks are:

1. Explore and integrate NanoParquet for Parquet-based datasets.
2. Implement task creation workflows.
3. Automatically select suitable backends based on dataset format.
4. Expand documentation and test coverage.
5. Improve interoperability with the broader mlr3 ecosystem.

Conclusion

At this stage, the project has reached a stable foundation. The core dataset discovery, downloading, and caching infrastructure is now in place. The next phase will focus on transforming cached datasets into fully functional mlr3 tasks with efficient backend support.

I am excited to continue development and share further updates as the project progresses.

Why Parquet?

One of the main reasons for choosing parquet is that it enables us to implement custom resampling strategies similar to those available in mlr3oml.

The parquet conversion branch contains datasets already organized into train, validation, and test splits, making it easier to create reproducible machine learning tasks and benchmarks.

Cache Architecture

To support dataset downloading and caching, I implemented download_hfhub() cache.R and download_parquet().You can visit my code. This function downloads and manages files inside a hub directory located under the main mlr3hf cache folder.

The cache structure contains:

• blobs
• snapshots
• refs

This design remains compatible across operating systems. Even when symbolic links are unavailable, files can still be downloaded and accessed correctly.

The implementation follows the general ideas used by Hugging Face Hub while adapting them to the needs of mlr3hf.

Comparison with Python’s load_dataset

Downloading component of mlr3hf is conceptually similar to Python’s load_dataset().

In the Python ecosystem:

• Datasets are cached in dedicated dataset and hub directories.
• The Hub cache contains structures similar to blobs, snapshots, and refs.
• Downloaded data is eventually converted into Arrow datasets and then accessed through libraries such as Pandas.

In mlr3hf:

• We currently do not support Xet-based storage.
• Parquet files are downloaded directly into the dataset cache.
• This approach will integrate naturally with future DuckDB backends, allowing efficient querying and analysis without additional conversion steps.

Acknowledgements

I would like to thank our mentors, Toby and Sebastian, for their guidance and design discussions.

I also took inspiration from:

• HF Hub a existing package of R hfhub
• Hugging Face cache management documentation

Progress So Far

The following files have been implemented:

• download_hfhub.R
• download_parquet.R
• cache_paruet.R
• cache_hfhub.R
• defaults.R
• utils.R

Next Steps

The next stage of development will focus on implementing:

• HFData.R
• HFObject.R

along with their supporting components. These classes will form the core engineering layer of the package.

Feedback

If anyone have any questions, suggestions, or would like to discuss the design, feel free to contact me at:

anjaninandan599@gmail.com

Cache Management Challenge

One of the major challenges is cache management.

Hugging Face datasets are associated with revisions such as commit hashes and branches, and users may want to download datasets from specific revisions rather than always using the latest version.

To address this, the current design explores Hugging Face’s storage structure involving:

• Snapshots
• Blobs
• Symbolic links (symlinks)

This design aims to minimise duplication while improving storage efficiency.

Exploring Parquet Infrastructure

Another area of investigation is Hugging Face’s Parquet infrastructure.

Hugging Face automatically converts many datasets into Parquet format through its Parquet conversion pipeline. This simplifies dataset access because users do not need to explicitly provide filenames or revision details.

While this endpoint offers convenience, the long-term objective remains implementing dataset downloading directly through the Hugging Face API:

https://huggingface.co/{repo_type}s/{repo_id}/resolve/{revision}/{filename}

This approach provides finer revision control and aligns more closely with the design goals of mlr3hf.

Project Objectives

The overall objective is to build a system that balances:

• User friendliness
• Efficient caching mechanisms
• Revision-aware dataset management
• Storage optimisation
• Smooth integration with the mlr3 workflow

Building these capabilities in a clean and scalable manner makes the project significantly more challenging than it initially appears.

Agenda for the Upcoming Two Weeks

(25 May 2026 – 08 June 2026)