Google Summer of Code 2026 – Progress Report (till Week 6)

Overview

During the first six weeks of GSoC, I focused on building the core infrastructure for mlr3hf, with the primary goal of providing efficient dataset downloading, caching, and metadata management for Hugging Face datasets in R.

---

1. Implemented a Hugging Face-style Caching System

I initially started by implementing cache.R, but after several design discussions with mentors, I redesigned the caching mechanism to closely follow the architecture used by the Hugging Face Hub. This provides a more robust and maintainable solution while remaining familiar to users already working with Hugging Face.

The cache is organized under ~/.cache/huggingface/ and contains two separate caches.

Hub Cache (Canonical Datasets)

~/.cache/huggingface/
└── hub/
    └── <repo_name>/
        ├── blobs/
        │   └── <etag>                  # actual downloaded file
        │
        ├── snapshots/
        │   └── <commit_hash>/
        │       └── <file_name>         # symlink/copy → ../../blobs/<etag>
        │
        └── refs/
            └── <branch_or_tag>         # contains commit hash

Features:

• Stores downloaded canonical dataset files.
• Uses blobs, snapshots, and refs similar to the official Hugging Face cache.
• Prevents duplicate downloads.
• Supports repository revisions through commit hashes.

Example:

dataset <- cache_hfhub(
  repo_id = "scikit-learn/iris",
  file_name = "Iris.csv"
)

Parquet Cache

~/.cache/huggingface/
└── datasets/
    └── <repo_folder>/
        └── <config>/
            ├── train/
            │   ├── blobs/
            │   │   └── <etag>
            │   └── snapshots/
            │       └── <commit_hash>/
            │           └── <filename>.parquet
            │
            ├── test/
            │   ├── blobs/
            │   └── snapshots/
            │
            └── validation/
                ├── blobs/
                └── snapshots/

Each dataset split maintains:

• blobs
• snapshots
• commit-based versioning

Example:

dataset <- cache_parquet(
  repo_id = "scikit-learn/iris",
  config = "default"
)

This implementation enables efficient storage, avoids redundant downloads, and follows the same overall design philosophy as Hugging Face’s caching system.

---

2. Implemented list_datasets()

I implemented list_datasets(), allowing users to search datasets directly from the Hugging Face Hub.

Example:

list_datasets(search = "machine-learning")

The function supports keyword-based dataset discovery and returns matching repositories, making it easier for users to explore datasets directly from R.

---

3. Implemented download_desc()

I implemented download_desc.R, which retrieves repository metadata before downloading any files.

The function provides useful information such as:

• Available files in the repository
• File names
• Total storage

This metadata will also be integrated into the HFData R6 class so that users can inspect repository details during object initialization.

---

4. Developed the Initial Version of HFData

I have completed a working implementation of HFData.R.

Before cleaning and refactoring the code, I will test it extensively on multiple Hugging Face datasets to identify implementation issues and edge cases. This testing phase helped improve the overall reliability of the backend before finalizing the public API.

---

5. Testing

A significant amount of work has gone into writing unit tests.

Current Test Coverage

File	Lines	Relevant	Covered	Missed	Coverage
R/cache_hfhub.R	192	83	43	40	51.81%
R/defaults.R	40	20	16	4	80.00%
R/cache_parquet.R	201	92	77	15	83.70%
R/utils.R	225	107	98	9	91.59%
R/download_desc.R	38	16	16	0	100.00%

Overall test coverage: 81.42%

My next target is to increase the overall test coverage to 90%+, with particular focus on improving coverage for cache_hfhub.R.

---

6. Development Infrastructure

In addition to implementing the package functionality, I also implemented overall development workflow by:

• Resolving R CMD check issues.
• Configuring automated documentation generation.
• Setting up pkgdown.
• Deploying the documentation using GitHub Pages and Netlify.
• Improving the package structure and development workflow.

---

Current Status

The core backend infrastructure is now functional.

The following components have been implemented:

• cache_parquet()
• cache_hfhub()
• list_datasets()
• download_desc()
• Unit testing framework
• Documentation infrastructure
• CI setup and package website deployment

The project has now reached the stage where the backend infrastructure is largely complete, allowing future work to focus on usability, benchmarking, and polishing the user experience.

---

Goals for the Next Phase

The next milestones are:

• HFData.R implementation.
• Increase unit test coverage to 90%+.
• Benchmark overall code implementations and evaluate their performance.
• Update and correct the documentation to reflect the latest API changes.
• Add more examples and improve the overall user documentation before the final release.