⬆️ dep-bump(deps): Bump the dependency-packages group across 1 directory with 2 updates

[dependabot]

Bumps the dependency-packages group with 2 updates in the / directory: datasets and transformers.

Updates datasets from 3.2.0 to 3.3.1

Release notes

Sourced from datasets's releases.

3.3.1

Bug fixes

• Fix filter speed regression by @​lhoestq in huggingface/datasets#7408

Full Changelog: huggingface/datasets@3.3.0...3.3.1

3.3.0

Dataset Features

• Support async functions in map() by @​lhoestq in huggingface/datasets#7384

  • Especially useful to download content like images or call inference APIs

  prompt = "Answer the following question: {question}. You should think step by step."
  async def ask_llm(example):
      return await query_model(prompt.format(question=example["question"]))
  ds = ds.map(ask_llm)

• Add repeat method to datasets by @​alex-hh in huggingface/datasets#7198

  ds = ds.repeat(10)

• Support faster processing using pandas or polars functions in IterableDataset.map() by @​lhoestq in huggingface/datasets#7370

  • Add support for "pandas" and "polars" formats in IterableDatasets
  • This enables optimized data processing using pandas or polars functions with zero-copy, e.g.

  ds = load_dataset("ServiceNow-AI/R1-Distill-SFT", "v0", split="train", streaming=True)
  ds = ds.with_format("polars")
  expr = pl.col("solution").str.extract("boxed\\{(.*)\\}").alias("value_solution")
  ds = ds.map(lambda df: df.with_columns(expr), batched=True)

• Apply formatting after iter_arrow to speed up format -> map, filter for iterable datasets by @​alex-hh in huggingface/datasets#7207

  • IterableDatasets with "numpy" format are now much faster

What's Changed

• don't import soundfile in tests by @​lhoestq in huggingface/datasets#7340
• minor video docs on how to install by @​lhoestq in huggingface/datasets#7341
• Fix typo in arrow_dataset by @​AndreaFrancis in huggingface/datasets#7328
• remove filecheck to enable symlinks by @​fschlatt in huggingface/datasets#7133
• Webdataset special columns in last position by @​lhoestq in huggingface/datasets#7349
• Bump hfh to 0.24 to fix ci by @​lhoestq in huggingface/datasets#7350
• fsspec 2024.12.0 by @​lhoestq in huggingface/datasets#7352
• changes to MappedExamplesIterable to resolve #7345 by @​vttrifonov in huggingface/datasets#7353
• Catch OSError for arrow by @​lhoestq in huggingface/datasets#7348
• Remove .h5 from imagefolder extensions by @​lhoestq in huggingface/datasets#7374
• Add Pandas, PyArrow and Polars docs by @​lhoestq in huggingface/datasets#7382
• Optimized sequence encoding for scalars by @​lukasgd in huggingface/datasets#7393
• Update docs by @​lhoestq in huggingface/datasets#7395

... (truncated)

Commits

• 4ead6ec Release: 3.3.1 (#7409)
• 704704d Fix filter speed regression (#7408)
• de062f0 set dev version (#7401)
• e9dae36 Release: 3.3.0 (#7398)
• 5ef1a6c Update README.md (#7396)
• 235df2d Update docs (#7395)
• a7cb715 Optimized sequence encoding for scalars (#7393)
• 339e9dc Support async functions in map() (#7384)
• 3a4e74a Add repeat method to datasets (#7198)
• 6fa8fb3 Add Pandas, PyArrow and Polars docs (#7382)
• Additional commits viewable in compare view

Updates transformers from 4.48.0 to 4.49.0

Release notes

Sourced from transformers's releases.

Patch release v4.48.3

This ends the python3.9 issues mostly!

• Add future import for Py < 3.10 (#35666) by @​Rocketknight1

For some very niche cases, the new rope embedding introduced device failures

• Fix device in rope module when using dynamic updates (#35608) by @​Cyrilvallez

Num items in batch

• Fix model kwargs (#35875) by @​muellerzr: this is long due, sorry that it took so long. Some models were not compatible with the num_items_in_batch

Finally the fix to Gemma2 is propagated to paligemma2!

• Paligemma: fix generation with Gemma2 (#36044) by @​zucchini-nlp

Patch release v4.48.2

Sorry because the fixes for num_items_in_batches are not done yet 😓 To follow along see this PR, a new patch will be available soon!

Now, we mostly had BC issue with python version 3.9:

• Restore is_torch_greater_or_equal_than for backward compatibility (#35734) by @​tlrmchlsmth
• Fix NoneType type as it requires py>=3.10 (#35843) by @​SunMarc

Then we had a small regression for DBRX saving:

• Fix: loading DBRX back from saved path (#35728) by @​zucchini-nlp

Finally we have a fix for gemma and the hybrid attention architectures:

• Fix mask slicing for models with HybridCache #35681 by @​Cyrilvallez

Miscellaneous:

• Fix is_causal being a tensor (#35791) by @​IlyasMoutawwakil

Patch release v4.48.1

Yet again we are dawned with a gradient accumulation fix! There is also a refactoring of the attention that let a small typo in, we made sure PHI is no longer broken!

Moonshine had a small issue when wrapping generate so we removed that!

• [Phi] bias should be True (#35650) @​ArthurZucker
• Fix condition when GA loss bug fix is not performed (#35651) @​techkang
• Patch moonshine (#35731) @​eustlb

🤗

Commits

• a22a437 v4.49.0
• 8018e0e add shared experts for upcoming Granite 4.0 language models (#35894)
• bcfc9d7 [Bugfix] Fix reloading of pixtral/llava configs (#36077)
• 0c78ef6 🔴 VLM: compile compatibility (#35724)
• b45cf0e Guard against unset resolved_archive_file (#35628)
• 96f01a3 Revert qwen2 breaking changes related to attention refactor (#36162)
• cb586a3 Add require_read_token to fp8 tests (#36189)
• 5f726f8 New HIGGS quantization interfaces, JIT kernel compilation support. (#36148)
• 15ec971 Prepare processors for VideoLLMs (#36149)
• 33d1d71 Add ImageProcessorFast to Qwen2.5-VL processor (#36164)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
• @dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
• @dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
• @dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
• @dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

[dependabot]

Superseded by #27.