Skip to content

Support Python 3.9 for PyIceberg#637

Merged
ehsantn merged 40 commits intomainfrom
ehsan/python_3_9
Jul 7, 2025
Merged

Support Python 3.9 for PyIceberg#637
ehsantn merged 40 commits intomainfrom
ehsan/python_3_9

Conversation

@ehsantn
Copy link
Copy Markdown
Collaborator

@ehsantn ehsantn commented Jul 5, 2025

Changes included in this PR

Adds back Python 3.9 support since PyIceberg's Poetry setup requires Python 3.9: apache/iceberg-python#2167
The main changes are avoiding structured pattern matching and adapting type annotations.

Testing strategy

Tested Iceberg (local, S3 Tables), dataframe library unit tests, and NYC Taxi JIT locally with Python 3.9.
Nightly run passed except some flaky issues (rerunning to be sure): https://dev.azure.com/bodo-inc/Bodo/_build/results?buildId=25853&view=results

User facing changes

Python 3.9 packages will be available.

Checklist

  • Pipelines passed before requesting review. To run CI you must include [run CI] in your commit message.
  • I am familiar with the Contributing Guide
  • I have installed + ran pre-commit hooks.

new_axis0 = pd.TimedeltaIndex(
index_data, name=head_axis0.name, unit=head_axis0.unit
)
elif isinstance(head_axis0, pd.Index):
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pd.Index needs to be moved last in the class matching case since it's the super class of others.

@ehsantn ehsantn marked this pull request as ready for review July 7, 2025 13:25
Copy link
Copy Markdown
Contributor

@scott-routledge2 scott-routledge2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @ehsantn !

Copy link
Copy Markdown
Contributor

@chrisoh7 chrisoh7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @ehsantn !

Copy link
Copy Markdown
Contributor

@IsaacWarren IsaacWarren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Ehsan!

- fsspec >=2021.09
- pyarrow =19.0
- numba 0.61.2
- numba >=0.60
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be numba >=0.60,<0.62 here and the other places we specify numba version?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifying upper version limits causes trouble for dependency solvers and is not great practice. We are less sensitive to Numba versions at this point and will upgrade quickly if necessary.

Copy link
Copy Markdown
Contributor

@IsaacWarren IsaacWarren Jul 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'd just be worried that a numba upgrade would break our old package versions forcing users to upgrade their bodo version but if new numba versions are unlikely to cause issues, makes sense

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 7, 2025

Codecov Report

Attention: Patch coverage is 36.64773% with 223 lines in your changes missing coverage. Please review.

Project coverage is 65.71%. Comparing base (18406a4) to head (0619286).
Report is 143 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #637      +/-   ##
==========================================
+ Coverage   65.42%   65.71%   +0.28%     
==========================================
  Files         176      179       +3     
  Lines       63691    65436    +1745     
  Branches     8915     9262     +347     
==========================================
+ Hits        41672    42998    +1326     
- Misses      19397    19790     +393     
- Partials     2622     2648      +26     

@ehsantn ehsantn merged commit 90daf28 into main Jul 7, 2025
51 of 53 checks passed
@ehsantn ehsantn deleted the ehsan/python_3_9 branch July 7, 2025 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants