[pearu]

Rationale for this change

CSV columns typed as date32 or date64 are parsed only from the ISO-8601 format, and ConvertOptions::timestamp_parsers applies only to timestamp columns, so other date formats need reading as a timestamp and a cast.

What changes are included in this PR?

• Date columns try timestamp_parsers after the ISO-8601 parser fails, and floor the parsed timestamp to the day boundary, as a cast from timestamp to date does. Values outside the range of date32/date64 are rejected.
• strptime parsers without a date directive are skipped, so that e.g. "%H:%M:%S" does not turn a time into 1900-01-01.
• With the ISO-8601 parser in timestamp_parsers, fractional seconds are accepted and discarded.
• Type inference of date columns is unchanged and uses ISO-8601 only, so a value with a time of day is not inferred as a date.

This PR is the date part of #50146.

Are these changes tested?

Yes, Date32Conversion.UserDefinedParsers and Date64Conversion.UserDefinedParsers in C++, and test_dates_with_timestamp_parsers in Python.

Are there any user-facing changes?

Yes, timestamp_parsers now also applies to date32/date64 columns given in column_types. This is documented in the C++ CSV docs and the ConvertOptions docstring.

AI usage disclosure

Developed with AI assistance (Claude Code): the implementation, tests, documentation and this description were AI-generated under my direction and reviewed line by line by me. I own and can debug these changes. No copyrighted material is included.

• GitHub Issue: [C++][Python] Use timestamp_parsers as a fallback for CSV date columns #54650

[github-actions]

Thanks for opening a pull request!

This pull request has been automatically closed because you currently have 4 open pull requests, which is more than the limit of 3.

Due to the increase in pull requests opened by AI bots, and in order to keep the review queue manageable, Apache Arrow limits contributors without repository access to at most 3 concurrently open pull requests. This helps make sure each pull request gets the attention it needs and that work in progress does not go stale.

Once one of your other open pull requests has been merged or closed, you are welcome to reopen this one.

See also:

• Contribution Guidelines - Limit on concurrent pull requests
• Contribution Guidelines - Contributing Overview

[github-actions]

⚠️ GitHub issue #54650 has been automatically assigned in GitHub to PR creator.

[pearu]

@kou @ianmcook could you re-open this PR? Once opened, I'll rebase and it will be ready for review.

[kou]

Could you rebase on main?

[pearu]

Could you rebase on main?

Done. Please review.

[kou]

If we have a separated date_parsers, can we simplify the current implementation?

[pearu]

I used an AI agent to analyze the suggested alternative for its pros and cons. Assuming the existing tests stay as representative samples of the feature, with a separate date_parsers:

Could go:

• StrptimeFormatCanProduce
• make_date_time_converter and the is_type_inference parameter of Converter::Make, but only if date_parsers is also used for type inference, which means a column matching a datetime format listed in both options would be inferred as a date (inference tries dates before timestamps); keeping them means date_parsers differs from timestamp_parsers in not affecting inference

Would come:

• the new option in C++ ConvertOptions and in pyarrow (constructor, property, equality, pickling, repr), with its tests and docs, likely later also in GLib and R
• if used for type inference: inference of non-ISO date columns as dates, with its own tests
• the same again with time_parsers for GH-54651: [C++][Python] Use timestamp_parsers as a fallback for CSV time columns #54662

Everything else would stay, as the tests use datetime formats, ISO8601() with fractional seconds and custom parsers, which the new option would have to accept as timestamp_parsers does; only the tests that a format without a date directive is not tried (and their counterparts in #54662) would be dropped, as that case only arises when date and timestamp columns share one option. Users with formats that contain both a date and a time would list them in both options.

The API change is the more important point: a new public option that is documented and kept from then on. So the implementation would get somewhat simpler, at the cost of a larger API. On this balance I'd keep the current approach, unless you see an API reason for the separate option.