API: rename mode.nan_is_na option to future.distinguish_nan_and_na

Follow-up on https://github.com/[USER_REDACTED]ndas-dev/pandas/pull/62040

See https://github.com/[USER_REDACTED]ndas-dev/pandas/pull/62040#issuecomment-3596823502 for some arguments why I think it fits better, but in general: if this option is for migrating to a future change in behaviour (and not to be kept as a choosable mode long term), then a `future` option fits better.

Regarding naming, I would really like to keep the pattern of "enabling" future options by [NAME_REDACTED] But because of that, `future.nan_is_na` would not work. Took `future.distinguish_nan_and_na`, but that is quite long, so if someone has a better idea, all ears.

@@ -547,29 +547,55 @@ small behavior differences as collateral:
 Changed treatment of NaN values in pyarrow and numpy-nullable floating dtypes
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

-Previously, when dealing with a nullable dtype (e.g. ``Float64Dtype`` or ``int64[pyarrow]``), ``NaN`` was treated as interchangeable with :class:`NA` in some circumstances but not others. This was done to make adoption easier, but caused some confusion (:issue:`32265`). In 3.0, an option ``"mode.nan_is_na"`` (default ``True``) controls whether to treat ``NaN`` as equivalent to :class:`NA`.
+Previously, when dealing with a nullable dtype (e.g. ``Float64Dtype`` or ``int64[pyarrow]``),
+``NaN`` was treated as interchangeable with :class:`NA` in some circumstances but not others.
+This was done to make adoption easier, but caused some confusion (:issue:`32265`).
+In 3.0, this behaviour is made consistent to by [NAME_REDACTED] ``NaN`` as equivalent
+to :class:`NA` in all cases.

-With ``pd.set_option("mode.nan_is_na", True)`` (again, this is the default), ``NaN`` can be passed to constructors, ``__setitem__``, ``__contains__`` and be treated the same as :class:`NA`. The only change users will see is that arithmetic and ``np.ufunc`` operations that previously introduced ``NaN`` entries produce :class:`NA` entries instead:
+By default, ``NaN`` can be passed to constructors, ``__setitem__``, ``__contains__``
+and will be treated the same as :class:`NA`. The only change users will see is
+that arithmetic and ``np.ufunc`` operations that previously introduced ``NaN``
+entries produce :class:`NA` entries instead.

 *Old behavior:*

 .. code-block:: ipython

-    In [2]: ser = pd.Series([0, None], dtype=pd.Float64Dtype())
+    # NaN in input gets converted to NA
+    In [1]: ser = pd.Series([0, np.nan], dtype=pd.Float64Dtype())
+    In [2]: ser
+    Out[2]:
+    0     0.0
+    1    <NA>
+    dtype: Float64
+    # NaN produced by arithmetic (0/0) remained [NAME_REDACTED] [3]: ser / 0
     Out[3]:
     0     NaN
     1    <NA>
     dtype: Float64
+    # the NaN value is not considered as missing
+    In [4]: (ser / 0).isna()
+    Out[4]:
+    0    False
+    1     True
+    dtype: bool

 *New behavior:*

 .. ipython:: python

-    ser = pd.Series([0, None], dtype=pd.Float64Dtype())
+    ser = pd.Series([0, np.nan], dtype=pd.Float64Dtype())
+    ser
     ser / 0
+    (ser / 0).isna()

-By contrast, with ``pd.set_option("mode.nan_is_na", False)``, ``NaN`` is always considered distinct and specifically as a floating-point value, so cannot be used with integer dtypes:
+In the future, the intention is to consider ``NaN`` and :class:`NA` as distinct
+values, and an option to control this behaviour is added in 3.0 through
+``pd.options.future.distinguish_nan_and_na``. When enabled, ``NaN`` is always
+considered distinct and specifically as a floating-point value. As a consequence,
+it cannot be used with integer dtypes.

 *Old behavior:*

@@ -583,13 +609,21 @@ By contrast, with ``pd.set_option("mode.nan_is_na", False)``, ``NaN`` is always

 .. ipython:: python

-    pd.set_option("mode.nan_is_na", False)
-    ser = pd.Series([1, np.nan], dtype=pd.Float64Dtype())
-    ser[1]
+    with pd.option_context("future.distinguish_nan_and_na", True):
+        ser = pd.Series([1, np.nan], dtype=pd.Float64Dtype())
+        print(ser[1])
+
+If we had passed ``pd.Int64Dtype()`` or ``"int64[pyarrow]"`` for the dtype in
+the latter example, this would raise, as a float ``NaN`` cannot be held by an
+integer dtype.

-If we had passed ``pd.Int64Dtype()`` or ``"int64[pyarrow]"`` for the dtype in the latter example, this would raise, as a float ``NaN`` cannot be held by [NAME_REDACTED]
+With ``"future.distinguish_nan_and_na"`` enabled, ``ser.to_numpy()`` (and
+``frame.values`` and ``np.asarray(obj)``) will convert to ``object`` dtype if
+:class:`NA` entries are present, where before they would coerce to
+``NaN``.  To retain a float numpy dtype, explicitly pass ``na_value=np.nan``
+to :meth:`Series.to_numpy`.

-With ``"mode.nan_is_na"`` set to ``False``, ``ser.to_numpy()`` (and ``frame.values`` and ``np.asarray(obj)``) will convert to ``object`` dtype if :class:`NA` entries are present, where before they would coerce to ``NaN``.  To retain a float numpy dtype, explicitly pass ``na_value=np.nan`` to :meth:`Series.to_numpy`.
+Note that the option is experimental and subject to change in future releases.

 The ``__module__`` attribute now points to public modules
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@@ -36,5 +36,5 @@ def using_string_dtype() -> bool:

 def is_nan_na() -> bool:
-    _mode_options = _global_config["mode"]
-    return _mode_options["nan_is_na"]
+    _mode_options = _global_config["future"]
+    return not _mode_options["distinguish_nan_and_na"]

@@ -2127,5 +2127,5 @@ def monkeysession():
 [USER_REDACTED].fixture(params=[True, False])
 def using_nan_is_na(request):
     opt = request.param
-    with pd.option_context("mode.nan_is_na", opt):
+    with pd.option_context("future.distinguish_nan_and_na", not opt):
         yield opt

@@ -429,15 +429,6 @@ def is_terminal() -> bool:
         validator=is_one_of_factory([True, False, "warn"]),
     )

-    cf.register_option(
-        "nan_is_na",
-        os.environ.get("PANDAS_NAN_IS_NA", "1") == "1",
-        "Whether to treat NaN entries as interchangeable with pd.NA in "
-        "numpy-nullable and pyarrow float dtypes. See discussion in "
-        "https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/32265",
-        validator=is_one_of_factory([True, False]),
-    )
-

 # user warnings
 chained_assignment = """
@@ -900,5 +891,17 @@ def register_converter_cb(key: str) -> None:
         validator=is_one_of_factory([True, False]),
     )

+    cf.register_option(
+        "distinguish_nan_and_na",
+        os.environ.get("PANDAS_FUTURE_DISTINGUISH_NAN_AND_NA", "0") == "1",
+        "Whether to treat NaN entries as distinct from pd.NA in "
+        "numpy-nullable and pyarrow float dtypes. By [NAME_REDACTED] "
+        "interchangeable as missing values (NaN will be coerced to NA). "
+        "See discussion in "
+        "https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/32265",
+        validator=is_one_of_factory([True, False]),
+    )
+
+
 # GH#59502
 cf.deprecate_option("future.no_silent_downcasting", Pandas4Warning)

@@ -997,7 +997,7 @@ def _read_ujson(self) -> DataFrame | Series:
         else:
             obj = self._get_object_parser(self.data)
         if self.dtype_backend is not lib.no_default:
-            with option_context("mode.nan_is_na", True):
+            with option_context("future.distinguish_nan_and_na", False):
                 return obj.convert_dtypes(
                     infer_objects=False, dtype_backend=self.dtype_backend
                 )
@@ -1075,7 +1075,7 @@ def __next__(self) -> DataFrame | Series:
             raise ex

         if self.dtype_backend is not lib.no_default:
-            with option_context("mode.nan_is_na", True):
+            with option_context("future.distinguish_nan_and_na", False):
                 return obj.convert_dtypes(
                     infer_objects=False, dtype_backend=self.dtype_backend
                 )

@@ -386,7 +386,7 @@ def parse_table_schema(json, precise_float: bool) -> DataFrame:
             'table="orient" can not yet read ISO-formatted Timedelta data'
         )

-    with option_context("mode.nan_is_na", True):
+    with option_context("future.distinguish_nan_and_na", False):
         df = df.astype(dtypes)

     if "primaryKey" in table["schema"]:

the description here seems to suggest that "True" means interchangeable, which is misleading

other than this, LGTM pending green


BUG: pytables with non-nano timedelta64

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

Surfaced by #63196, which currently xfails test_preserve_timedeltaindex_type.  This implements a fix.

@@ -1192,6 +1192,7 @@ MultiIndex
 I/O
 ^^^
 - Bug in :class:`DataFrame` and :class:`Series` ``repr`` of :py:class:`collections.abc.Mapping` elements. (:issue:`57915`)
+- Bug in :meth:`DataFrame.to_hdf` and :func:`read_hdf` with ``timedelta64`` dtypes with non-nanosecond resolution failing to round-trip correctly (:issue:`63239`)
 - Fix bug in ``on_bad_lines`` callable when returning too many fields: now emits
   ``ParserWarning`` and truncates extra fields regardless of ``index_col`` (:issue:`61837`)
 - Bug in :func:`pandas.json_normalize` inconsistently handling non-dict items in ``data`` when ``max_level`` was set. The function will now raise a ``TypeError`` if ``data`` is a list containing non-dict items (:issue:`62829`)

@@ -13,6 +13,7 @@
     Any,
     ClassVar,
     Self,
+    cast,
 )

 import numpy as np
@@ -44,7 +45,10 @@
 )

 if TYPE_CHECKING:
-    from pandas._typing import npt
+    from pandas._typing import (
+        TimeUnit,
+        npt,
+    )

 class PyTablesScope(_scope.Scope):
@@ -225,15 +229,19 @@ def stringify(value):
             if conv_val.tz is not None:
                 conv_val = conv_val.tz_convert("UTC")
             return TermValue(conv_val, conv_val._value, kind)
-        elif kind in ("timedelta64", "timedelta"):
+        elif kind.startswith("timedelta"):
+            unit = "ns"
+            if "[" in kind:
+                unit = cast("TimeUnit", kind.split("[")[-1][:-1])
             if isinstance(conv_val, str):
                 conv_val = Timedelta(conv_val)
             elif lib.is_integer(conv_val) or lib.is_float(conv_val):
                 conv_val = Timedelta(conv_val, unit="s")
             else:
                 conv_val = Timedelta(conv_val)
-            conv_val = conv_val.as_unit("ns")._value
+            conv_val = conv_val.as_unit(unit)._value
             return TermValue(int(conv_val), conv_val, kind)
+
         elif meta == "category":
             metadata = extract_array(self.metadata, extract_numpy=True)
             result: npt.NDArray[np.intp] | np.intp | int

@@ -2702,8 +2702,12 @@ def convert(self, values: np.ndarray, nan_rep, encoding: str, errors: str):
             # recreate with tz if indicated
             converted = _set_tz(converted, tz, dtype)

-        elif dtype == "timedelta64":
-            converted = np.asarray(converted, dtype="m8[ns]")
+        elif dtype.startswith("timedelta64"):
+            if dtype == "timedelta64":
+                # from before we started storing timedelta64 unit
+                converted = np.asarray(converted, dtype="m8[ns]")
+            else:
+                converted = np.asarray(converted, dtype=dtype)
         elif dtype == "date":
             try:
                 converted = np.asarray(
@@ -3086,8 +3090,13 @@ def read_array(self, key: str, start: int | None = None, stop: int | None = None
                 tz = getattr(attrs, "tz", None)
                 ret = _set_tz(ret, tz, dtype)

-            elif dtype == "timedelta64":
-                ret = np.asarray(ret, dtype="m8[ns]")
+            elif dtype and dtype.startswith("timedelta64"):
+                if dtype == "timedelta64":
+                    # This was written back before we started writing
+                    # timedelta64 units
+                    ret = np.asarray(ret, dtype="m8[ns]")
+                else:
+                    ret = np.asarray(ret, dtype=dtype)

         if transposed:
             return ret.T
@@ -3324,7 +3333,7 @@ def write_array(
             node._v_attrs.value_type = f"datetime64[{value.dtype.unit}]"
         elif lib.is_np_dtype(value.dtype, "m"):
             self._handle.create_array(self.group, key, value.view("i8"))
-            getattr(self.group, key)._v_attrs.value_type = "timedelta64"
+            getattr(self.group, key)._v_attrs.value_type = str(value.dtype)
         elif isinstance(value, BaseStringArray):
             vlarr = self._handle.create_vlarray(self.group, key, _tables().ObjectAtom())
             vlarr.append(value.to_numpy())
@@ -5175,8 +5184,12 @@ def _unconvert_index(data, kind: str, encoding: str, errors: str) -> np.ndarray
             index = DatetimeIndex(data)
         else:
             index = DatetimeIndex(data.view(kind))
-    elif kind == "timedelta64":
-        index = TimedeltaIndex(data)
+    elif kind.startswith("timedelta64"):
+        if kind == "timedelta64":
+            # created before we stored resolution information
+            index = TimedeltaIndex(data)
+        else:
+            index = TimedeltaIndex(data.view(kind))
     elif kind == "date":
         try:
             index = np.asarray([date.fromordinal(v) for v in data], dtype=object)
@@ -5413,7 +5426,7 @@ def _dtype_to_kind(dtype_str: str) -> str:
     elif dtype_str.startswith("datetime64"):
         kind = dtype_str
     elif dtype_str.startswith("timedelta"):
-        kind = "timedelta64"
+        kind = dtype_str
     elif dtype_str.startswith("bool"):
         kind = "bool"
     elif dtype_str.startswith("category"):

@@ -848,7 +848,7 @@ def test_append_raise(tmp_path, using_infer_string):
             store.append("df", df)

-def test_append_with_timedelta(tmp_path):
+def test_append_with_timedelta(tmp_path, unit):
     # GH 3577
     # append timedelta

@@ -860,6 +860,7 @@ def test_append_with_timedelta(tmp_path):
         }
     )
     df["C"] = df["A"] - df["B"]
+    df["C"] = df["C"].astype(f"m8[{unit}]")
     df.loc[3:5, "C"] = np.nan

     path = tmp_path / "test_append_with_timedelta.h5"

@@ -1017,11 +1017,12 @@ def test_duplicate_column_name(tmp_path, setup_path):
     assert other.equals(df)

-[USER_REDACTED].mark.xfail(reason="non-nano TimedeltaIndex does not round-trip")
-def test_preserve_timedeltaindex_type(setup_path):
+def test_preserve_timedeltaindex_type(setup_path, unit):
     # GH9635
     df = DataFrame(np.random.default_rng(2).normal(size=(10, 5)))
-    df.index = timedelta_range(start="0s", periods=10, freq="1s", name="example")
+    df.index = timedelta_range(
+        start="0s", periods=10, freq="1s", name="example", unit=unit
+    )

     with ensure_clean_store(setup_path) as store:
         store["df"] = df

Looks good!

The consequence is that a file written with 3.0 (with a timedelta column) won't be readable by [NAME_REDACTED]?
(although right now if you write non-nanos, they just come back as wrong values)

Can you add a whatsnew note as you had in https://github.com/[USER_REDACTED]ndas-dev/pandas/pull/55622?

Could parametrize `test_append_with_timedelta` for the different units? (or is there already other coverage?)

good idea, updated.


DOC: 62437 Replace [USER_REDACTED] and [USER_REDACTED] in core/groupby/groupby.py

I had issues updating my original PR (#63201), I redid it to account for formatting fixes. File: pandas/core/groupby/groupby.py

- [ ] closes #62437 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

@@ -64,7 +64,6 @@ class providing the base-class of operations.
     Pandas4Warning,
 )
 from pandas.util._decorators import (
-    Appender,
     Substitution,
     cache_readonly,
     doc,
@@ -738,11 +737,65 @@ def pipe(
         **kwargs: Any,
     ) -> T: ...

-    [USER_REDACTED](
-        klass="GroupBy",
-        examples=dedent(
-            """\
-        >>> df = pd.DataFrame({'A': 'a b a b'.split(), 'B': [1, 2, 3, 4]})
+    def pipe(
+        self,
+        func: Callable[Concatenate[Self, P], T] | tuple[Callable[..., T], str],
+        *args: Any,
+        **kwargs: Any,
+    ) -> T:
+        """
+        Apply a ``func`` with arguments to this GroupBy object and return its result.
+
+        Use `.pipe` when you want to improve readability by [NAME_REDACTED]
+        functions that expect Series, DataFrames, GroupBy or Resampler objects.
+        Instead of writing
+
+        >>> h = lambda x, arg2, arg3: x + 1 - arg2 * arg3
+        >>> g = lambda x, arg1: x * 5 / arg1
+        >>> f = lambda x: x**4
+        >>> df = pd.DataFrame([["a", 4], ["b", 5]], columns=["group", "value"])
+        >>> h(g(f(df.groupby("group")), arg1=1), arg2=2, arg3=3)  # doctest: +SKIP
+
+        You can write
+
+        >>> (
+        ...     df.groupby("group").pipe(f).pipe(g, arg1=1).pipe(h, arg2=2, arg3=3)
+        ... )  # doctest: +SKIP
+
+        which is much more readable.
+
+        Parameters
+        ----------
+        func : callable or tuple of (callable, str)
+            Function to apply to this GroupBy object or, alternatively,
+            a `(callable, data_keyword)` tuple where `data_keyword` is a
+            string indicating the keyword of `callable` that expects the
+            GroupBy object.
+        *args : iterable, optional
+            Positional arguments passed into `func`.
+        **kwargs : dict, optional
+            A dictionary of keyword arguments passed into `func`.
+
+        Returns
+        -------
+        GroupBy
+            The return type of `func`.
+
+        See Also
+        --------
+        Series.pipe : Apply a function with arguments to a series.
+        DataFrame.pipe : Apply a function with arguments to a dataframe.
+        apply : Apply function to each group instead of to the
+            full GroupBy object.
+
+        Notes
+        -----
+        See more `here
+        <https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#piping-function-calls>`_
+
+        Examples
+        --------
+        >>> df = pd.DataFrame({"A": "a b a b".split(), "B": [1, 2, 3, 4]})
         >>> df
            A  B
         0  a  1
@@ -753,20 +806,12 @@ def pipe(
         To get the difference between each groups maximum and minimum value in one
         pass, you can do

-        >>> df.groupby('A').pipe(lambda x: x.max() - x.min())
+        >>> df.groupby("A").pipe(lambda x: x.max() - x.min())
            B
         A
         a  2
-        b  2"""
-        ),
-    )
-    [USER_REDACTED](_pipe_template)
-    def pipe(
-        self,
-        func: Callable[Concatenate[Self, P], T] | tuple[Callable[..., T], str],
-        *args: Any,
-        **kwargs: Any,
-    ) -> T:
+        b  2
+        """
         return com.pipe(self, func, *args, **kwargs)

     [USER_REDACTED]

Looking good!

lgtm

This change is wrong, the columns should be adjusted. Is pre-commit doing this with the ruff formater?

Ditto.

Fixed! I think this is a holdover from the contents of the generated docstring I printed to console.

I misunderstood the example. Fixed alignment of A B to match columns

Thank you for the feedback!

I tried to combine lines 810/811 (i.e. `A B`) but that was causing `[NAME_REDACTED]/Docstrings` to fail. I reverted to the two lines and it's passing now, but doesn't look right in the file. I've not been able to find other examples that follow a pattern like that to see what others have done

`A B` would be the right thing if they were both columns. But the output has an index named A and a column named B, what you have currently is correct.

```python
df = pd.DataFrame({"A": "a b a b".split(), "B": [1, 2, 3, 4]})
print(df.groupby("A").pipe(lambda x: x.max() - x.min()))
#    B
# A
# a  2
# b  2
```


DEPR: unused Copy-on-Write option

- [x] closes #63230 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

@@ -72,6 +72,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
         -i "pandas.Series.dt PR01" `# Accessors are implemented as classes, but we do not document the Parameters section` \
         -i "pandas.Period.freq GL08" \
         -i "pandas.Period.ordinal GL08" \
+        -i "pandas.errors.ChainedAssignmentError SA01" \
         -i "pandas.errors.IncompatibleFrequency SA01,SS06,EX01" \
         -i "pandas.api.extensions.ExtensionArray.value_counts EX01,RT03,SA01" \
         -i "pandas.api.typing.DataFrameGroupBy.plot PR02" \

@@ -117,6 +117,9 @@ process in more detail.

     `PDEP-7: Consistent copy/view semantics in pandas with Copy-on-Write <https://pandas.pydata.org/pdeps/0007-copy-on-write.html>`__

+Setting the option ``mode.copy_on_write`` no longer has any impact. The option is deprecated
+and will be removed in pandas 4.0.
+
 .. _whatsnew_300.enhancements.col:

 ``pd.col`` syntax can now be used in :meth:`DataFrame.assign` and :meth:`DataFrame.loc`

@@ -408,12 +408,11 @@ def is_terminal() -> bool:
     cf.register_option("sim_interactive", False, tc_sim_interactive_doc)

-# TODO better name?
 copy_on_write_doc = """
 : bool
-    Use new copy-view behaviour using Copy-on-Write. Defaults to False,
-    unless overridden by the 'PANDAS_COPY_ON_WRITE' environment variable
-    (if set to "1" for True, needs to be set before pandas is imported).
+    Use new copy-view behaviour using Copy-on-Write. No longer used,
+    pandas now always uses Copy-on-Write behavior. This option will
+    be removed in pandas 4.0.
 """

@@ -902,3 +901,11 @@ def register_converter_cb(key: str) -> None:

 # GH#59502
 cf.deprecate_option("future.no_silent_downcasting", Pandas4Warning)
+cf.deprecate_option(
+    "mode.copy_on_write",
+    Pandas4Warning,
+    msg=(
+        "Copy-on-Write can no longer be disabled, setting to False has no impact. "
+        "This option will be removed in pandas 4.0."
+    ),
+)

@@ -668,7 +668,7 @@ class ChainedAssignmentError(Warning):
     """
     Warning raised when trying to set using chained assignment.

-    When the ``mode.copy_on_write`` option is enabled, chained assignment can
+    With Copy-on-Write now always enabled, chained assignment can
     never work. In such a situation, we are always setting into a temporary
     object that is the result of an indexing operation (getitem), which under
     Copy-on-Write always behaves as a copy. Thus, assigning through a chain
@@ -677,18 +677,11 @@ class ChainedAssignmentError(Warning):
     For more information on Copy-on-Write,
     see :ref:`the user guide<copy_on_write>`.

-    [NAME_REDACTED]
-    --------
-    options.mode.copy_on_write : Global setting for enabling or disabling
-        Copy-on-Write behavior.
-
     Examples
     --------
-    >>> pd.options.mode.copy_on_write = True
     >>> df = pd.DataFrame({"A": [1, 1, 1, 2, 2]}, columns=["A"])
     >>> df["A"][0:3] = 10  # doctest: +SKIP
     ... # ChainedAssignmentError: ...
-    >>> pd.options.mode.copy_on_write = False
     """

@@ -89,3 +89,12 @@ def test_copy_deprecation_merge_concat():
         Pandas4Warning, match="copy", check_stacklevel=False
     ):
         concat([df, df], copy=False)
+
+
+[USER_REDACTED].mark.parametrize("value", [False, True, "warn"])
+def test_copy_on_write_deprecation_option(value):
+    msg = "Copy-on-Write can no longer be disabled"
+    # stacklevel points to contextlib due to use of context manager.
+    with tm.assert_produces_warning(Pandas4Warning, match=msg, check_stacklevel=False):
+        with pd.option_context("mode.copy_on_write", value):
+            pass

Could we use `option_context` just to be safe that this call won't ever modify state of another test?

Doh - of course, thanks. Using the context manager the stacklevel points to contextlib. I think we can fix this, but would like to do in a separate PR. I've opened #63235.


TST: Add regression test for infer_freq stateful behavior (#55794)

- [x] closes #55794
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [x] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

This PR adds a regression test for #55794.

The bug was that `infer_freq` had stateful behavior - calling it on a full index with duplicates would affect the result of subsequent calls on slices without duplicates. This has been fixed on main and this test ensures the fix is not regressed.

@@ -553,3 +553,19 @@ def test_infer_freq_pyarrow():
     assert frequencies.infer_freq(pd_series.values) == "30s"
     assert frequencies.infer_freq(pd_index) == "30s"
     assert frequencies.infer_freq(pd_series) == "30s"
+
+
+def test_infer_freq_no_stateful_behavior():
+    # GH#55794 infer_freq should not have stateful behavior
+    # calling infer_freq on a full index with duplicates should not
+    # affect the result of calling it on a slice without duplicates
+    times = to_datetime(["2019-01-01", "2019-01-02", "2019-01-03", "2019-01-03"])
+
+    # Before calling infer_freq on full index
+    assert frequencies.infer_freq(times[:3]) == "D"
+
+    # Call on full index (which has duplicates, so returns None)
+    assert frequencies.infer_freq(times) is None
+
+    # After calling on full index, slice should still return "D"
+    assert frequencies.infer_freq(times[:3]) == "D"

```suggestion
    assert frequencies.infer_freq(times) is None
```

Fixed, thanks.


TST: Add regression test for DataFrame.where with nullable dtype (#49052)

- [x] closes #49052
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [x] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

This PR adds a regression test for #49052.

The bug was that `DataFrame.where` would return `object` dtype when `other` was a `Series` with nullable dtype (e.g., `Int64`), while `Series.where` correctly returned the nullable dtype. This has been fixed on main and this test ensures the fix is not regressed.

@@ -1063,3 +1063,13 @@ def test_where_inplace_no_other():
     assert result is df
     expected = DataFrame({"a": [1, np.nan], "b": [np.nan, "y"]})
     tm.assert_frame_equal(df, expected)
+
+
+def test_where_other_nullable_dtype():
+    # GH#49052 DataFrame.where should return nullable dtype when
+    # other is a Series with nullable dtype, matching Series.where behavior
+    df = DataFrame([1, 2, 3], dtype="int64")
+    other = Series([pd.NA, pd.NA, pd.NA], dtype="Int64")
+    result = df.where(df > 1, other, axis=0)
+    expected = DataFrame({0: Series([pd.NA, 2, 3], dtype="Int64")})
+    tm.assert_frame_equal(result, expected)

```suggestion
    df = DataFrame([1, 2, 3], dtype="int64")
```

Fixed, thanks.


API: Timedelta constructor from keywords give microseconds

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

Discussed in today's dev meeting.  There is one test that is currently failing but will pass once #63018 is merged

@@ -2114,7 +2114,7 @@ class Timedelta(_Timedelta):
                     int(ns)
                     + int(us * 1_000)
                     + int(ms * 1_000_000)
-                    + seconds
+                    + seconds, "ns"
                 )
             except OverflowError as err:
                 # GH#55503
@@ -2124,6 +2124,13 @@ class Timedelta(_Timedelta):
                 )
                 raise OutOfBoundsTimedelta(msg) from err

+            if (
+                "nanoseconds" not in kwargs
+                and cnp.get_timedelta64_value(value) % 1000 == 0
+            ):
+                # If possible, give a microsecond unit
+                value = value.astype("m8[us]")
+
         disallow_ambiguous_unit(unit)

         cdef:

@@ -938,7 +938,7 @@ def rand_series_with_duplicate_datetimeindex() -> Series:
             Timestamp("2011-01-01", tz="US/Eastern").as_unit("s"),
             DatetimeTZDtype(unit="s", tz="US/Eastern"),
         ),
-        (Timedelta(seconds=500), "timedelta64[ns]"),
+        (Timedelta(seconds=500), "timedelta64[us]"),
     ]
 )
 def ea_scalar_and_dtype(request):

@@ -223,8 +223,8 @@ def test_div_td64arr(self, left, box_cls):
     [USER_REDACTED].mark.parametrize(
         "scalar_td",
         [
-            Timedelta(days=1),
-            Timedelta(days=1).to_timedelta64(),
+            Timedelta(days=1).as_unit("ns"),
+            Timedelta(days=1).as_unit("ns").to_timedelta64(),
             Timedelta(days=1).to_pytimedelta(),
             Timedelta(days=1).to_timedelta64().astype("timedelta64[s]"),
             Timedelta(days=1).to_timedelta64().astype("timedelta64[ms]"),
@@ -235,7 +235,9 @@ def test_numeric_arr_mul_tdscalar(self, scalar_td, numeric_idx, box_with_array):
         # GH#19333
         box = box_with_array
         index = numeric_idx
-        expected = TimedeltaIndex([Timedelta(days=n) for n in range(len(index))])
+        expected = TimedeltaIndex(
+            [Timedelta(days=n) for n in range(len(index))], dtype="m8[ns]"
+        )
         if isinstance(scalar_td, np.timedelta64):
             dtype = scalar_td.dtype
             expected = expected.astype(dtype)
@@ -254,9 +256,9 @@ def test_numeric_arr_mul_tdscalar(self, scalar_td, numeric_idx, box_with_array):
     [USER_REDACTED].mark.parametrize(
         "scalar_td",
         [
-            Timedelta(days=1),
-            Timedelta(days=1).to_timedelta64(),
-            Timedelta(days=1).to_pytimedelta(),
+            Timedelta(days=1).as_unit("ns"),
+            Timedelta(days=1).as_unit("ns").to_timedelta64(),
+            Timedelta(days=1).as_unit("ns").to_pytimedelta(),
         ],
         ids=lambda x: type(x).__name__,
     )

@@ -1255,8 +1255,14 @@ def test_td64arr_add_sub_tdi(self, box_with_array, names):

         tdi = TimedeltaIndex(["0 days", "1 day"], name=names[1])
         tdi = np.array(tdi) if box in [tm.to_array, pd.array] else tdi
-        ser = Series([Timedelta(hours=3), Timedelta(hours=4)], name=names[0])
-        expected = Series([Timedelta(hours=3), Timedelta(days=1, hours=4)], name=exname)
+        ser = Series(
+            [Timedelta(hours=3), Timedelta(hours=4)], name=names[0], dtype="m8[ns]"
+        )
+        expected = Series(
+            [Timedelta(hours=3), Timedelta(days=1, hours=4)],
+            name=exname,
+            dtype="m8[ns]",
+        )

         ser = tm.box_expected(ser, box)
         expected = tm.box_expected(expected, box)
@@ -1270,7 +1276,9 @@ def test_td64arr_add_sub_tdi(self, box_with_array, names):
         assert_dtype(result, "timedelta64[ns]")

         expected = Series(
-            [Timedelta(hours=-3), Timedelta(days=1, hours=-4)], name=exname
+            [Timedelta(hours=-3), Timedelta(days=1, hours=-4)],
+            name=exname,
+            dtype="m8[ns]",
         )
         expected = tm.box_expected(expected, box)

@@ -1666,7 +1674,7 @@ def test_td64arr_mul_masked(self, dtype, box_with_array):
         other = Series(np.arange(5), dtype=dtype)
         other = tm.box_expected(other, box_with_array)

-        expected = Series([Timedelta(hours=n**2) for n in range(5)])
+        expected = Series([Timedelta(hours=n**2) for n in range(5)], dtype="m8[ns]")
         expected = tm.box_expected(expected, box_with_array)
         if dtype == "int64[pyarrow]":
             expected = expected.astype("duration[ns][pyarrow]")

@@ -109,7 +109,9 @@ def test_sum_2d_skipna_false(self):
         assert result is pd.NaT

         result = tda.sum(axis=0, skipna=False)
-        expected = pd.TimedeltaIndex([Timedelta(seconds=12), pd.NaT])._values
+        expected = pd.TimedeltaIndex(
+            [Timedelta(seconds=12), pd.NaT], dtype="m8[ns]"
+        )._values
         tm.assert_timedelta_array_equal(result, expected)

         result = tda.sum(axis=1, skipna=False)
@@ -119,7 +121,8 @@ def test_sum_2d_skipna_false(self):
                 Timedelta(seconds=5),
                 Timedelta(seconds=9),
                 pd.NaT,
-            ]
+            ],
+            dtype="m8[ns]",
         )._values
         tm.assert_timedelta_array_equal(result, expected)

@@ -359,15 +359,15 @@ def test_value_counts_object_inference_deprecated():
                 + [Timestamp("2016-01-02")]
                 + [Timestamp("2016-01-01") + Timedelta(days=i) for i in range(1, 5)]
             ),
-            DatetimeIndex(pd.date_range("2016-01-01", periods=5, freq="D", unit="ns")),
+            DatetimeIndex(pd.date_range("2016-01-01", periods=5, freq="D", unit="us")),
         ],
         [
             TimedeltaIndex(
                 [Timedelta(hours=i) for i in range(1)]
                 + [Timedelta(hours=1)]
                 + [Timedelta(hours=i) for i in range(1, 5)],
             ),
-            TimedeltaIndex(pd.timedelta_range(Timedelta(0), periods=5, freq="h")),
+            TimedeltaIndex(pd.timedelta_range(Timedelta(hours=0), periods=5, freq="h")),
         ],
         [
             DatetimeIndex(

@@ -55,7 +55,8 @@ def test_sub_datetime_preserves_freq_across_dst(self):
                 Timedelta(days=1),
                 Timedelta(days=2),
                 Timedelta(days=2, hours=23),
-            ]
+            ],
+            dtype="m8[ns]",
         )
         tm.assert_index_equal(res, expected)
         assert res.freq == expected.freq

@@ -104,7 +104,9 @@ def test_float64_ns_rounded(self):
     def test_float64_unit_conversion(self):
         # GH#23539
         tdi = to_timedelta([1.5, 2.25], unit="D")
-        expected = TimedeltaIndex([Timedelta(days=1.5), Timedelta(days=2.25)])
+        expected = TimedeltaIndex(
+            [Timedelta(days=1.5), Timedelta(days=2.25)], dtype="m8[ns]"
+        )
         tm.assert_index_equal(tdi, expected)

     def test_construction_base_constructor(self):

@@ -10,29 +10,29 @@ class TestAsUnit:
     def test_as_unit(self):
         td = Timedelta(days=1)

-        assert td.as_unit("ns") is td
+        assert td.as_unit("us") is td

-        res = td.as_unit("us")
-        assert res._value == td._value // 1000
-        assert res._creso == NpyDatetimeUnit.NPY_FR_us.value
+        res = td.as_unit("ns")
+        assert res._value == td._value * 1000
+        assert res._creso == NpyDatetimeUnit.NPY_FR_ns.value

-        rt = res.as_unit("ns")
+        rt = res.as_unit("us")
         assert rt._value == td._value
         assert rt._creso == td._creso

         res = td.as_unit("ms")
-        assert res._value == td._value // 1_000_000
+        assert res._value == td._value // 1_000
         assert res._creso == NpyDatetimeUnit.NPY_FR_ms.value

-        rt = res.as_unit("ns")
+        rt = res.as_unit("us")
         assert rt._value == td._value
         assert rt._creso == td._creso

         res = td.as_unit("s")
-        assert res._value == td._value // 1_000_000_000
+        assert res._value == td._value // 1_000_000
         assert res._creso == NpyDatetimeUnit.NPY_FR_s.value

-        rt = res.as_unit("ns")
+        rt = res.as_unit("us")
         assert rt._value == td._value
         assert rt._creso == td._creso

@@ -818,11 +818,11 @@ def test_mod_numeric(self):
         assert isinstance(result, Timedelta)
         assert result == Timedelta(0)

-        result = td % 1e12
+        result = td % 1e9
         assert isinstance(result, Timedelta)
         assert result == Timedelta(minutes=3, seconds=20)

-        result = td % int(1e12)
+        result = td % int(1e9)
         assert isinstance(result, Timedelta)
         assert result == Timedelta(minutes=3, seconds=20)

@@ -876,8 +876,8 @@ def test_divmod_numeric(self):
         # GH#19365
         td = Timedelta(days=2, hours=6)

-        result = divmod(td, 53 * 3600 * 1e9)
-        assert result[0] == Timedelta(1, unit="ns")
+        result = divmod(td, 53 * 3600 * 1e6)
+        assert result[0] == Timedelta(1, unit="us").as_unit("us")
         assert isinstance(result[1], Timedelta)
         assert result[1] == Timedelta(hours=1)

@@ -19,6 +19,21 @@
 import pandas._testing as tm

+class TestTimedeltaConstructorKeywordBased:
+    # Tests for constructing a Timedelta from keywords like the pytimedelta
+    # base class
+    def test_nanosecond_keyword(self):
+        # GH#63216
+        td = Timedelta(nanoseconds=1000)
+        assert td.unit == "ns"
+
+    def test_noninteger_microseconds(self):
+        # GH#63216
+        td = Timedelta(microseconds=1.5)
+        assert td.unit == "ns"
+        assert td == Timedelta(nanoseconds=1500)
+
+
 class TestTimedeltaConstructorUnitKeyword:
     [USER_REDACTED].mark.parametrize("unit", ["Y", "y", "M"])
     def test_unit_m_y_raises(self, unit):
@@ -272,14 +287,14 @@ def test_construction():
     assert Timedelta(10, unit="D")._value == expected
     assert Timedelta(10.0, unit="D")._value == expected
     assert Timedelta("10 days")._value == expected // 1000
-    assert Timedelta(days=10)._value == expected
-    assert Timedelta(days=10.0)._value == expected
+    assert Timedelta(days=10)._value == expected // 1000
+    assert Timedelta(days=10.0)._value == expected // 1000

     expected += np.timedelta64(10, "s").astype("m8[ns]").view("i8")
     assert Timedelta("10 days 00:00:10")._value == expected // 1000
-    assert Timedelta(days=10, seconds=10)._value == expected
-    assert Timedelta(days=10, milliseconds=10 * 1000)._value == expected
-    assert Timedelta(days=10, microseconds=10 * 1000 * 1000)._value == expected
+    assert Timedelta(days=10, seconds=10)._value == expected // 1000
+    assert Timedelta(days=10, milliseconds=10 * 1000)._value == expected // 1000
+    assert Timedelta(days=10, microseconds=10 * 1000 * 1000)._value == expected // 1000

     # rounding cases
     assert Timedelta(82739999850000)._value == 82739999850000
@@ -411,7 +426,7 @@ def test_construction():
 def test_td_construction_with_np_dtypes(npdtype, item):
     # GH#8757: test construction with np dtypes
     pykwarg, npkwarg = item
-    expected = np.timedelta64(1, npkwarg).astype("m8[ns]").view("i8")
+    expected = np.timedelta64(1, npkwarg).astype("m8[us]").view("i8")
     assert Timedelta(**{pykwarg: npdtype(1)})._value == expected

@@ -665,7 +665,7 @@ def test_resolution_deprecated(self):
         # GH#21344
         td = Timedelta(days=4, hours=3)
         result = td.resolution
-        assert result == Timedelta(nanoseconds=1)
+        assert result == Timedelta(microseconds=1)

         # Check that the attribute is available on the class, mirroring
         #  the stdlib timedelta behavior

@@ -145,12 +145,12 @@ def test_apply_out_of_range(request, tz_naive_fixture, _offset):
         elif (
             isinstance(tz, tzlocal)
             and is_platform_windows()
-            and _offset in (QuarterEnd, BQuarterBegin, BQuarterEnd)
+            and _offset in (QuarterEnd, BQuarterBegin, BQuarterEnd, FY5253Quarter)
         ):
             request.applymarker(
                 pytest.mark.xfail(reason="After GH#49737 t.tzinfo is None on CI")
             )
-        assert str(t.tzinfo) == str(result.tzinfo)
+        assert str(t.tzinfo) == str(result.tzinfo), (t.tzinfo, result.tzinfo)

     except OutOfBoundsDatetime:
         pass

Looks good!

The actual Timedelta constructor with kwargs, is that tested much?
I was checking `pandas/tests/scalar/timedelta/test_constructors.py`, and there is `test_td_constructor_on_nanoseconds`, but maybe you could at least add an explicit test for `Timedelta(nanoseconds=1000)` still having nanoseconds, as well as something like `Timedelta(microseconds=1.5)` to cover the check whether to convert to microseconds or not?

We should probably create some todo to try to parse to microseconds if this is out of bounds for nanoseconds?

I was wondering why this would be needed, but I suppose it is because the expected will have ns. But now the array inference PR is merged, this change can be (or needs to be) reverted?


API: partial revert of returning read-only EAs from .array/.values

See https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099 for context.

This reverts the behaviour change of https://github.com/[USER_REDACTED]ndas-dev/pandas/pull/61925 partially, to at least ensure that `.array` and `.values` still returns the underlying ExtensionArray (if the Series is backed by [NAME_REDACTED]). Other ways to get a numpy array are still left to return a read-only ndarray for now.

@@ -5023,8 +5023,9 @@ def array(self) -> ExtensionArray:
             from pandas.core.arrays.numpy_ import NumpyExtensionArray

             array = NumpyExtensionArray(array)
-        array = array.view()
-        array._readonly = True
+        # TODO decide on read-only https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099
+        # array = array.view()
+        # array._readonly = True
         return array

     [USER_REDACTED]

@@ -2387,7 +2387,9 @@ def external_values(values: ArrayLike) -> ArrayLike:
         values.flags.writeable = False
     else:
         # ExtensionArrays
-        values = values.view()
-        values._readonly = True
+        # TODO decide on read-only https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099
+        # values = values.view()
+        # values._readonly = True
+        pass

     return values

@@ -821,8 +821,9 @@ def _references(self) -> BlockValuesRefs:
     [USER_REDACTED]
     def array(self) -> ExtensionArray:
         arr = self._mgr.array_values()
-        arr = arr.view()
-        arr._readonly = True
+        # TODO decide on read-only https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099
+        # arr = arr.view()
+        # arr._readonly = True
         return arr

     def __len__(self) -> int:

@@ -25,12 +25,21 @@
     ],
     ids=["values", "array", "np.asarray", "np.array"],
 )
-def test_series_values(method):
+def test_series_values(request, method):
     ser = Series([1, 2, 3], name="name")
     ser_orig = ser.copy()

     arr = method(ser)

+    if request.node.callspec.id == "array":
+        # https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099
+        # .array for now does not return a read-only view
+        assert arr.flags.writeable is True
+        # updating the array updates the series
+        arr[0] = 0
+        assert ser.iloc[0] == 0
+        return
+
     # .values still gives a view but is read-only
     assert np.shares_memory(arr, get_array(ser, "name"))
     assert arr.flags.writeable is False
@@ -109,20 +118,42 @@ def test_series_to_numpy():
 [USER_REDACTED].mark.parametrize(
     "method",
     [
+        lambda ser: np.asarray(ser.values),
         lambda ser: np.asarray(ser.array),
         lambda ser: np.asarray(ser),
         lambda ser: np.asarray(ser, dtype="int64"),
         lambda ser: np.array(ser, copy=False),
     ],
-    ids=["array", "np.asarray", "np.asarray-dtype", "np.array"],
+    ids=["values", "array", "np.asarray", "np.asarray-dtype", "np.array"],
 )
-def test_series_values_ea_dtypes(method):
+def test_series_values_ea_dtypes(request, method):
     ser = Series([1, 2, 3], dtype="Int64")
+    ser_orig = ser.copy()
+
     arr = method(ser)

+    if request.node.callspec.id in ("values", "array"):
+        # https://github.com/[USER_REDACTED]ndas-dev/pandas/issues/63099
+        # .array/values for now does not return a read-only view
+        assert arr.flags.writeable is True
+        # updating the array updates the series
+        arr[0] = 0
+        assert ser.iloc[0] == 0
+        return
+
+    # conversion to ndarray gives a view but is read-only
     assert np.shares_memory(arr, get_array(ser))
     assert arr.flags.writeable is False

+    # mutating series through arr therefore doesn't work
+    with pytest.raises(ValueError, match="read-only"):
+        arr[0] = 0
+    tm.assert_series_equal(ser, ser_orig)
+
+    # mutating the series itself still works
+    ser.iloc[0] = 0
+    assert ser.values[0] == 0
+

 [USER_REDACTED].mark.parametrize(
     "method",

is the idea to leave this commented-out indefinitely?  or will be cleaned up before merging?

id prefer to remove but not going to make a stink about it.  merge on green(ish)


removed [USER_REDACTED]

- [ ] fix #62437
Replace [USER_REDACTED] with specific code comments on these files.

- pandas/core/indexes/datetimelike.py
- pandas/core/indexes/datetimes.py

@@ -36,9 +36,7 @@
     NullFrequencyError,
 )
 from pandas.util._decorators import (
-    Appender,
     cache_readonly,
-    doc,
 )

 from pandas.core.dtypes.common import (
@@ -57,12 +55,10 @@
     PeriodArray,
     TimedeltaArray,
 )
-from pandas.core.arrays.datetimelike import DatetimeLikeArrayMixin
 import pandas.core.common as com
 import pandas.core.indexes.base as ibase
 from pandas.core.indexes.base import (
     Index,
-    _index_shared_docs,
 )
 from pandas.core.indexes.extension import NDArrayBackedExtensionIndex
 from pandas.core.indexes.range import RangeIndex
@@ -92,8 +88,51 @@ class DatetimeIndexOpsMixin(NDArrayBackedExtensionIndex, ABC):
     _can_hold_strings = False
     _data: DatetimeArray | TimedeltaArray | PeriodArray

-    [USER_REDACTED](DatetimeLikeArrayMixin.mean)
     def mean(self, *, skipna: bool = True, axis: int | None = 0):
+        """
+        Return the mean value of the Array.
+
+        Parameters
+        ----------
+        skipna : bool, default True
+            Whether to ignore any NaT elements.
+        axis : int, optional, default 0
+            Axis for the function to be applied on.
+
+        Returns
+        -------
+        scalar
+            Timestamp or Timedelta.
+
+        See Also
+        --------
+        numpy.ndarray.mean : Returns the average of array elements along a given axis.
+        Series.mean : Return the mean value in a Series.
+
+        Notes
+        -----
+        mean is only defined for Datetime and Timedelta dtypes, not for Period.
+
+        Examples
+        --------
+        For :class:`pandas.DatetimeIndex`:
+
+        >>> idx = pd.date_range("2001-01-01 00:00", periods=3)
+        >>> idx
+        DatetimeIndex(['2001-01-01', '2001-01-02', '2001-01-03'],
+                      dtype='datetime64[us]', freq='D')
+        >>> idx.mean()
+        Timestamp('2001-01-02 00:00:00')
+
+        For :class:`pandas.TimedeltaIndex`:
+
+        >>> tdelta_idx = pd.to_timedelta([1, 2, 3], unit="D")
+        >>> tdelta_idx
+        TimedeltaIndex(['1 days', '2 days', '3 days'],
+                        dtype='timedelta64[ns]', freq=None)
+        >>> tdelta_idx.mean()
+        Timedelta('2 days 00:00:00')
+        """
         return self._data.mean(skipna=skipna, axis=axis)

     [USER_REDACTED]
@@ -136,8 +175,37 @@ def asi8(self) -> npt.NDArray[np.int64]:
         return self._data.asi8

     [USER_REDACTED]
-    [USER_REDACTED](DatetimeLikeArrayMixin.freqstr)
     def freqstr(self) -> str:
+        """
+        Return the frequency object as a string if it's set, otherwise None.
+
+        See Also
+        --------
+        DatetimeIndex.inferred_freq : Returns a string representing a frequency
+            generated by infer_freq.
+
+        Examples
+        --------
+        For DatetimeIndex:
+
+        >>> idx = pd.DatetimeIndex(["1/1/2020 10:00:00+00:00"], freq="D")
+        >>> idx.freqstr
+        'D'
+
+        The frequency can be inferred if there are more than 2 points:
+
+        >>> idx = pd.DatetimeIndex(
+        ...     ["2018-01-01", "2018-01-03", "2018-01-05"], freq="infer"
+        ... )
+        >>> idx.freqstr
+        '2D'
+
+        For PeriodIndex:
+
+        >>> idx = pd.PeriodIndex(["2023-1", "2023-2", "2023-3"], freq="M")
+        >>> idx.freqstr
+        'M'
+        """
         from pandas import PeriodIndex

         if self._data.freqstr is not None and isinstance(
@@ -153,8 +221,10 @@ def freqstr(self) -> str:
     def _resolution_obj(self) -> Resolution: ...

     [USER_REDACTED]_readonly
-    [USER_REDACTED](DatetimeLikeArrayMixin.resolution)
     def resolution(self) -> str:
+        """
+        Returns day, hour, minute, second, millisecond or microsecond
+        """
         return self._data.resolution

     # ------------------------------------------------------------------------
@@ -199,8 +269,40 @@ def equals(self, other: Any) -> bool:

         return np.array_equal(self.asi8, other.asi8)

-    [USER_REDACTED](Index.__contains__.__doc__)
     def __contains__(self, key: Any) -> bool:
+        """
+        Return a boolean indicating whether the provided key is in the index.
+
+        Parameters
+        ----------
+        key : label
+            The key to check if it is present in the index.
+
+        Returns
+        -------
+        bool
+            Whether the key search is in the index.
+
+        Raises
+        ------
+        TypeError
+            If the key is not hashable.
+
+        See Also
+        --------
+        Index.isin : Returns an ndarray of boolean dtype indicating whether the
+            list-like key is in the index.
+
+        Examples
+        --------
+        >>> idx = pd.Index([1, 2, 3, 4])
+        >>> idx
+        Index([1, 2, 3, 4], dtype='int64')
+        >>> 2 in idx
+        True
+        >>> 6 in idx
+        False
+        """
         hash(key)
         try:
             self.get_loc(key)
@@ -243,8 +345,19 @@ def _format_attrs(self):
                 attrs.append(("freq", freq))
         return attrs

-    [USER_REDACTED](Index._summary.__doc__)
     def _summary(self, name=None) -> str:
+        """
+        Return a summarized representation.
+
+        Parameters
+        ----------
+        name : str
+            name to use in the summary representation
+
+        Returns
+        -------
+        String with a summarized representation of the index
+        """
         result = super()._summary(name=name)
         if self.freq:
             result += f"\nFreq: {self.freqstr}"
@@ -405,8 +518,10 @@ def shift(self, periods: int = 1, freq=None) -> Self:

     # --------------------------------------------------------------------

-    [USER_REDACTED](Index._maybe_cast_listlike_indexer)
     def _maybe_cast_listlike_indexer(self, keyarr):
+        """
+        Analogue to maybe_cast_indexer for get_indexer instead of get_loc.
+        """
         try:
             res = self._data._validate_listlike(keyarr, allow_object=True)
         except (ValueError, TypeError):
@@ -497,8 +612,32 @@ def values(self) -> np.ndarray:
         data.flags.writeable = False
         return data

-    [USER_REDACTED](DatetimeIndexOpsMixin.shift)
     def shift(self, periods: int = 1, freq=None) -> Self:
+        """
+        Shift index by [NAME_REDACTED] frequency increments.
+        This method is for shifting the values of datetime-like indexes
+        by a specified time increment a given number of times.
+
+        Parameters
+        ----------
+        periods : int, default 1
+            Number of periods (or increments) to shift by,
+            can be positive or negative.
+        freq : pandas.DateOffset, pandas.Timedelta or string, optional
+            Frequency increment to shift by.
+            If None, the index is shifted by [NAME_REDACTED] `freq` attribute.
+            Offset aliases are valid strings, e.g., 'D', 'W', 'M' etc.
+
+        Returns
+        -------
+        pandas.DatetimeIndex
+            Shifted index.
+
+        See Also
+        --------
+        Index.shift : Shift values of Index.
+        PeriodIndex.shift : Shift values of PeriodIndex.
+        """
         if freq is not None and freq != self.freq:
             if isinstance(freq, str):
                 freq = to_offset(freq)
@@ -524,8 +663,38 @@ def shift(self, periods: int = 1, freq=None) -> Self:
         return type(self)._simple_new(result, name=self.name)

     [USER_REDACTED]_readonly
-    [USER_REDACTED](DatetimeLikeArrayMixin.inferred_freq)
     def inferred_freq(self) -> str | None:
+        """
+        Return the inferred frequency of the index.
+
+        Returns
+        -------
+        str or None
+            A string representing a frequency generated by ``infer_freq``.
+            Returns ``None`` if the frequency cannot be inferred.
+
+        See Also
+        --------
+        DatetimeIndex.freqstr : Return the frequency object as a string if it's set,
+            otherwise ``None``.
+
+        Examples
+        --------
+        For ``DatetimeIndex``:
+
+        >>> idx = pd.DatetimeIndex(["2018-01-01", "2018-01-03", "2018-01-05"])
+        >>> idx.inferred_freq
+        '2D'
+
+        For ``TimedeltaIndex``:
+
+        >>> tdelta_idx = pd.to_timedelta(["0 days", "10 days", "20 days"])
+        >>> tdelta_idx
+        TimedeltaIndex(['0 days', '10 days', '20 days'],
+                       dtype='timedelta64[ns]', freq=None)
+        >>> tdelta_idx.inferred_freq
+        '10D'
+        """
         return self._data.inferred_freq

     # --------------------------------------------------------------------
@@ -816,14 +985,66 @@ def _get_insert_freq(self, loc: int, item):
                     freq = self.freq
         return freq

-    [USER_REDACTED](NDArrayBackedExtensionIndex.delete)
     def delete(self, loc) -> Self:
+        """
+        Make new Index with passed location(-s) deleted.
+
+        Parameters
+        ----------
+        loc : int or list of int
+            Location of item(-s) which will be deleted.
+            Use a list of locations to delete more than one value at the same time.
+
+        Returns
+        -------
+        Index
+            Will be same type as self, except for RangeIndex.
+
+        See Also
+        --------
+        numpy.delete : Delete any rows and column from NumPy array (ndarray).
+
+        Examples
+        --------
+        >>> idx = pd.Index(["a", "b", "c"])
+        >>> idx.delete(1)
+        Index(['a', 'c'], dtype='str')
+        >>> idx = pd.Index(["a", "b", "c"])
+        >>> idx.delete([0, 2])
+        Index(['b'], dtype='str')
+        """
         result = super().delete(loc)
         result._data._freq = self._get_delete_freq(loc)
         return result

-    [USER_REDACTED](NDArrayBackedExtensionIndex.insert)
     def insert(self, loc: int, item):
+        """
+        Make new Index inserting new item at location.
+        [NAME_REDACTED] numpy.insert semantics for negative values.
+
+        Parameters
+        ----------
+        loc : int
+            The integer location where the new item will be inserted.
+        item : object
+            The new item to be inserted into the Index.
+
+        Returns
+        -------
+        Index
+            Returns a new Index object resulting from inserting the specified item at
+            the specified location within the original Index.
+
+        See Also
+        --------
+        Index.append : Append a collection of Indexes together.
+
+        Examples
+        --------
+        >>> idx = pd.Index(["a", "b", "c"])
+        >>> idx.insert(1, "x")
+        Index(['a', 'x', 'b', 'c'], dtype='str')
+        """
         result = super().insert(loc, item)
         if isinstance(result, type(self)):
             # i.e. parent class method did not cast
@@ -833,7 +1054,6 @@ def insert(self, loc: int, item):
     # --------------------------------------------------------------------
     # NDArray-[NAME_REDACTED]

-    [USER_REDACTED](_index_shared_docs["take"] % _index_doc_kwargs)
     def take(
         self,
         indices,
@@ -842,6 +1062,47 @@ def take(
         fill_value=None,
         **kwargs,
     ) -> Self:
+        """
+        Return a new Index of the values selected by [NAME_REDACTED]
+        For internal compatibility with numpy arrays.
+
+        Parameters
+        ----------
+        indices : array-like
+            Indices to be taken.
+        axis : {0 or 'index'}, optional
+            The axis over which to select values, always 0 or 'index'.
+        allow_fill : bool, default True
+            How to handle negative values in `indices`.
+            * False: negative values in `indices` indicate positional indices
+                from the right (the default). This is similar to
+                :func:`numpy.take`.
+            * True: negative values in `indices` indicate
+                missing values. These values are set to `fill_value`. Any other
+                other negative values raise a ``ValueError``.
+        fill_value : scalar, default None
+            If allow_fill=True and fill_value is not None, indices specified by
+            -1 are regarded as NA. If Index doesn't hold NA, raise ValueError.
+        **kwargs
+            Required for compatibility with numpy.
+
+        Returns
+        -------
+        Index
+            An index formed of elements at the given indices. Will be the same
+            type as self, except for RangeIndex.
+
+        See Also
+        --------
+        numpy.ndarray.take: Return an array formed from the
+            elements of a at the given indices.
+
+        Examples
+        --------
+        >>> idx = pd.Index(["a", "b", "c"])
+        >>> idx.take([2, 2, 1, 2])
+        Index(['c', 'c', 'b', 'c'], dtype='str')
+        """
         nv.validate_take((), kwargs)
         indices = np.asarray(indices, dtype=np.intp)

@@ -33,7 +33,6 @@
 from pandas.errors import Pandas4Warning
 from pandas.util._decorators import (
     cache_readonly,
-    doc,
     set_module,
 )
 from pandas.util._exceptions import find_stack_level
@@ -275,40 +274,382 @@ def _engine_type(self) -> type[libindex.DatetimeEngine]:
     # --------------------------------------------------------------------
     # methods that dispatch to DatetimeArray and wrap result

-    [USER_REDACTED](DatetimeArray.strftime)
     def strftime(self, date_format) -> Index:
+        """
+        Convert to Index using specified date_format.
+
+        Return an Index of formatted strings specified by date_format, which
+        supports the same string format as the python standard library. Details
+        of the string format can be found in `python string format
+        doc <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>`__.
+
+        Formats supported by the C `strftime` API but not by [NAME_REDACTED]
+        doc (such as `"%R"`, `"%r"`) are not officially supported and should be
+        preferably replaced with their supported equivalents (such as `"%H:%M"`,
+        `"%I:%M:%S %p"`).
+        Note that `PeriodIndex` support additional directives, detailed in
+        `Period.strftime`.
+
+        Parameters
+        ----------
+        date_format : str
+            Date format string (e.g. "%Y-%m-%d").
+
+        Returns
+        -------
+        ndarray[object]
+            NumPy ndarray of formatted strings.
+
+        See Also
+        --------
+        to_datetime : Convert the given argument to datetime.
+        DatetimeIndex.normalize : [NAME_REDACTED] with times to midnight.
+        DatetimeIndex.round : Round the DatetimeIndex to the specified freq.
+        DatetimeIndex.floor : Floor the DatetimeIndex to the specified freq.
+        Timestamp.strftime : Format a single Timestamp.
+        Period.strftime : Format a single Period.
+
+        Examples
+        --------
+        >>> rng = pd.date_range(pd.Timestamp("2018-03-10 09:00"), periods=3, freq="s")
+        >>> rng.strftime("%B %d, %Y, %r")
+        Index(['March 10, 2018, 09:00:00 AM', 'March 10, 2018, 09:00:01 AM',
+               'March 10, 2018, 09:00:02 AM'],
+                    dtype='str')
+        """
         arr = self._data.strftime(date_format)
         return Index(arr, name=self.name, dtype=arr.dtype)

-    [USER_REDACTED](DatetimeArray.tz_convert)
     def tz_convert(self, tz) -> Self:
+        """
+        Convert tz-aware [NAME_REDACTED]/Index from one time zone to another.
+
+        Parameters
+        ----------
+        tz : str, zoneinfo.ZoneInfo, pytz.timezone, dateutil.tz.tzfile, datetime.tzinfo or None
+            Time zone for time. Corresponding timestamps would be converted
+            to this time zone of the [NAME_REDACTED]/Index. A `tz` of None will
+            convert to UTC and remove the timezone information.
+
+        Returns
+        -------
+        Array or Index
+            [NAME_REDACTED]/Index with target `tz`.
+
+        Raises
+        ------
+        TypeError
+            If Datetime Array/Index is tz-naive.
+
+        See Also
+        --------
+        DatetimeIndex.tz : A timezone that has a variable offset from UTC.
+        DatetimeIndex.tz_localize : Localize tz-naive DatetimeIndex to a
+            given time zone, or remove timezone from a tz-aware DatetimeIndex.
+
+        Examples
+        --------
+        With the `tz` parameter, we can change the DatetimeIndex
+        to other time zones:
+
+        >>> dti = pd.date_range(
+        ...     start="2014-08-01 09:00", freq="h", periods=3, tz="Europe/Berlin"
+        ... )
+
+        >>> dti
+        DatetimeIndex(['2014-08-01 09:00:00+02:00',
+                       '2014-08-01 10:00:00+02:00',
+                       '2014-08-01 11:00:00+02:00'],
+                      dtype='datetime64[ns, Europe/Berlin]', freq='h')
+
+        >>> dti.tz_convert("US/Central")
+        DatetimeIndex(['2014-08-01 02:00:00-05:00',
+                       '2014-08-01 03:00:00-05:00',
+                       '2014-08-01 04:00:00-05:00'],
+                      dtype='datetime64[ns, US/Central]', freq='h')
+
+        With the ``tz=None``, we can remove the timezone (after converting
+        to UTC if necessary):
+
+        >>> dti = pd.date_range(
+        ...     start="2014-08-01 09:00", freq="h", periods=3, tz="Europe/Berlin"
+        ... )
+
+        >>> dti
+        DatetimeIndex(['2014-08-01 09:00:00+02:00',
+                       '2014-08-01 10:00:00+02:00',
+                       '2014-08-01 11:00:00+02:00'],
+                        dtype='datetime64[ns, Europe/Berlin]', freq='h')
+
+        >>> dti.tz_convert(None)
+        DatetimeIndex(['2014-08-01 07:00:00',
+                       '2014-08-01 08:00:00',
+                       '2014-08-01 09:00:00'],
+                        dtype='datetime64[ns]', freq='h')
+        """  # noqa: E501
         arr = self._data.tz_convert(tz)
         return type(self)._simple_new(arr, name=self.name, refs=self._references)

-    [USER_REDACTED](DatetimeArray.tz_localize)
     def tz_localize(
         self,
         tz,
         ambiguous: TimeAmbiguous = "raise",
         nonexistent: TimeNonexistent = "raise",
     ) -> Self:
+        """
+        Localize tz-naive [NAME_REDACTED]/Index to tz-aware [NAME_REDACTED]/Index.
+
+        This method takes a time zone (tz) naive [NAME_REDACTED]/Index object
+        and makes this time zone aware. It does not move the time to another
+        time zone.
+
+        This method can also be used to do the inverse -- to create a time
+        zone unaware object from an aware object. To that end, pass `tz=None`.
+
+        Parameters
+        ----------
+        tz : str, zoneinfo.ZoneInfo,, pytz.timezone, dateutil.tz.tzfile, datetime.tzinfo or None
+            Time zone to convert timestamps to. Passing ``None`` will
+            remove the time zone information preserving local time.
+        ambiguous : 'infer', 'NaT', bool array, default 'raise'
+            When clocks moved backward due to DST, ambiguous times may arise.
+            For example in [NAME_REDACTED] (UTC+01), when going from
+            03:00 DST to 02:00 non-DST, 02:30:00 local time occurs both at
+            00:30:00 UTC and at 01:30:00 UTC. In such a situation, the
+            `ambiguous` parameter dictates how ambiguous times should be
+            handled.
+
+            - 'infer' will attempt to infer fall dst-transition hours based on
+              order
+            - bool-ndarray where True signifies a DST time, False signifies a
+              non-DST time (note that this flag is only applicable for
+              ambiguous times)
+            - 'NaT' will return NaT where there are ambiguous times
+            - 'raise' will raise a ValueError if there are ambiguous
+              times.
+
+        nonexistent : 'shift_forward', 'shift_backward, 'NaT', timedelta, \
+        default 'raise'
+            A nonexistent time does not exist in a particular timezone
+            where clocks moved forward due to DST.
+
+            - 'shift_forward' will shift the nonexistent time forward to the
+              closest existing time
+            - 'shift_backward' will shift the nonexistent time backward to the
+              closest existing time
+            - 'NaT' will return NaT where there are nonexistent times
+            - timedelta objects will shift nonexistent times by [NAME_REDACTED]
+            - 'raise' will raise a ValueError if there are
+              nonexistent times.
+
+        Returns
+        -------
+        Same type as self
+            Array/Index converted to the specified time zone.
+
+        Raises
+        ------
+        TypeError
+            If the [NAME_REDACTED]/Index is tz-aware and tz is not None.
+
+        See Also
+        --------
+        DatetimeIndex.tz_convert : Convert tz-aware DatetimeIndex from
+            one time zone to another.
+
+        Examples
+        --------
+        >>> tz_naive = pd.date_range('2018-03-01 09:00', periods=3)
+        >>> tz_naive
+        DatetimeIndex(['2018-03-01 09:00:00', '2018-03-02 09:00:00',
+                       '2018-03-03 09:00:00'],
+                      dtype='datetime64[ns]', freq='D')
+
+        [NAME_REDACTED] in US/Eastern time zone:
+
+        >>> tz_aware = tz_naive.tz_localize(tz='US/Eastern')
+        >>> tz_aware
+        DatetimeIndex(['2018-03-01 09:00:00-05:00',
+                       '2018-03-02 09:00:00-05:00',
+                       '2018-03-03 09:00:00-05:00'],
+                      dtype='datetime64[ns, US/Eastern]', freq=None)
+
+        With the ``tz=None``, we can remove the time zone information
+        while keeping the local time (not converted to UTC):
+
+        >>> tz_aware.tz_localize(None)
+        DatetimeIndex(['2018-03-01 09:00:00', '2018-03-02 09:00:00',
+                       '2018-03-03 09:00:00'],
+                      dtype='datetime64[ns]', freq=None)
+
+        Be careful with DST changes. When there is sequential data, pandas can
+        infer the DST time:
+
+        >>> s = pd.to_datetime(pd.Series(['2018-10-28 01:30:00',
+        ...                               '2018-10-28 02:00:00',
+        ...                               '2018-10-28 02:30:00',
+        ...                               '2018-10-28 02:00:00',
+        ...                               '2018-10-28 02:30:00',
+        ...                               '2018-10-28 03:00:00',
+        ...                               '2018-10-28 03:30:00']))
+        >>> s.dt.tz_localize('CET', ambiguous='infer')
+        0   2018-10-28 01:30:00+02:00
+        1   2018-10-28 02:00:00+02:00
+        2   2018-10-28 02:30:00+02:00
+        3   2018-10-28 02:00:00+01:00
+        4   2018-10-28 02:30:00+01:00
+        5   2018-10-28 03:00:00+01:00
+        6   2018-10-28 03:30:00+01:00
+        dtype: datetime64[s, CET]
+
+        In some cases, inferring the DST is impossible. In such cases, you can
+        pass an ndarray to the ambiguous parameter to set the DST explicitly
+
+        >>> s = pd.to_datetime(pd.Series(['2018-10-28 01:20:00',
+        ...                               '2018-10-28 02:36:00',
+        ...                               '2018-10-28 03:46:00']))
+        >>> s.dt.tz_localize('CET', ambiguous=np.array([True, True, False]))
+        0   2018-10-28 01:20:00+02:00
+        1   2018-10-28 02:36:00+02:00
+        2   2018-10-28 03:46:00+01:00
+        dtype: datetime64[s, CET]
+
+        If the DST transition causes nonexistent times, you can shift these
+        dates forward or backwards with a timedelta object or `'shift_forward'`
+        or `'shift_backwards'`.
+
+        >>> s = pd.to_datetime(pd.Series(['2015-03-29 02:30:00',
+        ...                               '2015-03-29 03:30:00'], dtype="M8[ns]"))
+        >>> s.dt.tz_localize('Europe/Warsaw', nonexistent='shift_forward')
+        0   2015-03-29 03:00:00+02:00
+        1   2015-03-29 03:30:00+02:00
+        dtype: datetime64[ns, Europe/Warsaw]
+
+        >>> s.dt.tz_localize('Europe/Warsaw', nonexistent='shift_backward')
+        0   2015-03-29 01:59:59.999999999+01:00
+        1   2015-03-29 03:30:00+02:00
+        dtype: datetime64[ns, Europe/Warsaw]
+
+        >>> s.dt.tz_localize('Europe/Warsaw', nonexistent=pd.Timedelta('1h'))
+        0   2015-03-29 03:30:00+02:00
+        1   2015-03-29 03:30:00+02:00
+        dtype: datetime64[ns, Europe/Warsaw]
+        """  # noqa: E501
         arr = self._data.tz_localize(tz, ambiguous, nonexistent)
         return type(self)._simple_new(arr, name=self.name)

-    [USER_REDACTED](DatetimeArray.to_period)
     def to_period(self, freq=None) -> PeriodIndex:
+        """
+        Cast to PeriodArray/PeriodIndex at a particular frequency.
+
+        [NAME_REDACTED]/Index to PeriodArray/PeriodIndex.
+
+        Parameters
+        ----------
+        freq : str or Period, optional
+            One of pandas' :ref:`period aliases <timeseries.period_aliases>`
+            or a Period object. Will be inferred by default.
+
+        Returns
+        -------
+        PeriodArray/PeriodIndex
+            Immutable ndarray holding ordinal values at a particular frequency.
+
+        Raises
+        ------
+        ValueError
+            When converting a DatetimeArray/Index with non-regular values,
+            so that a frequency cannot be inferred.
+
+        See Also
+        --------
+        PeriodIndex: Immutable ndarray holding ordinal values.
+        DatetimeIndex.to_pydatetime: [NAME_REDACTED] as object.
+
+        Examples
+        --------
+        >>> df = pd.DataFrame(
+        ...     {"y": [1, 2, 3]},
+        ...     index=pd.to_datetime(
+        ...         [
+        ...             "2000-03-31 00:00:00",
+        ...             "2000-05-31 00:00:00",
+        ...             "2000-08-31 00:00:00",
+        ...         ]
+        ...     ),
+        ... )
+        >>> df.index.to_period("M")
+        PeriodIndex(['2000-03', '2000-05', '2000-08'],
+                    dtype='period[M]')
+
+        Infer the daily frequency
+
+        >>> idx = pd.date_range("2017-01-01", periods=2)
+        >>> idx.to_period()
+        PeriodIndex(['2017-01-01', '2017-01-02'],
+                    dtype='period[D]')
+        """
         from pandas.core.indexes.api import PeriodIndex

         arr = self._data.to_period(freq)
         return PeriodIndex._simple_new(arr, name=self.name)

-    [USER_REDACTED](DatetimeArray.to_julian_date)
     def to_julian_date(self) -> Index:
+        """
+        [NAME_REDACTED] to a [NAME_REDACTED].
+
+        This method returns the number of days as a float since noon January 1, 4713 BC.
+
+        https://en.wikipedia.org/wiki/Julian_day
+
+        Returns
+        -------
+        ndarray or Index
+            Float values that represent each date in [NAME_REDACTED].
+
+        See Also
+        --------
+        Timestamp.to_julian_date : Equivalent method on ``Timestamp`` objects.
+
+        Examples
+        --------
+        >>> idx = pd.DatetimeIndex(["2028-08-12 00:54", "2028-08-12 02:06"])
+        >>> idx.to_julian_date()
+        Index([2461995.5375, 2461995.5875], dtype='float64')
+        """
         arr = self._data.to_julian_date()
         return Index._simple_new(arr, name=self.name)

-    [USER_REDACTED](DatetimeArray.isocalendar)
     def isocalendar(self) -> DataFrame:
+        """
+        Calculate year, week, and day according to the ISO 8601 standard.
+        Returns
+        -------
+        DataFrame
+            With columns year, week and day.
+        See Also
+        --------
+        Timestamp.isocalendar : Function return a 3-tuple containing ISO year,
+            week number, and weekday for the given Timestamp object.
+        datetime.date.isocalendar : Return a named tuple object with
+            three components: year, week and weekday.
+
+        Examples
+        --------
+        >>> idx = pd.date_range(start="2019-12-29", freq="D", periods=4)
+        >>> idx.isocalendar()
+                    year  week  day
+        2019-12-29  2019    52    7
+        2019-12-30  2020     1    1
+        2019-12-31  2020     1    2
+        2020-01-01  2020     1    3
+        >>> idx.isocalendar().week
+        2019-12-29    52
+        2019-12-30     1
+        2019-12-31     1
+        2020-01-01     1
+        Freq: D, Name: week, dtype: UInt32
+        """
         df = self._data.isocalendar()
         return df.set_index(self)

@@ -645,8 +986,25 @@ def get_loc(self, key):
         except KeyError as err:
             raise KeyError(orig_key) from err

-    [USER_REDACTED](DatetimeTimedeltaMixin._maybe_cast_slice_bound)
     def _maybe_cast_slice_bound(self, label, side: str):
+        """
+        This function should be overloaded in subclasses that allow non-trivial
+        casting on label-slice bounds, e.g. datetime-like indices allowing
+        strings containing formatted datetimes.
+
+        Parameters
+        ----------
+        label : object
+        side : {'left', 'right'}
+
+        Returns
+        -------
+        label : object
+
+        Notes
+        -----
+        Value of `side` parameter should be validated in caller.
+        """
         # GH#42855 handle date here instead of get_slice_bound
         if isinstance(label, dt.date) and not isinstance(label, dt.datetime):
             # Pandas supports slicing with dates, treated as datetimes at midnight.

This docstring needs spaces between sections

These indentations look incorrect

Hi [USER_REDACTED], I think I missed it accidentally. I've already fixed it, thanks for the review.


BUG: Inconsistent behavior of Groupby with None values with filter (#…

…62501)

- [x] closes #62501
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.4.rst` file if fixing a bug or adding a new feature.

@@ -1422,6 +1422,7 @@ Other
 - Bug in :meth:`DataFrame.apply` where passing ``engine="numba"`` ignored ``args`` passed to the applied function (:issue:`58712`)
 - Bug in :meth:`DataFrame.eval` and :meth:`DataFrame.query` which caused an exception when using NumPy attributes via ``@`` notation, e.g., ``df.eval("[USER_REDACTED].floor(a)")``. (:issue:`58041`)
 - Bug in :meth:`DataFrame.eval` and :meth:`DataFrame.query` which did not allow to use ``tan`` function. (:issue:`55091`)
+- Bug in :meth:`DataFrame.groupby` with ``None`` values with filter (:issue:`62501`)
 - Bug in :meth:`DataFrame.query` where using duplicate column names led to a ``TypeError``. (:issue:`59950`)
 - Bug in :meth:`DataFrame.query` which raised an exception or produced incorrect results when expressions contained backtick-quoted column names containing the hash character ``#``, backticks, or characters that fall outside the ASCII range (U+0001..U+007F). (:issue:`59285`) (:issue:`49633`)
 - Bug in :meth:`DataFrame.query` which raised an exception when querying integer column names using backticks. (:issue:`60494`)

@@ -636,7 +636,7 @@ def indices(self) -> dict[Hashable, npt.NDArray[np.intp]]:
         return self._grouper.indices

     [USER_REDACTED]
-    def _get_indices(self, names):
+    def _get_index(self, name):
         """
         Safe get multiple indices, translate keys for
         datelike to underlying repr.
@@ -652,23 +652,24 @@ def get_converter(s):
             else:
                 return lambda key: key

-        if len(names) == 0:
-            return []
+        if isna(name):
+            return self.indices.get(np.nan, [])
+        if isinstance(name, tuple):
+            name = tuple(np.nan if isna(comp) else comp for comp in name)

         if len(self.indices) > 0:
             index_sample = next(iter(self.indices))
         else:
             index_sample = None  # Dummy sample

-        name_sample = names[0]
         if isinstance(index_sample, tuple):
-            if not isinstance(name_sample, tuple):
+            if not isinstance(name, tuple):
                 msg = "must supply a tuple to get_group with multiple grouping keys"
                 raise ValueError(msg)
-            if not len(name_sample) == len(index_sample):
+            if not len(name) == len(index_sample):
                 try:
                     # If the original grouper was a tuple
-                    return [self.indices[name] for name in names]
+                    return self.indices[name]
                 except KeyError as err:
                     # turns out it wasn't a tuple
                     msg = (
@@ -678,23 +679,12 @@ def get_converter(s):
                     raise ValueError(msg) from err

             converters = (get_converter(s) for s in index_sample)
-            names = (
-                tuple(f(n) for f, n in zip(converters, name, strict=True))
-                for name in names
-            )
-
+            name = tuple(f(n) for f, n in zip(converters, name, strict=True))
         else:
             converter = get_converter(index_sample)
-            names = (converter(name) for name in names)
-
-        return [self.indices.get(name, []) for name in names]
+            name = converter(name)

-    [USER_REDACTED]
-    def _get_index(self, name):
-        """
-        Safe get index, translate keys for datelike to underlying repr.
-        """
-        return self._get_indices([name])[0]
+        return self.indices.get(name, [])

     [USER_REDACTED]
     [USER_REDACTED]_readonly

@@ -652,9 +652,25 @@ def indices(self) -> dict[Hashable, npt.NDArray[np.intp]]:
         """dict {group name -> group indices}"""
         if len(self.groupings) == 1 and isinstance(self.result_index, CategoricalIndex):
             # This shows unused categories in indices GH#38642
-            return self.groupings[0].indices
-        codes_list = [ping.codes for ping in self.groupings]
-        return get_indexer_dict(codes_list, self.levels)
+            result = self.groupings[0].indices
+        else:
+            codes_list = [ping.codes for ping in self.groupings]
+            result = get_indexer_dict(codes_list, self.levels)
+        if not self.dropna:
+            has_mi = isinstance(self.result_index, MultiIndex)
+            if not has_mi and self.result_index.hasnans:
+                result = {
+                    np.nan if isna(key) else key: value for key, value in result.items()
+                }
+            elif has_mi:
+                # MultiIndex has no efficient way to tell if there are NAs
+                result = {
+                    # error: "Hashable" has no attribute "__iter__" (not iterable)
+                    tuple(np.nan if isna(comp) else comp for comp in key): value  # type: ignore[attr-defined]
+                    for key, value in result.items()
+                }
+
+        return result

     [USER_REDACTED]
     [USER_REDACTED]_readonly

@@ -606,3 +606,33 @@ def test_filter_consistent_result_before_after_agg_func():
     grouper.sum()
     result = grouper.filter(lambda x: True)
     tm.assert_frame_equal(result, expected)
+
+
+def test_filter_with_non_values():
+    # GH 62501
+    df = DataFrame(
+        [
+            [1],
+            [None],
+        ],
+        columns=["a"],
+    )
+
+    result = df.groupby("a", dropna=False).filter(lambda x: True)
+    tm.assert_frame_equal(result, df)
+
+
+def test_filter_with_non_values_multi_index():
+    # GH 62501
+    df = DataFrame(
+        [
+            [1, 2],
+            [3, None],
+            [None, 4],
+            [None, None],
+        ],
+        columns=["a", "b"],
+    )
+
+    result = df.groupby(["a", "b"], dropna=False).filter(lambda x: True)
+    tm.assert_frame_equal(result, df)

Thanks for the PR! Please always add tests. Does this also handle the tuple case on L667?

> I am not familiar with the (groupby) ASVs, but I guess it referring to this: https://pandas.pydata.org/community/benchmarks.html. Help would be greatly appreciated, although I will through the docs by [NAME_REDACTED] - if you're using conda for your virtual environment, then this should be sufficient:

    asv continuous -f 1.1 upstream/main HEAD -b ^groupby

Apologies [USER_REDACTED] - just one request.

lgtm

This check is expensive - this function is only ever called currently with `names` a list of length 1, and the rest of the method is O(1) in terms of `self.indices`. It's called from the inner loop of `DataFramGroupBy.fitler` as we're iterating over each group. This seems avoidable.

I believe we could change this function to just accept a single `name` (rather than a list) and then have a special case:

```python
if isna(name):
    return self.indices.get(np.nan, [])
```

It seems better to do this on indices cached property directly, and only in the case where there is a NaN value with `if not self.dropna and self.result_index.hasnans`.

Good point, will adjust.

I think `self.indices.get(np.nan, [])` won't work as the `Nan` value in the `self.indices` can not be accessed reliable before changing the keys from `Nan` to `np.nan`. I think I have a working solution though. Will supply the updated version of the PR tomorrow.

> I think `self.indices.get(np.nan, [])` won't work as the `Nan` value in the `self.indices` can not be accessed reliable before changing the keys from `Nan` to `np.nan`.

Isn't this what I suggested to do in https://github.com/[USER_REDACTED]ndas-dev/pandas/pull/63178#discussion_r2554233008

I think I misread your first comment from two days ago. To make sure we are on the same page, we can change the function `_get_indices(self, names)` to `_get_indices(self, name)`. Changing the list for a single name?

Yes! It is only ever used with a single name today.

Can you rename this `_get_index` and remove the other `_get_index` entirely.

```suggestion
                    # error: "Hashable" has no attribute "__iter__" (not iterable)
                    tuple(np.nan if isna(comp) else comp for comp in key): value  # type: ignore[attr-defined]
```

Added the change.

Done!

Can you move the note to 3.0.0


BUG: ensure to always return new objects in Index set operations (avoid metadata mutation)

…n) (#63169)

- [x] closes #63169
- [x] [Tests added and passed]
- [x] All [code checks passed]
- [ ] Added [type annotations]
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

Previously, `Index.intersection` and `Index.union` (including `MultiIndex` implementations) would return a direct reference to `self` (or `other`) if the result was identical to the input. This allowed side effects where mutating the metadata (e.g. `.name`) of the result would corrupt the original index.

This change forces a shallow copy (`deep=False`) in these cases. This preserves the performance benefit of sharing the underlying data array while ensuring the Index container itself is a distinct object with independent metadata.

@@ -1153,6 +1153,7 @@ Indexing
 - Bug in :meth:`DataFrame.loc.__getitem__` and :meth:`DataFrame.iloc.__getitem__` with a :class:`CategoricalDtype` column with integer categories raising when trying to index a row containing a ``NaN`` entry (:issue:`58954`)
 - Bug in :meth:`Index.__getitem__` incorrectly raising with a 0-dim ``np.ndarray`` key (:issue:`55601`)
 - Bug in :meth:`Index.get_indexer` not casting missing values correctly for new string datatype (:issue:`55833`)
+- Bug in :meth:`Index.intersection`, :meth:`Index.union`, :meth:`MultiIndex.intersection`, and :meth:`MultiIndex.union` returning a reference to the original Index instead of a new instance when operating on identical indexes, which could cause metadata corruption when modifying the result (:issue:`63169`)
 - Bug in adding new rows with :meth:`DataFrame.loc.__setitem__` or :class:`Series.loc.__setitem__` which failed to retain dtype on the object's index in some cases (:issue:`41626`)
 - Bug in indexing on a :class:`DatetimeIndex` with a ``timestamp[pyarrow]`` dtype or on a :class:`TimedeltaIndex` with a ``duration[pyarrow]`` dtype (:issue:`62277`)

@@ -2975,13 +2975,12 @@ def __bool__(self) -> NoReturn:
     def _get_reconciled_name_object(self, other):
         """
         If the result of a set operation will be self,
-        return self, unless the name changes, in which
-        case make a shallow copy of self.
+        return a shallow copy of self.
         """
         name = get_op_result_name(self, other)
         if self.name is not name:
             return self.rename(name)
-        return self
+        return self.copy(deep=False)

     [USER_REDACTED]
     def _validate_sort_keyword(self, sort) -> None:

@@ -4082,13 +4082,12 @@ def _is_comparable_dtype(self, dtype: DtypeObj) -> bool:
     def _get_reconciled_name_object(self, other) -> MultiIndex:
         """
         If the result of a set operation will be self,
-        return self, unless the names change, in which
-        case make a shallow copy of self.
+        return a shallow copy of self.
         """
         names = self._maybe_match_names(other)
         if self.names != names:
             return self.rename(names)
-        return self
+        return self.copy(deep=False)

     def _maybe_match_names(self, other):
         """

@@ -727,7 +727,7 @@ def test_intersection(self, index, sort):

         # Corner cases
         inter = first.intersection(first, sort=sort)
-        assert inter is first
+        assert inter is not first

     [USER_REDACTED].mark.parametrize(
         "index2_name,keeps_name",
@@ -812,16 +812,16 @@ def test_union_identity(self, index, sort):
         first = index[5:20]

         union = first.union(first, sort=sort)
-        # i.e. identity is not preserved when sort is True
-        assert (union is first) is (not sort)
+        # GH#63169 - identity is not preserved to prevent shared mutable state
+        assert union is not first

         # This should no longer be the same object, since [] is not consistent,
         # both objects will be recast to dtype('O')
         union = first.union(Index([], dtype=first.dtype), sort=sort)
-        assert (union is first) is (not sort)
+        assert union is not first

         union = Index([], dtype=first.dtype).union(first, sort=sort)
-        assert (union is first) is (not sort)
+        assert union is not first

     [USER_REDACTED].mark.parametrize("index", ["string"], indirect=True)
     [USER_REDACTED].mark.parametrize("second_name,expected", [(None, None), ("name", "name")])
@@ -984,3 +984,83 @@ def test_union_pyarrow_timestamp(self):
         res = left.union(right)
         expected = Index(["2020-01-01", "2020-01-02"], dtype=left.dtype)
         tm.assert_index_equal(res, expected)
+
+
+def test_intersection_mutation_safety():
+    # GH#63169
+    index1 = Index([0, 1], name="original")
+    index2 = Index([0, 1], name="original")
+
+    result = index1.intersection(index2)
+
+    assert result is not index1
+    assert result is not index2
+
+    tm.assert_index_equal(result, index1)
+    assert result.name == "original"
+
+    index1.name = "changed"
+
+    assert result.name == "original"
+    assert index1.name == "changed"
+
+
+def test_union_mutation_safety():
+    # GH#63169
+    index1 = Index([0, 1], name="original")
+    index2 = Index([0, 1], name="original")
+
+    result = index1.union(index2)
+
+    assert result is not index1
+    assert result is not index2
+
+    tm.assert_index_equal(result, index1)
+    assert result.name == "original"
+
+    index1.name = "changed"
+
+    assert result.name == "original"
+    assert index1.name == "changed"
+
+
+def test_union_mutation_safety_other():
+    # GH#63169
+    index1 = Index([0, 1], name="original")
+    index2 = Index([0, 1], name="original")
+
+    result = index1.union(index2)
+
+    assert result is not index2
+
+    tm.assert_index_equal(result, index2)
+    assert result.name == "original"
+
+    index2.name = "changed"
+
+    assert result.name == "original"
+    assert index2.name == "changed"
+
+
+def test_multiindex_intersection_mutation_safety():
+    # GH#63169
+    mi1 = MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"])
+    mi2 = MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"])
+
+    result = mi1.intersection(mi2)
+    assert result is not mi1
+
+    mi1.names = ["changed1", "changed2"]
+    assert result.names == ["x", "y"]
+
+
+def test_multiindex_union_mutation_safety():
+    # GH#63169
+    mi1 = MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"])
+    mi2 = MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"])
+
+    result = mi1.union(mi2)
+    assert result is not mi1
+
+    mi1.names = ["changed1", "changed2"]
+    assert result.names == ["x", "y"]

@@ -114,7 +114,7 @@ def test_intersection_bug_1708(self):

     def test_intersection_equal(self, sort):
         # GH 24471 Test intersection outcome given the sort keyword
-        # for equal indices intersection should return the original index
+        # GH#63169 intersection returns a copy to prevent shared mutable state
         first = timedelta_range("1 day", periods=4, freq="h")
         second = timedelta_range("1 day", periods=4, freq="h")
         intersect = first.intersection(second, sort=sort)
@@ -124,7 +124,8 @@ def test_intersection_equal(self, sort):

         # Corner cases
         inter = first.intersection(first, sort=sort)
-        assert inter is first
+        assert inter is not first
+        tm.assert_index_equal(inter, first)

     [USER_REDACTED].mark.parametrize("period_1, period_2", [(0, 4), (4, 0)])
     def test_intersection_zero_length(self, period_1, period_2, sort):

Looking good, would like a 2nd eye on this. cc [USER_REDACTED]

Looks good!
Could you add a note about this in the 3.0.0.rst whatsnew file? (eg in the indexing subsection in the bug fixes section)

Thanks!

We'd like to move away from having test classes in cases where the class itself isn't useful. Can you just make these functions.

Got it. I'll convert them to functions shortly.

```suggestion
        return self.copy(deep=False)
```

I know that is the default, but for code readability I might add it explicitly (given the default for Series/DataFrame is True, one might expect that here as well; I was at least confused about when first looking at this)

```suggestion
        assert inter is not first
        tm.assert_index_equal(inter, first)
```

Just to ensure it are still the same values


ENH: use stable sort in value_counts

- [x] closes pandas-dev/pandas#63155
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
  - It is not too clear to me how I can test the stability of sorting.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
  - The failing parts are not related to this PR.
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
  - Not applicable.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
  - Added in docstring.
- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.
  - Only natual intelligence is involved.

@@ -948,7 +948,7 @@ def value_counts_internal(
             result = Series(counts, index=idx, name=name, copy=False)

     if sort:
-        result = result.sort_values(ascending=ascending)
+        result = result.sort_values(ascending=ascending, kind="stable")

     if normalize:
         result = result / counts.sum()

@@ -993,7 +993,12 @@ def value_counts(
             If True then the object returned will contain the relative
             frequencies of the unique values.
         sort : bool, default True
-            Sort by [NAME_REDACTED] the order of the data when False.
+            Stable sort by [NAME_REDACTED] the order of the data
+            when False.
+
+            .. versionchanged:: 3.0.0
+
+                Prior to 3.0.0, the sort was unstable.
         ascending : bool, default False
             Sort in ascending order.
         bins : int, optional

@@ -7761,11 +7761,16 @@ def value_counts(
         normalize : bool, default False
             Return proportions rather than frequencies.
         sort : bool, default True
-            Sort by [NAME_REDACTED] the order of the data when False.
+            Stable sort by [NAME_REDACTED] the order of the data
+            when False.

             .. versionchanged:: 3.0.0

                 Prior to 3.0.0, ``sort=False`` would sort by [NAME_REDACTED]
+
+            .. versionchanged:: 3.0.0
+
+                Prior to 3.0.0, the sort was unstable.
         ascending : bool, default False
             Sort in ascending order.
         dropna : bool, default True
@@ -7875,7 +7880,7 @@ def value_counts(
         counts.name = name

         if sort:
-            counts = counts.sort_values(ascending=ascending)
+            counts = counts.sort_values(ascending=ascending, kind="stable")
         if normalize:
             counts /= counts.sum()

@@ -2776,8 +2776,8 @@ def value_counts(
         normalize : bool, default False
             Return proportions rather than frequencies.
         sort : bool, default True
-            Sort by [NAME_REDACTED] False, non-grouping columns will appear
-            in the order they occur in within groups.
+            Stable sort by [NAME_REDACTED] False, non-grouping
+            columns will appear in the order they occur in within groups.

             .. versionchanged:: 3.0.0

@@ -1446,6 +1446,19 @@ def test_value_counts_series(self):
         )
         tm.assert_series_equal(result, expected)

+    def test_value_counts_stability(self):
+        # GH 63155
+        arr = np.random.default_rng(2).integers(0, 32, 64)
+        result = algos.value_counts_internal(arr, sort=True)
+
+        value_counts = Series(arr).value_counts(sort=False)
+        expected = value_counts.sort_values(ascending=False, kind="stable")
+        tm.assert_series_equal(result, expected)
+
+        unstable_sorted = value_counts.sort_values(ascending=False, kind="quicksort")
+        with pytest.raises(AssertionError):
+            tm.assert_series_equal(result, unstable_sorted)
+

 class TestDuplicated:
     def test_duplicated_with_nas(self):

Change you Series.value_counts docstring too and add a test - something like

```python
ser = pd.Series(np.random.randint(0, 100, 10000))
result = ser.value_counts()
expected = ser.value_counts(sort=False).sort_values(ascending=False, kind="stable")
tm.assert_series_equal(result, expected)
```

seems to fairly reliably fail without this PR (100000 out of 100000 times). Should set the NumPy random seed in the test.

I don't think this is a breaking change since unstable sorting doesn't guarantee anything about the output order. Would also like another eye on this.

lgtm

Can we test with less values, like less than 100?

Hi, may I ask for a review? I wish we can see the patch in 3.0. Thank you!