**Title:** Decouple projection command serialization and align command payloads

**Background**
The `ProjectionManagerCommandWriter` currently implements command serialization inline. It directly calls a private `PublishCommand` method that is declared with `throw new NotImplementedException()`. To enable proper testing of the command logic and to prepare for a real backing writer, the serialization/write concern should be extracted behind an `ICommandWriter` interface.

Additionally, the `CreatePrepared` management message carries a `ProjectionSourceDefinition` while the projection core service expects a `QuerySourcesDefinition` in the serialized command. A mismatch here leads to incorrect command payloads being generated.

**Ask**
1. Introduce an `ICommandWriter` interface with a single method `PublishCommand(string command, Guid workerId, object body)`. Provide a skeleton `CommandWriter` class (implementation may be stubbed) and inject it into `ProjectionManagerCommandWriter` via the constructor.
2. Change `ProjectionManagerCommandWriter` to delegate command publishing to `ICommandWriter` instead of using its own private method.
3. Correct the `CreatePrepared` management message so that it uses a `QuerySourcesDefinition` to represent the source definition, both as a property and in its constructor.
4. In `ProjectionCoreServiceCommandReader`, make the inner command types (`LoadStoppedCommand`, `StartCommand`, `StopCommand`, `KillCommand`, `DisposeCommand`, `CreatePreparedCommand`, `CreateAndPrepareCommand`, `CreateAndPrepareSlaveCommand`, `SpoolStreamReadingCommand`) public so they can be referenced by the writer and tests.
5. Introduce a serialization-safe version struct `PersistedProjectionVersion` with implicit conversions to/from `ProjectionVersion`. Update the `CreatePreparedCommand`, `CreateAndPrepareCommand`, and `CreateAndPrepareSlaveCommand` to use `PersistedProjectionVersion` (instead of `ProjectionVersion`) for their `Version` property, and `QuerySourcesDefinition` (instead of `ProjectionSourceDefinition`) for their `SourceDefinition` property where applicable.
6. Adjust the caller in `ManagedProjection` so that when constructing the `CreatePrepared` message it passes a `QuerySourcesDefinition` derived from the persisted state’s `SourceDefinition`.

**Constraints/Edge Cases**
- The `ICommandWriter` implementation does not need a real backing store yet; a throwing implementation is acceptable.
- No behavioral change is expected in the command content after this patch; serialization must round-trip correctly with the new types.
- All existing handlers that construct `CreatePrepared` must be updated accordingly (only `ManagedProjection` in the current codebase).
- Ensure that the `PersistedProjectionVersion` conversion does not lose information and matches the fields (`Id`, `Epoch`, `Version`) already used in `ProjectionVersion`.

Progress: tests on projection manager command writer


EmittedStream must prevent events from being emitted out of causal order.
Accept a new `from` CheckpointTag at construction that represents the last known position of the stream. On every `EmitEvents` call, validate that each event’s `CausedByTag` is not less than the tag of the previous event (or `from` for the first event). If an event with an earlier tag is passed, throw `InvalidOperationException`. Also ensure `from` and `zeroPosition` are non‑null at construction. This guard is needed for both normal operation and recovery, where a stale position could otherwise slip through.

Added: EmittedStream now validates event order


The `BatchAppend` gRPC method silently changes the wire format of the correlation ID it returns to clients. When a client sends a request with a string-formatted `UUID` (e.g. `"550e8400-e29b-41d4-a716-446655440000"`), the server always responds with a structured `UUID` (most/least significant bits), discarding the original format. This prevents clients that rely on string correlation IDs from properly matching responses to requests, and breaks the expectation that the correlation ID is opaquely echoed.

**Acceptance Criteria**

1. **Format preservation**
   When the server processes a `BatchAppendReq`, the resulting `BatchAppendResp` must contain a `correlation_id` field that uses the **same UUID variant** (structured or string) as the corresponding request.
   - If the request carries a structured UUID, the response returns a structured UUID with the same most/least significant bits.
   - If the request carries a string UUID, the response returns a string UUID with the same textual representation.

2. **Value fidelity**
   The correlation identifier value (the actual UUID bytes or string representation) must be identical between the request and its corresponding response, irrespective of format.

3. **Compatibility**
   Existing clients that use structured UUIDs must see no behavioral change; the structured-to-structured round-trip must continue to work identically.

4. **Multi-batch correlation**
   In a streaming `BatchAppend` call with multiple requests (each with its own correlation ID and format), each response must preserve the format of the request that triggered it, enabling the caller to distinguish responses by the original correlation method they used.

5. **No extra server-side conversions**
   The server must not arbitrarily re-encode or reinterpret the UUID. The exact bytes received for the `CorrelationId` protobuf field (including the `value` oneof choice) must be echoed back in the response without transformation.

fixed batch append not using requested uuid format


The static constructor of `MessageHierarchy` currently enumerates every assembly in the `AppDomain`, searching for types derived from `Message`. This unconditional scan leads to:

1. **Startup latency** – especially when many assemblies are loaded (e.g., from NuGet packages, system assemblies, or test runners), the reflection‑heavy walk adds noticeable delay.
2. **Fragility** – some unrelated assemblies may trigger `TypeLoadException` or `ReflectionTypeLoadException` during `Assembly.GetTypes()`, causing the entire hierarchy initialization to fail and preventing EventStore from starting.

In practice, only a handful of assemblies (e.g., `EventStore.Core`, `EventStore.Projections.Core`, and their test suites) actually contain message types. There is no declarative way to distinguish them from all other loaded assemblies.

**Expected behavior:**
The `MessageHierarchy` scanner should only process assemblies that have been explicitly marked as containing message types. This can be achieved by defining a custom assembly‑level attribute (for example, `[MessageContainer]`) and filtering `AppDomain.CurrentDomain.GetAssemblies()` to those decorated with it.

**Acceptance criteria:**
- Assemblies that contain `Message` subclasses must apply the marker attribute (e.g., `[assembly: MessageContainer]`).
- The `MessageHierarchy` static initializer must skip any assembly that does not have the attribute.
- The resulting parent/descendant tables and `MaxMsgTypeId` must remain identical to the current behavior for the set of marked assemblies.
- Startup time under typical development/production loads should be reduced, and the initialization must no longer fail due to problems in unrelated assemblies.

**Impact:**
Without this change, every deployment and every test run spends extra time scanning irrelevant assemblies, and certain environments (particularly with dynamic plugin loading or instrumentation libraries) may fail to start entirely.

Changed: message hierarchy initialization scans only assemblies marked


The following bug report describes the observed problems and the intended behavioral changes.

---

**Title:** Telemetry metrics contain misleading memory defaults, ambiguous names, incompatible gRPC call counters, and double-counted indexed events during index rebuild

**Description:**

Several issues in the telemetry metric configuration and implementation cause incorrect or confusing monitoring data:

1. **Default Memory Metrics Are Misleading**
   The `MemPagedBytes` and `MemVirtualBytes` process metrics are enabled by default in `telemetryconfig.json`.
   - `MemPagedBytes` uses `Process.PagedMemorySize64`, which reports the amount of pagable memory – not the number of bytes actually paged. This does not represent actual paging activity.
   - `MemVirtualBytes` uses `Process.VirtualMemorySize64`, which does not show the bytes that are actually allocated in the virtual address space (e.g., reserved versus committed).
   Because these values do not convey the intended information, they should be disabled by default so users do not rely on them.

2. **Incoming gRPC Call Metric Mixes Gauges and Cumulative Counters**
   The metric currently exposed as `eventstore-incoming-grpc-calls` carries a dimension for `kind` that includes `current` (a gauge-like up‑down counter) alongside cumulative counters (`total`, `failed`, `unimplemented`, `deadline-exceeded`).
   - Aggregating across `kind` values is natural for cumulative counters, but the `current` value behaves as a gauge and cannot be meaningfully summed or averaged with the others.
   - The single metric name also makes it unclear that `current` is a different instrument type.
   The fix is to:
     * Place the `current` call count into its own, separate metric (e.g., `eventstore-current-incoming-grpc-calls`) with no extra dimensions.
     * Keep the cumulative counters in a properly named metric (e.g., `eventstore-incoming-grpc-calls`) that only contains the cumulative counter dimensions.

3. **Generic Duration/Latency Metric Names**
   Two duration metrics have overly generic names:
   - `eventstore-duration` is actually used for gRPC method call durations.
   - `eventstore-latency` is actually used for gossip pull/push latencies.
   These should be renamed to `eventstore-grpc-method-duration` and `eventstore-gossip-latency` respectively, so operators can distinguish them at a glance.

4. **Indexed Events Counter Double-Counts During Rebuild**
   The metric that tracks the number of events added to the index (used to monitor indexing throughput) is incorrectly incremented when the index is being rebuilt. During a rebuild the same events are re‑indexed from the transaction log, causing the counter to double‑count events that were already indexed before the rebuild.
   The counter should only increment when a new event is committed to the index during normal operation, i.e., when the index is not in a rebuild state. This prevents inflated counts and makes the metric reflect only newly available events.

**Expected Behavior**
- `MemPagedBytes` and `MemVirtualBytes` are disabled by default.
- Incoming gRPC “current” call count is reported as a standalone metric with no dimensions, while the cumulative totals are reported in a separate, clearly named metric.
- gRPC method durations use a metric named `eventstore-grpc-method-duration`; gossip latencies use `eventstore-gossip-latency`.
- The indexed‑events counter does **not** increment during index rebuild; it only increments for events that are being indexed as part of normal streaming writes.

**Impact**
Without these changes:
- Operators may misinterpret memory usage and paging behavior.
- Monitoring dashboards that aggregate gRPC call metrics across all `kind` values will receive nonsensical results when including the `current` value.
- Ambiguous metric names make it difficult to identify what a given duration histogram represents.
- Indexed‑event throughput appears artificially high after an index rebuild, masking real performance characteristics.


The database open validation should tolerate a single extraneous chunk file located immediately after the last known chunk. This can happen when a new chunk is created but the writer checkpoint has not yet been flushed. In that case, the file must be automatically removed rather than causing a startup error.

Specifically, when enumerating chunks after the lastChunkNum:
- If exactly one LatestVersion file has start = lastChunkNum + 1, start == end, and version == 0, delete it silently.
- For any other extraneous files (additional files beyond that one, non-sequential start numbers, non-zero versions, or start != end), throw CorruptDatabaseException with an inner ExtraneousFileFoundException as before.

Update `EnsureNoExcessiveChunks` in `TFChunkDb` accordingly. No changes to the public API or configuration are required.

Revert "Revert the change allowing an extra chunk at startup (DB-64)"


**Title:** Persistent subscription incorrect retry/parking behavior and incomplete replay of parked messages

**Area:** `EventStore.Core.Services.PersistentSubscription`

**Report:**

Two related defects in the persistent subscription implementation:

1. **Explicit NAK retry does not respect `MaxRetryCount`**
   When a client sends an explicit NAK with `NakAction.Retry` (or `NakAction.Unknown` which also retries), the message is unconditionally retried without checking whether the configured maximum retry limit has been reached. This allows clients to bypass the retry policy, effectively causing an infinite retry loop for a message that should instead be parked after exhausting its retry attempts. Timeout-based retries already enforce the limit; explicit NAK retries must follow the same contract.

2. **Replay of all parked messages misses the last parked event and uses wrong version tracking**
   `RetryAllParkedMessages` replays the parked event stream, but:
   - The end-of-stream query reads the *next expected version* instead of the *last event number*, so the final parked message is never replayed.
   - During replay, the stop marker is compared against the link event’s `EventNumber` rather than the parked message’s original event number. Combined with the off-by-one endpoint, this causes the replay to terminate prematurely, missing one or more events, and may push events to clients in an incorrect order.
   - The parked message metadata does not carry the original event number, making it impossible to correctly identify the replay boundary.

**Expected behavior:**

- **Max retry enforcement on explicit NAK:** Before retrying a message because of an explicit NAK, the subscription must check whether the retry count has already reached `MaxRetryCount`. If the limit has been hit, the message must be parked immediately (via `ActionTakenForRetriedMessage` / `ParkMessage`), not retried again.
- **Complete parked message replay:**
  - Reading the end of the parked stream must return the last event number (not next expected version), and the replay range must be computed so that the final event is included.
  - The replay loop must stop based on the original event number of the parked message, which must be persisted in the parked event’s metadata.
  - After a complete replay, the subscription must correctly transition out of the replay state and resume normal processing.

**Non‑obvious constraints:**
- The `GetLinkToFor` link creation continues to link to the resolved event’s actual stream/number (`Event.EventNumber@Event.EventStreamId`). The replay boundary uses the *original event number* stored in the metadata, not the link’s own event number.
- The fix should not alter behavior for `NakAction.Park` or `NakAction.Skip`.
- The `ParkedMessageMetadata` structure must be extended with a field (e.g. `SubscriptionEventNumber`) to record the original event number at the time of parking.

**Implementation hints (if helpful):**
- In `NotAcknowledgeMessagesProcessed`, call `ActionTakenForRetriedMessage` before `RetryMessage` for the relevant NAK actions, mirroring the pattern already used in `NotifyClockTick`.
- In `BeginParkMessage`, store `ev.OriginalEventNumber` in the metadata.
- In `BeginReadEndSequence`, use `comp.LastEventNumber` instead of `comp.NextEventNumber`.
- In `RetryAllParkedMessages`, pass `end.Value + 1` as the stop parameter to `TryReadingParkedMessagesFrom`, and in `HandleParkedReadCompleted` compare `ev.OriginalEventNumber` against `stopAt` to detect the replay end.

Correct the replay parked event version tracking and pushing to


**Bug: PTable merge during hash upgrade truncates enumeration and mis-orders collision entries**

**Observed behavior**
When merging PTables that contain streams with hash collisions into a higher-version PTable (e.g., upgrading from 32-bit to 64-bit hashes), the resulting PTable can be missing entire streams past a certain point, and streams affected by hash collisions may appear in the wrong sorted order. This manifests as incomplete index data and/or incorrect event number ranges returned by stream reads.

**Expected behavior**
The merging process should process all index entries from the source PTables, re-hash them with the new hash function, and ensure the final PTable is both complete and correctly sorted by the new hash. Hash collisions (where two different stream IDs share the same old hash but have distinct new hashes) should be sorted independently so that all entries end up in their proper new-hash groups in the output.

**Impact / Risk**
- After a hash upgrade, some streams may become effectively invisible to the index, causing `EventStore` to fail to read those streams or report them as missing.
- Streams whose entries are incorrectly ordered due to collision misplacement can lead to corrupted event number ranges and incorrect read results.
- The bug is deterministic and affects any upgrade scenario where the source PTables contain entries that share a common old hash but map to different new hashes, and where some source entries are filtered out (e.g., soft-deleted) during enumeration.

**Root cause (summary)**
Two flaws were discovered in the enumerator that reads entries group-by-old-hash during `PTable.MergeTo`:

1. **Premature enumeration end:** When all entries in a source hash group are filtered out (e.g., because the records no longer exist), the group returns an empty list. The calling `MoveNext()` interprets this as the end of the entire source enumerator and stops merging, skipping all subsequent hash groups.

2. **Entry leak between hash groups:** The logic that reads entries until the hash changes accidentally includes the *first* entry of the next hash group in the current group’s list. This entry then gets sorted with the current group based on its upgraded hash. When the merged PTable is later enumerated, that entry is returned before its proper hash group, breaking the global sort order. The problem is masked unless the leaked entry would have appeared later in the sorted output—which happens precisely when its upgraded hash collides with the streams of the current group, or when it would otherwise sort after some entries of the current group.

**Desired fix**
Ensure the merge enumerator:
- Continues to the next group even when the current group yields zero visible entries.
- Stops reading a group strictly at the boundary where the stream hash changes, without consuming the next group’s first entry.
- Sorts only the entries that truly belong to the current group by their upgraded hashes before returning them.

This will make the upgraded PTable complete and correctly sorted, regardless of hash collisions or filtered-out entries.

i) Bugfix that may cause PTable enumeration to end earlier when upgrading hashes (test provided by @lscpike):


Add support for two new log record types: PartitionType and StreamType.
Each record must have a sub-header containing only a single `Guid` partition ID (16 bytes).
The payload is a dynamically sized UTF-8 string (no BOM) representing a name.
Enforce a maximum name length of 100 UTF-8 bytes; throw if exceeded.

The record type identifiers are:
- `PartitionType = 3`
- `StreamType = 4`

Both types use log record version `0`.

Provide creation APIs that accept:
- timestamp
- log position
- record ID (partition type ID / stream type ID)
- partition ID
- name string

The creations must populate the common record header fields (`Type`, `Version`, `TimeStamp`, `RecordId`, `LogPosition`) and the sub-header’s partition ID, then write the UTF-8 encoded name into the payload.

Return the created records in a form that exposes the header, sub-header, and the string payload through a read-only view (similar to the existing epoch record flow but with string payloads).

Add PartitionType and StreamType structs and creation methods (#2918)


Title: Catalan anglicisms rule mishandles multi-word English expressions in context

When checking Catalan text, the anglicisms rule (which suggests Catalan alternatives for English words) should suppress matches for English words that appear inside an English sentence. This works correctly for single‑word anglicisms, but multi‑word anglicisms (for example, “community manager”) are **incorrectly flagged even when the whole sentence is in English**. Conversely, a multi‑word anglicism that should be flagged inside a Catalan sentence may be incorrectly ignored.

**Steps to reproduce**
1. Set the language to Catalan.
2. Check `This is the community manager.` – the rule produces one or more matches, but it should produce **none** because the entire phrase is in an English context (surrounded by tokens marked as `_english_ignore_`).
3. Check `Aquest és el community manager.` – the rule should report the anglicism (since the sentence is Catalan), but it may report nothing.

**Expected behaviour**
- The “English context” exception must apply to the **whole matched phrase**, not just individual tokens. A match should only be discarded when the entire multi‑word anglicism is flanked by tokens that are tagged as English.
- Token‑level exceptions for proper nouns, immunised tokens, and `speller`‑ignored tokens must still be honoured for each token in the matched span.
- The fix must handle boundary cases (matched phrase at sentence start/end) correctly.
- The change should not alter behaviour for other replacement rules or for single‑word anglicisms, and must not break other languages.

[ca] improve exceptions in anglicisms rule


**Arabic: spelling error positions are incorrect when text contains Tashkeel**

When Arabic text includes diacritical marks (Tashkeel), the Hunspell‑based spelling rule reports error offsets that do not match the actual character positions in the original input. For example, a misspelled word may be underlined at a shifted location, or an error that should span a certain range of characters is instead reported at a different range.

**Desired behavior**
- Spelling‑error offsets (`fromPos`/`toPos`) must be calculated against the original sentence text, with Tashkeel characters treated as ordinary characters that contribute to the position count.
- The spell‑check itself (lookup against the dictionary, ignore‑word lists, etc.) should ignore Tashkeel: a word with diacritics and the same word without diacritics should be considered identical for spelling purposes.
- The same stripping‑for‑lookup should apply when the tagger analyses words for additional disambiguation – but the token text passed to the spelling and tagging stages must retain the original Tashkeel so that position information stays accurate.

**Acceptance criteria**
1. In a sentence containing Tashkeel, a misspelled word is flagged at the character offsets that correspond to its actual location in the full Unicode string (not at a shortened position).
2. Correctly‑spelled words that carry Tashkeel do not trigger false errors.
3. Words that are correctly spelled without Tashkeel are still accepted when presented with any combination of the diacritical marks listed in the Arabic language module.

[ar] Fix wrong error position when text has Tashkeel  (#2273)


```
Improve multiword disambiguation in Catalan: when a token has multiple possible
multiword readings (opening tags like <...> that expect a closing tag </...>),
the hybrid disambiguator currently picks the first one, which may correspond to
a shorter chunk. It should instead select the reading that spans the longest
distance in the sentence, i.e., the one whose matching closing tag appears
farthest to the right among the upcoming tokens. If a reading's closing tag is
not found at all, that reading should not be chosen.
```


**Title:** TokenAgreementVerbNounRule produces false errors for several valid Ukrainian constructions

**Description:**
The Ukrainian token agreement rule (`TokenAgreementVerbNounRule`) incorrectly flags a number of grammatically correct sentences. The rule needs to be extended to recognize the following legitimate patterns and stop reporting them as subject‑verb / noun‑agreement errors:

1. **Quantitative subjects (numerals and indefinite quantities)**
   Sentences like `боротиметься кілька однопартійців`, `входило двоє студентів`, `буде лежати двоє хворих`, `не виїхало більшість автобусів`, `сидять понад міру`. The verb may be singular/plural neuter or other forms that do not create a true agreement mistake when the subject is a quantity word.

2. **Predicative / impersonal constructions with *бути***
   Combinations such as `буде видно тільки супутники` (бути + predicative) and `треба буде ще склянку` / `потрібно було більше часу` (predicative + бути) should be accepted. The rule currently treats the noun after such chains as a mismatched subject.

3. **Adjective + *бути* + genitive complement**
   For example, `повна була образів` – the adjective agrees with the subject, while the complement is in the genitive case. This should not trigger a disagreement error.

4. **Impersonal verb + *бути* + noun**
   `відкрито було журнал` and similar impersonal‑voice constructions must be allowed.

5. **Verb + genitive object (partitive)**
   `нарубав лісу` – the genitive object after certain verbs should not be mistaken for a subject.

6. **Fixed adverbial expressions**
   The phrases `страх як` and `жах як` (meaning “very”) should be recognized and not cause false positives.
   Also, patterns like `триває повним ходом` must pass.

7. **Incomplete set of modal adjectives**
   The list of modal adjectives that govern certain agreement patterns is missing `вимушений`, `ладний` and `радий`. These should be treated like the already‑handled `змушений`, `здатний`, `готовий` etc.

The expected outcome: all the example sentences above (and similar constructions) must be accepted without the rule raising an error. Any fix should use the existing exception‑handling infrastructure so that the rule only fires on genuine agreement problems.


The internal language detection routine accepts raw input text and applies a cleaning step that removes URLs, email addresses, Twitter handles, non-breaking spaces (`\u00A0`), and zero-width spaces (`\uFEFF`) before classification. Several algorithm-selection and scoring decisions (e.g., fastText vs. ngram, short-text confidence special-casing, preferred-lang-only filtering) are gated on the length of the *raw* text. For texts that contain a significant amount of removable content, the cleaned text is much shorter than the raw text, leading to incorrect threshold decisions and degraded detection accuracy. Ensure that every length-based threshold inside the detection pipeline uses the length of the cleaned text, not the raw input.

properly use the length of the cleaned text for threshold decision


**Request:**

In Ukrainian, the noun‑verb agreement rule incorrectly flags phrases like «намагання спам’ятати» (efforts to remember) as errors.
The noun «намагання» governs an infinitive, so the construction is grammatically correct.
Fix the false positive so that the rule does **not** report an agreement error when an infinitive verb follows a noun that can govern an infinitive, irrespective of the noun’s other morphological tags (e.g., inanimate, plural, nominative).
All other existing correct detections must be preserved.


[de] Agreement rule and tokenizer: avoid false alarms

Several German input patterns incorrectly trigger an agreement warning. Please suppress them:

* **Adverbs/indefinites before nouns** – Words like `Meist`, `Ausreichend`, `Genügend`, `Reichlich`, `Regelmäßig`, `Hauptsächlich` followed by a noun (e.g., *Ausreichend Bananen essen.*) should **not** produce an agreement error.

* **Adjective–nominalized adjective pairs** – Combinations such as *psychisch Kranke*, *sportlich Aktive*, *gesetzlich Versicherte* are being flagged; these are correct and should be treated as anti‑patterns.

* **Sentence‑final *typisch* + noun** – Constructions like *Typisch November!* (where *typisch* is followed by a noun and then a sentence‑end punctuation mark) are false positives.

* **Tokenizer splitting of “Damenshorts”** – The compound tokenizer splits *Damenshorts* into *Damen* and *Shorts*, which leads to downstream false agreement warnings. Add an exception so the word stays unsplit (i.e., *Damen* + *shorts*).


The Spanish tagger's `-mente`‑adverb heuristic only fires for lowercase (or mixed‑case) words. All‑caps forms like `DESTACADAMENTE` are silently missed.
Make the detection case‑insensitive: compare the lowercased token against the `"mente"` suffix so that any capitalisation yields an adverb analysis (lemma still derived from the lowercased form).


Improve the `EN_A_VS_AN` rule's handling of words that begin with the letter `u`. Right now, the rule seems to apply a "starts with vowel letter → use `an`" test too broadly, so it flags "a unibody" or "a Userset" (where `u` is pronounced as a consonant `yoo`) and likely misses truly incorrect "a unidentifiable" / "a uninteresting" (where `u` is pronounced as a vowel `uh`). The fix should make the determiner choice (`a` vs `an`) follow pronunciation rather than just the initial character:

- Treat `u`-initial words that start with a `yoo` sound (e.g., `uni…`, `eu…`, `one…`, and similar) as requiring **`a`**, not `an`.
- Ensure known `un…` words where the first syllable is unstressed and the `u` is a vowel sound (e.g., `unidentifiable`, `uninteresting`, `unimportant`) still require **`an`**.

The change should only affect the English `A vs. An` rule; all existing correct cases must keep working.


Bug: In the Ukrainian language module, analyzing text that contains a standalone combining accent character (for example, U+0301 COMBINING ACUTE ACCENT) throws a `StringIndexOutOfBoundsException`. The crash occurs because the disambiguation logic calls `charAt(0)` on an empty clean token (the ignored character results in an empty string after cleaning). Other ignored characters that produce an empty clean token can trigger the same problem.

Fix: Modify the affected disambiguation code so that tokens whose clean token string is null or empty are safely skipped, preventing the out-of-bounds access. After the fix, analyzing such input should complete without an exception and produce a well‑formed analyzed sentence (typically an empty sentence with only start/end markers).


```text
Bug: Dutch compound checker rejects valid compounds with lowercase acronyms

1. Summary
   LanguageTool’s Dutch spell checker incorrectly marks a number of common, correct compounds
   as spelling errors. The problem affects compounds whose first part is a lowercase or
   mixed-case acronym, as well as the word “vip” used as a compound head without an ‘s’.

2. Observed behaviour
   • The following are reported as misspelled, but are correct:
        zzp-ondernemertje
        cd-spelertje
        btw-toevoegingen
        tv-poppetjes
        vipbehandeling
   • Older all-caps variants (e.g. ZZP-ondernemertje, CD-spelertje, BTW-toevoegingen, TV-poppetjes)
     were already rejected and should stay rejected.

3. Expected behaviour
   a) The compound-acceptance logic should recognise a wider set of lowercase/mixed-case acronyms
      (such as “zzp”, “cd”, “btw”, “tv”, “AMvB”, etc.) when they are written exactly as they
      appear in standard Dutch and are followed by a hyphen.
   b) All-uppercase versions of those same acronyms must still be treated as incorrect.
   c) The word “vip” should be allowed as the first part of a hyphenless compound
      (e.g. “vipbehandeling”), in the same way that other everyday Dutch words are allowed
      without requiring an intervening ‘s’.

4. Impact
   • False positives reduce user trust and can distract or mislead the writer.
   • The fix should only affect the Dutch (nl) language module; no other languages should change.
```

[nl] allow special cases in CompoundAcceptor


**Title:** `GRPCPostProcessing` mutates fallback `RuleMatch` list when the remote call fails

**Background**
`GRPCPostProcessing.filter()` is called with a list of matches; if the gRPC request fails (timeout, circuit breaker open, etc.) it is supposed to return exactly that list. The code even has a comment `// don't modify passed rule matches list, used as fallback`. However, the request-building step adjusts offsets on the live `RuleMatch` objects, so the fallback ends up carrying corrupted positions.

**What needs to happen**
- Ensure that when the remote call cannot complete, the returned list is the unmodified input list and none of the `RuleMatch` objects are mutated.
- The network call should be separable into a method that can be overridden in a test subclass so we can write a unit test for this behaviour. The constructor must be accessible to allow that subclassing.
- While touching this area, make the postprocessing more robust:
  - Handle the case where no circuit breaker is configured (null) without crashing.
  - Catch a broader set of exceptions (not only `InterruptedException`) so that unexpected errors also fall back gracefully to the unmodified input list instead of letting the exception propagate.

**Acceptance criteria (observable)**
- When any exception (including a timeout) occurs during gRPC postprocessing, the original `ruleMatches` list and its elements are returned unaltered – positions, messages, and all other fields are identical to what was passed in.

fix modification of matches in GRPCPostProcessing


The unpaired-brackets rule (`GenericUnpairedBracketsRule`) currently flags an opening bracket as an error as soon as it's unmatched within a single sentence. This is too aggressive: a writer might still be composing the rest of the sentence, or the bracket may be matched in a later sentence. Make the rule only report an unmatched *opening* bracket once the sentence that contains it is clearly finished.

An opening bracket should only be flagged when:
- the sentence ends with a sentence-ending punctuation mark (`.`, `?`, `!`), or
- there is at least one more sentence *after* the current one in the analyzed text (so the text clearly continues and the bracket remains unmatched across a sentence boundary).

Unmatched *closing* brackets (e.g., `)` appearing without a preceding `(`, `»` appearing without a corresponding `«`) should still be reported immediately, because they are always errors.

No false negatives should be introduced for clear multi-sentence mismatches, and no regressions for existing correct matches that already respect sentence boundaries.

make unpaired brackets rule less aggressive, so it only complains once the current sentence is finished


The Dutch spell checker is incorrectly accepting compound words that contain internal uppercase letters. For example, the word "kinderenHet" is treated as correct but should be considered a spelling error. The compound splits into "kinder" + "enHet". When the part "enHet" is lowercased to "enhet", it may match a dictionary entry, causing the whole compound to be wrongly accepted. Fix the compound-acceptance logic so that a candidate non-compound part that does not follow normal word casing (i.e., it contains an uppercase letter after the first character) is rejected, rather than being lowercased and checked against the dictionary. The rule should remain safe: perfectly normal lowercase or capitalized words (e.g., "kind", "Huis") must still be accepted as compound parts.


In the multitoken spell checker for Spanish and Catalan, a misspelled phrase produces an incorrect suggestion: `"Franco more"` is being matched to the correct form `"Frank Gore"`. The expected behavior is that no suggestion should be returned for this phrase, because there is no real multitoken match—this is a false positive.

Investigation of the matching logic shows that the too-broad similar‑character normalization `k → c` is responsible. However, simply removing that normalization would break a valid suggestion that should still work: `"Nicolai Rimski-Kórsakov"` must continue to be matched to `"Nikolai Rimski-Kórsakov"`.

Refine the similar‑character replacement so that it only maps `"ko"` to `"co"` instead of replacing every `'k'` with `'c'`. This narrower rule preserves the valid mapping between `"Nikolai"` and `"Nicolai"` while preventing the false match between `"Franco"` and `"Frank"`.

**Acceptance criteria**
1. For Spanish and Catalan, calling `getMultitokenSpeller().getSuggestions("Franco more")` returns an empty list (`[]`).
2. For Spanish and Catalan, calling `getMultitokenSpeller().getSuggestions("Nicolai Rimski-Kórsakov")` still returns `["Nikolai Rimski-Kórsakov"]`.
3. The fix is restricted to the shared multitoken‑spelling normalization code; no per‑language workarounds.

multitoken spelling: revert configuration


**Title:** Ukrainian tokenizer: improve handling of `+` and apostrophe before parenthesized word

**Background**
The `UkrainianWordTokenizer` currently fails to split the `+` character when it appears adjacent to Ukrainian letters (e.g., `прислівник+займенник`), while it should continue to keep `+` attached to numbers (like `+20`) or a trailing `+` with no following letter (e.g., `Роттердам+ `). Additionally, the tokenization of an apostrophe immediately before an opening parenthesis followed by a Ukrainian letter (e.g., `’(є)`) is broken: the apostrophe is incorrectly merged with the word inside the parentheses by the brace‑in‑word pattern.

**What to change**

1. **Plus sign `+`**
   When `+` is directly followed by a Ukrainian letter (Cyrillic), it should be tokenized as a separate token. This means that the `+` must be split from both the preceding and following Ukrainian letters.
   - `прислівник+займенник` → tokens: `прислівник`, `+`, `займенник`
   - `+займенник` → tokens: `+`, `займенник`
   - `+20` must stay intact as a single token.
   - `Роттердам+ ` must keep `Роттердам+` as a single token (plus followed by a space or end of input is not a letter).

2. **Apostrophe before parenthesized Ukrainian word**
   The existing brace‑in‑word rule should not match when the character before `(` is an apostrophe. For instance, `’(є)` should tokenize as four tokens: `’`, `(`, `є`, `)`. The apostrophe must be detached from the following parenthesized group.

**Constraints**
- The fixes must not break existing tokenization of other punctuation, dates, abbreviations, or apostrophe‑end patterns.
- The solution must respect the tokenizer’s internal BREAKING_PLACEHOLDER mechanism and ensure that the resulting token list follows the standard `tokenize` contract for the module.
- Only Ukrainian‑specific character ranges should be affected; changes must not interfere with other languages.


Update the Dutch compound acceptance rules to always require a hyphen for compounds where the first part is any of the following words: adjunct, assistent, aspirant, collega, leerling, student, substituut.

When one of these words appears as the first part of a compound, the hyphenated form (e.g., "collega-burgemeester") must be accepted as a valid compound. The solid writing without a hyphen (e.g., "aspirantbuschauffeur") should be rejected.

Hyphenated compounds with these first parts must also be recognized as compound nouns by the Dutch part-of-speech tagger (e.g., "aspirant-burgemeestertje" should tag correctly).

[nl] add special cases to CompoundAcceptor


```
Fix a false positive in the English unpaired-brackets rule:

When a sentence contains a digit immediately followed by a double-quote character (e.g. `3"`,
`No. 3"`), the rule should not flag any unpaired quote in that sentence.  The pattern
digit‑followed‑by‑double‑quote usually indicates an inch measurement or a quoted number, and
it should be treated as safe – treat the entire sentence as if no unpaired brackets were found.
```

[en] prevent false alarm for quotes that end with a number


**Title:** Use language-specific quotation marks for suggestions in command-line output

**Description:**
When LanguageTool’s command-line interface prints a message that contains a suggestion, the suggestion text is currently enclosed in hardcoded straight single quotes (`'...'`). This does not respect the typographic conventions of the language being checked (e.g., French expects `« ... »`, Polish expects `„...”`, German expects `„...“`, etc.).

Other parts of LanguageTool (such as the GUI) already wrap suggestions using the language’s own opening and closing quotation marks, obtained from `Language.getOpeningQuote()` and `Language.getClosingQuote()`. The command-line output should follow the same pattern so that suggestions appear in a way that is natural for the target language.

**Acceptance criteria:**
- For a message that contains `<suggestion>word</suggestion>`, the output replaces the tags with the opening and closing quote characters of the language the text is being checked in, instead of the current single quote.
- The change must not affect any other output formatting.
- The output for Polish (as an example) should show `„word”` (or the appropriate Polish quotes) instead of `'word'`.

**Additional context:**
The language is already available in the code path that constructs the message output; the fix should use that language object to retrieve the correct quote characters.

use language-specific quotes for message here, too


**Bug Report: Advanced typography quote conversion mistakes**

When LanguageTool’s “advanced typography” feature converts straight ASCII quotes to typographic quotes, the following incorrect transformations occur:

- **Single‑character quoted text** – a string where one character is surrounded by single quotes (e.g. `'a'`, `'"'`, or `'3'`) is not properly converted. The opening and closing single quotes should be turned into their typographic counterparts, while the inner character should remain unchanged. In German input, for example, `: '"'` should become `: ‚"‘` (German single quotes around the straight double‑quote character), but the current output is wrong for such patterns.

- **English possessive `'s`** – the genitive suffix `'s` (apostrophe‑s, as in “friend’s”) is sometimes changed to an opening single quote (`‘s`) instead of the correct closing‑single‑quote (`’s`). The conversion should always render this sequence as `’s` (right single quotation mark) in English text.

Please fix the quote‑conversion logic so that these cases produce the typographically expected results. No other aspect of the advanced‑typography behavior should be altered.

advanced typography: fix #4412, fix #4880


We need to split the monolithic Arabic punctuation whitespace rule (`ArabicPunctuationWhitespaceRule`) into separate, focused rules for each punctuation type. Currently that single rule handles whitespace around commas, question marks, semicolons, brackets, and quotes all in one place. We already have a dedicated `ArabicCommaWhitespaceRule` that extends `CommaWhitespaceRule` and only deals with the Arabic comma (U+060C). We want the same treatment for the Arabic question mark (U+061F) and the Arabic semicolon (U+061B).

**Acceptance criteria:**

1. **New rules**
   - Create a rule for whitespace around the Arabic question mark. It should behave like `ArabicCommaWhitespaceRule` but target `؟` (U+061F).
   - Create a rule for whitespace around the Arabic semicolon. It should behave like `ArabicCommaWhitespaceRule` but target `؛` (U+061B).
   - Each new rule must have a unique rule ID (e.g., `ARABIC_QM_WHITESPACE` for the question mark, `ARABIC_SC_WHITESPACE` for the semicolon).

2. **Remove the old monolithic rule**
   - Delete `ArabicPunctuationWhitespaceRule` completely (class and all its methods).
   - Remove it from the Arabic language rule registration.

3. **Restore lost generic whitespace checks**
   The old monolithic rule also handled whitespace around brackets, quotes, dots, and the Latin comma. Those checks must continue to work. They are already implemented in the base `CommaWhitespaceRule` (which is already used by `ArabicCommaWhitespaceRule`). Register a `CommaWhitespaceRule` instance with quotes-whitespace checking enabled, so no functional regression occurs for those non-Arabic-punctuation cases.

4. **Registration**
   In the Arabic language class, register:
   - The existing `ArabicCommaWhitespaceRule`
   - The new `ArabicQuestionMarkWhitespaceRule`
   - The new `ArabicSemiColonWhitespaceRule`
   - A `CommaWhitespaceRule` with quotes-whitespace handling enabled

5. **Behavioral invariants**
   - Whitespace directly before an Arabic question mark or semicolon should be flagged, and the suggested fix should remove that space.
   - Missing space after an Arabic question mark or semicolon (when followed by a non-whitespace, non-quote token) should be flagged, and the suggested fix should insert a space.
   - Correct combinations (no space before, one space after) must not produce a match.
   - Existing punctuation/whitespace logic (including exceptions for file paths, digits, ellipsis, and bracket pairs like `[ ]`) must continue to work for the characters that `CommaWhitespaceRule` already covers.

6. **No other changes**
   Do not introduce any other behavioral changes; the split must be a pure refactoring that keeps all existing whitespace-check functionality intact.

[ar] replace ArabicPunctuationWhitespaceRule by ArabicCommaWhitespaceRule, ArabicQuestionMarkWhitespaceRule and ArabicSemiColonWhitespaceRule


**Title:** False positive in German missing-comma rule for sentences with "mit dem/der X verbindet"

The German rule that checks for missing commas before relative clauses (`MissingCommaRelativeClauseRule`) incorrectly flags a missing comma in sentences like:

> Es ist die Wiederkehr der Panikmache, die der neue Nationalismus mit dem der Sprachreiniger verbindet und die Geschichte der Sprachreinigung zu einem Lehrstück macht.

The rule believes a comma is needed before the second "der", even though "der" is not a relative pronoun here—it’s part of the prepositional phrase *mit dem der Sprachreiniger*. The same false positive should be avoided for similar constructions where a preposition like *mit* is followed by an inflected article (*dem*, *der*, *des*) and then a noun, followed later by a conjugated verb such as *verbindet*.

**Acceptance criteria**
- The rule must **not** suggest a comma before *der* in the example sentence above, and no missing-comma warning should appear for the segment `mit dem der Sprachreiniger verbindet`.
- The rule must **continue** to correctly detect missing commas in genuine relative‑clause contexts (e.g. “Das Auto, das am Straßenrand steht, parkt im Halteverbot.”).
- The fix should be robust enough to suppress the false positive for similar verb‑final patterns involving the preposition *mit* + article + noun, without making the rule miss real errors.
- Existing test cases for this rule must keep passing, so existing true positives are not degraded.


Please find and fix a false alarm in the German case-checking rule. The symptom: `"Liebe Kund:in"` is flagged as an error, but it is a correct German salutation using a gender-gap (gender-neutral) form.

What’s happening under the hood:
- The tokenizer splits `"Kund:in"` into three tokens: the stem `"Kund"`, the colon `":"`, and the suffix `"in"` (or `"innen"`).
- The capitalization rule then perceives the stem `"Kund"` as a standalone word following `"Liebe"` and incorrectly suggests it should be lowercased.
- The false alarm should be suppressed whenever a token is immediately followed by `":"` and then `"in"` or `"innen"` (as consecutive tokens), because that pattern represents a valid gender-gap construction.

Acceptance criteria:
- Do not report a capitalization/uppercase error for any word that is directly followed by `":"` and then `"in"` or `"innen"`.
- The fix must not affect legitimate case errors and must not break sentences where a colon appears in other contexts (e.g., before an explanatory clause).
- Boundary safety: handle positions near the end of the token array correctly (no out-of-bounds access).
- Apply the suppression to the existing case rule’s logic that already considers colon-prefixed exceptions (there is a guard for a preceding colon; the new condition should be added analogously but for a following colon+suffix).

[de] no false alarm for "Liebe Kund:in" (#311)


**Title:** Allow phrases (multi-word terms) in user dictionary

**Description:**
Users want to add custom phrases (e.g. “New York”, “open source”, “aa bb”) to their personal dictionary so that those exact sequences are not flagged as spelling errors. Currently the user dictionary only accepts single words; any entry containing a space is immediately rejected by the server with an error message (e.g. “Invalid word, you can only add words that don‘t contain spaces”). Even if that validation were removed, the spell checker would still report individual spelling errors for each token inside the phrase — so simply loosening the validation is not enough.

**Expected behavior:**
When a user has added a phrase like `"aa bb"` to their dictionary, the system should suppress spelling errors for `aa` and `bb` when they appear consecutively in the text, separated by a single space, and in the same order as the phrase. The filter must be careful **not** to suppress matches for isolated occurrences of the same tokens elsewhere (e.g. `aa` followed by something else, or `bb` preceded by anything other than `aa`). It should only apply to matches that come from dictionary‑based spelling rules (those that check a word list), not grammar or other rule types.

A phrase may also overlap with longer phrases that are present in the dictionary — the filter must handle that cleanly.

**Implementation hints:**
- The existing `DictionaryMatchFilter` already removes matches for single dictionary words, but it doesn’t handle multi‑word entries. A similar mechanism suitable for phrases would be the right approach.
- The core logic needs to scan through a list of `RuleMatch` objects, identify consecutive spelling matches (where the `fromPos` of one is exactly `prevToPos + 1`, i.e. one space apart), and check whether the concatenated text equals a known user phrase.
- The set of user phrases (dictionary entries containing spaces) can be derived from the existing user‑specific words stored in `UserConfig`.
- The new filter should be registered in the server pipeline (`PipelinePool`) so that it runs on every check request, and ideally also be available to other LanguageTool consumers that use a `UserConfig`.

**Validation note (server side):**
The current server‑side word validation in `DatabaseAccess` explicitly rejects words with spaces. That check should eventually be relaxed once the filtering logic is in place. For now it can stay with a TODO comment explaining that it will be removed when the phrase filter is ready.

**Impact:**
This is an often‑requested improvement for users dealing with technical terminology, multi‑word proper nouns, or domain‑specific jargon who want a quick way to whitelist those phrases without creating complex exception rules or submitting dictionary updates.

prepare allowing to add phrases (words with spaces) to the user dictionary


```
Title: Invalid language codes in HTTP check parameters cause 500 instead of 400

The server’s text-checking endpoint currently returns an HTTP 500 Internal
Server Error when a request contains an unrecognized language code in any of
these parameters:

- language
- motherTongue
- altLanguages
- sourceLanguage
- preferredVariants (via the preferredVariants list)
- fallback (used during language detection)

Under the hood, `Languages.getLanguageForShortCode` throws an
`IllegalArgumentException` for unknown codes, which escapes the handlers and
turns into a 500.  According to the HTTP API contract, this should be a 400
Bad Request with a descriptive message.

We need to ensure that every place the server resolves a language code from
user input catches the `IllegalArgumentException` and re-throws it as a
`BadRequestException` so the caller gets a proper 4xx response.  The behaviour
should be consistent across all the parameters listed above.  The error
message can be whatever `getLanguageForShortCode` originally produced.
```

fix w/ wrapper for getLanguageForShortCode in TextChecker


**Title**: Multilanguage text detection misses short English sentences and capital-case phrases in spell check

**Background**
When `HunspellRule` spell-checks a sentence, it tracks how many misspelled tokens are actually valid English words. If enough of the text looks English, the rule can suppress the errors as "wrong language" rather than flooding the user with false positives. This multilingual shortcut is currently too conservative and misses several common real-world patterns.

**Observed behavior**
Using the German (`de-DE`) spell checker, the following sentences are **not** recognized as English, and each token is reported as a spelling error:
- `He is a very cool guy from Poland.`
- `How are you?`
- `CONFIDENTIALITY NOTICE:`

**Expected behavior**
These sentences should be treated as English text and flagged with `errorLimitLang = "en"`, so that downstream consumers (e.g., the too-many-errors limiter) can act accordingly rather than reporting spelling errors for every word.

**Probable cause / investigation hints**
1. The current threshold (`sentLength > 3`) prevents detection on short sentences: `How are you?` has exactly 3 non-whitespace tokens and falls through.
2. The English ratio uses `sentLength` as the denominator. Some short English words are also valid in the target language dictionary (e.g., `he` and `a` are dictionary hits in German). When these appear in an otherwise-English sentence, they count against the English ratio, pulling it below the 0.66 threshold.
3. Purely uppercase "headline" sentences (like `CONFIDENTIALITY NOTICE:`) may behave differently due to case folding in the lookup path, and may also be affected by the too-small-sentence guard.

**Acceptance criteria**
- The sentences listed above are classified as English (`errorLimitLang = "en"`) when checked by the German spell checker (`de-DE`).
- The existing detection for longer sentences and other languages must not regress.
- Rational and minimal change to the heuristic — avoid adding complex hardcoded word lists for other language pairs unless absolutely necessary.


# False positive: sentence-start capitalization rule flags numbered list items

## Problem

When a user writes a plain text document containing a numbered list like:

```
1. first item
2. second item
3. third item
```

LanguageTool reports a false positive on every item after the first, claiming the word after the number should be capitalized (e.g., flagging "second" in `2. second item`).

The root issue is that the built-in sentence splitter does not treat numbered-list continuations as part of the same logical list context. Each `"\n2. …"` fragment gets treated as an independent sentence, and when its first word starts with a lowercase letter, the capitalization rule fires.

## Desired behavior

The sentence-start uppercase rule should not fire a match on a sentence whose first token is lowercase when the immediately preceding sentence (i) ends with a line break followed by a numbered list marker (`<digit>+. `), or (ii) is itself a line consisting of a numbered list marker followed by content.

In other words, if the previous "sentence" looks like it belongs to a numbered list, the current sentence should be exempt from the usual start-of-sentence capitalization check, just as URLs and email addresses are already exempt.

## Scope and safety constraints

- The exemption should apply **only** to plain-text numbered lists (e.g., `1.`, `2.`, `10.`). Bullet lists, lettered lists, and other list styles are out of scope for now.
- The rule should still flag a truly missing capital at the start of a regular sentence, even if that sentence happens to begin with a digit (e.g., `"42 is the answer."` should not be affected).
- The change must not weaken detection of genuine missing-capital errors in ordinary prose that follows a numbered-list sentence.
- The behavior should work correctly in the English language module (where the problem was reported), but ideally the logic lives in the core rule so other language modules benefit from it automatically.
- The list-detection should handle both single-line list items and items that span multiple sentences within the same numbered point (the marker is only on the first line of the item).

be less strict about numbered lists, it's okay when the list item text starts with a lowercase character


**Stop ConfusionProbabilityRule from flagging single-word sentences**
The confusion-probability rule (ConfusionProbabilityRule) currently reports a potential confusion error when a sentence consists of only a single word like "Their" or "There". This happens because the only context token available is the implicit sentence-start marker, not a real surrounding word. The rule should not fire when the tokenized sentence contains only the start-of-sentence token and one word token; without at least one real neighbour token there is no meaningful confusion signal. Multi-word sentences must continue to work as before (they will have at least three tokens: _START, word1, word2).

don't trigger confusion rule if there's no context other than the sentence start token


Title: `ShortDescriptionProvider` static caches cause flaky `ClassCastException` under concurrent access
Type: Bug / Thread-safety
Affected: `languagetool-core` — `org.languagetool.ShortDescriptionProvider`
Priority: High (causes runtime failures in multi-threaded server deployments)

### Problem
`ShortDescriptionProvider` maintains two static, non-synchronized collections (`wordToDef` and `initializedLangs`) to lazily load and cache word definitions per language. When the provider is called from multiple threads (as happens in the HTTP server), concurrent modifications and concurrent reads can corrupt the internal structures of `HashMap` and `HashSet`, leading to flaky `ClassCastException` or other undefined behaviour. This makes the proofreading server unreliable under load.

### Expected behaviour
- The provider must be safe for concurrent use without requiring callers to perform external locking.
- Descriptions for a language are still loaded from the resource file (`word_definitions.txt`) on first access and cached for the lifetime of the JVM.
- The public method `getShortDescription(String word, Language lang)` must remain functional and thread-safe.
- The existing `getAllDescriptions(Language lang)` method (used by tests) must continue to return a view of the cached data for a given language.

### Notes
- Only the word-definition caching modelled by this class is in scope; other parts of the proofreading engine are not affected.
- The resource format (tab-separated, ignoring comments / blank lines) and the file location (`/<lang>/word_definitions.txt`) remain unchanged.

### Acceptance Criteria
- No `ClassCastException` or other concurrent-modification failures in a simple multi-threaded stress test repeatedly calling `getShortDescription` for the same language.
- Existing behaviour for single-threaded use and for languages without a definitions file (should return null / empty) is preserved.

make ShortDescriptionProvider thread-safer and shorter


**Title:** MultitokenSpeller ignores alternative perfect matches that differ only by casing/diacritics

**Context**
The `MultitokenSpeller` (used e.g. by `FrenchMultitokenSpeller`) provides spelling suggestions for multi-word expressions like “gréco-romain”. Currently when the speller finds one exact match after lowercase/diacritic normalization (distance 0), it stops and returns only that candidate. But there can be several valid forms that are all exact matches and differ only in casing or diacritic marks, e.g. “Gréco-Romain” and “gréco-romain”. The first one encountered wins, which may not match the user’s intended capitalization.

**Desired behavior**
`getSuggestions()` should be able to return multiple candidates that are all zero-distance matches, so that both “Gréco-Romain” and “gréco-romain” appear when the input is “greco-romain”. Collecting a small bounded number (e.g., up to 2) of zero-distance matches before stopping is sufficient—unlimited collection is not needed and could harm performance. Candidates with non-zero distance should continue to be filtered as before, and the existing weighting logic should still apply: only candidates with the same minimal distance as the first added candidate should appear in the final suggestion list.

**Acceptance criteria**
1. For an input like “greco-romain”, the output includes both “Gréco-Romain” and “gréco-romain” (order is not critical).
2. The total number of returned suggestions for a given input remains small—no regression that causes dozens of near-identical entries.
3. The change is isolated to the zero-distance handling in `MultitokenSpeller.getSuggestions()`; everything else (distance calculation, weighting, final filtering of non-zero-distance candidates) stays unchanged.

multitoken speller: allow several candidates with different casing


## Bug Report: DE_AGREEMENT false positives with color names and conjunctions

The German grammar rule `DE_AGREEMENT` incorrectly flags several valid sentences as having noun-phrase agreement errors. This makes the tool unreliable for texts where such constructions are common.

### Steps to reproduce
Check the following sentences with LanguageTool's German grammar checker (e.g. via the public HTTP API or the LibreOffice integration):

1. `Das Orange ist schön.`
2. `Dieses rötliche Orange gefällt mir am besten.`
3. `Für die Religiösen ist das Blasphemie und führt zu Aufständen.`

### Expected behavior
All three sentences are grammatically correct German and should not be marked with an agreement error.

### Actual behavior
Each sentence triggers a false positive under `DE_AGREEMENT`. LanguageTool reports e.g. a mismatch between “Das” and “Orange” (example 1), between “Dieses / rötliche” and “Orange” (example 2), and between “das” and “Blasphemie” when the sentence continues with a conjunction (example 3). In contrast, the sentence `Für die Religiösen ist das Blasphemie.` (without the continuation) is already handled correctly.

### Impact
These false positives degrade the user experience for anyone writing about colors or using constructions where a predicate noun is followed by a conjunction. The rule should be adjusted to recognize that:

- Color names like “Orange” can be used with neuter determiners and adjectives without agreement violation.
- A predicate noun like “Blasphemie” in the given pattern should also be allowed when the sentence continues with a conjunction (`und`, etc.), not only at a hard sentence boundary.

[de] improve DE_AGREEMENT (closes #1591)


**Title:** Catalan unpaired ¡/¿ rules don't suggest the right placement when sentences start with dialogue dashes

**Description**
For Catalan, the `CA_UNPAIRED_EXCLAMATION` and `CA_UNPAIRED_QUESTION` rules are meant to flag sentences that end with `!` or `?` but lack the opening `¡` or `¿`, and to propose adding the missing mark. Currently these rules either suggest the wrong insertion position or fail to trigger when the sentence begins with a punctuation character like a hyphen or dash (commonly used for dialogue).

The same problem exists in the Spanish `QuestionMarkRule` (which also covers exclamations): when a sentence such as `—Hola!` is missing the opening `¡`, the suggestion incorrectly places the mark before the dash (`¡—Hola`) instead of before the first word (`¡Hola`).

**Expected behavior**
When a sentence is missing an opening exclamation or question mark and the sentence begins with any punctuation (e.g., `-`, `—`, etc.), the system must place the missing mark immediately before the first *word* token, skipping all leading punctuation.

**Concrete examples**
- Catalan "-Com estàs?" → suggest `¿Com` (not `¿-Com` or no suggestion)
- Catalan "-Benvinguda!" → suggest `¡Benvinguda`
- Spanish "—Hola!" → suggest `¡Hola` (not `¡—Hola`)

**Acceptance criteria**
1. For Catalan, `CA_UNPAIRED_QUESTION` and `CA_UNPAIRED_EXCLAMATION` must produce the same correct placement as the Spanish rule: the inverted mark is placed before the first word after any leading punctuation.
2. For Spanish, the existing `QuestionMarkRule` must skip leading punctuation when determining the insertion point, so that marks are not inserted before dashes, hyphens, or other punctuation at the start of a sentence.
3. All other cases (no leading punctuation) must continue to work as before.

[es,ca] improve suggestions: question/excamation mark


**Title:** Spanish/Catalan inverted punctuation rules misbehave on non‑final `?`/`!` and duplicate logic

**Affected rules**
- `QuestionMarkRule` (Spanish)
- `CatalanUnpairedQuestionMarksRule`
- `CatalanUnpairedExclamationMarksRule`

**Observed problem**
When a sentence contains a `?` or `!` that is **not the end** of the sentence (e.g. inside a quoted phrase, a parenthetical remark, or where sentence splitting incorrectly keeps following text), the rule sometimes:

- Suggests inserting an opening inverted mark at the very beginning of the sentence instead of at the actual clause that needs it.
- Produces false‑positives (flagging a missing opener when one isn’t needed) or misses real missing openers.

**Expected behaviour**
If a `?` or `!` token is not a sentence-ending punctuation (i.e. it is not marked as `isSentenceEnd`), the search for the correct insertion point should restart **after** that token. This avoids anchoring the suggestion to the wrong part of the sentence and prevents incorrect matches when internal punctuation doesn’t actually require an opening mark.

**Additional note (Catalan)**
`CatalanUnpairedExclamationMarksRule` currently duplicates the matching logic from `CatalanUnpairedQuestionMarksRule` with hard‑coded `¡`/`!` instead of `¿`/`?`. Any improvement should be applied to both punctuation types, and ideally the exclamation rule should inherit the common behaviour to avoid future drift.

**Goal**
- Improve the inverted‑mark detection so that non‑final `?`/`!` do not cause misplacement or spurious warnings.
- Reduce duplication in the Catalan rules by sharing the core logic between question and exclamation variants.


```
Title: [LO extension] DocumentCache incorrectly marks paragraphs as different when footnotes have no visible marker

The LibreOffice extension’s document cache compares a flat paragraph representation (which uses zero-width spaces for footnote anchors) with the text paragraph (which may contain digits or Roman numerals for footnotes). The comparison currently assumes every footnote anchor corresponds to at least one visible character in the text paragraph. This is not always true: when a footnote anchor appears immediately before a character like '[' or in other positions where the footnote number is not shown, the comparison fails and considers the paragraph changed even though it is identical.

This causes the cache to be bypassed, leading to spurious re-checking of unchanged text and degraded performance.

The fix should be in the DocumentCache class, specifically the logic that enumerates possible lengths of the visible footnote representation. It must allow a footnote anchor to be matched with zero visible characters so that such paragraphs are correctly recognized as equal.
```

[LO extension] fix of problem in document cache generation - related to issue #8012


Fix the English translation inflector so it never silently drops a token it cannot inflect. Right now, when the German POS tag is `null`, empty, or any tag not covered by an `if`/`else if` branch, the result is an empty list and the token is omitted from translation suggestions. Expected behaviour: in all of those cases the uninflected English token should be preserved as a fallback form.

Additionally, the comparative-adjective branch only matches the exact pattern `ADJ:PRD:KOM`. German tags frequently use other sub-formats (e.g., with case/number/gender attributes before the `KOM` segment, like `ADJ:AKK:SIN:FEM:KOM:DEF`). Broaden that comparison so all comparative adjective tags are captured and produce the English comparative form, instead of falling through and being dropped.


# Optimize text matching in PatternToken by extracting and interning matchers

**Background**
When rules are loaded from XML, each `PatternToken` creates its own text matcher (regex or plain string). Many token definitions share identical `(pattern, isRegExp, caseSensitive)` triples, but this duplication is not exploited. Furthermore, the current matching logic could be made faster for common simple patterns: e.g., alternation (`foo|bar`), character classes (`[abc]`), and optional parts (`a?`) can be precomputed into a finite set of possible values, allowing set lookup instead of full regex matching.

**Request**
Extract the text matching logic out of `PatternToken` into a dedicated, shareable matcher object. The key behaviours to deliver:

1. **Reuse identical matchers**
   In `XMLRuleHandler`, intern the matchers so that any two tokens with the same `(pattern, isRegExp, caseSensitive)` use a single matcher instance.

2. **Optimized matching strategy**
   The matcher should internally select the cheapest correct matching algorithm based on the pattern:
   - Plain string (or regex that reduces to a single literal) → string equality (case-sensitive or case-insensitive).
   - Simple regex that can be expanded into a finite set of strings → set membership check (case-sensitive or case-insensitive).
   - General regex → standard `Pattern.compile` + `Matcher.matches()` (with the existing `InterruptibleCharSequence` wrapper for interruptibility).
   - The special pattern `\0` must be handled correctly (treated as a plain literal, not a regex).

3. **Preserve contract**
   All existing matching semantics must remain identical, including the handling of case sensitivity, the inflected/lemma test token selection, negation, and the `testString` flag.

**Non-goals**
- This ticket does not ask to change the public API of `PatternToken` beyond what is necessary for the extraction; `getString()`, `isCaseSensitive()`, and `isRegularExpression()` should remain available.

**Location hints**
- The current text matching setup lives in `PatternToken` (fields `stringToken`, `caseSensitive`, `stringRegExp`, `stringMatcher`; methods `createMatcher()`, `stringEquals()`, `isStringTokenMatched()`, `getPossibleRegexpValues()`, etc.).
- Rule loading and token construction happen in `XMLRuleHandler` (methods `finalizeExceptions`, `finalizeTokens`).

extract text matching with regexp optimizations out of PatternToken


Title: Unpaired brackets false positive for uppercase list markers in multi-line input

GenericUnpairedBracketsRule currently raises a false “unpaired bracket” error for closing parentheses in lists that use uppercase letters (e.g., `A)`, `B)`, `C)`).  This happens when the text is checked as a whole document (the text-level rule path, e.g. via `analyzeText`) rather than as an isolated sentence.  The rule already correctly suppresses these false positives for lowercase list markers (`a)`, `b)`, …) in the same scenario.

**Example input that should be accepted without errors:**
```
A) New York
B) Boston
C) Chicago
```

**Expected behaviour:**
The suppression logic for list-style closing parentheses must treat uppercase and lowercase letters identically.  It should work for both sentence‑level and document‑level analysis.

**Background / suspected cause:**
The existing suppression code extracts a context string from the *current sentence’s* text, but the position offset (`startPos`) is relative to the *entire* analysed text (all sentences).  This mismatch means the context lookup can fail for list markers that appear after the first sentence or when the full text contains leading newlines, causing the rule to emit a false positive for uppercase letters while lowercase letters still happen to pass through another branch.  The fix should make the context extraction use the complete text so the offset is always valid.


**Title:** Add POS tag support for nominalized separable verbs from the spelling dictionary

**Description:**
The German spelling dictionary (`spelling.txt`) contains separable verb entries such as `herum_geben` and `mit_managen`. The tagger already uses these to recognise verb forms like `herumzugeben`, but the corresponding nominalized forms are not tagged at all:

- `Herumgeben` / `Mitmanagen`  (das Herumgeben, das Mitmanagen)
- `Herumgebens` / `Mitmanagens`  (des Herumgebens, des Mitmanagens)

**Acceptance Criteria:**
1. For any separable verb entry in `spelling.txt` (format `prefix_verb` where the prefix starts with a lowercase letter), the tagger must recognise the capitalised nominalized infinitive (e.g., `Mitmanagen`) and its genitive (e.g., `Mitmanagens`) as nouns.
2. The base form without `s` (e.g., `Mitmanagen`) shall receive readings with the following POS tags:
   - `SUB:NOM:SIN:NEU:INF`
   - `SUB:AKK:SIN:NEU:INF`
   - `SUB:DAT:SIN:NEU:INF`
3. The genitive form (base + `s`, e.g., `Mitmanagens`) shall receive a reading with:
   - `SUB:GEN:SIN:NEU:INF`
4. The lemma for all readings must be the base nominalized form (e.g., `Mitmanagen`).
5. Only words with a capitalised prefix should be tagged as nouns; lowercase forms (e.g., intermediate verb stems) must not receive these noun tags.
6. The existing recognition of verb forms like `herumzugeben` must continue to work unchanged.

**Edge cases / safety:**
- Entries in `spelling.txt` that contain an underscore but are not separable verbs (for example `Manager_in`) must not generate nominalized noun readings.
- The implementation should not break any existing tagging behaviour for unrelated words.

[de] add POS tags for spelling.txt entries like "herum_geben" -> "(das) Herumgeben", "(des) Herumgebens"


**Title**: Filter out newline and invisible separator characters from rule match boundaries and suggestions

**Background**
Some rules produce `RuleMatch` instances where the matched text (and the corresponding `fromPos`/`toPos`) inadvertently includes leading or trailing newline (`\n`) characters or the invisible separator character `\u2063` (U+2063, INVISIBLE SEPARATOR). This can happen, for example, when a match spans across sentence boundaries or when markup replacements leave these characters adjacent to the actual error. The result is unhelpful suggestions (e.g., suggesting to replace a word together with surrounding newlines) and mis-aligned highlights.

**Request**
Add a new `RuleMatchFilter` implementation that post-processes all matches from `JLanguageTool` and:

1. Trims any leading or trailing `\n` or `\u2063` characters from the match text (as reported by the original text) AND from each suggestion replacement string.
2. Adjusts the match’s `fromPos`/`toPos` accordingly, so the match offset now points only to the remaining content.
3. If trimming causes `toPos ≤ fromPos` (i.e., the entire match was composed only of newline and/or invisible separator characters), the match must be **removed** entirely.
4. If after trimming the (single) suggestion replacement equals the trimmed match text, the match must also be **removed** (it would be a no-op correction anyway).
5. If the match range originally extends beyond the length of `AnnotatedText#getOriginalText()`, leave the match untouched to avoid out-of-bounds errors.
6. The filter must be active by default – it should be added to the list of `matchFilters` inside `JLanguageTool` (alongside any existing filters) so that it is automatically applied when calling `check()`.

The filter should use the original text obtained from the `AnnotatedText` object to extract the match substring and perform the trimming logic. The invisible separator character to recognize is the single Unicode code point `\u2063`. The trimming must be applied to both the beginning and end of the matched substring and of each suggestion, with “endsWith” and “startsWith” checks repeated until no more removable characters are found (this ensures consecutive newlines/separators are all stripped).

**Existing interface**
`RuleMatchFilter` (see `DictionarySpellMatchFilter` for an example) provides the method `List<RuleMatch> filter(List<RuleMatch> ruleMatches, AnnotatedText text)`. The new filter should implement that interface.

**Acceptance criteria (observable behavior)**
- A `RuleMatch` that spans `"Headline\n\n\n\n\u2063\n\n\n\n"` in the original text should be trimmed to span only `"Headline"`, and any suggestion like `"Headline\n\n\n\n\u2063\n\n\n\n"` should become `"Headline"`.
- A match whose text is `"Heeadline\n\n\n\n\u2063\n\n\n\n"` where the suggestion is `"Headline\n\n\n\n\u2063\n\n\n\n"` should be dropped because after trimming the suggestion matches the trimmed match text ("Heeadline" vs "Headline" — not dropped because they differ; but if suggestion after trimming equals the match after trimming, it is dropped).
- A match consisting entirely of newlines and invisible separators should be removed from the result list entirely.
- A match where the original text length is shorter than `toPos` should be kept as-is.

For implementation, the filter will likely need to use `text.getOriginalText().substring(from, to)` to extract the matched portion, then perform trimming loops and finally call `ruleMatch.setOffsetPosition` and `ruleMatch.setSuggestedReplacements` on the mutated match objects.

Add NewLineMatchFilter to filter rule matches containing trailing/leading newlines or invisible separators


Subject: Invalid regex patterns not rejected by LanguageTool and possible mismatches with Java's standard regex engine

Description:

When pattern rules contain invalid regular expressions (for example, a trailing `?` alone like `tú|?`), LanguageTool silently accepts them instead of raising a `PatternSyntaxException`. This can happen when users define custom patterns or when existing rules accidentally contain broken regex syntax.

Additionally, I noticed that the optimized string matching for regex tokens sometimes produces results that differ from what `java.util.regex.Pattern` would return for the same input. This leads to subtle false positives/negatives that are hard to diagnose because the mismatch depends on the specific input string.

Expected behavior:

1. `StringMatcher.create()` (or the underlying factory) must validate the regex pattern early and throw a `PatternSyntaxException` if the pattern is invalid, before any matching is attempted.
2. For any valid regex pattern and any candidate string, the match result produced by LanguageTool’s matcher should be identical to the result obtained from `Pattern.compile(...).matcher(...).matches()`, taking case sensitivity into account. In particular:
   - When case-sensitive matching is used, the behavior must match `String.matches(regexp)`.
   - When case-insensitive matching is used, the behavior must match `Pattern.compile(regexp, Pattern.CASE_INSENSITIVE | Pattern.UNICODE_CASE).matcher(input).matches()`.
3. The helper that extracts possible regexp values should not attempt to process invalid patterns; instead, validation should occur first.

Impact:

- Rule authors are not warned about syntax mistakes, making debugging more time-consuming.
- Inconsistent matching can lead to incorrect error detection results (missed errors or false alarms), degrading the proofreading quality.
- The issue affects all languages that use pattern rules with regex tokens.

I kindly ask to ensure that all regex patterns accepted by LanguageTool are syntactically valid and that the match semantics are fully aligned with Java’s standard regular expression engine.

StringMatcher: assert only correct regexps are used, add more equivalence checks in tests


supervision_value: 2
difficulty: 2
reasoning: The fix requires a small, localized change in one method plus a test. An agent would run the failing test, examine `parseBuffer`, notice the trailing‑newline edge case in splitting and multiple‑line detection, apply a trim and adjust the newline check, then rerun tests — a narrow debugging workflow with limited broader investigation value.

---

**Implementation request:**

`FastText.parseBuffer` currently mishandles buffers that end with a newline character. The trailing `\n` causes `split("\\s+")` to produce a trailing empty element, which can trigger a false “multiple lines” warning or a parsing error. Adjust the method so that:

- Leading and trailing whitespace (including leading/trailing newlines) does not affect tokenisation or the extraction of label–value pairs.
- The warning about multiple lines is emitted only when a `\n` appears *before* the last non‑whitespace character in the buffer — a single trailing newline should be treated as a single‑line output from FastText.


The Ukrainian word tokenizer does not split the multi-abbreviation `т. зв.` (так званий/so-called) correctly.
For instance, the phrase `до т. зв. сальону` should tokenize into `до`, `т.`, `зв.`, `сальону` — mirroring the existing handling of `і т. д.` and similar compound abbreviations.
The pattern must account for regular spaces, non-breaking spaces (`\u00A0`), and narrow no-break spaces (`\u202F`) between the two abbreviation parts, consistent with how other multi‑abbreviation patterns are implemented.


**Request:**
Fix two issues in the German speller's split-word error handling:

1. When a split-word error (`g anz` → `ganz`) occurs right after an ignored token (e.g. a function word on the ignore list), the error span’s start position is wrong. The suggestion is either missing or its reported offset doesn’t match the actual token position.
2. The suggested replacement string sometimes has a trailing space when one of the joined suggestion parts is empty.

Ensure that the correct span is reported and the suggestion string is clean (no trailing/leading whitespace). The fix should apply to the general split-word suggestion logic used by the Hunspell-based rules.

fix suggestion for one more case (#1729)


In the German verb agreement rule, sort the suggested corrections by how similar each suggestion is to the user's original text (the subject–verb span, including the surrounding words when the error spans multiple tokens). Use an edit-distance metric so that suggestions closer to the original are listed first. Currently suggestions appear in an arbitrary or alphabetical order, which often pushes the most natural fix far down the list. This sorting should happen both when the subject precedes the verb and when the verb precedes the subject.

[de] sort suggestions by distance to user's text


**Title:** Dutch compound acceptance falsely accepts words with 3 or more sub‑parts

**Component:** `languagetool-language-modules/nl` (compound word checking)

**Observable problem:**
The Dutch proofreader currently accepts some very long compound words that should be rejected. For example, the nonsense word `gezondheidsomlijningssvervangingsinfluencers` is accepted even though it contains more than two distinct word parts. Only two‑part compounds should be allowed; longer chains must be flagged as errors.

**Root cause suspected:**
The compound acceptor seems to recursively attempt to split a candidate into two parts, and for each part it checks whether that part is itself an acceptable compound. This recursion permits arbitrarily long compound chains and, in some paths, the tagger used inside those checks can itself synthesise compound analyses, creating an unintended loop that further inflates false acceptances.

**Acceptance criteria / required behaviour:**
- Any compound consisting of two lexical parts that satisfy the existing Dutch compound rules should still be accepted.
- A compound made of three or more lexical parts must be **rejected** (i.e. the spell‑checking rule should flag it).
- The internal sub‑word existence checks (`isNoun`, `isExistingWord`) must not recursively split a word into further compounds; they should only perform a raw dictionary/lemma lookup, avoiding any compound logic. This prevents both the looping issue and the false acceptance of longer compounds.
- Adjust error‑prone branches in the compound validation, notably:
  - When a part ends with `s` and should have its trailing `s` stripped for the lookup, ensure the casing of the lookup string matches the lower‑case variant so that tests using mixed‑case suffixes still pass.
  - In the branch for parts where no linking `s` is allowed but a colliding vowel is not detected, remove the duplicate `spellingOk(part2)` check – it is already guarded by `isNoun(part2)`.
- No changes to the public API of `CompoundAcceptor` or `DutchTagger` beyond what is needed to support the above; existing tests for valid two‑part compounds must continue to pass.

**Background context:**
- `CompoundAcceptor` is the class responsible for deciding whether a candidate word is a valid Dutch compound.
- `DutchTagger` (the language‑specific tagger) has its own compound‑handling code that may produce compound analyses; this is not appropriate for the raw existence check needed inside `CompoundAcceptor`.
- The link to the relevant tagger dictionaries and the logic for stripping interfixes (`s`, `e`, `en`, etc.) remains unchanged.

**Deliverable:**
A fix that implements the constraints above. Please ensure the unit tests cover the rejection of a multi‑part compound (e.g. `gezondheidsomlijningssvervangingsinfluencers`) while still accepting reasonable two‑part compounds like `webschoolboek`.

[nl] prevent 2+ part compound words from being accepted


When performing a hard reset (`GIT_RESET_HARD`), files that are only in the index as unresolved conflicts (stages 1, 2, or 3 without a stage-0 entry) are left behind in the working directory instead of being removed. A hard reset is expected to discard all local changes, including ongoing merges, resulting in a clean working tree that matches the target commit.

**Acceptance criteria:**
1. After a hard reset on a repository with one or more unmerged (conflicted) files, the working directory must no longer contain those files.
2. The cleanup must happen for any combination of conflict stages present—ancestor, ours, theirs—regardless of which stages exist for a given file path.
3. The behavior must treat an unmerged file as a tracked-and-dirty path, and therefore subject to forced removal (consistent with `GIT_CHECKOUT_FORCE` implied by hard reset).
4. Existing behavior for normal tracked files and untracked files must remain unchanged.
5. The reset operation must succeed, and subsequent inspection of the working directory (e.g., `git_path_exists` for the previously conflicted path) must indicate that the file no longer exists.


Revwalk uses a homegrown stop condition that re-scans the pending set multiple times and can bail out prematurely. Rework the walk preparation to behave like git:

- Process the initial set of user-supplied tips and hidden boundary commits in a single loop.
- When a commit is uninteresting, propagate `uninteresting` to its ancestors recursively without revisiting already-marked commits.
- Do not add the same commit to the pending list more than once during preparation (track visitation with a flag).
- Introduce a “slop” window: when no interesting commits remain, keep scanning candidates for up to 5 more rounds. If an earlier-dated commit reappears in that window, continue walking. This prevents missing objects in complex branch topologies.
- Ensure hide callbacks and first-parent simplification are applied during the preparation pass, so hidden commits are excluded and only the first parent is followed when requested.
- Preserve the invariant that the walk returns exactly the set of commits reachable from the tips, excluding hidden ones and their uninteresting ancestors. When sorting is unspecified, the output order may vary between runs, but the set of returned commits must be correct.


We need `git_revparse` to hand back resolved objects instead of just OIDs, so callers don’t have to perform an extra `git_object_lookup` / `git_object_free` pair themselves. Currently `git_revparse_single` returns a `git_object **`, but `git_revparse` still returns `git_oid` outputs, forcing every consumer to separately look up and free the objects. This causes unnecessary work and leads to repetitive lookup/free patterns throughout the codebase.

**Desired behaviour**
- `git_revparse` should accept `git_object **left` and `git_object **right` instead of `git_oid *left` and `git_oid *right`.
- On success the function stores a fully resolved object in each non‑NULL output pointer. The caller becomes the owner of those objects and must free them with `git_object_free` when done.
- The objects are the exact same ones that `git_revparse_single` would return for the left and right specifiers; no additional lookup should be needed.
- The `flags` output parameter remains unchanged and describes whether the spec targeted a single object, a range, or a merge‑base operation.

**Acceptance criteria**
1. After a successful call with a single‑spec (no `..` / `...`), `*left` is a valid `git_object *` representing the resolved object, and `*right` is `NULL` (if `right` was passed as `NULL`) or left unchanged.
2. After a successful range specification like `A..B`, both `*left` and `*right` are valid objects corresponding to the two sides of the range.
3. On error, no objects are returned (outputs should be left untouched or set to `NULL` as appropriate).
4. Existing callers that were doing `git_revparse` → `git_oid` → `git_object_lookup` can drop the lookup step entirely and use the objects directly, reducing code and avoiding an extra lookup/free cycle.
5. Internal uses (e.g., `git_revwalk_push_range`) benefit from the same simplification without changing walk behaviour.

**Safety / constraints**
- The function must not leave objects behind on partial success. If an error occurs after one object is resolved, that object should be freed before returning, or the function must guarantee that no outputs are set until both are ready.
- If `right` is `NULL`, the function works as a single‑object parse but still returns the object in `*left`.
- The change must not alter the spec‑parsing rules; only the output type changes.
- Memory ownership convention matches `git_revparse_single`: caller frees the returned objects.

This aligns `git_revparse` with `git_revparse_single` and eliminates a class of double lookups across the library and its examples.

Change git_revparse to output git_object pointers


Currently `git_reset` cannot operate when `HEAD` is an orphaned (unborn) branch – for example right after `git init` before any commit.
`git_repository_head` returns `GIT_EORPHANEDHEAD`, and the subsequent attempt to update the head reference fails because the branch does not exist yet.

Make `git_reset` handle this case for all reset types (soft, mixed, hard):

- If `HEAD` is orphaned, read the symbolic target of `HEAD` (the branch name it points to).
- Create a new direct reference under that name pointing to the target commit’s OID.
- `HEAD` must remain a valid symbolic reference to that branch; the repository is no longer orphaned after the call.
- If `HEAD` is *not* orphaned, behaviour is unchanged.

Constraints / safety:

- Support non-chained symbolic references only (e.g. `HEAD` → `refs/heads/master`). Chained indirection can be left for later.
- Clear the orphaned-head error state before proceeding; do not conflate it with other errors.
- The branch must be created with the exact OID of the commit object (peeling tags if needed), as the caller already resolved the target to a commit before handing it to the head-update routine.

reset: make git_reset() cope with an orphaned HEAD


The smart protocol negotiation buffer is not reset between successive negotiation steps in `git_smart__negotiate_fetch`. After each `git_smart__negotiation_step` the accumulator `data` retains the previous step’s output, which corrupts subsequent negotiation rounds. This manifests most visibly in shallow clones (e.g., depth=1) where multiple ACK/NAK iterations occur, often causing the fetch to stall or fail with protocol errors. Clear the `git_str` immediately after each negotiation step so that each round starts with a fresh buffer.

Additionally, the preprocessor guard for in-memory SSH credentials incorrectly uses `GIT_SSH_MEMORY_CREDENTIALS` instead of the correct `GIT_SSH_LIBSSH2_MEMORY_CREDENTIALS`. That guard must be corrected everywhere it appears so that features dependent on libssh2’s memory credential support are not inadvertently excluded.


The following request describes a set of missing and broken behaviors around user-defined diff drivers. All observations are about externally visible results—diff output, attribute mapping, and driver caching.

---

When a repository configures a diff driver (for example, via `diff.<name>.xfuncname` and `diff.<name>.funcname`), the diff output should include function-context headers that match those patterns. Right now it appears that custom patterns are never used; the hunk headers always fall back to the default.

Similarly, a driver defined solely to force text (i.e. `diff.<name>.binary = false`) should produce a normal text diff for files that use that driver, regardless of file content. Currently such a configuration seems to have no effect—files do not get forced to text.

Once a driver is loaded from the config, it should be cached so that subsequent diffs against files with the same driver name do not re-parse the config each time. The current behavior appears to re-load (or leak) drivers on every lookup.

There is also evidence that the config keys for the word-regex (`diff.<name>.wordregex`), function-name patterns (`diff.<name>.funcname` and `diff.<name>.xfuncname`), and the binary flag are not all read correctly because the key buffer may not be reset between reads, potentially leading to only the first key being fetched successfully.

The mapping of the `diff` attribute should be fully supported:
- Unspecified (`diff` not set) → use the default “auto” driver.
- `diff=false` → force binary diff.
- `diff=true` → force text diff.
- Any other value → treat as a driver name, load the corresponding driver from config, and fall back to auto if the named driver cannot be found.

The function-context matching loop that iterates over the user-provided patterns appears to have an inverted comparison that prevents it from ever matching a single pattern, rendering user-defined function headers completely broken.

The required fix should ensure that all the above behaviors work correctly: patterns actually match, binary/text forcing takes effect, drivers are cached, keys are read properly, and the attribute-to-driver mapping covers all four cases.


When `git_submodule_add_setup` (or similar submodule-add operations) receives a relative URL (e.g., `"../AnotherRepo"`), the library must resolve it to an absolute URL and store that resolved value in `.gitmodules`.  The resolution base must be determined with descending priority, matching upstream git behavior:

1. If HEAD is attached to a local branch that has a remote-tracking branch, use the URL of that remote.
2. Otherwise, if a remote named `"origin"` exists, use the URL of `"origin"`.
3. Otherwise, use the repository’s working directory path as the base.

The resolution must also be performed by `git_submodule_resolve_url` when it is called with a relative URL.

All other existing submodule functionality that relies on the stored URL (e.g., sync, open) should work transparently, as the resolved absolute URL is recorded in the submodule’s configuration entry (`submodule.<name>.url`).

**Edge cases / safety constraints:**
- If HEAD is detached or its branch has no upstream, the fallback chain must be followed gracefully without error (i.e., do not fail only because HEAD does not point to a tracking branch).
- If there is no `"origin"` remote, fall back to the workdir path without reporting a fatal error.
- The working directory base must be suitable for resolving relative file-system submodule paths (e.g., `"./"` resolves to the repository’s top-level directory).
- The resolved URL must be absolute; the stored value must never be a relative path.

git_submodule_resolve_url supports relative urls


The URL parser must reject '@' in the user part of the authority for every scheme except SSH.  RFC 3986 does not allow '@' in userinfo, yet the parser currently accepts it for all schemes.  That breaks correct handling of HTTPS (and other non-SSH) URLs, because a username like `user@domain` creates an ambiguous authority segment.

Only the SSH scheme needs the non-standard leniency: some SSH services (e.g., Google’s source.developers.google.com) rely on email-style usernames that include an '@'.  For SSH, continue to allow '@' in the username.  For every other scheme, encountering an '@' while parsing the userinfo part must produce a “malformed hostname” error.

Comparison of the scheme must be case-insensitive ("ssh", "SSH", "Ssh", etc.) and must not affect the delimiter '@' that separates userinfo from the host.

url: only allow @s in usernames for ssh urls


When a repository has shallow roots from a prior fetch and a subsequent fetch does not encounter those roots during negotiation, the still-valid shallow roots are dropped. The `out->size` field is never assigned in `setup_shallow_roots`, so the negotiated shallow-root array reports zero elements regardless of what was copied into its buffer. This causes the stored shallow file to lose unrelated roots across fetches, shrinking the shallow set incorrectly and potentially unshallowing commits that should remain cut off.

**Acceptance criteria**
- After a shallow fetch for a specific commit, a second shallow fetch for a different, unrelated commit must retain the shallow roots from both fetches.
- After a deep fetch that includes ancestors of one root but not another, only the newly satisfied root is removed from the shallow set; the unrelated root is preserved.
- The `shallow_roots` array returned from negotiation must accurately report the count of entries written into it.
- Unshallowing (depth = `GIT_FETCH_DEPTH_UNSHALLOW`) must clear all shallow roots and leave the repository non-shallow, verified by checking both the shallow state and the root count.
- After a fetch that brings in the full history of some previously shallow root, that root must no longer appear in the shallow root set, while any other shallow roots remain intact.