Case study: using Claude to remediate the eMudhra CA compromise across a Domino estate

by | Oct 28, 2026 | Dev Tips, Admin Tips | 0 comments

This is an example of how we used Claude to update our Domino servers following the eMudhra CA compromise.

It meant that this could be done across a fleet of different Domino servers from a central point relatively quickly. In reality it was a bit of a learning experience and took longer than it should, but it was still good and was useful learning in using AI to administer Domino servers.

Summary

HCL published CVE-2026-108014 (KB0134742) on 28 October 2026. The private keys of eight eMudhra intermediate CAs had been compromised, so a Domino server that trusts those chains can have its outbound TLS intercepted. No fixed release was available. The manual remediation, KB0134844, edits two trust stores on every server:

  • the JVM cacerts
  • the libcurl cacert.pem in each data directory

There were also some very useful articles by Daniel Nashed.

A managed-service provider used Claude Code, Anthropic’s AI coding agent, to do this work on its Azure-hosted Windows Domino servers. Claude ran on an administrator’s laptop with the Azure CLI and a Jira connector. It worked through these stages:

  1. Read the HCL material.
  2. Ran read-only discovery on each server.
  3. Wrote and tested a remediation script.
  4. Ran dry runs.
  5. Ran the live changes, including the Domino stops and restarts.
  6. Kept the Jira tickets up to date throughout.

An administrator approved each step that changed anything.

The fix worked, but not on the first attempt. Several problems came up along the way, and two of them were Claude’s own mistakes. Each one would have left servers exposed while looking finished. They are described in detail below, because they are the useful part of this account.

Background: where Domino keeps its trust

KB0134844 lists three places that can trust the compromised CAs.

StoreLocationUsed byKB0134844 action
JVM trust store<program dir>\jvm\lib\security\cacerts (one per install, shared by every partition)Java agents, XPages, anything on the Domino JVMkeytool -delete emSign ECC Root CA – G3
libcurl bundle<data dir>\cacert.pem (one per partition)LotusScript NotesHTTPRequest, OIDC, Domino IQDelete emSign ECC Root CA – G3, emSign Root CA – C1 and emSign ECC Root CA – C3, plus Comodo AAA “in older versions”. Restart.
Domino databasesnames.nsf Internet Certifiers, certstore.nsfDomino’s own TLS client featuresEither remove manually added eMudhra certifiers or use Claude to do this using the C API (more another time)

Two details matter in practice:

  • The cross-signed path. The compromised TLS intermediates chain to emSign ECC Root CA – G3. They are also cross-signed by Sectigo and chain to AAA Certificate Services (Comodo). Removing the emSign roots alone does not block the cross-signed path.
  • The test site https://dv-g3b-aaa-emudhra.we-have-a-problem.com/ serves exactly that chain: leaf → EM DV TLS CA – G3B → AAA Certificate Services.

The estate

  • All the servers are Azure Windows VMs running Domino 14.0 FP5 (14.0.005) with IBM Semeru Java 17.0.17, installed in E:\IBM\Domino.
  • Most are partitioned: Data1, Data2 and sometimes Data3, each its own HCLDominoServer(EIBMDominoDataN) service running as LocalService. On several servers only Data1 is running.
  • Every cacert.pem was the same 2023-11-09 file: 137 certificates, including all four targets.
  • Every JVM cacerts had 146 entries, including G3.
  • The work was split into four phases of 1, 5, 4 and 4 servers.

How Claude was used

Tools

  • Claude Code on the administrator’s laptop, with local PowerShell, Git Bash, Python, OpenSSL and a JDK keytool.
  • Azure CLI az vm run-command invoke to run PowerShell on each VM as SYSTEM. There was no RDP and no file copying; all server work went through Run Command.
  • The Atlassian connector, to create, update and link Jira tickets.

Workflow

  1. Ticket and research. Claude created the Jira ticket and read the bulletin. The KB0134844 page is a JavaScript single-page app (ServiceNow), so a normal fetch returns an empty shell. Claude retrieved the article body through ServiceNow’s page API (/api/now/sp/page?id=kb_article&sysparm_article=…). That gave the exact commands, aliases and PEM blocks.
  2. Planning only. The administrator stopped Claude from changing anything (“planning only — make no changes”). The plan was then reviewed and adjusted: backups on the server rather than locally, waits of 5 minutes with a check every 20 seconds, and verbose logs.
  3. Read-only discovery on each server. This listed:
    – the services, accounts and start types
    – the Domino and Java versions
    – the data directories
    – which of the four target certificates were present in each store, matched by SHA-256 fingerprint
  4. Script development with local tests. Claude ran the script against copies of the real files on the laptop before it ran anywhere else. The tests covered a dry run, a live run, a repeat run, a locked file and the restore path.
  5. Dry run on each server. This did everything except overwrite the live files. Backups were written and the changes staged and verified, but the live hashes were proved unchanged.
  6. Live run on each server, after explicit approval and at a time the administrator agreed for any customer outage.
  7. Post-checks.
    – The run log was checked for warnings.
    – Console errors since the restart were compared with the previous two days’ console logs, to separate new problems from existing ones.
    – Results were posted to the server’s Jira ticket.

The remediation script

remediate-emudhra-KB0134844.ps1 in the project folder. Its default mode is a dry run.

PhaseWhat it does
0: checks (read-only)Logs the versions, services and processes. Hashes the live files. Lists the keystore. Works out the PEM edit in memory. Any failure stops the run before anything is written.
1: backupsWrites <file>.pre-<ticket> next to each file, plus a copy in E:\Backup\<ticket>\ and a SHA-256 manifest. Never overwrites an existing backup. If a backup already holds an older version, it adds a dated one.
2: stage and verify (servers still running)Runs keytool -delete on a copy and checks that exactly one alias went. Rebuilds the PEM without the target blocks and checks the result is the original certificate set minus the targets, byte for byte (Latin-1 round trip, original line endings kept).
3: stopStops every running partition and waits until both the service and its processes are gone.
4: install (live only)Overwrites each live file in place, checks its hash, and logs its ACL before and after. On any error it restores from the backup.
5: startAlways runs, even after an error. Restarts what it stopped, in reverse order, and waits for “Server started on physical node” from the new nserver.exe PID.
6: final stateRe-hashes everything and reports the service states.

The script is safe to rerun. It skips any target that is already absent, and if nothing needs changing it does not stop Domino.

Problems encountered

These are listed roughly in the order they happened.

1. The wrong file name: cacerts.pem versus cacert.pem

What happened: the first searches looked for cacerts*.pem, with an “s”. They found nothing, and the ticket recorded “no cacerts.pem on the server”. The libcurl bundle is actually called cacert.pem, and every partition had one.

How it was caught: by a purely manual test, not by Claude. The administrator ran a server-side agent, based on Daniel Nashed’s test script, that opens the test page https://dv-g3b-aaa-emudhra.we-have-a-problem.com/. It still connected, which showed that LotusScript was trusting the chain through a store nobody had touched. A search using the correct name then found three copies of cacert.pem.

Lessons:

  1. Always test.
  2. “Not found” from a search is a finding about the search as much as about the server. Claude had added a positive control to the file search (it also searched for names.nsf, which must exist). That proved the search ran, but it couldn’t prove the file name was right.

2. AAA was in cacert.pem, and Claude said it wasn’t (Claude’s error)

What happened: Claude checked each PEM for AAA by searching the text for AAA Certificate Services.

  • A PEM file has no readable subject: the subject is inside the base64. The only readable text is curl’s heading above each block.
  • For AAA, that heading is “Comodo AAA Services root”. So the check would have reported “absent” whether or not AAA was there.
  • The emSign checks worked by luck, because their headings match their subjects.
  • Relying on this, Claude:
  • reported “AAA: absent” as a fact
  • wrote in the plan that HCL’s “older versions: also delete Comodo AAA” step did not apply
  • later argued that, because AAA wasn’t in cacert.pem, LotusScript’s trust must be coming from the NSFs

How it showed up: again through the administrator’s manual test. After the first live run, the server-side test agent on Server A, based on Daniel Nashed’s script, still connected to the eMudhra test site.

Diagnosis:
1. A byte search of nnotes.dll showed that libcurl 8.2.1 and OpenSSL 3.1.2 are built in, and that the binary refers to cacert.pem.
2. With set config Debug_NotesHTTPRequest=1, the console trace showed CAfile: e:\IBM\Domino\Data1\cacert.pem followed by SSL certificate verify ok for the G3B chain.
3. The same trace showed untrusted-root.badssl.com being rejected, so verification was clearly active.
4. A fingerprint check of cacert.pem then found AAA (D7A7A0FB…9EF4) in every file.

Fix and proof:
– The script now matches only by SHA-256 fingerprint and includes AAA as a fourth target.
– OpenSSL showed the effect directly. Verifying the live test chain against the previous file gives cert1.pem: OK. Against the new file it gives error 20 … unable to get local issuer certificate.

Cost: an extra restart of the first server, and time spent investigating the Domino Directory and certstore.nsf before the real cause was found.

Lessons:
– Identify certificates by fingerprint, never by name.
– Treat a negative result as unproven until the same check has found something you know is there.
– When a test contradicts the expected result, recheck your own earlier claims first.

3. Deleting AAA from the Domino Directory didn’t change the LotusScript result

Following a community write-up, the administrator deleted the AAA Internet Certifier from Server A’s names.nsf and restarted HTTP. The ping test still connected. The trace later explained why: on 14.0 FP5, NotesHTTPRequest checks against cacert.pem in the data directory. Directory certifiers matter for other parts of Domino, so the NSF check is still needed, but it does not fix LotusScript.

4. A running Domino locks its cacert.pem

The administrator asked for a “files only, no restart” approach, since changes only take effect after a restart anyway. The JVM cacerts swap worked, but the Data1\cacert.pem swap failed with “The process cannot access the file because it is being used by another process.” The script’s rollback restored the JVM file it had already changed.

The design then changed to stop every running partition first. A running partition locks its PEM, and the JVM store is shared, which is also why HCL says to shut Domino down first.

5. Swapping files changed the permissions

The first install method used [IO.File]::Replace, an atomic swap. That left the target with the staging folder’s permissions: two “ALL APPLICATION PACKAGES” read entries were lost. The restore that followed was an in-place copy, so it kept the wrong permissions.

The original permissions were all inherited, and icacls <file> /reset restored them exactly. Claude checked this entry by entry against the backup’s permissions. The script now overwrites in place, which keeps the target’s permissions, and logs the SDDL before and after every write. Permissions were unchanged in every later run.

A smaller trap found in local testing: in Windows PowerShell, passing $null to a .NET string parameter sends an empty string, so File.Replace(src, dst, $null) fails with “The path is not of a legal form”. The fix is [NullString]::Value.

6. keytool -cacerts waits for a password

HCL’s 14.x command (keytool -delete -alias "…" -cacerts) prompts for the store password. Under Run Command there is no console to answer it. The script uses -keystore <path> -storepass changeit instead. It always runs keytool -list first and copies the exact alias.

The aliases differ between stores. The 14.x JVM uses cn_emsign_ecc_root_ca__g3,o_emudhra_technologies_limited,ou_emsign_pki,c_in [jdk]. The older IA_jre store spells it c__in,_ou__emsign_pki,….

7. Run Command can’t send Domino console commands

nserver.exe =<notes.ini> -c "set config …" returned exit code 272 and did nothing. The Domino services run as LocalService, and Run Command runs as SYSTEM, so the console command can’t reach the running server. Claude stopped retrying. The other routes, such as editing notes.ini behind a running server or impersonating its account, were less safe than asking the administrator to type the command in the Domino Administrator console. The administrator did that.

8. Detecting “server started” reliably

Domino renames console.log when it starts, so “look for new lines after the old length” can miss the startup line. Each console line starts with [<nserver PID in hex>:…]. The script therefore waits for a Server started on physical node line from the new nserver.exe‘s PID, which it reads from the process list.

On these servers each partition took about 40 seconds to stop. Starting took between about 20 seconds and 2 minutes 15 seconds.

9. The debug trace writes credentials to the console log

With Debug_NotesHTTPRequest=1, libcurl’s verbose output goes to console.log. That includes the full URL with any embedded user:password@ and the response bodies. The ping utility used credentialled URLs.

Claude masked the credentials when it read the trace. It deleted the extract it had saved on the server and advised switching the trace off straight away. Don’t leave this setting on.

10. Other mistakes and slips by Claude

  • A paraphrase presented as a quotation. Early on, Claude put a “quote” from a community blog post into a ticket comment. It came from a web tool’s summary, not the original text. The administrator couldn’t find it. The comment was rewritten to cite a primary source (Apple’s support note) word for word. Lesson: a summarising fetch is not a quotation.
  • A scope edit that didn’t take effect. To make a “Server A only” live copy of the script, Claude used a sed command that changed $DryRun but not the scope line, because of an escaping error. Claude then told the administrator the run was Server A only. The run failed safely on the first file before reaching the other partitions’ files, but the claim was wrong. Since then, every live copy has been made with a check that the replacement matched exactly once, and the diff is shown before running.
  • A truncated summary. Run Command returns only about 4 KB of output. One verification run lost its key lines off the top of the output and had to be repeated. The script now writes a verbose log on the server and prints only a short summary.

11. Bad advice in a merged session

Notes from a separate Claude session were merged in partway through. They included Jira comments advising administrators to import the eMudhra intermediate into cacerts. That is the opposite of the fix: it adds trust. The comments also described the JVM’s trust in the test site as a good thing. These were corrected on the ticket.

Lesson: review AI-written content on the ticket for its direction, adding trust or removing it, as well as for its facts.

12. Smaller problems

  • Client-side test results meant nothing. The administrator’s Notes client failed HTTPS from LotusScript to every site, so a test that failed against the eMudhra site proved nothing. All testing moved to the server.
  • Windows line endings broke the server search. In Git Bash on Windows, the Azure CLI’s tab-separated output ends each line with a carriage return. A loop over subscription IDs then reported “Subscription not recognized” and found no VMs, until the carriage returns were stripped.
  • The Jira “internal note” setting was ignored. The Jira project is a standard software project, not a service desk, so the setting was silently dropped. Claude had already said in the comment that it was internal, and had to correct that.
  • A VM name differed from the server name. One server on the list had an Azure VM name with an extra -1 suffix, so a search by the listed name found nothing.

Results

PhaseServer (VM)Outcome
1VM 1About 3 minutes of downtime each time.
2VM 2About 2 minutes of downtime.
2VM 3About 1 minute of downtime.
2VM 4About 3 minutes of downtime.
2VM 5About 1½ minutes of downtime.
2VM 6About 1½ minutes of downtime.
3Phase 3
4Phase 4

VMs 3 to 6 were done in parallel, each after its own dry run. Their stopped partitions (Data2 and Data3) also had their cacert.pem fixed and were left stopped.

On every server that has been fixed:

  • JVM cacerts went from 146 to 145 entries.
  • Every cacert.pem went from 137 to 133 certificates and is now byte-identical across servers (SHA-256 8C61C1EF…5B64).
  • Permissions are unchanged, and the original files are kept on the server for rollback.

Still open:

  • Confirm the fix with the ping test on each server, after the AAA fix. OpenSSL confirms the trust change, but the production LotusScript test still needs to be run.
  • Check the NSFs (names.nsf and certstore.nsf).
  • Test the JVM change from Java code.
  • Decide what to do about the installer JRE’s IA_jre cacerts. The KB doesn’t mention it.

Advice for Domino administrators doing the same fix

  1. Check every partition’s cacert.pem, not just the JVM store. LotusScript, OIDC and Domino IQ use libcurl, and that reads the data directory’s PEM.
  2. Remove Comodo AAA from cacert.pem as well, at least on 14.0 FP5 with the 2023 bundle. The eMudhra test chain depends on it. Check first whether your own outbound integrations use AAA-anchored chains. In our traces they didn’t: we use Amazon, Let’s Encrypt and Starfield.
  3. Identify certificates by SHA-256 fingerprint.
    – G3: 86A1ECBA…BE6B
    – C1: 125609AA…608F
    – C3: BC4D809B…7EB3
    – AAA: D7A7A0FB…9EF4

The full values are in the child tickets.
4. Stop all partitions before changing the files. The JVM store is shared, and a running server locks its PEM.
5. Overwrite files in place rather than swapping them, or reset the permissions afterwards.
6. Use -keystore … -storepass changeit in scripts, not -cacerts.
7. Prove the result with a control. The eMudhra test site should fail, untrusted-root.badssl.com should fail, and your normal endpoints should still pass. Debug_NotesHTTPRequest=1 shows exactly which CA file was used and why, but switch it off straight away, because it logs credentials.
8. Expect the next HCL fix pack to replace these files. Re-check after upgrading.

What worked well with Claude, and what needed supervision

Worked well:

  • Finding and reading the primary sources, including the KB behind a JavaScript page.
  • Discovery across many servers in one pass.
  • Writing a careful, rerunnable script with backups, staging, rollback and verbose logs, and testing it locally before it went near a server, including the failure paths.
  • Keeping the Jira tickets accurate as the work went on, including correcting its own earlier statements.

Needed a human:

  • Every change, and every customer outage window, was approved by the administrator. The administrator moved Claude into planning-only mode when needed and narrowed or widened the scope.
  • Both trust-store gaps were found by the administrator’s manual testing, not by Claude. A server-side agent based on Daniel Nashed’s test script opened the eMudhra test page and still connected. Claude’s own checks had passed, but they were checking the wrong things. A real end-to-end test from the server is what showed the fix was incomplete.
  • Claude made confident statements that turned out to be wrong: “no cacerts.pem”, “AAA absent”, and the paraphrased quote. Each came from a check that couldn’t fail, or a source that wasn’t primary. The habits that caught them were the administrator’s insistence on dry runs and real-world tests, and Claude’s practice, once corrected, of verifying by fingerprint and with a known-positive control.

Appendix: the remediation script

⚠️ No warranty. Use at your own risk.

This code is provided “as is”, without warranty of any kind, express or implied, including but not limited to fitness for a particular purpose, correctness, or freedom from defects. The authors and their organisation accept no liability for any outage, data loss, security exposure or other damage arising from its use.

  • It stops and starts Domino servers and changes trust stores. Read it and understand it before running it.
  • It was written for one specific estate: Windows, Domino 14.0 FP5, E:\IBM\Domino, services named HCLDominoServer(EIBMDominoDataN). Check every path, service name and fingerprint against your own servers.
  • It runs as a dry run by default. Run it that way first, read the log, and test on a non-production server before any live run.
  • Have your own backups. Check the result with a real end-to-end test, such as NotesHTTPRequest against the eMudhra test page.

This copy has been anonymised: only the header comment, the backup folder and suffix, and the names of the local-test environment variables differ from the version that was run. It runs from Azure Run Command (az vm run-command invoke --command-id RunPowerShellScript --scripts @remediate-emudhra-KB0134844.ps1) or from an elevated Windows PowerShell 5.1 prompt on the server. Set $DryRun = $false only for the live run.

# eMudhra compromised CA remediation for HCL Domino on Windows, following HCL KB0134844
#
#   JVM trust store : delete emSign ECC Root CA - G3 from <Domino>\jvm\lib\security\cacerts
#   libcurl store   : delete emSign ECC Root CA - G3, emSign Root CA - C1, emSign ECC Root CA - C3
#                     and (KB: "older versions") Comodo AAA Certificate Services from <Data dir>\cacert.pem
#
# Certificates are matched by SHA-256 fingerprint only, never by name: the AAA root's heading in
# cacert.pem is "Comodo AAA Services root", which a name search for its subject misses.
# Safe to rerun: anything already removed is skipped, and a file needing no change is not touched.
#
# Order: checks -> backups -> build and verify staged copies (servers still running) ->
# stop EVERY running partition on the server -> copy the verified content over the live files
# (LIVE only) -> start the partitions it stopped, always, even after an error.
#
# All running partitions are stopped because the JVM cacerts is shared by every partition and
# HCL says to shut Domino down before changing it; a running partition also holds its cacert.pem
# open. Partitions that were already stopped are left stopped.
# Files are overwritten in place, which keeps their permissions; each is hash-checked afterwards.
# DRY RUN does everything except the overwrite; it stops and starts the partitions only when
# $DryRunRestarts is $true, otherwise it logs which partitions it would stop.
#
# Verbose log: <BackupRoot>\run-<stamp>.log on the server. Only a short summary is printed.

$DryRun       = $true            # $false = change the live files
$DataDirNames = @('Data1', 'Data2', 'Data3')   # partitions to process; a missing cacert.pem is skipped
$DryRunRestarts = $false         # $true = a dry run also stops and starts the partitions

# Environment overrides exist only for local testing off the server.
$DominoDir      = if ($env:EMUDHRA_DOMINODIR)  { $env:EMUDHRA_DOMINODIR }  else { 'E:\IBM\Domino' }
$BackupRoot     = if ($env:EMUDHRA_BACKUPROOT) { $env:EMUDHRA_BACKUPROOT } else { 'E:\Backup\eMudhra-KB0134844' }
$Keytool        = if ($env:EMUDHRA_KEYTOOL)    { $env:EMUDHRA_KEYTOOL }    else { Join-Path $DominoDir 'jvm\bin\keytool.exe' }
$ManageServices = -not $env:EMUDHRA_NOSERVICES
if ($env:EMUDHRA_LIVE -eq '1') { $DryRun = $false }

$ErrorActionPreference = 'Stop'
$Ticket         = 'eMudhra-KB0134844'
$RunStamp       = Get-Date -Format 'yyyyMMdd-HHmmss'
$Mode           = if ($DryRun) { 'DRY RUN' } else { 'LIVE' }
$PollSeconds    = 20
$MaxWaitSeconds = 300
$JvmCacerts     = Join-Path $DominoDir 'jvm\lib\security\cacerts'
$JvmAlias       = 'cn_emsign_ecc_root_ca__g3,o_emudhra_technologies_limited,ou_emsign_pki,c_in [jdk]'
$JvmAliasSha256 = '86A1ECBA089C4A8D3BBE2734C612BA341D813E043CF9E8A862CD5C57A36BBE6B'
$PemTargets = [ordered]@{
    '86A1ECBA089C4A8D3BBE2734C612BA341D813E043CF9E8A862CD5C57A36BBE6B' = 'emSign ECC Root CA - G3'
    '125609AA301DA0A249B97A8239CB6A34216F44DCAC9F3954B14292F2E8C8608F' = 'emSign Root CA - C1'
    'BC4D809B15189D78DB3E1D8CF4F9726A795DA1643CA5F1358E1DDB0EDC0D7EB3' = 'emSign ECC Root CA - C3'
    'D7A7A0FB5D7E2731D771E9484EBCDEF71D5F0C3E0A2948782BC83EE0EA699EF4' = 'AAA Certificate Services (Comodo AAA Services root)'
}
$ServiceNames = @{
    Data1 = 'HCLDominoServer(EIBMDominoData1)'
    Data2 = 'HCLDominoServer(EIBMDominoData2)'
    Data3 = 'HCLDominoServer(EIBMDominoData3)'
}
$StopOrder    = @('Data2', 'Data1', 'Data3')   # start order is the reverse
$BackupSuffix = ".pre-$Ticket"
$StageDir     = Join-Path $BackupRoot ('{0}-{1}' -f $(if ($DryRun) { 'dryrun' } else { 'live' }), $RunStamp)
$LogFile      = Join-Path $BackupRoot "run-$RunStamp.log"
$Manifest     = Join-Path $BackupRoot 'manifest-sha256.txt'
$Latin1       = [Text.Encoding]::GetEncoding(28591)   # byte-for-byte round trip of the PEM files
$Summary      = New-Object System.Collections.Generic.List[string]

New-Item -ItemType Directory -Force -Path $BackupRoot | Out-Null

# ---------------------------------------------------------------- helpers

function Log([string]$Message, [switch]$Important) {
    $line = '{0:yyyy-MM-dd HH:mm:ss.fff} {1}' -f (Get-Date), $Message
    Add-Content -LiteralPath $LogFile -Value $line -Encoding UTF8
    if ($Important) { $script:Summary.Add($line) }
}

function Get-Sha256([string]$Path) { (Get-FileHash -LiteralPath $Path -Algorithm SHA256).Hash }

function Describe([string]$Path) {
    $i = Get-Item -LiteralPath $Path
    '{0} bytes, modified {1:yyyy-MM-dd HH:mm:ss}, SHA256 {2}' -f $i.Length, $i.LastWriteTime, (Get-Sha256 $Path)
}

function Invoke-Native([string]$Exe, [string[]]$Arguments) {
    # Windows PowerShell turns native stderr into errors; collect it as text instead.
    $old = $ErrorActionPreference
    $ErrorActionPreference = 'Continue'
    try {
        $out  = & $Exe @Arguments 2>&1 | ForEach-Object { "$_" }
        $code = $LASTEXITCODE
    } finally { $ErrorActionPreference = $old }
    [pscustomobject]@{ ExitCode = $code; Output = @($out) }
}

function Read-FileShared([string]$Path, [long]$Offset = 0) {
    # Domino keeps console.log open, so read with full sharing.
    $fs = [IO.File]::Open($Path, 'Open', 'Read', 'ReadWrite, Delete')
    try {
        if ($Offset -gt $fs.Length) { $Offset = 0 }
        [void]$fs.Seek($Offset, 'Begin')
        $buf = New-Object byte[] ($fs.Length - $Offset)
        $read = 0
        while ($read -lt $buf.Length) {
            $n = $fs.Read($buf, $read, $buf.Length - $read)
            if ($n -le 0) { break }
            $read += $n
        }
        $Latin1.GetString($buf, 0, $read)
    } finally { $fs.Dispose() }
}

function Get-ConsoleLog([string]$DataDirName) { Join-Path $DominoDir "$DataDirName\IBM_TECHNICAL_SUPPORT\console.log" }

function Log-ConsoleTail([string]$DataDirName, [int]$Lines = 25) {
    $path = Get-ConsoleLog $DataDirName
    if (-not (Test-Path -LiteralPath $path)) { Log "  console.log not found: $path"; return }
    $len  = (Get-Item -LiteralPath $path).Length
    $text = Read-FileShared $path ([Math]::Max(0, $len - 12000))
    Log "  Last $Lines lines of $path ($len bytes):"
    $text -split "`r?`n" | Where-Object { $_ -ne '' } | Select-Object -Last $Lines | ForEach-Object { Log "    | $_" }
}

function Get-PartitionProcesses([string]$DataDirName) {
    $pattern = [regex]::Escape((Join-Path $DominoDir "$DataDirName\notes.ini"))
    @(Get-CimInstance Win32_Process | Where-Object { $_.CommandLine -and $_.CommandLine -match $pattern })
}

function Format-Processes($Processes) {
    if (-not $Processes -or $Processes.Count -eq 0) { return '(none)' }
    ($Processes | Sort-Object Name, ProcessId | ForEach-Object { "$($_.Name)[$($_.ProcessId)]" }) -join ', '
}

function Wait-Until([string]$What, [scriptblock]$Probe) {
    # Probe returns @{ Done; Detail }. Checks every $PollSeconds for at most $MaxWaitSeconds.
    $deadline = (Get-Date).AddSeconds($MaxWaitSeconds)
    $check = 0
    while ($true) {
        $check++
        $state = & $Probe
        Log "  [$What] check $check : $($state.Detail)"
        if ($state.Done) { return $true }
        if ((Get-Date) -ge $deadline) { return $false }
        Start-Sleep -Seconds $PollSeconds
    }
}

function Get-KeystoreEntries([string]$Path) {
    $r = Invoke-Native $Keytool @('-list', '-v', '-keystore', $Path, '-storepass', 'changeit')
    if ($r.ExitCode -ne 0) { throw "keytool -list failed on $Path (exit $($r.ExitCode)): $($r.Output -join ' | ')" }
    $entries = New-Object System.Collections.Generic.List[object]
    $alias = $null
    foreach ($l in $r.Output) {
        if ($l -match '^Alias name: (.*)$') { $alias = $Matches[1].Trim() }
        elseif ($alias -and $l -match '^\s*SHA256: ([0-9A-F:]+)') {
            $entries.Add([pscustomobject]@{ Alias = $alias; Sha256 = ($Matches[1] -replace ':', '') })
            $alias = $null
        }
    }
    $declared = $r.Output | Where-Object { $_ -match '^Your keystore contains (\d+) entr' } | ForEach-Object { [int]$Matches[1] } | Select-Object -First 1
    $type     = $r.Output | Where-Object { $_ -match '^Keystore type:' } | Select-Object -First 1
    if ($declared -ne $entries.Count) { throw "keytool reported $declared entries but $($entries.Count) were parsed from $Path" }
    [pscustomobject]@{ Type = $type; Count = $declared; Entries = $entries }
}

function Get-PemCerts([string]$Text) {
    $sha = [Security.Cryptography.SHA256]::Create()
    $rx  = [regex]'-----BEGIN CERTIFICATE-----(?<b64>[^-]*)-----END CERTIFICATE-----'
    foreach ($m in $rx.Matches($Text)) {
        $der = [Convert]::FromBase64String(($m.Groups['b64'].Value -replace '[^A-Za-z0-9+/=]', ''))
        $fp  = -join ($sha.ComputeHash($der) | ForEach-Object { $_.ToString('X2') })
        $subject = try { ([Security.Cryptography.X509Certificates.X509Certificate2]::new($der)).Subject } catch { '?' }
        [pscustomobject]@{ Index = $m.Index; Length = $m.Length; Sha256 = $fp; Subject = $subject }
    }
}

function Get-PemPatch([string]$Path) {
    # Works out the exact byte ranges to remove; changes nothing on disk.
    $bytes = [IO.File]::ReadAllBytes($Path)
    $text  = $Latin1.GetString($bytes)
    $certs = @(Get-PemCerts $text)
    $crlf  = ([regex]::Matches($text, "`r`n")).Count
    $lf    = ([regex]::Matches($text, "`n")).Count
    Log "  $Path : $($bytes.Length) bytes, $($certs.Count) certificates, line endings CRLF=$crlf LF-only=$($lf - $crlf)"

    $spans = New-Object System.Collections.Generic.List[object]
    foreach ($fp in $PemTargets.Keys) {
        $name = $PemTargets[$fp]
        $hits = @($certs | Where-Object { $_.Sha256 -eq $fp })
        if ($hits.Count -eq 0) { Log "  '$name' ($fp): not present - nothing to remove"; continue }
        if ($hits.Count -gt 1) { throw "Found '$name' ($fp) $($hits.Count) times in $Path - expected at most one" }
        $c     = $hits[0]
        $start = $c.Index
        $end   = $c.Index + $c.Length
        $tail  = [regex]::Match($text.Substring($end), '\A\r?\n')
        if ($tail.Success) { $end += $tail.Length }
        # Remove the "<blank line><name>\n====\n" heading above the block as well.
        $header = [regex]::Match($text.Substring(0, $c.Index), '(?<=\n)\r?\n(?<name>[^\r\n]+)\r?\n=+\r?\n\z')
        $headerName = '(no heading found - removing the certificate block only)'
        if ($header.Success) { $start = $header.Index; $headerName = $header.Groups['name'].Value }
        Log "  Target '$name' found: subject '$($c.Subject)', SHA256 $fp"
        Log "    heading in file: '$headerName'; removing chars $start-$end ($($end - $start) bytes)"
        $spans.Add([pscustomobject]@{ Name = $name; Sha256 = $fp; Start = $start; End = $end })
    }

    $ordered = @($spans | Sort-Object Start)
    for ($i = 1; $i -lt $ordered.Count; $i++) {
        if ($ordered[$i].Start -lt $ordered[$i - 1].End) { throw "Removal ranges overlap in $Path" }
    }
    $sb = New-Object System.Text.StringBuilder
    $pos = 0
    foreach ($s in $ordered) {
        Log "    --- removed text for '$($s.Name)' ---"
        $text.Substring($s.Start, $s.End - $s.Start) -split "`r?`n" | ForEach-Object { Log "    - $_" }
        [void]$sb.Append($text, $pos, $s.Start - $pos)
        $pos = $s.End
    }
    [void]$sb.Append($text, $pos, $text.Length - $pos)
    $newText  = $sb.ToString()
    $newBytes = $Latin1.GetBytes($newText)

    # Checks: three fewer certificates, every other certificate untouched, size adds up.
    $newCerts = @(Get-PemCerts $newText)
    $removed  = 0
    foreach ($s in $ordered) { $removed += $s.End - $s.Start }
    $removedFps = @($ordered | ForEach-Object Sha256)
    $expect   = @($certs | Where-Object { $removedFps -notcontains $_.Sha256 } | ForEach-Object Sha256 | Sort-Object)
    $actual   = @($newCerts | ForEach-Object Sha256 | Sort-Object)
    if ($newCerts.Count -ne $certs.Count - $ordered.Count) { throw "Patched $Path would have $($newCerts.Count) certificates, expected $($certs.Count - $ordered.Count)" }
    if (Compare-Object $expect $actual) { throw "Patched $Path does not contain exactly the original certificates minus the removed targets" }
    if ($newBytes.Length -ne $bytes.Length - $removed) { throw "Patched $Path size mismatch" }
    $sha = [Security.Cryptography.SHA256]::Create()
    $newHash = -join ($sha.ComputeHash($newBytes) | ForEach-Object { $_.ToString('X2') })
    Log "  Patched result: $($newBytes.Length) bytes, $($newCerts.Count) certificates, SHA256 $newHash"
    [pscustomobject]@{ Bytes = $newBytes; Sha256 = $newHash; Before = $certs.Count; After = $newCerts.Count; Changed = ($ordered.Count -gt 0); Removed = (@($ordered | ForEach-Object Name) -join ', ') }
}

function Backup-File([string]$Source, [string]$CentralName) {
    # Earlier backups are never overwritten. If the standard backup holds an earlier version of
    # the file, the current file is backed up again under a dated name; rollback uses that one.
    $srcHash = Get-Sha256 $Source
    $suffix  = $BackupSuffix
    $primary = "$Source$BackupSuffix"
    if ((Test-Path -LiteralPath $primary) -and (Get-Sha256 $primary) -ne $srcHash) {
        $suffix = "$BackupSuffix.$RunStamp"
        Log "  $primary holds an earlier version of $Source (kept unchanged); backing up the current file as $Source$suffix"
    }
    $local   = "$Source$suffix"
    $central = Join-Path $BackupRoot ($CentralName + $suffix.Substring($BackupSuffix.Length))
    foreach ($dest in @($local, $central)) {
        if (Test-Path -LiteralPath $dest) {
            $h = Get-Sha256 $dest
            if ($h -ne $srcHash) { throw "Backup $dest already exists and differs from $Source (backup $h, source $srcHash). Not overwriting." }
            Log "  Backup already present and identical: $dest"
        } else {
            Copy-Item -LiteralPath $Source -Destination $dest
            $h = Get-Sha256 $dest
            if ($h -ne $srcHash) { throw "Backup copy $dest does not match $Source" }
            Log "  Backup written: $dest ($((Get-Item -LiteralPath $dest).Length) bytes, SHA256 $h)"
        }
    }
    $script:RestoreFrom[$Source] = $local
    Add-Content -LiteralPath $Manifest -Encoding UTF8 -Value ('{0:yyyy-MM-dd HH:mm:ss}  {1}  {2}  -> {3} and {4}' -f (Get-Date), $srcHash, $Source, $local, $central)
}

function Install-Staged([string]$Staged, [string]$Live, [string]$ExpectedLiveBefore) {
    # Overwrite the live file in place with the verified staged content. Overwriting (rather than
    # swapping files) keeps the live file's permissions.
    $now = Get-Sha256 $Live
    if ($now -ne $ExpectedLiveBefore) { throw "$Live changed since the pre-check (now $now)" }
    $stagedHash = Get-Sha256 $Staged
    $aclBefore = (Get-Acl -LiteralPath $Live).Sddl
    Log "  Overwriting $Live with verified staged file $Staged (SHA256 $stagedHash)"
    Log "    ACL before: $aclBefore"
    $script:touched.Add($Live)
    Copy-Item -LiteralPath $Staged -Destination $Live -Force
    $aclAfter = (Get-Acl -LiteralPath $Live).Sddl
    Log "    ACL after : $aclAfter"
    if ($aclAfter -ne $aclBefore) { Log "  WARNING: permissions on $Live changed (see ACL lines above)" -Important }
    $after = Get-Sha256 $Live
    if ($after -ne $stagedHash) { throw "$Live SHA256 $after does not match the staged file after overwrite" }
    Log "  Live file updated: $Live ($(Describe $Live))" -Important
}

function Restore-File([string]$Live) {
    $backup = $script:RestoreFrom[$Live]
    if ((Get-Sha256 $Live) -eq (Get-Sha256 $backup)) { Log "  $Live already matches its backup - nothing to restore" -Important; return }
    Log "  RESTORING $Live from $backup" -Important
    Copy-Item -LiteralPath $backup -Destination $Live -Force
    if ((Get-Sha256 $Live) -ne (Get-Sha256 $backup)) { throw "Restore of $Live failed: hash does not match backup" }
    Log "  Restored and verified: $Live ($(Describe $Live))" -Important
}

function Get-LiveHashes {
    $h = [ordered]@{}
    $h[$JvmCacerts] = Get-Sha256 $JvmCacerts
    foreach ($d in 'Data1', 'Data2', 'Data3') {
        $p = Join-Path $DominoDir "$d\cacert.pem"
        if (Test-Path -LiteralPath $p) { $h[$p] = Get-Sha256 $p }
    }
    $h
}

# ---------------------------------------------------------------- run

$stopped    = New-Object System.Collections.Generic.List[string]
$touched    = New-Object System.Collections.Generic.List[string]
$RestoreFrom = @{}
$jvmNeeded  = $true
$stopOk     = $true
$changeOk   = $false
$failed     = $false
$liveBefore = $null

Log "===== eMudhra remediation (HCL KB0134844) - $Mode =====" -Important
Log "Computer $env:COMPUTERNAME, user $([Security.Principal.WindowsIdentity]::GetCurrent().Name), PowerShell $($PSVersionTable.PSVersion)"
Log "Scope: JVM cacerts (shared by all partitions) + cacert.pem in $($DataDirNames -join ', '); services managed: $ManageServices; poll every $PollSeconds s, max wait $MaxWaitSeconds s"
Log "Domino dir $DominoDir; keytool $Keytool; backups $BackupRoot; staging $StageDir; log $LogFile"

try {
    # ---- Phase 0: read-only checks. Any failure here stops the run before anything is written.
    Log '--- Phase 0: environment and pre-checks (read-only)'
    $nserver = Join-Path $DominoDir 'nserver.exe'
    if (Test-Path -LiteralPath $nserver) { Log "  nserver.exe version $((Get-Item -LiteralPath $nserver).VersionInfo.FileVersion)" }
    $java = Join-Path $DominoDir 'jvm\bin\java.exe'
    if (Test-Path -LiteralPath $java) { (Invoke-Native $java @('-version')).Output | ForEach-Object { Log "  java: $_" } }
    if (-not (Test-Path -LiteralPath $Keytool)) { throw "keytool not found: $Keytool" }

    foreach ($d in 'Data1', 'Data2', 'Data3') {
        $svc = Get-Service -Name $ServiceNames[$d] -ErrorAction SilentlyContinue
        $svcText = if ($svc) { "$($svc.Status), start type $($svc.StartType)" } else { 'not installed' }
        Log "  $d service $($ServiceNames[$d]): $svcText; processes: $(Format-Processes (Get-PartitionProcesses $d))"
    }

    $liveBefore = Get-LiveHashes
    foreach ($k in $liveBefore.Keys) { Log "  Live file before: $k ($(Describe $k)); ACL $((Get-Acl -LiteralPath $k).Sddl)" }

    $ks = Get-KeystoreEntries $JvmCacerts
    Log "  JVM cacerts: $($ks.Type); $($ks.Count) entries"
    $ks.Entries | Where-Object { $_.Alias -match 'emsign' } | ForEach-Object { Log "    emSign entry: '$($_.Alias)' SHA256 $($_.Sha256)" }
    $g3   = @($ks.Entries | Where-Object { $_.Alias -eq $JvmAlias })
    $g3fp = @($ks.Entries | Where-Object { $_.Sha256 -eq $JvmAliasSha256 })
    if ($g3.Count -eq 0 -and $g3fp.Count -eq 0) {
        $jvmNeeded = $false
        Log "  JVM cacerts: emSign ECC Root CA - G3 not present (checked by alias and by fingerprint) - no change needed" -Important
    } else {
        if ($g3.Count -ne 1 -or $g3fp.Count -ne 1) { throw "emSign ECC Root CA - G3: alias found $($g3.Count) times, fingerprint found $($g3fp.Count) times in $JvmCacerts - expected one of each" }
        if ($g3[0].Sha256 -ne $JvmAliasSha256) { throw "Alias '$JvmAlias' has SHA256 $($g3[0].Sha256), expected $JvmAliasSha256" }
        Log "  Target alias present with expected fingerprint: '$JvmAlias'" -Important
    }

    $pemPlans = [ordered]@{}
    foreach ($d in $DataDirNames) {
        $pem = Join-Path $DominoDir "$d\cacert.pem"
        if (-not (Test-Path -LiteralPath $pem)) { Log "  $d skipped: $pem not present" -Important; continue }
        Log "  Planning PEM edit for $pem"
        $plan = Get-PemPatch $pem
        if (-not $plan.Changed) { Log "  $d cacert.pem: none of the targets present - no change needed" -Important; continue }
        $pemPlans[$d] = $plan
        Log "  $d cacert.pem: $($plan.Before) -> $($plan.After) certificates (removing $($plan.Removed)); expected patched SHA256 $($plan.Sha256)" -Important
    }

    if (-not $jvmNeeded -and $pemPlans.Count -eq 0) {
        Log '  Nothing to change: every target is already absent. Domino is not stopped.' -Important
        $changeOk = $true
    } else {
        # ---- Phase 1: backups
        Log '--- Phase 1: backups'
        if ($jvmNeeded) { Backup-File $JvmCacerts 'jvm_lib_security_cacerts' }
        foreach ($d in $pemPlans.Keys) { Backup-File (Join-Path $DominoDir "$d\cacert.pem") "${d}_cacert.pem" }
        Log "  Backups complete; manifest $Manifest" -Important

        # ---- Phase 2: build and verify staged copies while the servers are still running
        Log '--- Phase 2: build and verify staged copies'
        New-Item -ItemType Directory -Force -Path $StageDir | Out-Null
        if ($jvmNeeded) {
            $ksStaged = Join-Path $StageDir 'cacerts'
            Copy-Item -LiteralPath $JvmCacerts -Destination $ksStaged
            $before = Get-KeystoreEntries $ksStaged
            Log "  keytool -delete -alias `"$JvmAlias`" -keystore $ksStaged"
            $r = Invoke-Native $Keytool @('-delete', '-alias', $JvmAlias, '-keystore', $ksStaged, '-storepass', 'changeit')
            $r.Output | ForEach-Object { Log "    keytool: $_" }
            if ($r.ExitCode -ne 0) { throw "keytool -delete exit code $($r.ExitCode)" }
            $after = Get-KeystoreEntries $ksStaged
            $gone  = @($before.Entries | Where-Object { $a = $_.Alias; -not ($after.Entries | Where-Object { $_.Alias -eq $a }) })
            $added = @($after.Entries | Where-Object { $a = $_.Alias; -not ($before.Entries | Where-Object { $_.Alias -eq $a }) })
            Log "  Keystore entries $($before.Count) -> $($after.Count); removed: $(($gone | ForEach-Object Alias) -join '; '); added: $($added.Count)"
            if ($after.Count -ne $before.Count - 1 -or $gone.Count -ne 1 -or $gone[0].Alias -ne $JvmAlias -or $added.Count -ne 0) { throw 'Keystore check failed after delete' }
            Log "  Staged JVM cacerts verified: $($before.Count) -> $($after.Count), only '$JvmAlias' removed ($(Describe $ksStaged))" -Important
        }

        $pemStaged = [ordered]@{}
        foreach ($d in $pemPlans.Keys) {
            $staged = Join-Path $StageDir "${d}_cacert.pem"
            [IO.File]::WriteAllBytes($staged, $pemPlans[$d].Bytes)
            $h = Get-Sha256 $staged
            $n = @(Get-PemCerts ($Latin1.GetString([IO.File]::ReadAllBytes($staged)))).Count
            if ($h -ne $pemPlans[$d].Sha256) { throw "Staged $staged SHA256 $h does not match expected $($pemPlans[$d].Sha256)" }
            Log "  Staged $d cacert.pem verified: $n certificates ($(Describe $staged))" -Important
            $pemStaged[$d] = $staged
        }

        # ---- Phase 3: stop the in-scope partitions that are running
        Log '--- Phase 3: stop Domino'
        $running = @($StopOrder | Where-Object { $svc = Get-Service -Name $ServiceNames[$_] -ErrorAction SilentlyContinue; $svc -and $svc.Status -eq 'Running' })
        $idle    = @($StopOrder | Where-Object { $running -notcontains $_ })
        Log "  Running partitions (all will be stopped): $(if ($running) { $running -join ', ' } else { '(none)' }); not running (left as they are): $(if ($idle) { $idle -join ', ' } else { '(none)' })" -Important
        if ($DryRun -and -not $DryRunRestarts) {
            Log "  Dry run: would stop $(($running | ForEach-Object { $ServiceNames[$_] }) -join ', ') - not stopping (DryRunRestarts is off)" -Important
        } elseif ($ManageServices) {
            foreach ($d in $running) {
                $svcName = $ServiceNames[$d]
                $svc = Get-Service -Name $svcName
                if ($svc.Status -ne 'Running') { Log "  $svcName is now $($svc.Status); not stopping it" -Important; continue }
                Log "  Stopping $svcName (processes: $(Format-Processes (Get-PartitionProcesses $d)))" -Important
                $svc.Stop()
                $stopped.Add($d)
                $probeName = $svcName; $probeDir = $d
                $ok = Wait-Until "stop $d" {
                    $s = (Get-Service -Name $probeName).Status
                    $p = Get-PartitionProcesses $probeDir
                    @{ Done = ($s -eq 'Stopped' -and $p.Count -eq 0); Detail = "service $s; processes $($p.Count): $(Format-Processes $p)" }
                }
                Log-ConsoleTail $d
                if (-not $ok) { Log "  $svcName did not stop within $MaxWaitSeconds s - no files will be changed" -Important; $stopOk = $false; break }
                Log "  $svcName stopped" -Important
            }
        } else { Log '  Service management disabled (local test)' }

        # ---- Phase 4: overwrite the live files (LIVE only)
        if (-not $stopOk) {
            Log '--- Phase 4: skipped - a partition did not stop' -Important
        } elseif ($DryRun) {
            Log '--- Phase 4: skipped (dry run) - live files left as they are' -Important
            $changeOk = $true
        } else {
            Log '--- Phase 4: overwrite live files with the verified copies'
            if ($jvmNeeded) {
                Install-Staged $ksStaged $JvmCacerts $liveBefore[$JvmCacerts]
                $check = Get-KeystoreEntries $JvmCacerts
                if ($check.Count -ne $ks.Count - 1 -or ($check.Entries | Where-Object { $_.Alias -eq $JvmAlias })) { throw 'Live JVM cacerts check failed after overwrite' }
                Log "  Live JVM cacerts now $($check.Count) entries; '$JvmAlias' absent" -Important
            }
            foreach ($d in $pemPlans.Keys) {
                $live = Join-Path $DominoDir "$d\cacert.pem"
                Install-Staged $pemStaged[$d] $live $liveBefore[$live]
                $n = @(Get-PemCerts ($Latin1.GetString([IO.File]::ReadAllBytes($live)))).Count
                Log "  Live $d cacert.pem now $n certificates" -Important
            }
            $changeOk = $true
        }
    }
} catch {
    $failed = $true
    Log "ERROR: $($_.Exception.Message)" -Important
    Log "  at: $($_.ScriptStackTrace -replace "`r?`n", ' <- ')"
    foreach ($f in $touched) {
        try { Restore-File $f } catch { Log "  Restore of $f FAILED: $($_.Exception.Message)" -Important }
    }
} finally {
    # ---- Phase 5: start whatever this run stopped, in reverse order
    if ($stopped.Count -gt 0) {
        Log '--- Phase 5: start Domino'
        $startOrder = @($stopped); [array]::Reverse($startOrder)
        foreach ($d in $startOrder) {
            $svcName = $ServiceNames[$d]
            try {
                $svc = Get-Service -Name $svcName
                if ($svc.Status -ne 'Stopped') { Log "  $svcName is $($svc.Status), not Stopped - cannot start it now; needs a follow-up check" -Important; continue }
                # Domino rotates console.log on start and prefixes each line with the nserver PID in hex,
                # so "started" = a "Server started on physical node" line from the NEW nserver process.
                $con = Get-ConsoleLog $d
                Log "  Starting $svcName (console.log present: $(Test-Path -LiteralPath $con))" -Important
                $svc.Start()
                $probeName = $svcName; $probeDir = $d; $probeCon = $con
                $ok = Wait-Until "start $d" {
                    $s = (Get-Service -Name $probeName).Status
                    $p = Get-PartitionProcesses $probeDir
                    $names = @($p | ForEach-Object Name)
                    $ns = @($p | Where-Object { $_.Name -eq 'nserver.exe' })
                    if ($ns.Count -eq 1 -and (Test-Path -LiteralPath $probeCon)) {
                        $hex = '{0:X}' -f [int]$ns[0].ProcessId
                        $line = (Read-FileShared $probeCon) -split "`r?`n" | Where-Object { $_ -match "^\[0*$hex`:" -and $_ -match 'Server started on physical node' } | Select-Object -First 1
                        $evidence = if ($line) { "console: '$($line.Trim())'" } else { "console: no 'Server started on physical node' line from nserver PID $($ns[0].ProcessId) (0x$hex) yet" }
                        $ready = [bool]$line
                    } elseif ($ns.Count -ne 1) {
                        $ready = $false
                        $evidence = "waiting for nserver.exe ($($ns.Count) found)"
                    } else {
                        $ready = $names -contains 'nhttp.exe'
                        $evidence = "no console.log, using nhttp.exe present = $ready"
                    }
                    @{ Done = ($s -eq 'Running' -and ($names -contains 'nserver.exe') -and $ready); Detail = "service $s; $evidence; processes $($p.Count): $(Format-Processes $p)" }
                }
                Log-ConsoleTail $d
                if ($ok) { Log "  $svcName started" -Important } else { Log "  $svcName NOT confirmed started within $MaxWaitSeconds s - needs a follow-up check" -Important }
            } catch { Log "  Starting $svcName FAILED: $($_.Exception.Message)" -Important }
        }
    }

    # ---- Phase 6: final state
    Log '--- Phase 6: final state'
    if ($liveBefore) {
        $liveAfter = Get-LiveHashes
        foreach ($k in $liveAfter.Keys) {
            $same = $liveAfter[$k] -eq $liveBefore[$k]
            Log "  Live file after: $k ($(Describe $k)) - $(if ($same) { 'unchanged' } else { 'CHANGED' })" -Important
            if ($DryRun -and -not $same) { Log "  WARNING: live file changed during a dry run: $k" -Important }
        }
    }
    foreach ($d in 'Data1', 'Data2', 'Data3') {
        $svc = Get-Service -Name $ServiceNames[$d] -ErrorAction SilentlyContinue
        if ($svc) { Log "  $d service $($ServiceNames[$d]): $($svc.Status)" -Important }
    }
    $result = if ($failed) { 'FAILED' } elseif (-not $stopOk) { 'ABORTED (shutdown timeout, no changes)' } elseif ($changeOk) { 'OK' } else { 'INCOMPLETE' }
    Log "===== Result: $result ($Mode) =====" -Important
}

$Summary | ForEach-Object { $_ }
"Full log: $LogFile"

Submit a Comment

Your email address will not be published. Required fields are marked *