turov_av/SDR-Rover
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harden RF handoff and stream OTA decode

Browse Source
This commit is contained in:
LittleSam129
2026-09-22 17:43:58 +03:00
parent 168e03d9e7
commit a94f2db4ee
17 changed files with 1548 additions and 497 deletions
Download Patch File Download Diff File Expand all files Collapse all files

358
docs/handoff/NEXT_RX_TASK.md
View File

@@ -1,7 +1,7 @@
TASK_ID = RF001A_OTA_CAPTURE_003 TASK_ID = RF001A_OTA_CAPTURE_004
TASK_TITLE = Start coordinated RF-001A common-clock OTA capture TASK_TITLE = Start hardened RF-001A common-clock OTA capture
SOURCE_BRANCH = video-development SOURCE_BRANCH = video-development
EXPECTED_HEAD = bc55d0516dc1ee36cefa1ff096e7d86f60645303 EXPECTED_HEAD = 168e03d9e778ab2ec63502552d385ec7be3ef677
REQUIRES_APPROVAL = YES REQUIRES_APPROVAL = YES
HARDWARE_ACCESS = YES HARDWARE_ACCESS = YES
RF_RX = YES RF_RX = YES
@@ -10,70 +10,79 @@ ALLOW_COMMIT = NO
ALLOW_PUSH = RX_RESULT_ONLY ALLOW_PUSH = RX_RESULT_ONLY
RISK_LEVEL = YELLOW RISK_LEVEL = YELLOW
This file is the authoritative current task for the RX laptop. ---
The RX assistant must read RX_NODE_STATE.md and RF_HANDOFF_CHANNEL_TEST.md first, then execute this file.
If chat history conflicts with these files, the shared RF-001A spec or current Git HEAD, stop and report the conflict.
# Current RX task # RF-001A RX capture 004
TASK = This file is the authoritative current task for the RX laptop. Read
START COORDINATED RF-001A COMMON-CLOCK OTA CAPTURE `RX_NODE_STATE.md` first. If chat history conflicts with these files or the
current Git HEAD, stop and report the conflict.
This is a new YELLOW task. Do not reuse approval or completion state from RF001A_OTA_CAPTURE_001, RF001A_OTA_CAPTURE_002 or any earlier task. This task is prepared only. A previous approval does not apply to this new
TASK_ID. Do not access HackRF until the user approves this exact YELLOW task.
One approval for this task covers the read-only bidirectional channel check, other read-only checks, one bounded RF RX capture and publication of the two small allowlisted result files to the existing rx-node branch. It never authorizes RF TX. ## 1. Software and Git gates
The preceding RF001A_OTA_CAPTURE_002 produced a real 120.1413067 s, 1920000000-byte capture with CLKIN detected before and after, zero reported overruns and SHA256 99a3e7cb51d3ad4e3b914b63da0695ed6658aa169110c32659aba995b542fa0c. Its READY publication failed, its exact child exit code was not recorded and no synchronized truth metadata was available. Treat that as historical evidence only. 1. Run `git pull --ff-only origin video-development`.
2. Require branch `video-development` and a clean tracked worktree.
3. Require local HEAD to equal `origin/video-development`; `EXPECTED_HEAD`
must be its ancestor.
4. Confirm the historical channel-test READY and COMPLETE acknowledgements
from `origin/tx-node` through the RX watcher.
5. Before opening HackRF, run the authenticated no-change publication check:
## 1. Bidirectional handoff channel gate python -B tools/rx_agent_watcher.py --check-result-write --result-branch rx-node
Before offering this task for YELLOW approval, the RX watcher must fetch origin/rx-node and origin/tx-node and scan the acknowledgement history from oldest to newest. It must confirm both exact records: Require `RX-NODE WRITE PREFLIGHT = PASS`. This command fetches the branch,
performs `git push --dry-run` of its existing HEAD to the same ref, and
confirms that the remote HEAD did not change. It creates no commit and no
production READY event.
6. Run the targeted tests, Ruff, Pyright and `git diff --check` according to
the tracked project instructions.
7. Validate `configs/rf/rf001a_truth_metadata.json`: waveform SHA256, seed,
PHY settings, 100 sequence numbers and 100 packet hashes must agree with
`configs/rf/rf001a_common_clock.json`.
STATE = CHANNEL_READY_OBSERVED If any gate fails, do not open HackRF and stop. In particular, a failed Git
SOURCE_RX_COMMIT = 4cb6c8cad1ab5194755c2740c2a5a83933d099d8 write preflight blocks the hardware task.
SOURCE_RX_DIGEST = 23fc54f1f788f068fd4a842daf0a445bd4ec192866b4d89406c74f1797a9952b
and: ## 2. Optional offline check of capture 003
STATE = CHANNEL_COMPLETE_OBSERVED If the existing `RF001A_OTA_CAPTURE_003` file is present locally, analyze it
SOURCE_RX_COMMIT = 83d91ff5309777ab639a24f774e9d0b8825a11e9 read-only with the streaming cs8 decoder before starting new hardware work:
SOURCE_RX_DIGEST = 7d98ed2e70e98a087b968c0d65fb1db4277b81f251e075d20583de2e5002b2d6
Both acknowledgements must use TASK_ID RF_HANDOFF_CHANNEL_TEST_001, the original shared source HEAD and RF_TX = 0. Required watcher output: python -B -X utf8 tools/qpsk_rf_rx.py --iq-file <capture-003-path>\rf001a_ota_capture.cs8 --iq-format cs8 --truth-metadata configs/rf/rf001a_truth_metadata.json --report <new-path-outside-git>\capture003_streaming_report.json
MAIN READY RECEIVED = CONFIRMED No RF-001A transmission occurred during capture 003. The expected exact
MAIN COMPLETE RECEIVED = CONFIRMED packet count is 0. CRC-valid background candidates must not be reported as
exact unless their complete packet SHA256 matches the tracked truth metadata.
The analysis must use memmap/chunks and must not materialize the full 1.92 GB
capture as complex64.
Do not rerun the channel test. If origin/tx-node is missing or either exact acknowledgement is absent or invalid, do not access HackRF, do not start capture and STOP with RF TX = 0. If capture 003 is unavailable, report `CAPTURE_003 ANALYSIS = NOT_RUN` and
continue only after all mandatory gates pass.
## 2. Preconditions ## 3. Hardware preflight after YELLOW approval
After the same [A] approval for this exact TASK_ID and after the channel gate passes: 1. Run `hackrf_info` and require RX serial
`000000000000000097c063dc2e5327d3`.
2. With `MASTER CLKOUT -> SLAVE CLKIN` connected, run:
1. Run git pull --ff-only origin video-development. hackrf_clock.exe -d 000000000000000097c063dc2e5327d3 -i
2. Require branch video-development.
3. Require git status --short --untracked-files=no to produce no output.
4. Require local HEAD to equal origin/video-development and EXPECTED_HEAD to be its ancestor.
5. Run hackrf_info and preserve complete stdout.
6. Require RX serial 000000000000000097c063dc2e5327d3.
7. Confirm that configs/rf/rf001a_common_clock.json and configs/rf/rf001a_truth_metadata.json exist. Require the tracked truth metadata test ID, seed, 100 packet hashes, sequence numbers, waveform SHA256 and PHY settings to agree with the shared RF-001A spec.
8. With MASTER CLKOUT physically connected to SLAVE CLKIN, run:
hackrf_clock.exe -d 000000000000000097c063dc2e5327d3 -i 3. Require `CLKIN status: clock signal detected`.
4. Repeat the Git write preflight immediately before the finite RX command.
5. Require antennas connected, approximately 2 m distance, clear LOS, and
video disabled.
6. Require at least 1500000000 free bytes on the capture volume and record the
measured value.
9. Preserve complete stdout and require the exact line: Do not reconfigure clocks. If CLKIN or the second Git preflight fails, stop
before capture.
CLKIN status: clock signal detected ## 4. Finite RX capture
10. Require the remote branch origin/rx-node to exist through both git fetch origin rx-node and git ls-remote --heads origin rx-node. Do not create it in this task. Use a unique directory outside Git. Never overwrite an earlier capture.
11. Create a unique capture directory outside Git. Refuse an existing directory and never overwrite an earlier capture.
12. Require at least 2500000000 free bytes on the target volume. Record the measured free-byte count.
If any condition fails, do not start capture. Report the failed condition, keep RF TX = 0, and STOP.
## 3. Authorized bounded RX capture
Only this finite receive operation is authorized:
RX serial = 000000000000000097c063dc2e5327d3 RX serial = 000000000000000097c063dc2e5327d3
center frequency = 1400000000 Hz center frequency = 1400000000 Hz
@@ -82,239 +91,114 @@ Only this finite receive operation is authorized:
RX VGA = 24 dB RX VGA = 24 dB
RF amp = OFF RF amp = OFF
antenna bias = OFF antenna bias = OFF
duration = 120 s duration = 60 s
sample count = 960000000 complex samples sample count = 480000000 complex samples
hardware format = cs8 hardware format = cs8
expected file size = 1920000000 bytes expected file size = 960000000 bytes
common clock = DETECTED
The hackrf_transfer.exe arguments are fixed: Use these fixed `hackrf_transfer.exe` arguments:
-d 000000000000000097c063dc2e5327d3 -d 000000000000000097c063dc2e5327d3
-r <unique_capture_path>\rf001a_ota_capture.cs8 -r <unique_capture_path>\rf001a_ota_capture.cs8
-f 1400000000 -f 1400000000
-s 8000000 -s 8000000
-n 960000000 -n 480000000
-l 24 -l 24
-g 24 -g 24
-a 0 -a 0
-p 0 -p 0
-B -B
Do not add repeat mode. Do not change gain, clock, frequency or sample rate. Use `tools/capture_process_runner.ps1` as a controlled background process.
Preserve its complete stdout, stderr, start JSON and result JSON outside Git.
The runner must start exactly one finite child, record the actual child PID and
UTC start time, set the expected end to start plus 60 seconds, wait for that
exact process, record its integer exit code, size and duration, and calculate
SHA256 after the output file closes. Never retry the capture automatically.
## 4. Background runner and durable evidence Within five seconds require the runner and child to be alive and the file to
exist and grow. Otherwise publish no READY and stop.
Use the tracked tools/capture_process_runner.ps1. Write the exact hackrf_transfer argument array to a unique local JSON file outside Git. Start the runner itself as a controlled background PowerShell process and pass unique paths for the capture, complete stdout, complete stderr, capture_started.json and capture_result.json. The runner must: ## 5. READY publication
- start exactly one hackrf_transfer child; Create the strict schema-version-1 READY JSON outside Git for TASK_ID
- record its PID and start time immediately; `RF001A_OTA_CAPTURE_004`. Use the actual child PID and UTC timestamps,
- record the expected end time as start time plus 120 seconds; `expected_capture_end = capture_started_at + 60 s`, current source HEAD,
- redirect complete stdout and stderr to unique local files; `common_clock_verified = true`, `rf_tx = 0`, and null final result fields.
- wait for that exact child, refresh its process object and read ExitCode only after WaitForExit;
- record its exact integer exit code, finish time, duration and output size;
- calculate SHA256 after the file is closed;
- write started and final JSON atomically;
- never start a second capture automatically.
Start the runner as a controlled background process. Within five seconds, require all four facts: Publish it once with:
runner process alive = YES python -B tools/rx_agent_watcher.py --publish-result <ready-json> --result-branch rx-node
hackrf_transfer child alive = YES
capture file created = YES
capture file growing = YES
If any fact is false, publish no READY state. Preserve local evidence, report RX CAPTURE STARTED = NO, keep RF TX = 0, and STOP. The publisher creates one immutable READY commit. On a recognized transient
network or authentication failure it checks the remote branch and may retry
the exact same commit once. It never creates a second READY commit or changes
the payload between attempts. If the remote already contains that commit, the
publication is successful. A second failure means:
## 5. Publish READY immediately READY publication = FAIL
RX READY FOR TX = NO
RF TX = 0
As soon as the child is alive and the file is growing, build a local JSON file outside Git with every field below: Do not publish READY after the capture window has ended. After the process
exits, publish only a `CAPTURE_COMPLETE` result with the actual exit code,
size, duration and error counters, including nonzero failure values.
{ After the remote HEAD confirms the READY commit, immediately print:
"schema_version": 1,
"task_id": "RF001A_OTA_CAPTURE_003",
"state": "RX_READY_FOR_TX",
"rx_ready_for_tx": true,
"process_alive": true,
"file_growing": true,
"capture_pid": <actual positive child PID>,
"published_at": "<current UTC ISO-8601 time>",
"capture_started_at": "<actual UTC ISO-8601 start time>",
"expected_capture_end": "<start time plus 120 seconds>",
"capture_path": "<absolute local capture path>",
"common_clock_verified": true,
"source_head": "<current local 40-character Git HEAD>",
"rf_tx": 0,
"exit_code": null,
"file_size": null,
"duration_seconds": null,
"overruns": null,
"usb_errors": null,
"file_errors": null,
"sha256": null,
"clkin_after": "NOT_RUN",
"rf_tx_observed": "UNKNOWN"
}
Publish it with the approved RX watcher state:
python -B tools/rx_agent_watcher.py --publish-result <ready-json-path> --result-branch rx-node
The publisher may change only:
docs/handoff/results/RX_RESULT.json
docs/handoff/results/RX_RESULT.md
The publisher checks origin/rx-node through the real origin of the main checkout. It creates a temporary Git worktree from that repository, stages exactly the two allowlisted result paths, pushes HEAD:rx-node, confirms the new remote HEAD with git ls-remote and removes the worktree. It must not switch the source checkout, push video-development, or add raw IQ, video, waveform or large logs.
Use this order without exception: verify CLKIN, start capture, verify the child is alive, verify the file is growing, publish READY, verify RX RESULT REMOTE HEAD equals the published commit, and only then report RX READY FOR TX = YES.
If publication or remote confirmation fails, show the complete error and report RX READY FOR TX = NO. The finite capture may finish normally, but TX coordination does not start and RF TX remains 0.
After successful publication, immediately return control to the user without waiting for capture completion:
TASK_ID =
RF001A_OTA_CAPTURE_003
RX CAPTURE STARTED =
YES
RX READY FOR TX =
YES
TASK_ID = RF001A_OTA_CAPTURE_004
RX CAPTURE STARTED = YES
RX READY FOR TX = YES
CAPTURE PID = CAPTURE PID =
CAPTURE PATH = CAPTURE PATH =
CAPTURE START TIME = CAPTURE START TIME =
EXPECTED CAPTURE END = EXPECTED CAPTURE END =
RX RESULT HEAD =
RF TX = 0
RX RESULT PUBLISH = Return control without waiting for capture completion.
PASS
RX RESULT REMOTE HEAD = ## 6. Completion and offline decode
RF TX = After the child exits, record exact exit code, actual size, duration, overruns,
0 USB errors, file-write errors and SHA256. A complete capture is exactly
960000000 bytes; a smaller file is incomplete. Run the CLKIN check again and
require detected status.
The READY state describes a live receiver only. It does not authorize RF TX on either computer. Publish an honest schema-version-1 COMPLETE result with final values. Keep IQ,
stdout, stderr and large logs outside Git.
## 6. Capture completion After `origin/tx-node` contains `TX_STARTED` and `TX_COMPLETE` for
`RF001A_OTA_TX_004`, run the streaming decoder with RX/TX timestamps:
After the child exits, read capture_result.json and require an integer exit_code recorded by the runner. Preserve all local logs and inspect every line for RX overruns, USB errors and file write errors. Do not infer zero when a tool did not report a counter; use JSON null and report NOT_REPORTED. python -B -X utf8 tools/qpsk_rf_rx.py --iq-file <capture-path>\rf001a_ota_capture.cs8 --iq-format cs8 --truth-metadata configs/rf/rf001a_truth_metadata.json --capture-started-at <CAPTURE_STARTED_AT> --tx-started-at <TX_STARTED_AT> --tx-finished-at <TX_FINISHED_AT> --report <new-path-outside-git>\rf001a_streaming_report.json
Run the read-only clock check again: The decoder uses a two-second margin before and after the timestamped burst.
Without complete timestamps it uses the streaming power locator. Chunk overlap
is derived from the real 256-byte RF-001 waveform and duplicate detections from
overlap are suppressed.
hackrf_clock.exe -d 000000000000000097c063dc2e5327d3 -i ## 7. Prohibited actions
Require CLKIN to remain detected. The expected complete capture has: RF TX, firmware update, CPLD update, Zadig, USB driver changes, clock
reconfiguration, commit or push to `video-development`, repeated capture, raw
IQ/video/waveform publication and large-log publication are forbidden.
exit code = 0 Final report:
file size = 1920000000 bytes
duration = consistent with 120 s
RX overruns = 0 or NOT_REPORTED
USB errors = 0 or NOT_REPORTED
file write errors = 0 or NOT_REPORTED
CLKIN after capture = DETECTED
A shorter file is incomplete. Any reported nonzero error remains evidence. Do not retry automatically.
## 7. Publish final result
Build a second local JSON document with the same identity and timing fields. Set:
state = CAPTURE_COMPLETE
rx_ready_for_tx = false
process_alive = false
file_growing = false
exit_code = <exact integer>
file_size = <actual integer bytes>
duration_seconds = <actual number>
overruns = <integer or null when not reported>
usb_errors = <integer or null when not reported>
file_errors = <integer or null when not reported>
sha256 = <lowercase 64-character SHA256>
clkin_after = DETECTED or NOT_DETECTED
rf_tx_observed = EXPECTED_EXTERNAL_TX or UNKNOWN
published_at = <current UTC ISO-8601 time>
rf_tx = 0
Use EXPECTED_EXTERNAL_TX only when the coordinated MAIN TX event is known to have occurred inside this capture window. Otherwise use UNKNOWN.
Publish with:
python -B tools/rx_agent_watcher.py --publish-result <final-json-path> --result-branch rx-node
Require RX RESULT REMOTE HEAD to equal the final result commit. If the push or confirmation fails, preserve the local JSON and logs, report the error, and do not claim publication success.
## 8. Offline decode
Use the tracked synchronized truth artifact without copying TX files manually:
python -B -X utf8 tools/qpsk_rf_rx.py --iq-file <capture-path> --iq-format cs8 --truth-metadata configs/rf/rf001a_truth_metadata.json --report <unique-path-outside-git>\rf001a_ota_decode.json
Report exact packets, missing packets, duplicates, BER and post-FEC errors from the generated report. Preserve a decoder failure as evidence; do not modify the receiver or invent results.
## 9. Prohibited actions
On the RX laptop, do not perform:
- RF TX;
- continuous RX;
- automatic capture retry;
- firmware or CPLD update;
- hackrf_spiflash;
- Zadig or USB driver changes;
- clock reconfiguration;
- commit or push to video-development;
- creation of rx-node;
- publication of raw IQ, video, waveform or large logs.
## 10. Final RX report
TASK_ID =
RF001A_OTA_CAPTURE_003
CHANNEL TEST =
RX CAPTURE COMPLETE =
EXIT CODE =
TASK_ID = RF001A_OTA_CAPTURE_004
RX-NODE WRITE PREFLIGHT = PASS / FAIL
COMMON CLOCK = VERIFIED / NOT_VERIFIED
RX CAPTURE COMPLETE = YES / NO
CAPTURE SIZE = CAPTURE SIZE =
CAPTURE DURATION = CAPTURE DURATION =
RX OVERRUNS = RX OVERRUNS =
USB ERRORS = USB ERRORS =
FILE WRITE ERRORS = FILE WRITE ERRORS =
RAW IQ SHA256 = RAW IQ SHA256 =
STREAMING DECODE = PASS / FAIL / NOT_RUN
CLKIN AFTER = EXACT RF001A PACKETS =
PEAK ANALYSIS MEMORY =
RF TX OBSERVED = RF TX = 0
FINAL RX RESULT PUBLISH =
FINAL RX RESULT REMOTE HEAD =
OFFLINE DECODE =
EXACT PACKETS =
MISSING =
DUPLICATES =
BER =
POST-FEC ERRORS =
RF TX =
0
STOP. STOP.

259
docs/handoff/NEXT_TX_TASK.md
View File

@@ -1,142 +1,96 @@
TASK_ID = RF001A_OTA_TX_003 TASK_ID = RF001A_OTA_TX_004
TASK_TITLE = Transmit one coordinated RF-001A QPSK burst TASK_TITLE = Send one finite RF-001A burst into RX capture 004
SOURCE_BRANCH = video-development SOURCE_BRANCH = video-development
EXPECTED_HEAD = bc55d0516dc1ee36cefa1ff096e7d86f60645303 EXPECTED_HEAD = 168e03d9e778ab2ec63502552d385ec7be3ef677
REQUIRES_APPROVAL = YES REQUIRES_APPROVAL = YES
HARDWARE_ACCESS = YES HARDWARE_ACCESS = YES
RF_RX = NO RF_RX = NO
RF_TX = YES RF_TX = YES
ALLOW_COMMIT = NO ALLOW_COMMIT = NO
ALLOW_PUSH = NO ALLOW_PUSH = TX_RESULT_ONLY
RISK_LEVEL = RED RISK_LEVEL = RED
This file is the authoritative current task for the MAIN PC TX node. ---
Read TX_NODE_STATE.md first. If chat history, this file, the shared RF-001A spec or current Git HEAD conflicts, stop and report the conflict.
# Current TX task # RF-001A TX 004
TASK = This task is prepared only. Do not execute it now. Old approval for any earlier
TRANSMIT ONE COORDINATED RF-001A QPSK BURST TASK_ID is invalid. The MAIN watcher waits for a fresh
`RF001A_OTA_CAPTURE_004` READY event from `origin/rx-node`.
This is a new RED task. Publication of this file is not permission to transmit. It does not inherit any approval from RF001A_OTA_TX_001, RF001A_OTA_TX_002, the RX task, a previous watcher prompt or a previous RF run. ## 1. Watcher gate
## 1. Coordination gate The watcher must require all of the following:
Start the local model-free watcher before the RX task begins: - RX TASK_ID = `RF001A_OTA_CAPTURE_004`;
- state = `RX_READY_FOR_TX`;
- source HEAD is the current `video-development` HEAD;
- COMMON_CLOCK_VERIFIED = YES;
- live child and growing capture are both true;
- planned RX duration is 59–61 seconds;
- READY is no more than 30 seconds old;
- at least 10 seconds remain in the capture window;
- the observed commit is the current valid `origin/rx-node` HEAD.
tools\start_tx_watcher.cmd The watcher then requests exactly:
The watcher polls origin/rx-node every 5 seconds. It processes every new result-branch commit from `last_processed_rx_node_commit` to the current `origin/rx-node` tip in chronological order. It accepts only a strict result for:
RX TASK_ID = RF001A_OTA_CAPTURE_003
STATE = RX_READY_FOR_TX
RX_READY_FOR_TX = YES
PROCESS_ALIVE = YES
FILE_GROWING = YES
COMMON_CLOCK_VERIFIED = YES
RF_TX = 0
The watcher records RF_HANDOFF_CHANNEL_TEST_001 READY and COMPLETE states without prompting for RED approval and never treats them as production readiness. A production READY publication must be no more than 30 seconds old. Its planned capture window must be 120 seconds. At least 10 seconds must remain. COMMON_CLOCK_VERIFIED must be YES, source_head must equal the expected video-development HEAD, and the result commit must be the current valid origin/rx-node HEAD.
When a valid new READY result appears, the watcher prints:
RX IS READY
TASK ID =
CAPTURE STARTED =
CAPTURE END EXPECTED =
TX TASK =
RF001A_OTA_TX_003
RF TX REQUIRES RED APPROVAL
Enter:
AUTHORIZE RF TX AUTHORIZE RF TX
Without the exact phrase AUTHORIZE RF TX, do not transmit. Any other response blocks TX. After the exact phrase, fetch `origin/rx-node`
again and repeat every commit, digest, task, source HEAD, freshness, common
clock and remaining-time check. A changed or stale result requires a new RX
capture and a new exact RED approval.
After the phrase, the watcher fetches rx-node again and repeats the result commit, digest, task ID, source HEAD, READY, freshness, common-clock and capture-window checks. If the result changed or less than 10 seconds remain, it refuses authorization and requires a new RX capture. On success publish a schema-version-2 `TX_AUTHORIZED` event to `origin/tx-node`
with the exact source RX commit and digest, TX TASK_ID `RF001A_OTA_TX_004`,
`RF_TX_AUTHORIZED = YES`, `RF_TX_STARTED = NO` and `RF_TX = 0`. Also write the
local authorization record. This event is permission evidence, not proof of a
transmission.
The watcher never opens HackRF hardware. On success it writes one local authorization record under: ## 2. Read-only TX preflight
.local\tx_authorizations\ Before opening TX hardware:
The record has rf_tx_started = false and rf_tx = 0. It is the handoff to the separately controlled TX executor. It is not evidence that transmission occurred. 1. Run `git pull --ff-only origin video-development`.
2. Require branch `video-development`, a clean tracked worktree and local HEAD
equal to `origin/video-development`; `EXPECTED_HEAD` must be its ancestor.
3. Run `hackrf_info`; require TX serial
`000000000000000097c063dc323b97d3`.
4. Require the recorded MASTER CLKOUT command = PASS, both antennas connected,
approximately 2 m distance, clear LOS, and video disabled.
5. Verify the waveform without radio access:
## 2. Fixed RF-001A parameters python -B -X utf8 tools/rf001a_prepare.py --verify-dir data/raw/rf001/rf001a_common_clock_001
center frequency = 1400000000 Hz 6. Require:
sample rate = 8000000 samples/s
modulation = QPSK
symbol rate = 500000 symbols/s
inner FEC = 2/3
payload = 256 bytes
packet count = 100
inter-packet gap = 5 ms
digital carrier offset = +125000 Hz
TX VGA = 20 dB
RF amp = OFF
antenna bias = OFF
common 10 MHz reference = VERIFIED
distance = approximately 2 m
LOS = YES
video = OFF
The waveform is one finite burst. Never add repeat mode. waveform path = data/raw/rf001/rf001a_common_clock_001/tx_waveform.cs8
waveform size = 17667656 bytes
## 3. Read-only preflight waveform SHA256 = 75f3fa39bfa4ee419f0676c54e4b9c486d7ff64bd0fe2fc06343b20c9d117ecd
sample count = 8833828 complex samples
Before the watcher can request RED approval: duration = 1.1042285 s
offline exact packets = 100/100
1. Run git pull --ff-only origin video-development.
2. Require branch video-development.
3. Require git status --short --untracked-files=no to produce no output.
4. Require local HEAD to equal origin/video-development and EXPECTED_HEAD to be its ancestor.
5. Run hackrf_info and preserve complete stdout.
6. Require TX serial 000000000000000097c063dc323b97d3.
7. Require the recorded MASTER CLKOUT command result = PASS.
8. Require confirmation that both antennas are connected, the nodes are approximately 2 m apart, and LOS is clear.
9. Verify the prepared waveform without radio access:
python -B -X utf8 tools/rf001a_prepare.py --verify-dir data/raw/rf001/rf001a_common_clock_001
10. Verify the actual cs8 bytes through the existing offline receiver:
python -B -X utf8 tools/qpsk_rf_rx.py --iq-file data/raw/rf001/rf001a_common_clock_001/tx_waveform.cs8 --iq-format cs8 --truth-metadata configs/rf/rf001a_truth_metadata.json --report <new-path-outside-git>\rf001a_tx_offline_verify.json
Require:
canonical waveform = PASS
hardware-format cs8 = PASS
offline hardware-format roundtrip = 100/100 exact
TX waveform path = data/raw/rf001/rf001a_common_clock_001/tx_waveform.cs8
TX waveform size = 17667656 bytes
TX waveform SHA256 = 75f3fa39bfa4ee419f0676c54e4b9c486d7ff64bd0fe2fc06343b20c9d117ecd
sample count = 8833828 complex samples
expected duration = 1.1042285 s
If any value differs, do not transmit. If any value differs, do not transmit.
## 4. Final check before opening TX hardware ## 3. Final executor gate
The controlled TX executor must read the authorization record immediately before TX and require: Immediately before hardware access, the controlled executor must read the
local authorization and re-fetch `origin/rx-node`. Require:
- task_id = RF001A_OTA_TX_003; - TX TASK_ID = `RF001A_OTA_TX_004`;
- authorization = AUTHORIZE RF TX; - RX TASK_ID = `RF001A_OTA_CAPTURE_004`;
- rx_task_id = RF001A_OTA_CAPTURE_003; - authorization phrase = `AUTHORIZE RF TX`;
- dry_run = false; - authorization binds the current READY commit and digest;
- action = MANUAL_EXECUTOR_GATE_OPEN; - `rf_tx_started = false` and `rf_tx = 0` in the authorization record;
- rf_tx_started = false; - at least 10 seconds remain in the same READY window.
- rf_tx = 0;
- the authorization references the latest fetched rx-node result commit and digest;
- current time is still inside the READY capture window with at least 10 seconds remaining.
Fetch origin/rx-node once more and compare its current result digest and commit with the authorization record. Re-run the READY and time checks. If anything differs, if the result is stale, or if fewer than 10 seconds remain, do not transmit and require a new RX capture and a new exact RED approval. If any check fails, do not open HackRF.
## 5. One authorized TX execution ## 4. One finite TX burst
Only after every preceding condition passes may the executor create a new unique evidence directory outside Git and run exactly one finite command: Only after all gates pass may the executor create a unique evidence directory
outside Git and run exactly one finite command:
hackrf_transfer.exe hackrf_transfer.exe
-d 000000000000000097c063dc323b97d3 -d 000000000000000097c063dc323b97d3
@@ -149,71 +103,48 @@ Only after every preceding condition passes may the executor create a new unique
-p 0 -p 0
-B -B
Run it once, synchronously, and preserve: Never add repeat mode and never retry automatically.
- exact process PID; At the exact process start, publish `TX_STARTED` to `origin/tx-node`, preserving
- start and end UTC timestamps; the authorization binding and recording `TX_STARTED_AT` as accurately as the
- exact exit code; host permits. Set `RF_TX_STARTED = YES` and `RF_TX = 1` because this event is
- complete stdout and stderr; evidence of an actual started process.
- every TX underrun line;
- every USB error;
- waveform path, size and SHA256;
- command parameters.
Set RF TX STOPPED = YES only after the finite process exits. Never retry automatically. Never start a second burst. Write the strict schema-version-2 event JSON outside Git and publish it with:
## 6. Final TX report python -B tools/tx_agent_watcher.py --publish-tx-event <started-json> --acknowledgement-branch tx-node
TASK_ID = After that exact process exits, publish `TX_COMPLETE` with `TX_FINISHED_AT`,
RF001A_OTA_TX_003 the integer process exit code, underrun count and error count. Preserve the
complete stdout and stderr outside Git. The event reports what happened; it
does not by itself prove that packets were received. Captured IQ remains the
source for exact packet verification.
RED AUTHORIZATION = Publish the completion JSON with the same `--publish-tx-event` command. The
AUTHORIZE RF TX / MISSING publisher requires the matching `TX_AUTHORIZED` and `TX_STARTED` history.
RX READY FOR TX = The only files permitted on `tx-node` are:
YES / NO
docs/handoff/results/TX_RESULT.json
docs/handoff/results/TX_RESULT.md
Do not publish waveform, IQ, video, source code, local state or large logs.
## 5. Final report
TASK_ID = RF001A_OTA_TX_004
RX READY FOR TX = YES / NO
RX RESULT HEAD = RX RESULT HEAD =
RX RESULT DIGEST = RX RESULT DIGEST =
TX AUTHORIZED EVENT = PASS / FAIL
TX SERIAL = TX STARTED AT =
TX FINISHED AT =
TX WAVEFORM SHA256 = TX PROCESS EXIT CODE =
TX START TIME =
TX END TIME =
TX EXIT CODE =
TX UNDERRUNS = TX UNDERRUNS =
TX ERRORS =
TX COMPLETE EVENT = PASS / FAIL
RF TX STOPPED = YES / NO
USB ERRORS = STOP after the one finite burst and event publication. Firmware, CPLD, Zadig,
USB driver changes, clock reconfiguration, a second burst, commit or push to
RF TX STOPPED = `video-development` are forbidden.
YES / NO
BURSTS TRANSMITTED =
0 / 1
RF TX =
0
At report time, RF TX = 0 means no transmission process remains active.
## 7. Prohibited actions
Do not:
- transmit without the fresh READY state and exact RED phrase;
- reuse an earlier authorization record;
- use repeat mode;
- transmit a second burst;
- retry automatically;
- change frequency, gains, clock configuration or waveform;
- update firmware or CPLD;
- use Zadig or change USB drivers;
- commit or push from this task.
STOP after the single finite burst or any failed gate.

30
docs/handoff/RX_AGENT_PROTOCOL.md
View File

@@ -12,7 +12,7 @@ The MAIN-to-RX acknowledgement channel uses the independent origin/tx-node branc
docs/handoff/NEXT_RX_TASK.md starts with these plain key-value fields: docs/handoff/NEXT_RX_TASK.md starts with these plain key-value fields:
TASK_ID = RF001A_OTA_CAPTURE_003 TASK_ID = RF001A_OTA_CAPTURE_004
TASK_TITLE = Start coordinated RF-001A common-clock OTA capture TASK_TITLE = Start coordinated RF-001A common-clock OTA capture
SOURCE_BRANCH = video-development SOURCE_BRANCH = video-development
EXPECTED_HEAD = <full 40-character base commit hash> EXPECTED_HEAD = <full 40-character base commit hash>
@@ -73,11 +73,11 @@ After the task actually finishes, mark the local ID complete:
The coordinated RF-001A result is a strict JSON object. Unknown or missing keys are rejected. READY requires: The coordinated RF-001A result is a strict JSON object. Unknown or missing keys are rejected. READY requires:
- task_id = RF001A_OTA_CAPTURE_003; - task_id = RF001A_OTA_CAPTURE_004;
- state = RX_READY_FOR_TX; - state = RX_READY_FOR_TX;
- rx_ready_for_tx, process_alive, file_growing and common_clock_verified all true; - rx_ready_for_tx, process_alive, file_growing and common_clock_verified all true;
- a positive capture_pid; - a positive capture_pid;
- 120-second planned duration; - 60-second planned duration;
- rf_tx = 0; - rf_tx = 0;
- final capture fields set to null. - final capture fields set to null.
@@ -95,6 +95,12 @@ Timestamps use timezone-aware ISO-8601. source_head is the actual local 40-chara
## Publishing to rx-node ## Publishing to rx-node
Before any production hardware access, run
`python -B tools/rx_agent_watcher.py --check-result-write`. It performs an
authenticated fetch, a no-change `git push --dry-run` of the existing rx-node
HEAD to the same ref, and a second remote HEAD check. Failure blocks capture.
The check creates no commit and no READY event.
Publish a prepared local JSON file with: Publish a prepared local JSON file with:
python -B tools/rx_agent_watcher.py --publish-result <result-json> --result-branch rx-node python -B tools/rx_agent_watcher.py --publish-result <result-json> --result-branch rx-node
@@ -114,7 +120,14 @@ It checks the branch with git fetch and git ls-remote through the real origin co
docs/handoff/results/RX_RESULT.json docs/handoff/results/RX_RESULT.json
docs/handoff/results/RX_RESULT.md docs/handoff/results/RX_RESULT.md
It checks the staged path set, commits, pushes HEAD:rx-node, and confirms that git ls-remote returns the new commit. It then removes the temporary worktree. It never switches the source checkout. It rejects any additional staged path and enforces a 64 KiB limit per result file. It checks the staged path set, creates one immutable commit, pushes that exact
commit hash to rx-node, and confirms that `git ls-remote` returns it. After a
recognized transient network or authentication error, it first checks whether
the commit already reached the remote. If not, it fetches and may retry that
same commit once. It never creates a second READY commit or changes the payload
between attempts. It then removes the temporary worktree. It never switches
the source checkout. It rejects any additional staged path and enforces a
64 KiB limit per result file.
Raw IQ, video, waveform files, large logs, secrets and machine-local state remain outside Git. A missing result branch or failed push is reported and never authorizes TX. Raw IQ, video, waveform files, large logs, secrets and machine-local state remain outside Git. A missing result branch or failed push is reported and never authorizes TX.
@@ -132,7 +145,12 @@ For each successfully handled channel-test event, the MAIN watcher publishes a s
The tx-node publisher stages exactly `docs/handoff/results/TX_RESULT.json` and `docs/handoff/results/TX_RESULT.md`, checks the staged path allowlist and enforces 64 KiB per file. A push failure prevents the rx-node cursor from advancing. Duplicate acknowledgements are suppressed by their source binding. No local state, code, IQ, video, waveform or large log is published. The tx-node publisher stages exactly `docs/handoff/results/TX_RESULT.json` and `docs/handoff/results/TX_RESULT.md`, checks the staged path allowlist and enforces 64 KiB per file. A push failure prevents the rx-node cursor from advancing. Duplicate acknowledgements are suppressed by their source binding. No local state, code, IQ, video, waveform or large log is published.
A successful gate writes one local authorization record under .local/tx_authorizations/. The record has rf_tx_started = false and rf_tx = 0. The separately controlled TX executor must re-fetch and compare the result commit and digest immediately before opening hardware. A successful gate publishes a schema-version-2 `TX_AUTHORIZED` event and writes
one local authorization record under `.local/tx_authorizations/`. The record
has `rf_tx_started = false` and `rf_tx = 0`. The separately controlled TX
executor must re-fetch and compare the result commit and digest immediately
before opening hardware. It then publishes `TX_STARTED` at process start and
`TX_COMPLETE` after exit with UTC timestamps, exit code, underruns and errors.
Duplicate result digests are not shown repeatedly. Duplicate result digests are not shown repeatedly.
@@ -147,6 +165,6 @@ Duplicate result digests are not shown repeatedly.
## Bidirectional channel and dry-run contracts ## Bidirectional channel and dry-run contracts
The completed preflight channel command published synthetic READY and COMPLETE results with TASK_ID RF_HANDOFF_CHANNEL_TEST_001 through origin/rx-node. The MAIN watcher observed both without a RED prompt. The RX watcher now requires exact matching acknowledgements in origin/tx-node before RF001A_OTA_CAPTURE_003 can pass its software gate. The completed preflight channel command published synthetic READY and COMPLETE results with TASK_ID RF_HANDOFF_CHANNEL_TEST_001 through origin/rx-node. The MAIN watcher observed both without a RED prompt. The RX watcher now requires exact matching acknowledgements in origin/tx-node before RF001A_OTA_CAPTURE_004 can pass its software gate.
Automated coordination tests use a temporary local bare Git repository and independent rx-node and tx-node branches. They cover exact commit and digest binding, duplicate suppression, two commits between polls, stale acknowledgement rejection, missing tx-node, push failure, non-fast-forward protection and a full fake two-way flow without accessing the real remote or SDR hardware and without transmitting RF. Automated coordination tests use a temporary local bare Git repository and independent rx-node and tx-node branches. They cover exact commit and digest binding, duplicate suppression, two commits between polls, stale acknowledgement rejection, missing tx-node, push failure, non-fast-forward protection and a full fake two-way flow without accessing the real remote or SDR hardware and without transmitting RF.

14
docs/handoff/RX_AGENT_USAGE.md
View File

@@ -10,7 +10,7 @@ The watcher stays in a visible console and polls origin/video-development every
## Idle ## Idle
While no new TASK_ID exists, the watcher performs only git fetch, reads the remote task file and compares IDs with .local/rx_agent_state.json. There is no model execution or token use. For RF001A_OTA_CAPTURE_003 it also verifies the required historical acknowledgements on origin/tx-node before offering the task for approval. While no new TASK_ID exists, the watcher performs only git fetch, reads the remote task file and compares IDs with .local/rx_agent_state.json. There is no model execution or token use. For RF001A_OTA_CAPTURE_004 it also verifies the required historical acknowledgements on origin/tx-node and performs the authenticated no-change rx-node write preflight before offering the task for approval.
## New task ## New task
@@ -41,7 +41,9 @@ When the task is actually complete, run:
## Publish READY ## Publish READY
For RF001A_OTA_CAPTURE_003, first require the watcher to print: For RF001A_OTA_CAPTURE_004, first require the watcher to print:
RX-NODE WRITE PREFLIGHT = PASS
MAIN READY RECEIVED = CONFIRMED MAIN READY RECEIVED = CONFIRMED
MAIN COMPLETE RECEIVED = CONFIRMED MAIN COMPLETE RECEIVED = CONFIRMED
@@ -65,6 +67,14 @@ The command fails if:
A failed publication is not READY evidence and cannot authorize TX. A failed publication is not READY evidence and cannot authorize TX.
Immediately before hardware access, repeat the same non-mutating check:
python -B tools/rx_agent_watcher.py --check-result-write --result-branch rx-node
The READY publisher retries only one recognized transient failure. It checks
the remote before retrying and pushes the same immutable commit; it does not
create a replacement READY commit.
## Verified model-free channel evidence ## Verified model-free channel evidence
The completed RF_HANDOFF_CHANNEL_TEST_001 evidence is bound to READY commit `4cb6c8cad1ab5194755c2740c2a5a83933d099d8` and COMPLETE commit `83d91ff5309777ab639a24f774e9d0b8825a11e9`. The RX watcher accepts the gate only after origin/tx-node contains the matching MAIN acknowledgements. This read-only check performs no capture and keeps RF TX at 0. The completed RF_HANDOFF_CHANNEL_TEST_001 evidence is bound to READY commit `4cb6c8cad1ab5194755c2740c2a5a83933d099d8` and COMPLETE commit `83d91ff5309777ab639a24f774e9d0b8825a11e9`. The RX watcher accepts the gate only after origin/tx-node contains the matching MAIN acknowledgements. This read-only check performs no capture and keeps RF TX at 0.

4
docs/handoff/RX_RESULT_TEMPLATE.md
View File

@@ -6,7 +6,7 @@ The publisher creates both docs/handoff/results/RX_RESULT.json and docs/handoff/
{ {
"schema_version": 1, "schema_version": 1,
"task_id": "RF001A_OTA_CAPTURE_003", "task_id": "RF001A_OTA_CAPTURE_004",
"state": "RX_READY_FOR_TX", "state": "RX_READY_FOR_TX",
"rx_ready_for_tx": true, "rx_ready_for_tx": true,
"process_alive": true, "process_alive": true,
@@ -14,7 +14,7 @@ The publisher creates both docs/handoff/results/RX_RESULT.json and docs/handoff/
"capture_pid": 1234, "capture_pid": 1234,
"published_at": "2026-09-21T12:00:02+00:00", "published_at": "2026-09-21T12:00:02+00:00",
"capture_started_at": "2026-09-21T12:00:00+00:00", "capture_started_at": "2026-09-21T12:00:00+00:00",
"expected_capture_end": "2026-09-21T12:02:00+00:00", "expected_capture_end": "2026-09-21T12:01:00+00:00",
"capture_path": "C:\\Users\\Professional\\Documents\\SDR_Rover_Captures\\<run>\\rf001a_ota_capture.cs8", "capture_path": "C:\\Users\\Professional\\Documents\\SDR_Rover_Captures\\<run>\\rf001a_ota_capture.cs8",
"common_clock_verified": true, "common_clock_verified": true,
"source_head": "<40-character lowercase Git HEAD>", "source_head": "<40-character lowercase Git HEAD>",

26
docs/handoff/TX_AGENT_USAGE.md
View File

@@ -44,19 +44,37 @@ The acknowledgement operation is ordinary Python and Git. It does not invoke Cha
RF_HANDOFF_CHANNEL_TEST_001 READY and COMPLETE states are recorded and acknowledged without a RED prompt and never open the production gate. RF_HANDOFF_CHANNEL_TEST_001 READY and COMPLETE states are recorded and acknowledged without a RED prompt and never open the production gate.
For a fresh valid RF001A_OTA_CAPTURE_003 READY result, the watcher checks the expected source HEAD, current origin/rx-node commit, common clock and capture window. It then prints the task ID, capture start and expected end and asks for: For a fresh valid RF001A_OTA_CAPTURE_004 READY result, the watcher checks the expected source HEAD, current origin/rx-node commit, common clock and capture window. It then prints the task ID, capture start and expected end and asks for:
AUTHORIZE RF TX AUTHORIZE RF TX
Any other input blocks authorization. Any other input blocks authorization.
After the exact phrase, the watcher fetches rx-node again. It requires the same result digest and commit, the expected source HEAD, a fresh publication, an active 120-second capture window, verified common clock and at least 10 seconds remaining. An expired historical READY is recorded as observed but cannot create an authorization. After the exact phrase, the watcher fetches rx-node again. It requires the same result digest and commit, the expected source HEAD, a fresh publication, an active 60-second capture window, verified common clock and at least 10 seconds remaining. An expired historical READY is recorded as observed but cannot create an authorization.
On success it writes: On success it writes:
.local\tx_authorizations\RF001A_OTA_TX_003_authorized.json .local\tx_authorizations\RF001A_OTA_TX_004_authorized.json
This record opens the gate for the separately controlled TX executor. The watcher itself does not run hackrf_transfer. The record explicitly says rf_tx_started = false and rf_tx = 0. The watcher also publishes `TX_AUTHORIZED` to origin/tx-node. This record opens
the gate for the separately controlled TX executor. The watcher itself does
not run hackrf_transfer. The record explicitly says `rf_tx_started = false`
and `rf_tx = 0`.
At actual process start the executor publishes `TX_STARTED` with the most
accurate available UTC timestamp. After process exit it publishes
`TX_COMPLETE` with finish time, integer exit code, underruns and errors. These
events preserve the exact RX READY commit and digest binding. They do not
replace verification of packets from captured IQ.
The executor writes each strict schema-version-2 event JSON outside Git, then
uses the model-free publisher:
python -B tools/tx_agent_watcher.py --publish-tx-event <event-json> --acknowledgement-branch tx-node
The publisher requires a clean, current `video-development` checkout. A
`TX_STARTED` event requires a matching `TX_AUTHORIZED` in tx-node. A
`TX_COMPLETE` event also requires the matching `TX_STARTED` timestamp.
The TX executor must re-fetch rx-node and compare the result commit and digest immediately before hardware access. The TX executor must re-fetch rx-node and compare the result commit and digest immediately before hardware access.

5
pyrightconfig.json
View File

@@ -22,11 +22,13 @@
"tools/qpsk_rf_common.py", "tools/qpsk_rf_common.py",
"tools/qpsk_rf_tx.py", "tools/qpsk_rf_tx.py",
"tools/qpsk_rf_rx.py", "tools/qpsk_rf_rx.py",
"tools/rf001a_streaming.py",
"tools/rf001a_prepare.py", "tools/rf001a_prepare.py",
"tools/rx_agent_watcher.py", "tools/rx_agent_watcher.py",
"tools/rf_handoff_channel.py", "tools/rf_handoff_channel.py",
"tools/rf_handoff_smoke.py", "tools/rf_handoff_smoke.py",
"tools/tx_agent_watcher.py", "tools/tx_agent_watcher.py",
"tools/tx_handoff_channel.py",
"tools/clock001.py", "tools/clock001.py",
"tests/test_video005_iq.py", "tests/test_video005_iq.py",
"tests/test_video005_video.py", "tests/test_video005_video.py",
@@ -34,6 +36,9 @@
"tests/test_rf001a_prepare.py", "tests/test_rf001a_prepare.py",
"tests/test_rx_agent_watcher.py", "tests/test_rx_agent_watcher.py",
"tests/test_rf_handoff_coordination.py", "tests/test_rf_handoff_coordination.py",
"tests/test_rf_handoff_reliability.py",
"tests/test_rf001a_streaming.py",
"tests/test_tx_handoff_acknowledgement.py",
"tests/test_capture_process_runner.py", "tests/test_capture_process_runner.py",
"tests/test_rf001a_truth_metadata.py", "tests/test_rf001a_truth_metadata.py",
"tests/test_clock001.py" "tests/test_clock001.py"

109
tests/test_rf001a_streaming.py Normal file
View File

@@ -0,0 +1,109 @@
"""Проверки оконного cs8-декодера RF-001A без SDR и RF TX."""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from pathlib import Path
import numpy as np
from protocol.video003_modem import build_frame
from protocol.video005_iq_waveform import shape_packet
from tools.qpsk_rf_common import SEED, payload_for_sequence
from tools.qpsk_rf_rx import decode_file
from tools.rf001a_streaming import (
MemoryPeakSampler,
SampleWindow,
count_cs8_clipping,
cs8_sample_count,
decode_cs8_streaming,
read_cs8_window,
receive_cs8_windows,
required_overlap_samples,
rf001_packet_waveform_samples,
timestamp_sample_window,
)
def _write_cs8(path: Path, iq: np.ndarray) -> None:
peak = float(np.max(np.abs(iq))) if iq.size else 1.0
normalized = iq / max(peak, 1e-12) * 0.75
interleaved = np.empty(iq.size * 2, dtype=np.int8)
interleaved[0::2] = np.rint(normalized.real * 127).astype(np.int8)
interleaved[1::2] = np.rint(normalized.imag * 127).astype(np.int8)
interleaved.tofile(path)
def test_timestamp_to_sample_window_uses_safe_margins() -> None:
capture = datetime(2026, 9, 22, 12, 0, tzinfo=timezone.utc)
window = timestamp_sample_window(
capture_started_at=capture.isoformat(),
tx_started_at=(capture + timedelta(seconds=10)).isoformat(),
tx_finished_at=(capture + timedelta(seconds=11.1042285)).isoformat(),
total_samples=160_000_000,
margin_seconds=2.0,
)
assert window.start == 64_000_000
assert window.stop == 104_833_824
def test_chunked_cs8_reading_from_large_sparse_capture(tmp_path: Path) -> None:
capture = tmp_path / "large_sparse.cs8"
with capture.open("wb") as stream:
stream.seek(1_920_000_000 - 1)
stream.write(b"\0")
raw = np.memmap(capture, dtype=np.int8, mode="r")
window = read_cs8_window(raw, 900_000_000, 900_001_000)
assert capture.stat().st_size == 1_920_000_000
assert cs8_sample_count(capture) == 960_000_000
assert window.dtype == np.complex64
assert window.size == 1_000
assert window.nbytes == 8_000
with MemoryPeakSampler() as memory:
assert count_cs8_clipping(capture) == 0
assert memory.additional_peak_bytes < 512_000_000
def test_packet_across_chunk_boundary_is_decoded_once(tmp_path: Path) -> None:
payload = payload_for_sequence(SEED, 7)
frame = build_frame(payload, "QPSK", "2/3", sequence_number=7)
waveform = shape_packet(frame)
chunk_samples = 120_000
packet_start = chunk_samples - waveform.size // 2
iq = np.zeros(chunk_samples * 2, dtype=np.complex64)
iq[packet_start:packet_start + waveform.size] = waveform
capture = tmp_path / "boundary.cs8"
_write_cs8(capture, iq)
result = receive_cs8_windows(
capture,
(SampleWindow(0, iq.size),),
decoder_backend="legacy",
chunk_samples=chunk_samples,
)
assert required_overlap_samples() == rf001_packet_waveform_samples() == 48_865
assert [packet.packet for packet in result.reception.packets] == [frame.packet]
assert result.maximum_loaded_samples < iq.size
def test_capture_without_rf001a_signal_has_zero_packets(tmp_path: Path) -> None:
capture = tmp_path / "noise_only.cs8"
np.zeros(400_000, dtype=np.int8).tofile(capture)
result = decode_cs8_streaming(capture, decoder_backend="legacy")
assert result.window_source == "STREAMING_POWER_LOCATOR"
assert result.windows == ()
assert result.reception.packets == ()
assert result.maximum_loaded_samples == 0
truth = Path(__file__).resolve().parents[1] / "configs/rf/rf001a_truth_metadata.json"
report = decode_file(
capture,
iq_format="cs8",
truth_metadata_path=truth,
decoder_backend="legacy",
)
assert report["exact_packets"] == 0
assert report["crc_valid_packets"] == 0
assert report["streaming_decoder"] is True
assert report["peak_additional_ram_bytes"] < 1_000_000_000

38
tests/test_rf_handoff_coordination.py
View File

@@ -31,6 +31,7 @@ from tools.rf_handoff_smoke import run_smoke
from tools.rx_agent_watcher import ( from tools.rx_agent_watcher import (
WatcherState as RxWatcherState, WatcherState as RxWatcherState,
publish_result, publish_result,
require_live_ready_window,
save_state as save_rx_state, save_state as save_rx_state,
) )
from tools.tx_agent_watcher import ( from tools.tx_agent_watcher import (
@@ -59,7 +60,7 @@ def ready_result(**changes: object) -> RxHandoffResult:
capture_pid=1234, capture_pid=1234,
published_at=NOW.isoformat(), published_at=NOW.isoformat(),
capture_started_at=NOW.isoformat(), capture_started_at=NOW.isoformat(),
expected_capture_end=(NOW + timedelta(seconds=120)).isoformat(), expected_capture_end=(NOW + timedelta(seconds=60)).isoformat(),
capture_path=r"C:\captures\rf001a_ota_capture.cs8", capture_path=r"C:\captures\rf001a_ota_capture.cs8",
common_clock_verified=True, common_clock_verified=True,
source_head=HEAD, source_head=HEAD,
@@ -97,10 +98,10 @@ def complete_result(
rx_ready_for_tx=False, rx_ready_for_tx=False,
process_alive=False, process_alive=False,
file_growing=False, file_growing=False,
published_at=(NOW + timedelta(seconds=121)).isoformat(), published_at=(NOW + timedelta(seconds=61)).isoformat(),
exit_code=0, exit_code=0,
file_size=1_920_000_000, file_size=960_000_000,
duration_seconds=120.01, duration_seconds=60.01,
overruns=0, overruns=0,
usb_errors=0, usb_errors=0,
file_errors=0, file_errors=0,
@@ -278,7 +279,7 @@ def test_complete_result_accepts_explicit_unreported_counters() -> None:
ready_result( ready_result(
published_at=(NOW - timedelta(seconds=31)).isoformat(), published_at=(NOW - timedelta(seconds=31)).isoformat(),
capture_started_at=(NOW - timedelta(seconds=31)).isoformat(), capture_started_at=(NOW - timedelta(seconds=31)).isoformat(),
expected_capture_end=(NOW + timedelta(seconds=89)).isoformat(), expected_capture_end=(NOW + timedelta(seconds=29)).isoformat(),
), ),
NOW, NOW,
"STALE_READY", "STALE_READY",
@@ -290,7 +291,7 @@ def test_complete_result_accepts_explicit_unreported_counters() -> None:
), ),
( (
ready_result( ready_result(
capture_started_at=(NOW - timedelta(seconds=111)).isoformat(), capture_started_at=(NOW - timedelta(seconds=51)).isoformat(),
expected_capture_end=(NOW + timedelta(seconds=9)).isoformat(), expected_capture_end=(NOW + timedelta(seconds=9)).isoformat(),
), ),
NOW, NOW,
@@ -329,7 +330,7 @@ def test_ready_evaluation_passes_with_more_than_ten_seconds() -> None:
) )
assert evaluation.ready assert evaluation.ready
assert evaluation.reason == "READY" assert evaluation.reason == "READY"
assert evaluation.seconds_remaining == pytest.approx(118.0) assert evaluation.seconds_remaining == pytest.approx(58.0)
def test_red_approval_and_duplicate_suppression(tmp_path: Path) -> None: def test_red_approval_and_duplicate_suppression(tmp_path: Path) -> None:
@@ -378,7 +379,7 @@ def test_window_and_commit_are_rechecked_after_exact_approval(tmp_path: Path) ->
digest = result_digest(result) digest = result_digest(result)
source = FakeSource(result, digest) source = FakeSource(result, digest)
executor = FakeExecutor(tmp_path / "fake_tx.txt") executor = FakeExecutor(tmp_path / "fake_tx.txt")
times = iter((NOW + timedelta(seconds=2), NOW + timedelta(seconds=111))) times = iter((NOW + timedelta(seconds=2), NOW + timedelta(seconds=51)))
status, _ = handle_result( status, _ = handle_result(
result, result,
@@ -600,7 +601,7 @@ def test_expired_historical_ready_is_observed_without_authorization(
started = NOW - timedelta(seconds=200) started = NOW - timedelta(seconds=200)
expired = ready_result( expired = ready_result(
capture_started_at=started.isoformat(), capture_started_at=started.isoformat(),
expected_capture_end=(started + timedelta(seconds=120)).isoformat(), expected_capture_end=(started + timedelta(seconds=60)).isoformat(),
published_at=(NOW - timedelta(seconds=100)).isoformat(), published_at=(NOW - timedelta(seconds=100)).isoformat(),
) )
expired_outcome = channel.publish(expired) expired_outcome = channel.publish(expired)
@@ -796,9 +797,16 @@ Fake task used only by the local coordination test.
) )
source_head = _git(["rev-parse", "HEAD"], work) source_head = _git(["rev-parse", "HEAD"], work)
fresh_start = datetime.now(timezone.utc)
fresh_result = ready_result(
source_head=source_head,
published_at=fresh_start.isoformat(),
capture_started_at=fresh_start.isoformat(),
expected_capture_end=(fresh_start + timedelta(seconds=60)).isoformat(),
)
result_path = tmp_path / "ready.json" result_path = tmp_path / "ready.json"
result_path.write_text( result_path.write_text(
canonical_result_json(ready_result(source_head=source_head)), canonical_result_json(fresh_result),
encoding="utf-8", encoding="utf-8",
) )
state_path = tmp_path / "rx_state.json" state_path = tmp_path / "rx_state.json"
@@ -824,4 +832,12 @@ Fake task used only by the local coordination test.
assert "RF TX = 0" in output assert "RF TX = 0" in output
published, _digest, _head = GitResultChannel(work).read() published, _digest, _head = GitResultChannel(work).read()
assert published == ready_result(source_head=source_head) assert published == fresh_result
def test_rx_publisher_refuses_ready_after_capture_window() -> None:
with pytest.raises(ValueError, match="window has already ended"):
require_live_ready_window(
ready_result(),
now=NOW + timedelta(seconds=60),
)

145
tests/test_rf_handoff_reliability.py Normal file
View File

@@ -0,0 +1,145 @@
"""Проверки повторной публикации READY и Git preflight без оборудования."""
from __future__ import annotations
from pathlib import Path
import subprocess
import pytest
from tools.rf_handoff_channel import GitResultChannel, ResultChannelError
OLD_HEAD = "1" * 40
READY_HEAD = "2" * 40
class SimulatedPushChannel(GitResultChannel):
def __init__(
self,
push_results: list[subprocess.CompletedProcess[str]],
observed_heads: list[str],
) -> None:
self.remote = "origin"
self.branch = "rx-node"
self.push_results = iter(push_results)
self.observed_heads = iter(observed_heads)
self.push_arguments: list[tuple[str, ...]] = []
self.fetch_calls = 0
def _run(
self,
arguments: list[str],
*,
cwd: Path | None = None,
check: bool = True,
) -> subprocess.CompletedProcess[str]:
del cwd, check
assert arguments[0] == "push"
self.push_arguments.append(tuple(arguments))
return next(self.push_results)
def remote_head(self) -> str:
return next(self.observed_heads)
def fetch(self) -> str:
self.fetch_calls += 1
return OLD_HEAD
def _push_result(returncode: int, stderr: str = "") -> subprocess.CompletedProcess[str]:
return subprocess.CompletedProcess(["git", "push"], returncode, "", stderr)
def _git(arguments: list[str], cwd: Path) -> str:
return subprocess.run(
["git", *arguments],
cwd=cwd,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
check=True,
).stdout.strip()
def test_transient_failure_retries_the_same_ready_commit_once() -> None:
channel = SimulatedPushChannel(
[_push_result(1, "Authentication failed"), _push_result(0)],
[OLD_HEAD, READY_HEAD],
)
assert channel._push_exact_commit(Path("checkout"), READY_HEAD, OLD_HEAD) == READY_HEAD
assert len(channel.push_arguments) == 2
assert channel.push_arguments[0] == channel.push_arguments[1]
assert channel.push_arguments[0][-1] == f"{READY_HEAD}:refs/heads/rx-node"
assert channel.fetch_calls == 1
def test_ambiguous_push_is_success_when_remote_already_has_ready() -> None:
channel = SimulatedPushChannel(
[_push_result(1, "TLS connection was non-properly terminated")],
[READY_HEAD],
)
assert channel._push_exact_commit(Path("checkout"), READY_HEAD, OLD_HEAD) == READY_HEAD
assert len(channel.push_arguments) == 1
assert channel.fetch_calls == 0
def test_second_transient_failure_does_not_create_another_ready() -> None:
channel = SimulatedPushChannel(
[
_push_result(1, "Authentication failed"),
_push_result(1, "Authentication failed"),
],
[OLD_HEAD, OLD_HEAD],
)
with pytest.raises(ResultChannelError, match="after one retry"):
channel._push_exact_commit(Path("checkout"), READY_HEAD, OLD_HEAD)
assert len(channel.push_arguments) == 2
assert channel.push_arguments[0] == channel.push_arguments[1]
def test_failed_write_preflight_stops_before_any_publication() -> None:
class FailingPreflight(GitResultChannel):
def __init__(self) -> None:
self.remote = "origin"
self.branch = "rx-node"
def fetch(self) -> str:
return OLD_HEAD
def remote_head(self) -> str:
raise AssertionError("remote confirmation must not follow failed dry-run")
def _run(
self,
arguments: list[str],
*,
cwd: Path | None = None,
check: bool = True,
) -> subprocess.CompletedProcess[str]:
del cwd, check
assert arguments[:3] == ["push", "--dry-run", "--porcelain"]
return _push_result(1, "Authentication failed")
with pytest.raises(ResultChannelError, match="push preflight failed"):
FailingPreflight().verify_write_access()
def test_write_preflight_checks_access_without_changing_remote(tmp_path: Path) -> None:
remote = tmp_path / "remote.git"
work = tmp_path / "work"
_git(["init", "--bare", str(remote)], tmp_path)
work.mkdir()
_git(["init"], work)
_git(["config", "user.name", "Test"], work)
_git(["config", "user.email", "test@example.invalid"], work)
(work / "seed.txt").write_text("seed\n", encoding="utf-8")
_git(["add", "seed.txt"], work)
_git(["commit", "-m", "Seed result branch"], work)
_git(["remote", "add", "origin", str(remote)], work)
_git(["push", "origin", "HEAD:refs/heads/rx-node"], work)
channel = GitResultChannel(work)
before = channel.remote_head()
assert channel.verify_write_access() == before
assert channel.remote_head() == before

96
tests/test_tx_handoff_acknowledgement.py
View File

@@ -26,10 +26,15 @@ from tools.tx_agent_watcher import TxWatcherState, process_history_cycle
from tools.tx_handoff_channel import ( from tools.tx_handoff_channel import (
CHANNEL_COMPLETE_OBSERVED, CHANNEL_COMPLETE_OBSERVED,
CHANNEL_READY_OBSERVED, CHANNEL_READY_OBSERVED,
TX_ACK_SCHEMA_VERSION,
TX_AUTHORIZED,
TX_COMPLETE,
TX_STARTED,
TX_RESULT_PATHS, TX_RESULT_PATHS,
GitAcknowledgementChannel, GitAcknowledgementChannel,
TxHandoffAcknowledgement, TxHandoffAcknowledgement,
acknowledgement_from_mapping, acknowledgement_from_mapping,
build_tx_execution_event,
channel_test_acknowledgements, channel_test_acknowledgements,
) )
@@ -91,7 +96,7 @@ def channel_ready(**changes: object) -> RxHandoffResult:
capture_pid=1234, capture_pid=1234,
published_at=NOW.isoformat(), published_at=NOW.isoformat(),
capture_started_at=NOW.isoformat(), capture_started_at=NOW.isoformat(),
expected_capture_end=(NOW + timedelta(seconds=120)).isoformat(), expected_capture_end=(NOW + timedelta(seconds=60)).isoformat(),
capture_path="MODEL_FREE_FAKE_CAPTURE", capture_path="MODEL_FREE_FAKE_CAPTURE",
common_clock_verified=False, common_clock_verified=False,
source_head=SOURCE_HEAD, source_head=SOURCE_HEAD,
@@ -175,6 +180,95 @@ def test_acknowledgement_schema_binds_digest_and_source_commit() -> None:
acknowledgement_from_mapping({**raw, "rf_tx_authorized": True}) acknowledgement_from_mapping({**raw, "rf_tx_authorized": True})
def test_tx_events_preserve_authorization_binding_and_timestamps() -> None:
authorization = TxHandoffAcknowledgement(
schema_version=TX_ACK_SCHEMA_VERSION,
task_id="RF001A_OTA_CAPTURE_004",
state=TX_AUTHORIZED,
source_rx_commit="1" * 40,
source_rx_digest="2" * 64,
source_head=SOURCE_HEAD,
observed_at=NOW.isoformat(),
main_watcher_pid=4321,
rf_tx_authorized=True,
rf_tx_started=False,
rf_tx=0,
tx_task_id="RF001A_OTA_TX_004",
)
started_at = (NOW + timedelta(seconds=1)).isoformat()
started = build_tx_execution_event(
authorization,
state=TX_STARTED,
observed_at=started_at,
tx_started_at=started_at,
)
finished_at = (NOW + timedelta(seconds=2.2)).isoformat()
complete = build_tx_execution_event(
authorization,
state=TX_COMPLETE,
observed_at=finished_at,
tx_started_at=started_at,
tx_finished_at=finished_at,
process_exit_code=0,
underruns=0,
errors=0,
)
assert started.source_rx_commit == authorization.source_rx_commit
assert started.source_rx_digest == authorization.source_rx_digest
assert started.rf_tx_started and started.rf_tx == 1
assert complete.tx_finished_at == finished_at
assert complete.tx_process_exit_code == 0
def test_execution_event_publication_requires_authorization_and_start(
tmp_path: Path,
) -> None:
_remote, work, _tx_root = repository_fixture(tmp_path)
channel = GitAcknowledgementChannel(work)
authorization = TxHandoffAcknowledgement(
schema_version=TX_ACK_SCHEMA_VERSION,
task_id="RF001A_OTA_CAPTURE_004",
state=TX_AUTHORIZED,
source_rx_commit="1" * 40,
source_rx_digest="2" * 64,
source_head=SOURCE_HEAD,
observed_at=NOW.isoformat(),
main_watcher_pid=4321,
rf_tx_authorized=True,
rf_tx_started=False,
rf_tx=0,
tx_task_id="RF001A_OTA_TX_004",
)
started_at = (NOW + timedelta(seconds=1)).isoformat()
finished_at = (NOW + timedelta(seconds=2.2)).isoformat()
started = build_tx_execution_event(
authorization,
state=TX_STARTED,
observed_at=started_at,
tx_started_at=started_at,
)
complete = build_tx_execution_event(
authorization,
state=TX_COMPLETE,
observed_at=finished_at,
tx_started_at=started_at,
tx_finished_at=finished_at,
process_exit_code=0,
underruns=0,
errors=0,
)
with pytest.raises(ResultChannelError, match="TX_AUTHORIZED"):
channel.publish_execution_event(started)
channel.publish(authorization)
with pytest.raises(ResultChannelError, match="TX_STARTED"):
channel.publish_execution_event(complete)
started_outcome = channel.publish_execution_event(started)
complete_outcome = channel.publish_execution_event(complete)
assert started_outcome.status == "PUBLISHED"
assert complete_outcome.status == "PUBLISHED"
def test_main_publish_uses_exact_allowlist_and_suppresses_duplicate( def test_main_publish_uses_exact_allowlist_and_suppresses_duplicate(
tmp_path: Path, tmp_path: Path,
) -> None: ) -> None:

75
tools/qpsk_rf_rx.py
View File

@@ -31,6 +31,12 @@ from tools.qpsk_rf_common import (
GAP_MS, PACKET_COUNT, PAYLOAD_BYTES, SEED, TEST_ID, GAP_MS, PACKET_COUNT, PAYLOAD_BYTES, SEED, TEST_ID,
payload_for_sequence, read_iq_file, sha256_file, payload_for_sequence, read_iq_file, sha256_file,
) )
from tools.rf001a_streaming import (
DEFAULT_CHUNK_SAMPLES,
MemoryPeakSampler,
cs8_sample_count,
decode_cs8_streaming,
)
def _bit_errors(observed: bytes, expected: bytes) -> tuple[int, int]: def _bit_errors(observed: bytes, expected: bytes) -> tuple[int, int]:
@@ -124,7 +130,7 @@ def summarize_result(
unique_sequences = set(sequences) unique_sequences = set(sequences)
cfo = [row["cfo_hz"] for row in rows] cfo = [row["cfo_hz"] for row in rows]
report: dict[str, Any] = { report: dict[str, Any] = {
"rf001_status": "PREPARED_NOT_RUN", "rf001_status": "OFFLINE_ANALYZED",
"test_id": truth.get("test_id") if truth is not None else None, "test_id": truth.get("test_id") if truth is not None else None,
"modulation": "QPSK", "modulation": "QPSK",
"symbol_rate": 500_000, "symbol_rate": 500_000,
@@ -242,10 +248,32 @@ def decode_file(
iq_format: str = "c64le", iq_format: str = "c64le",
truth_metadata_path: Path | None = None, truth_metadata_path: Path | None = None,
decoder_backend: str = "numba", decoder_backend: str = "numba",
capture_started_at: str | None = None,
tx_started_at: str | None = None,
tx_finished_at: str | None = None,
chunk_samples: int = DEFAULT_CHUNK_SAMPLES,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Открывает только файл; truth читается после вызова рабочего приёмника.""" """Декодирует c64le целиком, а большие cs8 — ограниченными окнами."""
iq, clipping_codes = read_iq_file(iq_path, iq_format)
reception = receive_iq_stream(iq, decoder_backend=decoder_backend) streaming = None
with MemoryPeakSampler() as memory:
if iq_format == "cs8":
streaming = decode_cs8_streaming(
iq_path,
capture_started_at=capture_started_at,
tx_started_at=tx_started_at,
tx_finished_at=tx_finished_at,
decoder_backend=decoder_backend,
chunk_samples=chunk_samples,
)
reception = streaming.reception
iq_sample_count = cs8_sample_count(iq_path)
clipping_codes = streaming.clipping_codes
iq = None
else:
iq, clipping_codes = read_iq_file(iq_path, iq_format)
reception = receive_iq_stream(iq, decoder_backend=decoder_backend)
iq_sample_count = int(iq.size)
truth = (json.loads(truth_metadata_path.read_text(encoding="utf-8")) truth = (json.loads(truth_metadata_path.read_text(encoding="utf-8"))
if truth_metadata_path is not None else None) if truth_metadata_path is not None else None)
if truth is not None and ( if truth is not None and (
@@ -262,17 +290,38 @@ def decode_file(
or truth.get("seed") != SEED or truth.get("seed") != SEED
): ):
raise ValueError("TX-ведомость не соответствует режиму RF-001") raise ValueError("TX-ведомость не соответствует режиму RF-001")
raw_bits = (_raw_ber_after_decode(iq, reception, int(truth["seed"]), decoder_backend) raw_bits = (
if truth is not None else None) _raw_ber_after_decode(iq, reception, int(truth["seed"]), decoder_backend)
return summarize_result( if truth is not None and iq is not None
else None
)
report = summarize_result(
reception, reception,
iq_sample_count=int(iq.size), iq_sample_count=iq_sample_count,
iq_sha256=sha256_file(iq_path), iq_sha256=sha256_file(iq_path),
iq_format=iq_format, iq_format=iq_format,
clipping_codes=clipping_codes, clipping_codes=clipping_codes,
truth=truth, truth=truth,
raw_bits=raw_bits, raw_bits=raw_bits,
) )
report["peak_analysis_memory_bytes"] = memory.peak_bytes
report["peak_additional_ram_bytes"] = memory.additional_peak_bytes
if streaming is not None:
report.update({
"streaming_decoder": True,
"window_source": streaming.window_source,
"analysis_windows": [
{"start_sample": window.start, "stop_sample": window.stop}
for window in streaming.windows
],
"overlap_samples": streaming.overlap_samples,
"maximum_loaded_complex_samples": streaming.maximum_loaded_samples,
"maximum_loaded_iq_bytes": streaming.maximum_loaded_samples * 8,
"raw_ber_scope": "NOT_COMPUTED_FOR_STREAMING_WINDOWS",
})
else:
report["streaming_decoder"] = False
return report
def main() -> None: def main() -> None:
@@ -282,6 +331,10 @@ def main() -> None:
parser.add_argument("--truth-metadata", type=Path) parser.add_argument("--truth-metadata", type=Path)
parser.add_argument("--report", required=True, type=Path) parser.add_argument("--report", required=True, type=Path)
parser.add_argument("--numba-packages", type=Path) parser.add_argument("--numba-packages", type=Path)
parser.add_argument("--capture-started-at")
parser.add_argument("--tx-started-at")
parser.add_argument("--tx-finished-at")
parser.add_argument("--chunk-samples", type=int, default=DEFAULT_CHUNK_SAMPLES)
arguments = parser.parse_args() arguments = parser.parse_args()
if arguments.report.exists(): if arguments.report.exists():
parser.error("отчёт уже существует; перезапись запрещена") parser.error("отчёт уже существует; перезапись запрещена")
@@ -290,7 +343,11 @@ def main() -> None:
parser.error("каталог пакетов Numba отсутствует") parser.error("каталог пакетов Numba отсутствует")
sys.path.insert(0, str(arguments.numba_packages.resolve())) sys.path.insert(0, str(arguments.numba_packages.resolve()))
result = decode_file(arguments.iq_file, iq_format=arguments.iq_format, result = decode_file(arguments.iq_file, iq_format=arguments.iq_format,
truth_metadata_path=arguments.truth_metadata) truth_metadata_path=arguments.truth_metadata,
capture_started_at=arguments.capture_started_at,
tx_started_at=arguments.tx_started_at,
tx_finished_at=arguments.tx_finished_at,
chunk_samples=arguments.chunk_samples)
arguments.report.parent.mkdir(parents=True, exist_ok=True) arguments.report.parent.mkdir(parents=True, exist_ok=True)
arguments.report.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", arguments.report.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8") encoding="utf-8")

363
tools/rf001a_streaming.py Normal file
View File

@@ -0,0 +1,363 @@
"""Потоковое чтение больших cs8-захватов и оконный приём RF-001A."""
from __future__ import annotations
from dataclasses import dataclass, replace
from datetime import datetime, timezone
import ctypes
from pathlib import Path
import threading
from typing import Iterable
import numpy as np
from protocol.video003_modem import SAMPLE_RATE_HZ, build_frame, samples_per_symbol
from protocol.video005_iq_waveform import NOMINAL_CARRIER_HZ, SYMBOL_RATE, shape_packet
from protocol.video005_receiver import IqStreamResult, ReceivedIqPacket, receive_iq_stream
from tools.qpsk_rf_common import PAYLOAD_BYTES, SEED, payload_for_sequence
DEFAULT_CHUNK_SAMPLES = 4_000_000
LOCATOR_BLOCK_SAMPLES = 400_000
TIMESTAMP_MARGIN_SECONDS = 2.0
@dataclass(frozen=True)
class SampleWindow:
start: int
stop: int
@property
def sample_count(self) -> int:
return self.stop - self.start
@dataclass(frozen=True)
class StreamingReception:
reception: IqStreamResult
windows: tuple[SampleWindow, ...]
window_source: str
overlap_samples: int
maximum_loaded_samples: int
clipping_codes: int
class MemoryPeakSampler:
"""Измеряет пиковый working set процесса во время анализа."""
def __init__(self, poll_seconds: float = 0.02) -> None:
self.poll_seconds = poll_seconds
self.baseline_bytes = current_process_rss_bytes()
self.peak_bytes = self.baseline_bytes
self._stop = threading.Event()
self._thread = threading.Thread(target=self._sample, daemon=True)
def _sample(self) -> None:
while not self._stop.wait(self.poll_seconds):
self.peak_bytes = max(self.peak_bytes, current_process_rss_bytes())
def __enter__(self) -> MemoryPeakSampler:
self._thread.start()
return self
def __exit__(self, *_exc: object) -> None:
self.peak_bytes = max(self.peak_bytes, current_process_rss_bytes())
self._stop.set()
self._thread.join(timeout=1.0)
@property
def additional_peak_bytes(self) -> int:
return max(0, self.peak_bytes - self.baseline_bytes)
def current_process_rss_bytes() -> int:
"""Возвращает текущий working set процесса без сторонних библиотек."""
if not hasattr(ctypes, "windll"):
return 0
class ProcessMemoryCounters(ctypes.Structure):
_fields_ = [
("cb", ctypes.c_ulong),
("page_fault_count", ctypes.c_ulong),
("peak_working_set_size", ctypes.c_size_t),
("working_set_size", ctypes.c_size_t),
("quota_peak_paged_pool_usage", ctypes.c_size_t),
("quota_paged_pool_usage", ctypes.c_size_t),
("quota_peak_non_paged_pool_usage", ctypes.c_size_t),
("quota_non_paged_pool_usage", ctypes.c_size_t),
("pagefile_usage", ctypes.c_size_t),
("peak_pagefile_usage", ctypes.c_size_t),
]
counters = ProcessMemoryCounters()
counters.cb = ctypes.sizeof(counters)
get_current_process = ctypes.windll.kernel32.GetCurrentProcess # type: ignore[attr-defined]
get_current_process.restype = ctypes.c_void_p
get_process_memory_info = ctypes.windll.psapi.GetProcessMemoryInfo # type: ignore[attr-defined]
get_process_memory_info.argtypes = [
ctypes.c_void_p,
ctypes.POINTER(ProcessMemoryCounters),
ctypes.c_ulong,
]
get_process_memory_info.restype = ctypes.c_int
succeeded = get_process_memory_info(
get_current_process(),
ctypes.byref(counters),
counters.cb,
)
return int(counters.working_set_size) if succeeded else 0
def cs8_sample_count(path: Path) -> int:
size = path.stat().st_size
if size == 0 or size % 2:
raise ValueError("cs8 должен содержать целое ненулевое число пар I/Q")
return size // 2
def read_cs8_window(raw: np.memmap, start: int, stop: int) -> np.ndarray:
"""Преобразует в complex64 только заданное окно cs8."""
total_samples = raw.size // 2
if start < 0 or stop <= start or stop > total_samples:
raise ValueError("границы окна cs8 выходят за пределы файла")
values = np.asarray(raw[start * 2:stop * 2], dtype=np.int8)
real = values[0::2].astype(np.float32)
imaginary = values[1::2].astype(np.float32)
return ((real + 1j * imaginary) / 128).astype(np.complex64)
def rf001_packet_waveform_samples() -> int:
"""Вычисляет длину волны из настоящего 256-байтного пакета RF-001."""
if PAYLOAD_BYTES != 256:
raise AssertionError("канонический RF-001 payload неожиданно изменился")
payload = payload_for_sequence(SEED, 0)
frame = build_frame(payload, "QPSK", "2/3", sequence_number=0)
return int(shape_packet(frame).size)
def required_overlap_samples() -> int:
"""Перекрывает целую максимальную волну одного пакета RF-001."""
return rf001_packet_waveform_samples()
def _parse_utc(value: str) -> datetime:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
if parsed.tzinfo is None:
raise ValueError("timestamp должен содержать часовой пояс")
return parsed.astimezone(timezone.utc)
def timestamp_sample_window(
*,
capture_started_at: str,
tx_started_at: str,
tx_finished_at: str,
total_samples: int,
sample_rate: int = SAMPLE_RATE_HZ,
margin_seconds: float = TIMESTAMP_MARGIN_SECONDS,
) -> SampleWindow:
"""Переводит UTC-границы TX в безопасное окно отсчётов RX."""
if total_samples <= 0 or sample_rate <= 0 or margin_seconds < 0:
raise ValueError("параметры timestamp-окна должны быть положительными")
capture = _parse_utc(capture_started_at)
started = _parse_utc(tx_started_at)
finished = _parse_utc(tx_finished_at)
if finished < started:
raise ValueError("TX_FINISHED_AT не может предшествовать TX_STARTED_AT")
margin_samples = round(margin_seconds * sample_rate)
start_offset = round((started - capture).total_seconds() * sample_rate)
stop_offset = round((finished - capture).total_seconds() * sample_rate)
start = max(0, start_offset - margin_samples)
stop = min(total_samples, stop_offset + margin_samples)
if stop <= start:
raise ValueError("TX timestamps не пересекаются с RX capture")
return SampleWindow(start, stop)
def _merge_windows(windows: Iterable[SampleWindow]) -> tuple[SampleWindow, ...]:
ordered = sorted(windows, key=lambda item: item.start)
merged: list[SampleWindow] = []
for window in ordered:
if not merged or window.start > merged[-1].stop:
merged.append(window)
else:
merged[-1] = SampleWindow(
merged[-1].start,
max(merged[-1].stop, window.stop),
)
return tuple(merged)
def locate_power_windows(
path: Path,
*,
block_samples: int = LOCATOR_BLOCK_SAMPLES,
padding_seconds: float = 1.5,
) -> tuple[SampleWindow, ...]:
"""Ищет повышенную мощность по всему cs8-файлу ограниченными блоками."""
total_samples = cs8_sample_count(path)
if block_samples <= 0 or padding_seconds < 0:
raise ValueError("параметры locator должны быть неотрицательными")
powers: list[float] = []
with path.open("rb") as stream:
while raw_bytes := stream.read(block_samples * 2):
values = np.frombuffer(raw_bytes, dtype=np.int8).astype(np.float32)
powers.append(float(np.mean(values * values)))
if not powers:
return ()
power_array = np.asarray(powers, dtype=np.float64)
median = float(np.median(power_array))
mad = float(np.median(np.abs(power_array - median)))
threshold = median + max(6.0 * mad, median * 0.15, 1.0)
active = np.flatnonzero(power_array > threshold)
if active.size == 0:
return ()
padding = round(padding_seconds * SAMPLE_RATE_HZ)
candidates = (
SampleWindow(
max(0, int(index) * block_samples - padding),
min(total_samples, (int(index) + 1) * block_samples + padding),
)
for index in active
)
return _merge_windows(candidates)
def count_cs8_clipping(path: Path, *, block_samples: int = DEFAULT_CHUNK_SAMPLES) -> int:
"""Считает rail-коды во всём файле, сохраняя ограниченный объём памяти."""
cs8_sample_count(path)
clipping = 0
with path.open("rb") as stream:
while raw_bytes := stream.read(block_samples * 2):
values = np.frombuffer(raw_bytes, dtype=np.int8)
clipping += int(np.count_nonzero((values == -128) | (values == 127)))
return clipping
def _is_overlap_duplicate(
packets: list[ReceivedIqPacket],
candidate: ReceivedIqPacket,
) -> bool:
tolerance = samples_per_symbol(SYMBOL_RATE)
return any(
packet.packet == candidate.packet
and abs(packet.start_sample - candidate.start_sample) <= tolerance
for packet in packets
)
def receive_cs8_windows(
path: Path,
windows: Iterable[SampleWindow],
*,
decoder_backend: str = "numba",
chunk_samples: int = DEFAULT_CHUNK_SAMPLES,
overlap_samples: int | None = None,
) -> StreamingReception:
"""Принимает заданные окна частями с перекрытием и удаляет повторы."""
selected = _merge_windows(windows)
overlap = required_overlap_samples() if overlap_samples is None else overlap_samples
if chunk_samples <= overlap or overlap < required_overlap_samples():
raise ValueError("chunk должен быть длиннее полного overlap RF-001")
raw = np.memmap(path, dtype=np.int8, mode="r")
packets: list[ReceivedIqPacket] = []
candidate_count = 0
rejected_candidates = 0
stage_time: dict[str, float] = {}
maximum_loaded = 0
for selected_window in selected:
for primary_start in range(selected_window.start, selected_window.stop, chunk_samples):
primary_stop = min(selected_window.stop, primary_start + chunk_samples)
read_start = max(selected_window.start, primary_start - overlap)
read_stop = min(selected_window.stop, primary_stop + overlap)
iq = read_cs8_window(raw, read_start, read_stop)
maximum_loaded = max(maximum_loaded, int(iq.size))
received = receive_iq_stream(
iq,
carrier_hz=NOMINAL_CARRIER_HZ,
decoder_backend=decoder_backend,
)
candidate_count += received.candidate_count
rejected_candidates += received.rejected_candidates
for name, duration in received.stage_time_s.items():
stage_time[name] = stage_time.get(name, 0.0) + duration
for packet in received.packets:
global_packet = replace(
packet,
start_sample=packet.start_sample + read_start,
)
if not primary_start <= global_packet.start_sample < primary_stop:
continue
if not _is_overlap_duplicate(packets, global_packet):
packets.append(global_packet)
del iq
packets.sort(key=lambda packet: packet.start_sample)
return StreamingReception(
IqStreamResult(
packets=tuple(packets),
candidate_count=candidate_count,
rejected_candidates=rejected_candidates,
stage_time_s=stage_time,
),
selected,
"EXPLICIT",
overlap,
maximum_loaded,
count_cs8_clipping(path),
)
def decode_cs8_streaming(
path: Path,
*,
capture_started_at: str | None = None,
tx_started_at: str | None = None,
tx_finished_at: str | None = None,
decoder_backend: str = "numba",
chunk_samples: int = DEFAULT_CHUNK_SAMPLES,
margin_seconds: float = TIMESTAMP_MARGIN_SECONDS,
) -> StreamingReception:
"""Выбирает timestamp-окно либо потоковый power locator и запускает приём."""
total_samples = cs8_sample_count(path)
timestamp_values = (capture_started_at, tx_started_at, tx_finished_at)
if all(value is not None for value in timestamp_values):
windows = (
timestamp_sample_window(
capture_started_at=str(capture_started_at),
tx_started_at=str(tx_started_at),
tx_finished_at=str(tx_finished_at),
total_samples=total_samples,
margin_seconds=margin_seconds,
),
)
source = "TX_TIMESTAMPS"
elif any(value is not None for value in timestamp_values):
raise ValueError("для timestamp-окна нужны все три времени RX/TX")
else:
windows = locate_power_windows(path)
source = "STREAMING_POWER_LOCATOR"
if not windows:
return StreamingReception(
IqStreamResult((), 0, 0, {}),
(),
source,
required_overlap_samples(),
0,
count_cs8_clipping(path),
)
reception = receive_cs8_windows(
path,
windows,
decoder_backend=decoder_backend,
chunk_samples=chunk_samples,
)
return replace(reception, window_source=source)

134
tools/rf_handoff_channel.py
View File

@@ -14,8 +14,9 @@ from typing import Any, Mapping
SCHEMA_VERSION = 1 SCHEMA_VERSION = 1
EXPECTED_RX_TASK_ID = "RF001A_OTA_CAPTURE_003" EXPECTED_RX_TASK_ID = "RF001A_OTA_CAPTURE_004"
EXPECTED_TX_TASK_ID = "RF001A_OTA_TX_003" EXPECTED_TX_TASK_ID = "RF001A_OTA_TX_004"
EXPECTED_CAPTURE_DURATION_SECONDS = 60.0
CHANNEL_TEST_TASK_ID = "RF_HANDOFF_CHANNEL_TEST_001" CHANNEL_TEST_TASK_ID = "RF_HANDOFF_CHANNEL_TEST_001"
READY_STATE = "RX_READY_FOR_TX" READY_STATE = "RX_READY_FOR_TX"
COMPLETE_STATE = "CAPTURE_COMPLETE" COMPLETE_STATE = "CAPTURE_COMPLETE"
@@ -25,6 +26,21 @@ RESULT_PATHS = frozenset((RESULT_JSON_PATH, RESULT_MARKDOWN_PATH))
MAX_RESULT_BYTES = 64 * 1024 MAX_RESULT_BYTES = 64 * 1024
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$") _SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
_HEAD_RE = re.compile(r"^[0-9a-f]{40}$") _HEAD_RE = re.compile(r"^[0-9a-f]{40}$")
_TRANSIENT_PUSH_ERRORS = (
"authentication failed",
"failed to authenticate",
"could not resolve host",
"connection reset",
"connection timed out",
"failed to connect",
"remote end hung up",
"rpc failed",
"schannel",
"ssl",
"tls",
"the requested url returned error: 5",
"unable to access",
)
class ResultChannelError(RuntimeError): class ResultChannelError(RuntimeError):
@@ -319,7 +335,11 @@ def evaluate_ready(
expected_end = _parse_utc(result.expected_capture_end, "expected_capture_end") expected_end = _parse_utc(result.expected_capture_end, "expected_capture_end")
published = _parse_utc(result.published_at, "published_at") published = _parse_utc(result.published_at, "published_at")
planned_duration = (expected_end - started).total_seconds() planned_duration = (expected_end - started).total_seconds()
if not 119.0 <= planned_duration <= 121.0: if not (
EXPECTED_CAPTURE_DURATION_SECONDS - 1.0
<= planned_duration
<= EXPECTED_CAPTURE_DURATION_SECONDS + 1.0
):
return ReadyEvaluation(False, "WRONG_CAPTURE_WINDOW") return ReadyEvaluation(False, "WRONG_CAPTURE_WINDOW")
age = (now_utc - published).total_seconds() age = (now_utc - published).total_seconds()
if age < -5.0: if age < -5.0:
@@ -386,6 +406,32 @@ class GitResultChannel:
def branch_exists(self) -> bool: def branch_exists(self) -> bool:
return self.remote_head() is not None return self.remote_head() is not None
def verify_write_access(self) -> str:
"""Проверить право записи dry-run push без коммита и изменения ветки."""
remote_head = self.fetch()
dry_run = self._run(
[
"push",
"--dry-run",
"--porcelain",
self.remote,
f"{remote_head}:refs/heads/{self.branch}",
],
check=False,
)
if dry_run.returncode != 0:
raise ResultChannelError(
"authenticated result-branch push preflight failed: "
f"{dry_run.stderr.strip()}"
)
confirmed_head = self.remote_head()
if confirmed_head != remote_head:
raise ResultChannelError(
"result branch changed during write preflight"
)
return remote_head
def _tracking_reference(self) -> str: def _tracking_reference(self) -> str:
return f"refs/remotes/{self.remote}/{self.branch}" return f"refs/remotes/{self.remote}/{self.branch}"
@@ -553,6 +599,66 @@ class GitResultChannel:
return commit return commit
return None return None
@staticmethod
def _is_transient_push_error(stderr: str) -> bool:
normalized = stderr.casefold()
return any(pattern in normalized for pattern in _TRANSIENT_PUSH_ERRORS)
def _push_exact_commit(
self,
checkout: Path,
commit: str,
remote_before: str,
) -> str:
"""Отправить неизменный коммит с одной безопасной повторной попыткой."""
push_arguments = [
"push",
"--quiet",
self.remote,
f"{commit}:refs/heads/{self.branch}",
]
first = self._run(push_arguments, cwd=checkout, check=False)
if first.returncode == 0:
remote_after = self.remote_head()
if remote_after != commit:
raise ResultChannelError(
"remote result HEAD does not confirm the published commit"
)
return commit
observed_after_failure = self.remote_head()
if observed_after_failure == commit:
return commit
if observed_after_failure != remote_before:
raise ResultChannelError(
"result push failed and remote branch changed to a conflicting "
f"commit: {remote_before} -> {observed_after_failure}"
)
if not self._is_transient_push_error(first.stderr):
raise ResultChannelError(f"result push failed: {first.stderr.strip()}")
fetched_head = self.fetch()
if fetched_head != remote_before:
raise ResultChannelError(
"remote result branch changed before the single retry"
)
second = self._run(push_arguments, cwd=checkout, check=False)
if second.returncode != 0:
observed_after_retry = self.remote_head()
if observed_after_retry == commit:
return commit
raise ResultChannelError(
"result push failed after one retry of the same commit: "
f"{second.stderr.strip()}"
)
remote_after = self.remote_head()
if remote_after != commit:
raise ResultChannelError(
"remote result HEAD does not confirm the retried commit"
)
return commit
def publish(self, result: RxHandoffResult) -> PublishOutcome: def publish(self, result: RxHandoffResult) -> PublishOutcome:
validate_result(result) validate_result(result)
remote_before = self.fetch() remote_before = self.fetch()
@@ -610,25 +716,11 @@ class GitResultChannel:
cwd=checkout, cwd=checkout,
) )
commit = self._run(["rev-parse", "HEAD"], cwd=checkout).stdout.strip() commit = self._run(["rev-parse", "HEAD"], cwd=checkout).stdout.strip()
push = self._run( remote_after = self._push_exact_commit(
[ checkout,
"push", commit,
"--quiet", remote_before,
self.remote,
f"HEAD:refs/heads/{self.branch}",
],
cwd=checkout,
check=False,
) )
if push.returncode != 0:
raise ResultChannelError(
f"result push failed: {push.stderr.strip()}"
)
remote_after = self.remote_head()
if remote_after is None or remote_after != commit:
raise ResultChannelError(
"remote result HEAD does not confirm the published commit"
)
return PublishOutcome("PUBLISHED", commit, remote_after) return PublishOutcome("PUBLISHED", commit, remote_after)
finally: finally:
if worktree_added: if worktree_added:

55
tools/rx_agent_watcher.py
View File

@@ -23,6 +23,8 @@ if str(ROOT) not in sys.path:
from tools.rf_handoff_channel import ( # noqa: E402 from tools.rf_handoff_channel import ( # noqa: E402
EXPECTED_RX_TASK_ID, EXPECTED_RX_TASK_ID,
GitResultChannel, GitResultChannel,
READY_STATE,
RxHandoffResult,
load_result, load_result,
) )
from tools.tx_handoff_channel import ( # noqa: E402 from tools.tx_handoff_channel import ( # noqa: E402
@@ -571,6 +573,7 @@ def publish_result(arguments: argparse.Namespace) -> int:
task = parse_task((repository / DEFAULT_TASK_PATH).read_text(encoding="utf-8-sig")) task = parse_task((repository / DEFAULT_TASK_PATH).read_text(encoding="utf-8-sig"))
state = load_state(arguments.state_file.resolve()) state = load_state(arguments.state_file.resolve())
result = load_result(arguments.publish_result.resolve()) result = load_result(arguments.publish_result.resolve())
require_live_ready_window(result)
if task.task_id != result.task_id: if task.task_id != result.task_id:
raise ValueError("result TASK_ID disagrees with current RX task") raise ValueError("result TASK_ID disagrees with current RX task")
if task.allow_push != "RX_RESULT_ONLY" or task.rf_tx: if task.allow_push != "RX_RESULT_ONLY" or task.rf_tx:
@@ -613,10 +616,55 @@ def publish_result(arguments: argparse.Namespace) -> int:
return 0 return 0
def require_live_ready_window(
result: RxHandoffResult,
*,
now: datetime | None = None,
) -> None:
"""Запретить READY после окончания конечного окна захвата."""
if result.state != READY_STATE:
return
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
expected_end = datetime.fromisoformat(
result.expected_capture_end.replace("Z", "+00:00")
).astimezone(timezone.utc)
if current >= expected_end:
raise ValueError("READY capture window has already ended")
def verify_result_channel_write_access(
repository: Path,
*,
remote: str = "origin",
result_branch: str = "rx-node",
output_function: Callable[[str], None] = print,
) -> str:
"""Проверить право публикации результата RX без изменения ветки."""
channel = GitResultChannel(
repository,
remote=remote,
branch=result_branch,
)
remote_head = channel.verify_write_access()
output_function("RX-NODE WRITE PREFLIGHT = PASS")
output_function(f"RX-NODE REMOTE HEAD = {remote_head}")
output_function("RF TX = 0")
return remote_head
def run_watcher(arguments: argparse.Namespace) -> int: def run_watcher(arguments: argparse.Namespace) -> int:
state_path = arguments.state_file.resolve() state_path = arguments.state_file.resolve()
if arguments.publish_result is not None: if arguments.publish_result is not None:
return publish_result(arguments) return publish_result(arguments)
if arguments.check_result_write:
verify_result_channel_write_access(
arguments.repository.resolve(),
remote=arguments.remote,
result_branch=arguments.result_branch,
)
return 0
if arguments.request_stop: if arguments.request_stop:
stopped = request_stop(state_path, arguments.stop_timeout) stopped = request_stop(state_path, arguments.stop_timeout)
print("WATCHER STOPPED" if stopped else "WATCHER STOP TIMEOUT") print("WATCHER STOPPED" if stopped else "WATCHER STOP TIMEOUT")
@@ -670,6 +718,12 @@ def run_watcher(arguments: argparse.Namespace) -> int:
task_text, source_head = source.read() task_text, source_head = source.read()
task = parse_task(task_text) task = parse_task(task_text)
if task.task_id == EXPECTED_RX_TASK_ID and not arguments.dry_run: if task.task_id == EXPECTED_RX_TASK_ID and not arguments.dry_run:
if is_new_task(task, state):
verify_result_channel_write_access(
arguments.repository.resolve(),
remote=arguments.remote,
result_branch=arguments.result_branch,
)
confirm_main_channel_test( confirm_main_channel_test(
arguments.repository.resolve(), arguments.repository.resolve(),
state, state,
@@ -726,6 +780,7 @@ def build_parser() -> argparse.ArgumentParser:
parser.add_argument("--stop-timeout", type=float, default=35.0) parser.add_argument("--stop-timeout", type=float, default=35.0)
parser.add_argument("--mark-completed") parser.add_argument("--mark-completed")
parser.add_argument("--publish-result", type=Path) parser.add_argument("--publish-result", type=Path)
parser.add_argument("--check-result-write", action="store_true")
parser.add_argument("--result-branch", default="rx-node") parser.add_argument("--result-branch", default="rx-node")
parser.add_argument("--acknowledgement-branch", default="tx-node") parser.add_argument("--acknowledgement-branch", default="tx-node")
return parser return parser

48
tools/tx_agent_watcher.py
View File

@@ -37,9 +37,13 @@ from tools.tx_handoff_channel import ( # noqa: E402
CHANNEL_COMPLETE_OBSERVED, CHANNEL_COMPLETE_OBSERVED,
CHANNEL_READY_OBSERVED, CHANNEL_READY_OBSERVED,
RX_READY_OBSERVED, RX_READY_OBSERVED,
TX_AUTHORIZATION_CREATED, TX_ACK_SCHEMA_VERSION,
TX_AUTHORIZED,
TX_COMPLETE,
TX_STARTED,
GitAcknowledgementChannel, GitAcknowledgementChannel,
TxHandoffAcknowledgement, TxHandoffAcknowledgement,
load_acknowledgement,
) )
@@ -445,12 +449,12 @@ def acknowledgement_for_result(
elif status == "RED_APPROVAL_BLOCKED": elif status == "RED_APPROVAL_BLOCKED":
acknowledgement_state = RX_READY_OBSERVED acknowledgement_state = RX_READY_OBSERVED
elif status == "TX_AUTHORIZATION_PREPARED": elif status == "TX_AUTHORIZATION_PREPARED":
acknowledgement_state = TX_AUTHORIZATION_CREATED acknowledgement_state = TX_AUTHORIZED
rf_tx_authorized = True rf_tx_authorized = True
if acknowledgement_state is None: if acknowledgement_state is None:
return None return None
return TxHandoffAcknowledgement( return TxHandoffAcknowledgement(
schema_version=1, schema_version=TX_ACK_SCHEMA_VERSION,
task_id=result.task_id, task_id=result.task_id,
state=acknowledgement_state, state=acknowledgement_state,
source_rx_commit=result_head, source_rx_commit=result_head,
@@ -461,6 +465,7 @@ def acknowledgement_for_result(
rf_tx_authorized=rf_tx_authorized, rf_tx_authorized=rf_tx_authorized,
rf_tx_started=False, rf_tx_started=False,
rf_tx=0, rf_tx=0,
tx_task_id=(EXPECTED_TX_TASK_ID if rf_tx_authorized else None),
) )
@@ -536,6 +541,42 @@ def process_history_cycle(
def run_watcher(arguments: argparse.Namespace) -> int: def run_watcher(arguments: argparse.Namespace) -> int:
state_path = arguments.state_file.resolve() state_path = arguments.state_file.resolve()
if arguments.publish_tx_event is not None:
if arguments.dry_run:
raise ValueError("--publish-tx-event cannot be combined with --dry-run")
repository = arguments.repository.resolve()
event = load_acknowledgement(arguments.publish_tx_event.resolve())
if event.state not in {TX_STARTED, TX_COMPLETE}:
raise ValueError("published execution event has the wrong state")
branch = _git_output(repository, ["branch", "--show-current"])
if branch != arguments.source_branch:
raise ValueError("TX event publisher is on the wrong source branch")
if _git_output(repository, ["status", "--porcelain", "--untracked-files=no"]):
raise ValueError("tracked worktree must be clean before TX event publication")
_git_output(
repository,
["fetch", "--quiet", arguments.remote, arguments.source_branch],
)
head = _git_output(repository, ["rev-parse", "HEAD"])
remote_head = _git_output(
repository,
["rev-parse", f"{arguments.remote}/{arguments.source_branch}"],
)
if head != remote_head:
raise ValueError("current HEAD disagrees with remote source branch")
if event.source_head != head:
raise ValueError("TX event SOURCE_HEAD disagrees with current HEAD")
publisher = GitAcknowledgementChannel(
repository,
remote=arguments.remote,
branch=arguments.acknowledgement_branch,
)
outcome = publisher.publish_execution_event(event)
print(f"TX EVENT PUBLISH = {outcome.status}")
print(f"TX EVENT STATE = {event.state}")
print(f"TX EVENT HEAD = {outcome.commit}")
print(f"TX EVENT REMOTE HEAD = {outcome.remote_head}")
return 0
if arguments.request_stop: if arguments.request_stop:
stopped = request_stop(state_path, arguments.stop_timeout) stopped = request_stop(state_path, arguments.stop_timeout)
print("TX WATCHER STOPPED" if stopped else "TX WATCHER STOP TIMEOUT") print("TX WATCHER STOPPED" if stopped else "TX WATCHER STOP TIMEOUT")
@@ -664,6 +705,7 @@ def build_parser() -> argparse.ArgumentParser:
parser.add_argument("--once", action="store_true") parser.add_argument("--once", action="store_true")
parser.add_argument("--dry-run", action="store_true") parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--result-file", type=Path) parser.add_argument("--result-file", type=Path)
parser.add_argument("--publish-tx-event", type=Path)
parser.add_argument("--approval", choices=("authorize", "deny")) parser.add_argument("--approval", choices=("authorize", "deny"))
parser.add_argument("--request-stop", action="store_true") parser.add_argument("--request-stop", action="store_true")
parser.add_argument("--stop-timeout", type=float, default=15.0) parser.add_argument("--stop-timeout", type=float, default=15.0)

286
tools/tx_handoff_channel.py
View File

@@ -2,7 +2,7 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import asdict, dataclass, fields from dataclasses import asdict, dataclass, fields, replace
from datetime import datetime, timezone from datetime import datetime, timezone
import hashlib import hashlib
import json import json
@@ -22,7 +22,8 @@ from tools.rf_handoff_channel import (
) )
TX_ACK_SCHEMA_VERSION = 1 TX_ACK_SCHEMA_VERSION = 2
LEGACY_TX_ACK_SCHEMA_VERSION = 1
TX_RESULT_JSON_PATH = "docs/handoff/results/TX_RESULT.json" TX_RESULT_JSON_PATH = "docs/handoff/results/TX_RESULT.json"
TX_RESULT_MARKDOWN_PATH = "docs/handoff/results/TX_RESULT.md" TX_RESULT_MARKDOWN_PATH = "docs/handoff/results/TX_RESULT.md"
TX_RESULT_PATHS = frozenset((TX_RESULT_JSON_PATH, TX_RESULT_MARKDOWN_PATH)) TX_RESULT_PATHS = frozenset((TX_RESULT_JSON_PATH, TX_RESULT_MARKDOWN_PATH))
@@ -32,6 +33,7 @@ CHANNEL_READY_OBSERVED = "CHANNEL_READY_OBSERVED"
CHANNEL_COMPLETE_OBSERVED = "CHANNEL_COMPLETE_OBSERVED" CHANNEL_COMPLETE_OBSERVED = "CHANNEL_COMPLETE_OBSERVED"
RX_READY_OBSERVED = "RX_READY_OBSERVED" RX_READY_OBSERVED = "RX_READY_OBSERVED"
TX_AUTHORIZATION_CREATED = "TX_AUTHORIZATION_CREATED" TX_AUTHORIZATION_CREATED = "TX_AUTHORIZATION_CREATED"
TX_AUTHORIZED = "TX_AUTHORIZED"
TX_STARTED = "TX_STARTED" TX_STARTED = "TX_STARTED"
TX_COMPLETE = "TX_COMPLETE" TX_COMPLETE = "TX_COMPLETE"
TX_ACK_STATES = frozenset( TX_ACK_STATES = frozenset(
@@ -40,6 +42,7 @@ TX_ACK_STATES = frozenset(
CHANNEL_COMPLETE_OBSERVED, CHANNEL_COMPLETE_OBSERVED,
RX_READY_OBSERVED, RX_READY_OBSERVED,
TX_AUTHORIZATION_CREATED, TX_AUTHORIZATION_CREATED,
TX_AUTHORIZED,
TX_STARTED, TX_STARTED,
TX_COMPLETE, TX_COMPLETE,
) )
@@ -74,6 +77,12 @@ class TxHandoffAcknowledgement:
rf_tx_authorized: bool rf_tx_authorized: bool
rf_tx_started: bool rf_tx_started: bool
rf_tx: int rf_tx: int
tx_task_id: str | None = None
tx_started_at: str | None = None
tx_finished_at: str | None = None
tx_process_exit_code: int | None = None
tx_underruns: int | None = None
tx_errors: int | None = None
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -99,8 +108,13 @@ def validate_acknowledgement(
) -> TxHandoffAcknowledgement: ) -> TxHandoffAcknowledgement:
if type(acknowledgement.schema_version) is not int: if type(acknowledgement.schema_version) is not int:
raise ValueError("schema_version must contain an integer") raise ValueError("schema_version must contain an integer")
if acknowledgement.schema_version != TX_ACK_SCHEMA_VERSION: if acknowledgement.schema_version not in {
raise ValueError(f"schema_version must be {TX_ACK_SCHEMA_VERSION}") LEGACY_TX_ACK_SCHEMA_VERSION,
TX_ACK_SCHEMA_VERSION,
}:
raise ValueError(
"schema_version must be a supported TX acknowledgement version"
)
string_values = ( string_values = (
acknowledgement.task_id, acknowledgement.task_id,
acknowledgement.state, acknowledgement.state,
@@ -121,7 +135,7 @@ def validate_acknowledgement(
raise ValueError("source_rx_digest must be a lowercase SHA256") raise ValueError("source_rx_digest must be a lowercase SHA256")
if not _HEAD_RE.fullmatch(acknowledgement.source_head): if not _HEAD_RE.fullmatch(acknowledgement.source_head):
raise ValueError("source_head must be a lowercase 40-character hash") raise ValueError("source_head must be a lowercase 40-character hash")
_parse_utc(acknowledgement.observed_at) observed_at = _parse_utc(acknowledgement.observed_at)
if ( if (
type(acknowledgement.main_watcher_pid) is not int type(acknowledgement.main_watcher_pid) is not int
or acknowledgement.main_watcher_pid <= 0 or acknowledgement.main_watcher_pid <= 0
@@ -133,6 +147,58 @@ def validate_acknowledgement(
raise ValueError("rf_tx_started must contain a boolean") raise ValueError("rf_tx_started must contain a boolean")
if type(acknowledgement.rf_tx) is not int or acknowledgement.rf_tx not in {0, 1}: if type(acknowledgement.rf_tx) is not int or acknowledgement.rf_tx not in {0, 1}:
raise ValueError("rf_tx must be 0 or 1") raise ValueError("rf_tx must be 0 or 1")
optional_integers = (
acknowledgement.tx_process_exit_code,
acknowledgement.tx_underruns,
acknowledgement.tx_errors,
)
if any(value is not None and type(value) is not int for value in optional_integers):
raise ValueError("TX counters and exit code must contain integers or null")
if any(
value is not None and value < 0
for value in (acknowledgement.tx_underruns, acknowledgement.tx_errors)
):
raise ValueError("TX error counters cannot be negative")
optional_strings = (
acknowledgement.tx_task_id,
acknowledgement.tx_started_at,
acknowledgement.tx_finished_at,
)
if any(value is not None and type(value) is not str for value in optional_strings):
raise ValueError("optional TX fields must contain strings or null")
if (
acknowledgement.tx_task_id is not None
and not _TASK_ID_RE.fullmatch(acknowledgement.tx_task_id)
):
raise ValueError("tx_task_id is invalid")
tx_started_at = (
_parse_utc(acknowledgement.tx_started_at)
if acknowledgement.tx_started_at is not None
else None
)
tx_finished_at = (
_parse_utc(acknowledgement.tx_finished_at)
if acknowledgement.tx_finished_at is not None
else None
)
if (
tx_started_at is not None
and tx_finished_at is not None
and tx_finished_at < tx_started_at
):
raise ValueError("tx_finished_at cannot precede tx_started_at")
if acknowledgement.schema_version == LEGACY_TX_ACK_SCHEMA_VERSION and any(
value is not None
for value in (
acknowledgement.tx_task_id,
acknowledgement.tx_started_at,
acknowledgement.tx_finished_at,
acknowledgement.tx_process_exit_code,
acknowledgement.tx_underruns,
acknowledgement.tx_errors,
)
):
raise ValueError("legacy acknowledgement cannot contain TX event metadata")
if ( if (
acknowledgement.state acknowledgement.state
@@ -153,41 +219,146 @@ def validate_acknowledgement(
CHANNEL_COMPLETE_OBSERVED, CHANNEL_COMPLETE_OBSERVED,
} and acknowledgement.task_id != CHANNEL_TEST_TASK_ID: } and acknowledgement.task_id != CHANNEL_TEST_TASK_ID:
raise ValueError("channel acknowledgement has the wrong task_id") raise ValueError("channel acknowledgement has the wrong task_id")
if ( if acknowledgement.state == TX_AUTHORIZATION_CREATED and (
acknowledgement.state == TX_AUTHORIZATION_CREATED acknowledgement.schema_version != LEGACY_TX_ACK_SCHEMA_VERSION
and ( or not acknowledgement.rf_tx_authorized
not acknowledgement.rf_tx_authorized or acknowledgement.rf_tx_started
or acknowledgement.rf_tx_started or acknowledgement.rf_tx != 0
or acknowledgement.rf_tx != 0 ):
) raise ValueError("legacy TX authorization state is inconsistent")
if acknowledgement.state == TX_AUTHORIZED and (
acknowledgement.schema_version != TX_ACK_SCHEMA_VERSION
or not acknowledgement.rf_tx_authorized
or acknowledgement.rf_tx_started
or acknowledgement.rf_tx != 0
or acknowledgement.tx_task_id is None
or acknowledgement.tx_started_at is not None
or acknowledgement.tx_finished_at is not None
or acknowledgement.tx_process_exit_code is not None
): ):
raise ValueError("TX authorization state is inconsistent") raise ValueError("TX authorization state is inconsistent")
if acknowledgement.state in {TX_STARTED, TX_COMPLETE} and not ( if acknowledgement.state == TX_STARTED and not (
acknowledgement.rf_tx_authorized acknowledgement.rf_tx_authorized
and acknowledgement.rf_tx_started and acknowledgement.rf_tx_started
and acknowledgement.rf_tx == 1 and acknowledgement.rf_tx == 1
and acknowledgement.tx_task_id is not None
and acknowledgement.tx_started_at is not None
and acknowledgement.tx_finished_at is None
and acknowledgement.tx_process_exit_code is None
): ):
raise ValueError("TX execution state is inconsistent") raise ValueError("TX start state is inconsistent")
if acknowledgement.state == TX_STARTED and (
tx_started_at is None or observed_at < tx_started_at
):
raise ValueError("TX start observation cannot precede process start")
if acknowledgement.state == TX_COMPLETE and not (
acknowledgement.rf_tx_authorized
and acknowledgement.rf_tx_started
and acknowledgement.rf_tx == 1
and acknowledgement.tx_task_id is not None
and acknowledgement.tx_started_at is not None
and acknowledgement.tx_finished_at is not None
and acknowledgement.tx_process_exit_code is not None
and acknowledgement.tx_underruns is not None
and acknowledgement.tx_errors is not None
):
raise ValueError("TX completion state is inconsistent")
if acknowledgement.state == TX_COMPLETE and (
tx_finished_at is None or observed_at < tx_finished_at
):
raise ValueError("TX completion observation cannot precede process finish")
if acknowledgement.state in {
CHANNEL_READY_OBSERVED,
CHANNEL_COMPLETE_OBSERVED,
RX_READY_OBSERVED,
} and any(
value is not None
for value in (
acknowledgement.tx_task_id,
acknowledgement.tx_started_at,
acknowledgement.tx_finished_at,
acknowledgement.tx_process_exit_code,
acknowledgement.tx_underruns,
acknowledgement.tx_errors,
)
):
raise ValueError("observation state cannot contain TX event metadata")
return acknowledgement return acknowledgement
def build_tx_execution_event(
authorization: TxHandoffAcknowledgement,
*,
state: str,
observed_at: str,
tx_started_at: str,
tx_finished_at: str | None = None,
process_exit_code: int | None = None,
underruns: int | None = None,
errors: int | None = None,
) -> TxHandoffAcknowledgement:
"""Построить строгое событие TX_STARTED или TX_COMPLETE из разрешения."""
validate_acknowledgement(authorization)
if authorization.state != TX_AUTHORIZED:
raise ValueError("TX execution event requires a TX_AUTHORIZED source")
if state not in {TX_STARTED, TX_COMPLETE}:
raise ValueError("TX execution event state is not supported")
event = replace(
authorization,
state=state,
observed_at=observed_at,
rf_tx_started=True,
rf_tx=1,
tx_started_at=tx_started_at,
tx_finished_at=tx_finished_at,
tx_process_exit_code=process_exit_code,
tx_underruns=underruns,
tx_errors=errors,
)
return validate_acknowledgement(event)
def acknowledgement_from_mapping( def acknowledgement_from_mapping(
raw: Mapping[str, Any], raw: Mapping[str, Any],
) -> TxHandoffAcknowledgement: ) -> TxHandoffAcknowledgement:
expected_fields = {field.name for field in fields(TxHandoffAcknowledgement)} expected_fields = {field.name for field in fields(TxHandoffAcknowledgement)}
optional_event_fields = {
"tx_task_id",
"tx_started_at",
"tx_finished_at",
"tx_process_exit_code",
"tx_underruns",
"tx_errors",
}
actual_fields = set(raw) actual_fields = set(raw)
if actual_fields != expected_fields: schema_version = raw.get("schema_version")
legacy_fields = expected_fields - optional_event_fields
if schema_version == LEGACY_TX_ACK_SCHEMA_VERSION and actual_fields == legacy_fields:
normalized = {**raw, **dict.fromkeys(optional_event_fields)}
elif actual_fields == expected_fields:
normalized = dict(raw)
else:
missing = sorted(expected_fields - actual_fields) missing = sorted(expected_fields - actual_fields)
extra = sorted(actual_fields - expected_fields) extra = sorted(actual_fields - expected_fields)
raise ValueError( raise ValueError(
f"acknowledgement fields disagree: missing={missing}, extra={extra}" f"acknowledgement fields disagree: missing={missing}, extra={extra}"
) )
acknowledgement = TxHandoffAcknowledgement( acknowledgement = TxHandoffAcknowledgement(
**{name: raw[name] for name in expected_fields} **{name: normalized[name] for name in expected_fields}
) )
return validate_acknowledgement(acknowledgement) return validate_acknowledgement(acknowledgement)
def load_acknowledgement(path: Path) -> TxHandoffAcknowledgement:
"""Прочитать строгое малое TX-событие из локального JSON."""
raw = json.loads(path.read_text(encoding="utf-8-sig"))
if not isinstance(raw, dict):
raise ValueError("TX acknowledgement root must be a JSON object")
return acknowledgement_from_mapping(raw)
def canonical_acknowledgement_json( def canonical_acknowledgement_json(
acknowledgement: TxHandoffAcknowledgement, acknowledgement: TxHandoffAcknowledgement,
) -> str: ) -> str:
@@ -226,6 +397,12 @@ def render_acknowledgement_markdown(
f"{'YES' if acknowledgement.rf_tx_authorized else 'NO'}", f"{'YES' if acknowledgement.rf_tx_authorized else 'NO'}",
f"RF_TX_STARTED = {'YES' if acknowledgement.rf_tx_started else 'NO'}", f"RF_TX_STARTED = {'YES' if acknowledgement.rf_tx_started else 'NO'}",
f"RF_TX = {acknowledgement.rf_tx}", f"RF_TX = {acknowledgement.rf_tx}",
f"TX_TASK_ID = {acknowledgement.tx_task_id}",
f"TX_STARTED_AT = {acknowledgement.tx_started_at}",
f"TX_FINISHED_AT = {acknowledgement.tx_finished_at}",
f"TX_PROCESS_EXIT_CODE = {acknowledgement.tx_process_exit_code}",
f"TX_UNDERRUNS = {acknowledgement.tx_underruns}",
f"TX_ERRORS = {acknowledgement.tx_errors}",
"", "",
"Raw IQ, video, waveforms, source code, local state and large logs " "Raw IQ, video, waveforms, source code, local state and large logs "
"are not stored in this branch.", "are not stored in this branch.",
@@ -368,10 +545,59 @@ class GitAcknowledgementChannel(GitResultChannel):
== acknowledgement.rf_tx_authorized == acknowledgement.rf_tx_authorized
and observed.rf_tx_started == acknowledgement.rf_tx_started and observed.rf_tx_started == acknowledgement.rf_tx_started
and observed.rf_tx == acknowledgement.rf_tx and observed.rf_tx == acknowledgement.rf_tx
and observed.tx_task_id == acknowledgement.tx_task_id
and observed.tx_started_at == acknowledgement.tx_started_at
and observed.tx_finished_at == acknowledgement.tx_finished_at
and observed.tx_process_exit_code
== acknowledgement.tx_process_exit_code
and observed.tx_underruns == acknowledgement.tx_underruns
and observed.tx_errors == acknowledgement.tx_errors
): ):
return commit return commit
return None return None
@staticmethod
def _same_execution_binding(
left: TxHandoffAcknowledgement,
right: TxHandoffAcknowledgement,
) -> bool:
return (
left.task_id == right.task_id
and left.source_rx_commit == right.source_rx_commit
and left.source_rx_digest == right.source_rx_digest
and left.source_head == right.source_head
and left.tx_task_id == right.tx_task_id
)
def publish_execution_event(
self,
event: TxHandoffAcknowledgement,
) -> PublishOutcome:
"""Опубликовать TX-событие только после связанного TX_AUTHORIZED."""
validate_acknowledgement(event)
if event.state not in {TX_STARTED, TX_COMPLETE}:
raise ResultChannelError("execution event must be TX_STARTED or TX_COMPLETE")
commits, _remote_head = self.commits_since(None)
authorization_found = False
start_found = event.state == TX_STARTED
for commit in commits:
observed, _digest, _head = self.read_commit(commit)
if not self._same_execution_binding(observed, event):
continue
if observed.state == TX_AUTHORIZED:
authorization_found = True
if (
observed.state == TX_STARTED
and observed.tx_started_at == event.tx_started_at
):
start_found = True
if not authorization_found:
raise ResultChannelError("matching TX_AUTHORIZED event is absent")
if not start_found:
raise ResultChannelError("matching TX_STARTED event is absent")
return self.publish(event)
def publish( def publish(
self, self,
acknowledgement: TxHandoffAcknowledgement, acknowledgement: TxHandoffAcknowledgement,
@@ -471,25 +697,11 @@ class GitAcknowledgementChannel(GitResultChannel):
raise ResultChannelError( raise ResultChannelError(
"committed tx-node tree violates the path allowlist" "committed tx-node tree violates the path allowlist"
) )
push = self._run( self._push_exact_commit(
[ checkout,
"push", commit,
"--quiet", remote_before,
self.remote,
f"HEAD:refs/heads/{self.branch}",
],
cwd=checkout,
check=False,
) )
if push.returncode != 0:
raise ResultChannelError(
f"acknowledgement push failed: {push.stderr.strip()}"
)
remote_after = self.remote_head()
if remote_after != commit:
raise ResultChannelError(
"remote tx-node HEAD does not confirm the published commit"
)
return PublishOutcome( return PublishOutcome(
"PUBLISHED", "PUBLISHED",
commit, commit,