diff --git a/PROJECT_LOG.md b/PROJECT_LOG.md index ac1a812..cd4c0b2 100644 --- a/PROJECT_LOG.md +++ b/PROJECT_LOG.md @@ -591,3 +591,43 @@ Lab042 завершена: кабельная петля через аттеню ## Следующий этап Lab043: передаёт PlutoSDR, принимает RTL-SDR. Общий опорный генератор исчезает, впервые появляется реальное рассогласование несущей и частоты дискретизации двух независимых устройств. Здесь понадобится защищённая частотная коррекция, перенесённая в Записи 019. + +# Запись 021 + +## Дата + +31 августа 2026 года + +## Тема + +Lab044 (произвольный protocol packet через реальный BPSK HackRF-канал): implementation prepared, hardware validation pending. + +## Статус + +Lab044 implementation prepared. Hardware experiment NOT YET RUN, RF TX count = 0. Реализация построена поверх рабочей точки Lab043 (checkpoint 4cee6e9) без изменения её RF-параметров, gain, PHY sync, PRBS7-преамбулы, CRC или независимого детектора. Три пробела, найденные аудитом (отсутствие теста на обрезанный protocol packet, отсутствие BEFORE/DURING/AFTER диагностики записи, отсутствие SHA-256 для всех бинарных RX/TX артефактов), закрыты и покрыты новыми тестами. Подробности — в docs/lab044_current_state.md. + +## Следующий этап + +Реальный аппаратный RF-прогон Lab044 (ровно один TX) должен выполнить локальный агент после полного PASS `python tools/quick_gate.py` и `python -m pytest tests/ -q` на машине с подключённым HackRF. + +# Запись 022 + +## Дата + +31 августа 2026 года + +## Тема + +Lab044: первый реальный аппаратный RF-прогон произвольного protocol packet через BPSK HackRF-канал (ровно один TX). + +## Выполнено + +Software gate пройден полностью на рабочем интерпретаторе машины (Python 3.13.1, numpy 2.2.6, scipy 1.18.0, pytest 9.1.1): `python -m pytest tests/ -q` = 373 passed, 0 failed, 0 skipped; `python tools/quick_gate.py` = OK; `git diff --check` чист. Выполнен ровно один аппаратный RF-эксперимент `python experiments/lab044_hackrf_bpsk_packet.py --hardware --i-understand-hardware`. RF TX count = 1, без retry. + +Common-clock preflight PASS: MASTER reg29 = 0x26, SLAVE reg29 = 0x26, SLAVE CLKIN = DETECTED за 69 мс (bounded polling 0.05/0.05/0.5 с). RX стартовал до TX. Логи TX/RX чистые: 0 underruns, 0 overruns. Clipping 0, перегрузки АЦП нет. + +Результат — **RESULT C, CRC NOT_REACHED (CRC не PASS)**. Независимый PRBS7-детектор нашёл преамбулу (peak/median +38.3 dB), PHY SYNC 0xD391 восстановлен верно. Physical-layer BER по преамбуле ≈ 0.625 % (8/1280), согласуется с Lab043. Но одиночные битовые ошибки попали в поля длины: PHY LENGTH декодировано 264 вместо 268 байт, а protocol payload_length — 2304 вместо 256, из-за чего protocol header дал PAYLOAD_LENGTH_MISMATCH и разбор пакета сорвался раньше CRC. Protocol SYNC 0xAA55, version, message type (TELEMETRY), sequence (44) при этом декодированы верно. Packet/payload BER не вычислялись как «после успешного decode»: RX-пакет 264 байта не сопоставим с TX 268 байт, `rx_payload.bin` не создан. SNR 7.19 dB, separation 4.574, CFO 0.101 Hz (общий опорник). Артефакты: `C:\Users\user\Documents\SDR_Rover_Lab044_raw\20260831_140142_bpsk_packet\`, SHA-256 всех реально созданных бинарных файлов пересчитаны независимо и совпали с metadata.json; фиктивный `rx_payload.bin` не создавался (SHA = null). Cleanup: MASTER/SLAVE CLKOUT OFF, зависших `hackrf_transfer.exe` нет. Пост-RF software gate повторно пройден (373 passed). Изменения ещё не коммитятся: сначала результат смотрит ChatGPT. Подробности и таблица BEFORE/DURING/AFTER — в docs/lab044_current_state.md. + +## Следующий этап + +Защитить критические поля кадра от одиночных битовых ошибок при BER ~0.6 %: FEC либо избыточное/помехоустойчивое кодирование PHY LENGTH и protocol-заголовка (ошибка сорвала разбор именно в длине, а не в CRC полезной нагрузки). Только после этого имеет смысл повторный аппаратный прогон Lab044 и переход к более длинным/структурированным передачам. diff --git a/docs/lab044_current_state.md b/docs/lab044_current_state.md new file mode 100644 index 0000000..a54af8f --- /dev/null +++ b/docs/lab044_current_state.md @@ -0,0 +1,225 @@ +# Lab044: текущее состояние + +## Статус + +Lab044 hardware experiment ВЫПОЛНЕН один раз. RF TX count = 1. +Результат: **RESULT C** (PHY sync восстановлен независимо, но protocol packet +не восстановлен, CRC NOT_REACHED). CRC **не** PASS — это честный +отрицательный, но научно полезный результат. + +Реализация подготовлена поверх рабочей точки Lab043 (checkpoint `4cee6e9`, +"Lab043: establish reproducible HackRF BPSK working point") и не меняет её +RF-параметры, gain, PHY sync, PRBS7-преамбулу, CRC или независимый детектор. + +## Фактический аппаратный результат (31 августа 2026 года) + +Каталог артефактов: +`C:\Users\user\Documents\SDR_Rover_Lab044_raw\20260831_140142_bpsk_packet\` + +- **RF TX count = 1** (ровно одна передача, `TransmitGuard`, без retry). +- **Common clock = PASS**: MASTER reg29 = 0x26, SLAVE reg29 = 0x26, + SLAVE CLKIN = DETECTED за 69 мс (1 опрос), bounded polling 0.05/0.05/0.5 с. +- **PHY SYNC = PASS**: независимый PRBS7-детектор преамбулы, + peak/median +38.3 dB, peak/second +11.6 dB, payload не использовался. +- **FRAME HEADER**: PHY SYNC 0xD391 декодирован верно, но поле PHY LENGTH + искажено битовой ошибкой — декодировано 264 вместо 268 байт. Структурно + `frame_header_pass=true` (sync совпал, длина в границах 1..1024), но длина + фактически неверна. +- **PROTOCOL HEADER = FAIL** (`PAYLOAD_LENGTH_MISMATCH`): protocol SYNC + 0xAA55, version 1, message type 2 (TELEMETRY), sequence 44 декодированы + верно, но payload_length = 2304 вместо 256 (битовая ошибка в старшем + байте длины). `expected_packet_bytes = 2316 ≠ 264`. +- **CRC = NOT_REACHED** (packet не собран корректно, до CRC decode не дошло). +- **Classification = C**. + +### Радиометрика + +- independent preamble detected: **да** +- preamble errors: **8 / 1280** (physical-layer BER ≈ **0.625 %**, + согласуется с ~0.4586 % Lab043) +- CFO: **0.101 Hz** (общий опорный генератор ⇒ почти нулевой сдвиг несущей) +- SNR: **7.19 dB** +- separation: **4.574** +- timing offset: **−3 samples** +- ADC peak: peak_complex **5.10** code, peak_component **5.0** code +- ADC p99.9: **3.606** code (headroom 122 code до предела int8) +- clipping: **0** (fraction 0.0), near-int8: 0 +- RX overruns: **0**, TX underruns: **0** (оба лога чистые) + +### Ошибки данных (только ПОСЛЕ независимого CRC decision) + +CRC-решение было NOT_REACHED, поэтому packet/payload BER **не вычисляются +как «после успешного decode»**. Диагностическое сравнение с TX-артефактами +показало: + +- packet: `comparable=false` — RX 264 байт против TX 268 байт (длины не + совпадают из-за искажения PHY LENGTH), побитовый BER не определён. +- payload: `comparable=false` — `rx_payload.bin` не создан (protocol header + не прошёл), payload не извлечён. + +Первичный вывод: канал по-прежнему даёт ~0.6 % BER (как в Lab043), и в этом +единственном кадре ошибки попали именно в поля длины (PHY LENGTH и protocol +payload_length), что структурно сорвало разбор пакета раньше CRC. Это не +новый дефект, а известная чувствительность незащищённого кадра к одиночным +битовым ошибкам в критических полях — прямой довод в пользу FEC/защиты +заголовка на следующем этапе. + +### BEFORE / DURING / AFTER + +Границы построены по независимо найденному preamble start (sample 6 122 876) +и декодированной PHY LENGTH. + +| Регион | Длительность | RMS (после DC) | p95 | p99 | p99.9 | peak | clipping | +|---|---|---|---|---|---|---|---| +| BEFORE | 0.765 с | 0.776 | 3.162 | 3.606 | 3.606 | 5.00 | 0 | +| DURING | 0.171 с | 0.833 | 3.162 | 3.606 | 3.606 | 4.47 | 0 | +| AFTER | 3.063 с | 0.782 | 3.162 | 3.606 | 3.606 | 5.10 | 0 | + +DURING отличается от фоновых BEFORE/AFTER повышенным RMS после удаления DC +(0.833 против 0.776/0.782) — то есть на интервале кадра действительно +присутствует переданный сигнал над фоном; хвосты амплитуды (p95/p99/p99.9) +и clipping идентичны, перегрузки АЦП нет. + +### Артефакты и SHA-256 (пересчитаны независимо, совпали с metadata.json) + +- `tx_payload.bin` (256 B): `aae1ee74…e27109` +- `tx_packet.bin` (268 B): `cc179e93…e4554b` +- `tx_bpsk_cs8.iq` (2 798 400 B): `15260dbc…f42e7` +- `rx_cs8.iq` (64 000 000 B): `8d5f4cd1…1019f1` (в Git не добавляется) +- `rx_packet.bin` (264 B): `b6ebbce3…c694e1` +- `rx_payload.bin`: **не создан** (SHA-256 = null, фиктивный файл не создаётся) + +### Cleanup + +MASTER CLKOUT OFF (SLAVE CLKIN = "no clock signal detected"), +SLAVE CLKOUT OFF, зависших `hackrf_transfer.exe` нет. + +## Цель Lab044 + +Впервые передать по реальному RF BPSK-пакет с произвольным payload, который +приёмник заранее не знает: `arbitrary bytes → protocol packet → PHY frame → +BPSK → RF → independent synchronization → demodulation → frame parsing → +CRC verification`. CRC PASS желателен, но не обязателен — CRC FAIL при +BER около 0.4586% (унаследованном из Lab043) является нормальным научно +полезным результатом (RESULT B). + +## Формат (подтверждён по фактическому коду, не принят на веру) + +- `protocol/packet.py`: `HEADER_FORMAT = ">HBBHH"` (8 байт: SYNC 0xAA55, + version, message type, sequence number, payload length), CRC-32 + (`zlib.crc32`, big-endian uint32, 4 байта), CRC покрывает header+payload. +- Payload = **256 bytes**, protocol packet = **268 bytes** + (8 + 256 + 4), сгенерирован через `secrets.token_bytes` — системный + источник случайности, один раз. +- PHY frame = `[1280-bit PRBS7 preamble Lab043][PHY SYNC 0xD391 + PHY LENGTH + uint16, ">HH", big-endian][protocol packet]` = **3456 BPSK-символов** + (1280 + 32 + 268×8). Преамбула — та же аппаратно проверенная + 1280-битная PRBS7 (`pilot_prbs7`, `PILOT_INITIAL_STATE=0x7F`) из + `experiments/lab043_hackrf_bpsk.py`, не generic-маркер из + `protocol/bpsk_radio.build_radio_frame()`. +- Waveform = **1,399,200 сэмплов** при 8 MS/s ⇒ длительность + **≈ 174.9 мс** (после RRC-формирования Lab042: rolloff 0.35, + span 10 символов, guard 16 символов). +- Bit order: MSB-first, явно (`np.unpackbits(..., bitorder="big")`). +- MAX_PACKET_BYTES = 1024 (fail-closed предел для PHY LENGTH). + +## RF-параметры (без изменений от Lab043) + +RF center 1080 MHz, sample rate 8 MS/s, symbol rate 20 ksym/s, digital +carrier offset +50 kHz, TX VGA = 10 dB, RX LNA = 24 dB, RX VGA = 24 dB, +RF amp OFF, common clock: JYEC TCXO 10 MHz MASTER → MASTER CLKOUT → SLAVE +CLKIN, bounded common-clock preflight (initial_settle=0.05s, +poll_interval=0.05s, timeout=0.5s), reg29 = 0x26 на обеих платах. + +## Пробелы аудита A/B/C — закрыты + +**A. TRUNCATED PROTOCOL PACKET TEST — FIXED.** Добавлен +`test_truncated_protocol_packet_is_flagged_without_exception` +(`tests/test_lab044_hackrf_bpsk_packet.py`): синтетический capture с +полной преамбулой и полным PHY header, но физически обрезанным (не +дополненным нулями) телом protocol packet. Ожидаемое и фактическое +поведение: `phy_sync_pass=True`, `frame_header_pass=False`, +`packet_received=False`, `frame_header["reason"]=="TRUNCATED_PACKET"`, +`crc=="NOT_REACHED"`, без исключений, без изменения рабочей +классификации. Исходный код `analyze_capture_independently` этот путь уже +предусматривал — не хватало только теста, подтверждающего его +достижимость и корректность. + +**B. BEFORE/DURING/AFTER DIAGNOSTICS — FIXED.** Добавлена функция +`_capture_diagnostics()` в `experiments/lab044_hackrf_bpsk_packet.py`, +переиспользующая `lab043._capture_intervals()` и `lab043.raw_statistics()` +без изменения самого Lab043 (совместимый набор полей: mean I/Q, +RMS после DC removal, p95/p99/p99.9 magnitude, peak, clipping +count/fraction — плюс добавленные `sample_range` и +`duration_seconds` на каждый регион). Граница DURING строится только по +`detection["refined_start_sample"]` (независимый preamble) и по PHY +LENGTH, если она уже декодирована; если PHY header ещё не восстановлен — +по максимально возможной длине кадра Lab044 (MAX_PACKET_BYTES), payload +никогда не используется. Если независимый preamble не найден вовсе, +DURING остаётся `None` с явной пометкой в поле `note`, сохраняется только +`whole_capture`. Результат добавлен как `analysis["capture_intervals"]` во +всех пяти точках возврата `analyze_capture_independently` (RESULT A/B/C/D). +Тесты: `test_capture_intervals_present_for_independently_decoded_frame`, +`test_capture_intervals_undefined_when_no_independent_start`. + +**C. SHA-256 ДЛЯ ВСЕХ BINARY ARTIFACTS — FIXED.** Добавлена функция +`_artifact_sha256()`, переиспользующая `lab043._sha256_file()`. После +`run_hardware_experiment()` в `metadata.json` появляется поле +`artifact_sha256` с хэшем каждого РЕАЛЬНО существующего файла: +`tx_payload.bin`, `tx_packet.bin`, `tx_bpsk_cs8.iq`, `rx_cs8.iq`, +`rx_packet.bin`, `rx_payload.bin`. Для `rx_packet.bin`/`rx_payload.bin`, +если decode не дошёл до их создания, значение — `None`; фиктивный файл не +создаётся (это поведение уже было в коде и не менялось). Тесты: +`test_artifact_sha256_recorded_for_every_real_rx_and_tx_file`, +`test_no_fake_rx_artifacts_or_hashes_when_packet_not_decoded`. + +Ничего из RF-параметров, gain, packet format, PHY sync, PRBS7-преамбулы, +payload size, CRC, независимого детектора или порогов Lab043 не менялось. +FEC/QPSK/JPEG/video не добавлялись. + +## Что реально проверено в этой сессии + +Эта сессия работает без `device_bash` на машине пользователя и без доступа +к сети из своей песочницы (нет pip/apt-репозитория), поэтому команда +`pytest` в буквальном смысле **NOT EXECUTED IN THIS SESSION**. + +Вместо этого фактически выполнено (все результаты — реальный вывод +интерпретатора, не предположение): + +- `ast.parse()` по `experiments/lab044_hackrf_bpsk_packet.py` и + `tests/test_lab044_hackrf_bpsk_packet.py` — синтаксис корректен. +- Полный импорт `experiments.lab044_hackrf_bpsk_packet` со всей цепочкой + зависимостей (`protocol.*`, `experiments.lab043_*`) — импортируется без + ошибок. +- Все 22 функции `test_*` из `tests/test_lab044_hackrf_bpsk_packet.py` + (17 существовавших + 5 новых для A/B/C) вызваны напрямую, как обычные + Python-функции, с самодельными подстановками фикстур `tmp_path` и + `monkeypatch` (реальный pytest недоступен для установки в этой + песочнице — нет сетевого доступа к PyPI/apt) — **22/22 PASS**. Это + честная проверка логики, но не замена запуску `pytest` как CLI-команды. +- `python tools/quick_gate.py` запущен реально: синтаксис и импорт + `protocol/*` прошли, единственная сообщённая ошибка — сам pytest не + установлен (`No module named pytest`), что ожидаемо и не является + дефектом кода. +- Аппаратный RF-эксперимент не запускался и не имитировался. + +## Что должен сделать локальный агент перед RF + +1. `python tools/quick_gate.py` — уже в этой сессии проходит синтаксис и + импорт; локально должен пройти целиком, включая `pytest tests -q`. +2. `python -m pytest tests/ -q` — полный реальный прогон (в этой сессии + недоступен физически, см. выше). Ожидается: все прежние тесты Lab043 + зелёные, все 22 теста `test_lab044_hackrf_bpsk_packet.py` зелёные. +3. `git diff --check` — конфликт-маркеры/whitespace. +4. Только после полного PASS — `python experiments/lab044_hackrf_bpsk_packet.py --hardware --i-understand-hardware`. +5. STOP conditions: common-clock preflight FAIL, RX завершился до TX, + TX/RX вернули ненулевой код, log содержит overrun/underrun, любое + исключение — во всех случаях RF TX должен остаться равным 0 (гарантия + `TransmitGuard`, покрыта тестами `test_common_clock_failure_means_zero_tx`, + `test_frame_validation_failure_means_zero_tx`, + `test_rx_startup_failure_means_zero_tx`). +6. Artifacts появятся в `~/Documents/SDR_Rover_Lab044_raw/_bpsk_packet/`: + `tx_payload.bin`, `tx_packet.bin`, `tx_bpsk_cs8.iq`, `rx_cs8.iq`, + `rx_packet.bin` (если извлечён), `rx_payload.bin` (если извлечён), + `metadata.json`, `analysis.json`. +7. Raw IQ (`rx_cs8.iq`, `tx_bpsk_cs8.iq`) в Git не добавлять. diff --git a/experiments/lab044_hackrf_bpsk_packet.py b/experiments/lab044_hackrf_bpsk_packet.py new file mode 100644 index 0000000..346130a --- /dev/null +++ b/experiments/lab044_hackrf_bpsk_packet.py @@ -0,0 +1,840 @@ +"""Lab044: один произвольный protocol packet через реальный BPSK HackRF link. + +Приёмный тракт принципиально не принимает ожидаемый payload и не читает +``tx_payload.bin``. Сравнение с переданными данными выполняется отдельной +диагностической стадией только после независимого frame/CRC-решения. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import math +import secrets +import struct +import sys +import time +from dataclasses import dataclass +from datetime import datetime +from pathlib import Path +from typing import Callable + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +import numpy as np +from scipy.signal import fftconvolve + +from experiments import lab043_hackrf_bpsk as lab043 +from experiments.lab043_hackrf_hardware import signed_iq_bytes_to_complex64 +from experiments.lab043_pluto_to_rtlsdr import pilot_prbs7, shape_frame_like_lab042 +from protocol import bpsk_radio as radio +from protocol.packet import ( + CRC_SIZE, + HEADER_FORMAT, + HEADER_SIZE, + MESSAGE_TYPE_ACK, + MESSAGE_TYPE_IMAGE_FRAGMENT, + MESSAGE_TYPE_TELEMETRY, + MESSAGE_TYPE_TEXT, + PROTOCOL_VERSION, + SYNC_WORD, + CRCError, + PacketError, + build_packet, + parse_packet, +) + + +# ============================================================ Параметры Lab044 + +PAYLOAD_SIZE_BYTES = 256 +MESSAGE_TYPE = MESSAGE_TYPE_TELEMETRY +SEQUENCE_NUMBER = 44 +MAX_PACKET_BYTES = 1024 + +PHY_SYNC = radio.RADIO_SYNC_WORD +PHY_HEADER_FORMAT = ">HH" +PHY_HEADER_SIZE_BYTES = struct.calcsize(PHY_HEADER_FORMAT) +PHY_HEADER_BIT_COUNT = PHY_HEADER_SIZE_BYTES * 8 +BIT_ORDER = "big" + +PREAMBLE_BIT_COUNT = lab043.PILOT_SYMBOL_COUNT +PREAMBLE_INITIAL_STATE = lab043.PILOT_INITIAL_STATE +DETECTION_GUARD_SYMBOLS = lab043.SECOND_PEAK_GUARD_SYMBOLS + +RF_HZ = lab043.RF_HZ +SAMPLE_RATE_HZ = lab043.SAMPLE_RATE_HZ +SYMBOL_RATE = lab043.SYMBOL_RATE +SAMPLES_PER_SYMBOL = lab043.SAMPLES_PER_SYMBOL +CARRIER_OFFSET_HZ = lab043.CARRIER_OFFSET_HZ +TARGET_RMS_CODE = lab043.TARGET_RMS_CODE +TX_VGA_DB = lab043.TX_VGA_DB +RX_LNA_DB = 24 +RX_VGA_DB = 24 +CAPTURE_SAMPLE_COUNT = lab043.CAPTURE_SAMPLE_COUNT +RX_LEAD_SECONDS = lab043.RX_LEAD_SECONDS +RX_WAIT_TIMEOUT_SECONDS = lab043.RX_WAIT_TIMEOUT_SECONDS + +KNOWN_MESSAGE_TYPES = { + MESSAGE_TYPE_TEXT, + MESSAGE_TYPE_TELEMETRY, + MESSAGE_TYPE_IMAGE_FRAGMENT, + MESSAGE_TYPE_ACK, +} + +CRC_PASS = "PASS" +CRC_FAIL = "FAIL" +CRC_NOT_REACHED = "NOT_REACHED" + + +@dataclass(frozen=True) +class TransmissionPlan: + payload: bytes + packet: bytes + phy_header: bytes + frame_bits: np.ndarray + tx_cs8: bytes + payload_sha256: str + packet_sha256: str + waveform_sha256: str + + @property + def frame_symbol_count(self) -> int: + return int(len(self.frame_bits)) + + @property + def waveform_sample_count(self) -> int: + return len(self.tx_cs8) // 2 + + +@dataclass(frozen=True) +class IndependentReception: + analysis: dict + packet_bytes: bytes | None + payload_bytes: bytes | None + + +PayloadFactory = Callable[[int], bytes] +Analyzer = Callable[[np.ndarray], IndependentReception] +DiagnosticComparer = Callable[[IndependentReception, Path, Path], dict] + + +# ============================================================ Bytes и frame + + +def bytes_to_bits(data: bytes) -> np.ndarray: + """Преобразовать bytes в биты MSB first с явным bitorder.""" + + if not isinstance(data, bytes): + raise TypeError("data должен иметь тип bytes") + return np.unpackbits(np.frombuffer(data, dtype=np.uint8), bitorder=BIT_ORDER) + + +def bits_to_bytes(bits: np.ndarray) -> bytes: + """Упаковать одномерные 0/1 биты MSB first.""" + + bits = np.asarray(bits, dtype=np.uint8) + if bits.ndim != 1: + raise ValueError("bits должен быть одномерным") + if len(bits) % 8: + raise ValueError("Количество битов должно быть кратно восьми") + if not np.all((bits == 0) | (bits == 1)): + raise ValueError("bits должен содержать только 0 и 1") + return np.packbits(bits, bitorder=BIT_ORDER).tobytes() + + +def preamble_bits() -> np.ndarray: + return pilot_prbs7(PREAMBLE_BIT_COUNT, PREAMBLE_INITIAL_STATE) + + +def encode_phy_header(packet_length_bytes: int) -> bytes: + if not 1 <= packet_length_bytes <= MAX_PACKET_BYTES: + raise ValueError("Длина protocol packet вне границ Lab044") + return struct.pack(PHY_HEADER_FORMAT, PHY_SYNC, packet_length_bytes) + + +def decode_phy_header(header_bits: np.ndarray) -> dict: + """Fail-closed разобрать PHY header, не выделяя память по принятой длине.""" + + bits = np.asarray(header_bits, dtype=np.uint8) + if len(bits) < PHY_HEADER_BIT_COUNT: + return {"passed": False, "reason": "TRUNCATED", "sync": None, "packet_bytes": None} + try: + sync, packet_bytes = struct.unpack( + PHY_HEADER_FORMAT, + bits_to_bytes(bits[:PHY_HEADER_BIT_COUNT]), + ) + except (ValueError, struct.error): + return {"passed": False, "reason": "DECODE_ERROR", "sync": None, "packet_bytes": None} + if sync != PHY_SYNC: + return { + "passed": False, + "reason": "SYNC_MISMATCH", + "sync": sync, + "packet_bytes": packet_bytes, + } + if not 1 <= packet_bytes <= MAX_PACKET_BYTES: + return { + "passed": False, + "reason": "LENGTH_OUT_OF_RANGE", + "sync": sync, + "packet_bytes": packet_bytes, + } + return {"passed": True, "reason": None, "sync": sync, "packet_bytes": packet_bytes} + + +def _scale_and_carrier(symbols: np.ndarray) -> np.ndarray: + shaped = shape_frame_like_lab042(symbols, SAMPLES_PER_SYMBOL) + indexes = np.arange(len(shaped), dtype=np.float64) + carriered = shaped * np.exp( + 2j * np.pi * CARRIER_OFFSET_HZ * indexes / SAMPLE_RATE_HZ + ) + rms = float(np.sqrt(np.mean(np.abs(carriered) ** 2))) + if not np.isfinite(rms) or rms <= 0.0: + raise ValueError("Нулевая или нечисловая мощность формы") + return carriered * (TARGET_RMS_CODE / rms) + + +def build_transmission_plan(payload: bytes) -> TransmissionPlan: + """Собрать и полностью проверить единственный произвольный Lab044 frame.""" + + if not isinstance(payload, bytes): + raise TypeError("payload должен иметь тип bytes") + if len(payload) != PAYLOAD_SIZE_BYTES: + raise ValueError(f"Lab044 требует ровно {PAYLOAD_SIZE_BYTES} payload bytes") + + packet = build_packet(payload, MESSAGE_TYPE, SEQUENCE_NUMBER) + phy_header = encode_phy_header(len(packet)) + frame_bits = np.concatenate(( + preamble_bits(), + bytes_to_bits(phy_header), + bytes_to_bits(packet), + )) + tx_cs8 = lab043.to_cs8_bytes(_scale_and_carrier(radio.bpsk_modulate(frame_bits))) + plan = TransmissionPlan( + payload=payload, + packet=packet, + phy_header=phy_header, + frame_bits=frame_bits, + tx_cs8=tx_cs8, + payload_sha256=hashlib.sha256(payload).hexdigest(), + packet_sha256=hashlib.sha256(packet).hexdigest(), + waveform_sha256=hashlib.sha256(tx_cs8).hexdigest(), + ) + validate_transmission_plan(plan) + return plan + + +def validate_transmission_plan(plan: TransmissionPlan) -> None: + """Закрыть TX-gate при любом внутреннем расхождении frame/packet.""" + + expected_packet_bytes = HEADER_SIZE + PAYLOAD_SIZE_BYTES + CRC_SIZE + if len(plan.packet) != expected_packet_bytes: + raise ValueError(f"Protocol packet имеет {len(plan.packet)}, ожидалось {expected_packet_bytes}") + if parse_packet(plan.packet).payload != plan.payload: + raise ValueError("Protocol packet round trip не восстановил payload") + if bits_to_bytes(bytes_to_bits(plan.packet)) != plan.packet: + raise ValueError("MSB-first packet bits round trip не совпал") + header = decode_phy_header(bytes_to_bits(plan.phy_header)) + if not header["passed"] or header["packet_bytes"] != len(plan.packet): + raise ValueError("PHY header round trip не совпал") + expected_frame_symbols = PREAMBLE_BIT_COUNT + PHY_HEADER_BIT_COUNT + len(plan.packet) * 8 + if plan.frame_symbol_count != expected_frame_symbols: + raise ValueError("Число frame symbols не совпало с форматом") + if len(plan.tx_cs8) != 2 * plan.waveform_sample_count or not plan.tx_cs8: + raise ValueError("Некорректная cs8-форма") + tx_codes = np.frombuffer(plan.tx_cs8, dtype=np.int8) + if int(np.max(np.abs(tx_codes.astype(np.int16)))) >= 127: + raise ValueError("TX waveform касается предела signed int8") + + +# ============================================================ Независимый RX + + +def _preamble_reference_samples() -> np.ndarray: + waveform = _scale_and_carrier(radio.bpsk_modulate(preamble_bits())) + return signed_iq_bytes_to_complex64(lab043.to_cs8_bytes(waveform)) + + +def _detect_preamble(samples: np.ndarray) -> dict: + reference = _preamble_reference_samples() + rx_decimated, reference_decimated = lab043._detection_streams(samples, reference) + scores = lab043._normalized_sliding_correlation(rx_decimated, reference_decimated) + guard = ( + DETECTION_GUARD_SYMBOLS + * SAMPLES_PER_SYMBOL + // lab043.DETECTION_DECIMATION + ) + metrics = lab043._peak_metrics(scores, guard) + detected = bool( + metrics["over_median_db"] >= lab043.PREAMBLE_OVER_MEDIAN_DB_MIN + and metrics["over_second_db"] >= lab043.PREAMBLE_OVER_SECOND_DB_MIN + ) + coarse_start = metrics["peak_index"] * lab043.DETECTION_DECIMATION + + low = max(0, coarse_start - 2_000) + high = min(len(samples), coarse_start + len(reference) + 2_000) + local = samples[low:high] - np.mean(samples) + centered_reference = reference - np.mean(reference) + if len(local) < len(centered_reference): + refined_start = coarse_start + else: + correlation = fftconvolve(local, np.conj(centered_reference[::-1]), mode="valid") + refined_start = low + int(np.argmax(np.abs(correlation))) + + return { + "preamble_detected": detected, + "method": "global PRBS7 preamble-only normalized correlation", + "payload_used": False, + "peak": metrics["peak"], + "median": metrics["median"], + "over_median_db": metrics["over_median_db"], + "over_second_db": metrics["over_second_db"], + "coarse_start_sample": coarse_start, + "refined_start_sample": refined_start, + } + + +def inspect_protocol_header(packet_bytes: bytes) -> dict: + """Проверить существующий protocol header отдельно от CRC.""" + + minimum = HEADER_SIZE + CRC_SIZE + if len(packet_bytes) < minimum: + return {"passed": False, "reason": "TRUNCATED", "payload_length": None} + try: + sync, version, message_type, sequence_number, payload_length = struct.unpack( + HEADER_FORMAT, + packet_bytes[:HEADER_SIZE], + ) + except struct.error: + return {"passed": False, "reason": "DECODE_ERROR", "payload_length": None} + expected_size = HEADER_SIZE + payload_length + CRC_SIZE + reason = None + if sync != SYNC_WORD: + reason = "SYNC_MISMATCH" + elif version != PROTOCOL_VERSION: + reason = "VERSION_MISMATCH" + elif message_type not in KNOWN_MESSAGE_TYPES: + reason = "MESSAGE_TYPE_INVALID" + elif expected_size != len(packet_bytes): + reason = "PAYLOAD_LENGTH_MISMATCH" + return { + "passed": reason is None, + "reason": reason, + "sync": sync, + "version": version, + "message_type": message_type, + "sequence_number": sequence_number, + "payload_length": payload_length, + "expected_packet_bytes": expected_size, + } + + +def _empty_analysis(raw: dict, detection: dict) -> dict: + return { + "phy_sync_pass": False, + "frame_header_pass": False, + "packet_received": False, + "protocol_header_pass": False, + "crc": CRC_NOT_REACHED, + "classification": "D", + "raw": raw, + "detection": detection, + "frame_header": None, + "protocol_header": None, + "carrier": None, + } + + +def _capture_diagnostics( + samples: np.ndarray, + detection: dict, + packet_bytes_hint: int | None, +) -> dict: + """BEFORE/DURING/AFTER статистика записи вокруг независимо найденного кадра. + + Переиспользует ``lab043._capture_intervals`` и ``lab043.raw_statistics`` + без изменения самого Lab043. Граница DURING строится только по + независимо найденному ``detection["refined_start_sample"]`` и по длине + кадра: если PHY LENGTH уже декодирован (``packet_bytes_hint``), длина + точная; иначе используется максимально возможная длина кадра Lab044. + Payload при этом никогда не используется. Если независимый preamble не + найден, DURING остаётся неопределённым и явно помечается как таковой. + """ + + if not detection.get("preamble_detected"): + return { + "before": None, + "during": None, + "after": None, + "frame_start_sample": None, + "frame_end_sample": None, + "note": "DURING interval не определён: независимый preamble не найден", + "whole_capture": lab043.raw_statistics(samples), + } + + frame_start = int(detection["refined_start_sample"]) + if packet_bytes_hint is not None and 1 <= packet_bytes_hint <= MAX_PACKET_BYTES: + frame_symbols = PREAMBLE_BIT_COUNT + PHY_HEADER_BIT_COUNT + packet_bytes_hint * 8 + note = "DURING построен по независимому preamble start и декодированной PHY LENGTH" + else: + frame_symbols = PREAMBLE_BIT_COUNT + PHY_HEADER_BIT_COUNT + MAX_PACKET_BYTES * 8 + note = ( + "DURING построен по независимому preamble start и максимально возможной " + "длине кадра Lab044 (PHY header ещё не восстановлен)" + ) + frame_length_samples = frame_symbols * SAMPLES_PER_SYMBOL + intervals = lab043._capture_intervals(samples, frame_start, frame_length_samples) + + sample_rate = float(SAMPLE_RATE_HZ) + region_bounds = { + "before": (0, intervals["frame_start_sample"]), + "during": (intervals["frame_start_sample"], intervals["frame_end_sample"]), + "after": (intervals["frame_end_sample"], int(len(samples))), + } + for region_key, (start, end) in region_bounds.items(): + stats = intervals[region_key] + if stats is None: + continue + stats["sample_range"] = [int(start), int(end)] + stats["duration_seconds"] = (end - start) / sample_rate + + intervals["note"] = note + return intervals + + +def _artifact_sha256(output_directory: Path, filenames: list[str]) -> dict[str, str | None]: + """SHA-256 каждого реально существующего файла; None для несозданных. + + Никогда не создаёт файл ради вычисления хэша: если файла нет на диске + (например, rx_payload.bin не был извлечён), значение — None. + Переиспользует lab043._sha256_file. + """ + + hashes: dict[str, str | None] = {} + for name in filenames: + path = output_directory / name + hashes[name] = lab043._sha256_file(path) if path.is_file() else None + return hashes + + +def analyze_capture_independently(samples: np.ndarray) -> IndependentReception: + """Принять Lab044 frame без ожидаемого packet или payload.""" + + samples = np.asarray(samples, dtype=np.complex64) + raw = lab043.raw_statistics(samples) + detection = _detect_preamble(samples) + if not detection["preamble_detected"]: + analysis = _empty_analysis(raw, detection) + analysis["capture_intervals"] = _capture_diagnostics(samples, detection, None) + return IndependentReception(analysis, None, None) + + maximum_frame_symbols = PREAMBLE_BIT_COUNT + PHY_HEADER_BIT_COUNT + MAX_PACKET_BYTES * 8 + maximum_waveform_samples = ( + 2 * 16 * SAMPLES_PER_SYMBOL + + maximum_frame_symbols * SAMPLES_PER_SYMBOL + + 10 * SAMPLES_PER_SYMBOL + ) + matched, nominal = lab043._matched_waveform( + samples, + detection["refined_start_sample"], + maximum_waveform_samples, + ) + expected_preamble = radio.bpsk_modulate(preamble_bits()) + timing, _ = lab043._timing_offsets(matched, nominal, expected_preamble) + carrier = lab043._evaluate_symbols( + matched, + nominal, + timing, + preamble_bits(), + PREAMBLE_BIT_COUNT, + ) + available_symbols = matched[ + nominal + timing : nominal + timing + maximum_frame_symbols * SAMPLES_PER_SYMBOL : SAMPLES_PER_SYMBOL + ].astype(np.complex128) + if carrier["abs_h"] == 0.0: + analysis = _empty_analysis(raw, detection) + analysis["classification"] = "C" + analysis["carrier"] = carrier + analysis["capture_intervals"] = _capture_diagnostics(samples, detection, None) + return IndependentReception(analysis, None, None) + + symbol_time = (np.arange(len(available_symbols)) + 0.5) / SYMBOL_RATE + normalized = ( + available_symbols + * np.exp(-2j * np.pi * carrier["cfo_hz"] * symbol_time) + / (carrier["abs_h"] * np.exp(1j * carrier["phase_rad"])) + ) + received_bits = (normalized.real >= 0.0).astype(np.uint8) + preamble_error_count = int(np.count_nonzero( + received_bits[:PREAMBLE_BIT_COUNT] != preamble_bits() + )) + phy_sync_pass = bool( + raw["clipping_count"] == 0 + and raw["near_int8_count"] == 0 + and len(received_bits) >= PREAMBLE_BIT_COUNT + PHY_HEADER_BIT_COUNT + ) + header_start = PREAMBLE_BIT_COUNT + header_end = header_start + PHY_HEADER_BIT_COUNT + frame_header = decode_phy_header(received_bits[header_start:header_end]) + if not frame_header["passed"]: + analysis = { + **_empty_analysis(raw, detection), + "phy_sync_pass": phy_sync_pass, + "classification": "C", + "frame_header": frame_header, + "carrier": carrier, + "preamble_errors": preamble_error_count, + "timing_offset_samples": timing, + "capture_intervals": _capture_diagnostics(samples, detection, frame_header.get("packet_bytes")), + } + return IndependentReception(analysis, None, None) + + packet_length = int(frame_header["packet_bytes"]) + packet_start = header_end + packet_end = packet_start + packet_length * 8 + if len(received_bits) < packet_end: + frame_header = {**frame_header, "passed": False, "reason": "TRUNCATED_PACKET"} + analysis = { + **_empty_analysis(raw, detection), + "phy_sync_pass": phy_sync_pass, + "classification": "C", + "frame_header": frame_header, + "carrier": carrier, + "preamble_errors": preamble_error_count, + "timing_offset_samples": timing, + "capture_intervals": _capture_diagnostics(samples, detection, packet_length), + } + return IndependentReception(analysis, None, None) + + packet_bytes = bits_to_bytes(received_bits[packet_start:packet_end]) + protocol_header = inspect_protocol_header(packet_bytes) + crc_status = CRC_NOT_REACHED + payload_bytes: bytes | None = None + if protocol_header["passed"]: + payload_length = int(protocol_header["payload_length"]) + payload_bytes = packet_bytes[HEADER_SIZE : HEADER_SIZE + payload_length] + try: + parsed = parse_packet(packet_bytes) + except CRCError: + crc_status = CRC_FAIL + except PacketError as error: + protocol_header = { + **protocol_header, + "passed": False, + "reason": f"PACKET_ERROR: {error}", + } + else: + crc_status = CRC_PASS + payload_bytes = parsed.payload + + if crc_status == CRC_PASS: + classification = "A" + elif crc_status == CRC_FAIL and protocol_header["passed"]: + classification = "B" + else: + classification = "C" + analysis = { + "phy_sync_pass": phy_sync_pass, + "frame_header_pass": True, + "packet_received": True, + "protocol_header_pass": bool(protocol_header["passed"]), + "crc": crc_status, + "classification": classification, + "raw": raw, + "detection": detection, + "frame_header": frame_header, + "protocol_header": protocol_header, + "carrier": carrier, + "preamble_errors": preamble_error_count, + "timing_offset_samples": timing, + "received_symbol_count": int(len(received_bits)), + "payload_known_before_decode": False, + "capture_intervals": _capture_diagnostics(samples, detection, packet_length), + } + return IndependentReception(analysis, packet_bytes, payload_bytes) + + +# ============================================================ Диагностика после CRC + + +def _comparison(received: bytes | None, expected: bytes) -> dict: + if received is None or len(received) != len(expected): + return { + "comparable": False, + "expected_bytes": len(expected), + "received_bytes": None if received is None else len(received), + "bit_errors": None, + "ber": None, + "byte_errors": None, + "byte_error_rate": None, + "first_differing_byte_positions": [], + "bit_error_run_lengths": [], + } + expected_array = np.frombuffer(expected, dtype=np.uint8) + received_array = np.frombuffer(received, dtype=np.uint8) + xor = np.bitwise_xor(expected_array, received_array) + differing_bytes = np.flatnonzero(xor) + error_bits = bytes_to_bits(xor.tobytes()) + error_positions = np.flatnonzero(error_bits) + runs: list[int] = [] + if len(error_positions): + run_length = 1 + for previous, current in zip(error_positions[:-1], error_positions[1:]): + if current == previous + 1: + run_length += 1 + else: + runs.append(run_length) + run_length = 1 + runs.append(run_length) + bit_errors = int(np.count_nonzero(error_bits)) + byte_errors = int(len(differing_bytes)) + return { + "comparable": True, + "expected_bytes": len(expected), + "received_bytes": len(received), + "bit_errors": bit_errors, + "ber": bit_errors / (8 * len(expected)) if expected else 0.0, + "byte_errors": byte_errors, + "byte_error_rate": byte_errors / len(expected) if expected else 0.0, + "first_differing_byte_positions": differing_bytes[:20].astype(int).tolist(), + "bit_error_run_lengths": runs, + } + + +def diagnostic_compare_after_decode( + reception: IndependentReception, + tx_payload_path: Path, + tx_packet_path: Path, +) -> dict: + """Только теперь прочитать TX artifacts и сравнить без исправления RX.""" + + tx_packet = tx_packet_path.read_bytes() + tx_payload = tx_payload_path.read_bytes() + return { + "stage": "AFTER_INDEPENDENT_CRC_DECISION", + "receiver_knew_payload_before_decode": False, + "packet": _comparison(reception.packet_bytes, tx_packet), + "payload": _comparison(reception.payload_bytes, tx_payload), + } + + +def analyze_then_compare( + samples: np.ndarray, + tx_payload_path: Path, + tx_packet_path: Path, + analyzer: Analyzer = analyze_capture_independently, + comparer: DiagnosticComparer = diagnostic_compare_after_decode, +) -> IndependentReception: + """Зафиксировать порядок: независимый decode, затем чтение TX artifacts.""" + + reception = analyzer(samples) + reception.analysis["diagnostic_comparison"] = comparer( + reception, + tx_payload_path, + tx_packet_path, + ) + return reception + + +# ============================================================ Hardware + + +def default_output_directory() -> Path: + timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") + return Path.home() / "Documents" / "SDR_Rover_Lab044_raw" / f"{timestamp}_bpsk_packet" + + +def run_hardware_experiment( + output_directory: Path, + runner: lab043.CommandRunner | None = None, + process_starter: lab043.ProcessStarter | None = None, + sleep_function: lab043.SleepFunction = time.sleep, + payload_factory: PayloadFactory = secrets.token_bytes, + analyzer: Analyzer = analyze_capture_independently, + comparer: DiagnosticComparer = diagnostic_compare_after_decode, +) -> dict: + """Один fail-closed hardware packet: preflight, RX, один TX, decode.""" + + runner = runner or lab043._default_command_runner + process_starter = process_starter or lab043._default_process_starter + payload = payload_factory(PAYLOAD_SIZE_BYTES) + plan = build_transmission_plan(payload) + validate_transmission_plan(plan) + + output_directory.mkdir(parents=True, exist_ok=True) + tx_payload_path = output_directory / "tx_payload.bin" + tx_packet_path = output_directory / "tx_packet.bin" + tx_waveform_path = output_directory / "tx_bpsk_cs8.iq" + tx_payload_path.write_bytes(plan.payload) + tx_packet_path.write_bytes(plan.packet) + tx_waveform_path.write_bytes(plan.tx_cs8) + + guard = lab043.TransmitGuard() + rx_process: lab043.RunningCommand | None = None + try: + preflight = lab043.common_clock_preflight(runner, guard) + rx_path = output_directory / "rx_cs8.iq" + rx_stdout_path = output_directory / "rx_stdout.txt" + rx_stderr_path = output_directory / "rx_stderr.txt" + tx_stdout_path = output_directory / "tx_stdout.txt" + tx_stderr_path = output_directory / "tx_stderr.txt" + transfer_tool = str(lab043.HACKRF_TOOLS_DIR / "hackrf_transfer.exe") + rx_command = [ + transfer_tool, + "-d", lab043.SLAVE_SERIAL, + "-r", str(rx_path), + "-f", str(RF_HZ), + "-s", str(SAMPLE_RATE_HZ), + "-n", str(CAPTURE_SAMPLE_COUNT), + "-l", str(RX_LNA_DB), + "-g", str(RX_VGA_DB), + "-a", "0", + "-p", "0", + "-B", + ] + rx_process = process_starter(rx_command, rx_stdout_path, rx_stderr_path) + sleep_function(RX_LEAD_SECONDS) + if rx_process.poll() is not None: + rx_process.close_logs() + detail = rx_stderr_path.read_text(encoding="utf-8", errors="replace") + raise RuntimeError(f"RX завершился до разрешения TX: {detail.strip()}") + + tx_command = [ + transfer_tool, + "-d", lab043.MASTER_SERIAL, + "-t", str(tx_waveform_path), + "-f", str(RF_HZ), + "-s", str(SAMPLE_RATE_HZ), + "-n", str(plan.waveform_sample_count), + "-x", str(TX_VGA_DB), + "-a", "0", + "-p", "0", + "-B", + ] + guard.register_transmit() + tx_result = runner(tx_command) + tx_stdout_path.write_text(tx_result.stdout or "", encoding="utf-8") + tx_stderr_path.write_text(tx_result.stderr or "", encoding="utf-8") + if tx_result.returncode != 0: + raise RuntimeError(f"Единственный TX завершился с кодом {tx_result.returncode}") + + rx_returncode = rx_process.wait(RX_WAIT_TIMEOUT_SECONDS) + if rx_returncode != 0: + raise RuntimeError(f"RX завершился с кодом {rx_returncode}") + expected_size = CAPTURE_SAMPLE_COUNT * 2 + if rx_path.stat().st_size != expected_size: + raise RuntimeError("Размер raw не совпал с числом запрошенных samples") + raw_sha_first = lab043._sha256_file(rx_path) + raw_sha_second = lab043._sha256_file(rx_path) + if raw_sha_first != raw_sha_second: + raise RuntimeError("SHA-256 raw изменился после закрытия") + rx_log = rx_stdout_path.read_text(encoding="utf-8", errors="replace") + rx_stderr_path.read_text( + encoding="utf-8", errors="replace" + ) + tx_log = (tx_result.stdout or "") + (tx_result.stderr or "") + if not lab043._transfer_log_is_clean(rx_log): + raise RuntimeError("RX log содержит overrun/underrun") + if not lab043._transfer_log_is_clean(tx_log): + raise RuntimeError("TX log содержит overrun/underrun") + + samples = lab043.load_cs8(rx_path) + reception = analyze_then_compare( + samples, + tx_payload_path, + tx_packet_path, + analyzer=analyzer, + comparer=comparer, + ) + if reception.packet_bytes is not None: + (output_directory / "rx_packet.bin").write_bytes(reception.packet_bytes) + if reception.payload_bytes is not None: + (output_directory / "rx_payload.bin").write_bytes(reception.payload_bytes) + + metadata = { + "payload_bytes": len(plan.payload), + "protocol_packet_bytes": len(plan.packet), + "phy_header_bits": PHY_HEADER_BIT_COUNT, + "total_frame_symbols": plan.frame_symbol_count, + "total_waveform_samples": plan.waveform_sample_count, + "tx_duration_seconds": plan.waveform_sample_count / SAMPLE_RATE_HZ, + "bit_order": "MSB first", + "byte_order": "big-endian/network order", + "max_packet_bytes": MAX_PACKET_BYTES, + "payload_sha256": plan.payload_sha256, + "packet_sha256": plan.packet_sha256, + "waveform_sha256": plan.waveform_sha256, + "raw": { + "bytes": expected_size, + "complex_samples": CAPTURE_SAMPLE_COUNT, + "duration_seconds": CAPTURE_SAMPLE_COUNT / SAMPLE_RATE_HZ, + "sha256_first": raw_sha_first, + "sha256_second": raw_sha_second, + }, + "rf": { + "center_hz": RF_HZ, + "sample_rate_hz": SAMPLE_RATE_HZ, + "symbol_rate": SYMBOL_RATE, + "digital_carrier_hz": CARRIER_OFFSET_HZ, + "tx_vga_db": TX_VGA_DB, + "rx_lna_db": RX_LNA_DB, + "rx_vga_db": RX_VGA_DB, + "rf_amp": False, + "antenna_power": False, + }, + "preflight": preflight, + "rf_tx_count": guard.transmit_calls, + "artifact_sha256": _artifact_sha256( + output_directory, + [ + "tx_payload.bin", + "tx_packet.bin", + "tx_bpsk_cs8.iq", + "rx_cs8.iq", + "rx_packet.bin", + "rx_payload.bin", + ], + ), + } + (output_directory / "metadata.json").write_text( + json.dumps(metadata, ensure_ascii=False, indent=2), encoding="utf-8" + ) + (output_directory / "analysis.json").write_text( + json.dumps(reception.analysis, ensure_ascii=False, indent=2), encoding="utf-8" + ) + return { + "classification": reception.analysis["classification"], + "analysis": reception.analysis, + "metadata": metadata, + "output_directory": str(output_directory), + } + finally: + if rx_process is not None: + try: + rx_process.terminate() + except Exception: + pass + lab043._disable_clock_outputs(runner) + + +def main() -> int: + parser = argparse.ArgumentParser(description="Lab044 arbitrary BPSK packet + CRC") + parser.add_argument("--hardware", action="store_true") + parser.add_argument("--i-understand-hardware", action="store_true") + parser.add_argument("--output-directory", type=Path) + arguments = parser.parse_args() + if not arguments.hardware: + parser.print_help() + return 0 + if not arguments.i_understand_hardware: + print("Аппаратный режим заблокирован без --i-understand-hardware") + return 2 + result = run_hardware_experiment(arguments.output_directory or default_output_directory()) + print(json.dumps(result, ensure_ascii=False, indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_lab044_hackrf_bpsk_packet.py b/tests/test_lab044_hackrf_bpsk_packet.py new file mode 100644 index 0000000..e28fa1a --- /dev/null +++ b/tests/test_lab044_hackrf_bpsk_packet.py @@ -0,0 +1,506 @@ +"""Синтетические и fail-closed проверки Lab044 без оборудования.""" + +from __future__ import annotations + +import hashlib +import json +import subprocess +import sys +from pathlib import Path + +import numpy as np +import pytest + +from experiments import lab043_hackrf_bpsk as lab043 +from experiments import lab044_hackrf_bpsk_packet as lab044 +from protocol.packet import CRCError, parse_packet + + +def deterministic_payload() -> bytes: + return bytes(range(256)) + + +def test_bytes_bits_round_trip_is_explicit_msb_first() -> None: + random_bytes = np.random.default_rng(44).integers(0, 256, 1024, dtype=np.uint8).tobytes() + bits = lab044.bytes_to_bits(random_bytes) + assert bits[:8].tolist() == [int(bit) for bit in f"{random_bytes[0]:08b}"] + assert lab044.bits_to_bytes(bits) == random_bytes + + +def test_phy_header_round_trip_and_valid_length() -> None: + decoded = lab044.decode_phy_header( + lab044.bytes_to_bits(lab044.encode_phy_header(268)) + ) + assert decoded == {"passed": True, "reason": None, "sync": 0xD391, "packet_bytes": 268} + + +def test_phy_header_rejects_zero_length() -> None: + header = lab044.bytes_to_bits(struct_pack_header(0)) + assert lab044.decode_phy_header(header)["reason"] == "LENGTH_OUT_OF_RANGE" + + +def test_phy_header_rejects_excessive_length() -> None: + header = lab044.bytes_to_bits(struct_pack_header(lab044.MAX_PACKET_BYTES + 1)) + assert lab044.decode_phy_header(header)["reason"] == "LENGTH_OUT_OF_RANGE" + + +def struct_pack_header(length: int) -> bytes: + import struct + + return struct.pack(lab044.PHY_HEADER_FORMAT, lab044.PHY_SYNC, length) + + +def test_truncated_phy_header_is_safe() -> None: + result = lab044.decode_phy_header(np.zeros(15, dtype=np.uint8)) + assert result["passed"] is False + assert result["reason"] == "TRUNCATED" + + +def test_arbitrary_payload_packet_and_frame_round_trip() -> None: + plan = lab044.build_transmission_plan(deterministic_payload()) + assert len(plan.packet) == 268 + assert plan.frame_symbol_count == 3456 + assert plan.waveform_sample_count == 1_399_200 + assert parse_packet(plan.packet).payload == deterministic_payload() + + +def test_crc_pass_and_one_payload_bit_crc_fail() -> None: + plan = lab044.build_transmission_plan(deterministic_payload()) + assert parse_packet(plan.packet).payload == deterministic_payload() + corrupted = bytearray(plan.packet) + corrupted[lab044.HEADER_SIZE + 7] ^= 0x01 + with pytest.raises(CRCError): + parse_packet(bytes(corrupted)) + + +def synthetic_capture(phase_rad: float = 0.0, cfo_hz: float = 0.0) -> tuple[np.ndarray, lab044.TransmissionPlan]: + plan = lab044.build_transmission_plan(deterministic_payload()) + tx = lab044.signed_iq_bytes_to_complex64(plan.tx_cs8) + indexes = np.arange(len(tx), dtype=np.float64) + impaired = tx * np.exp(1j * (phase_rad + 2 * np.pi * cfo_hz * indexes / lab044.SAMPLE_RATE_HZ)) + capture = np.concatenate(( + np.zeros(60_000, dtype=np.complex64), + impaired.astype(np.complex64), + np.zeros(60_000, dtype=np.complex64), + )) + return capture, plan + + +def test_ideal_bpsk_receiver_recovers_unknown_payload_and_crc() -> None: + capture, plan = synthetic_capture() + reception = lab044.analyze_capture_independently(capture) + assert reception.analysis["phy_sync_pass"] is True + assert reception.analysis["frame_header_pass"] is True + assert reception.analysis["packet_received"] is True + assert reception.analysis["protocol_header_pass"] is True + assert reception.analysis["crc"] == lab044.CRC_PASS + assert reception.payload_bytes == plan.payload + + +def test_polarity_inversion_is_resolved_only_by_preamble() -> None: + capture, plan = synthetic_capture(phase_rad=np.pi) + reception = lab044.analyze_capture_independently(capture) + assert reception.analysis["crc"] == lab044.CRC_PASS + assert reception.payload_bytes == plan.payload + + +def test_synthetic_cfo_and_timing_case() -> None: + capture, plan = synthetic_capture(phase_rad=0.7, cfo_hz=0.2) + reception = lab044.analyze_capture_independently(capture) + assert reception.analysis["crc"] == lab044.CRC_PASS + assert reception.payload_bytes == plan.payload + assert reception.analysis["detection"]["payload_used"] is False + + +def test_truncated_protocol_packet_is_flagged_without_exception() -> None: + """Gap A: valid preamble + valid PHY header + physically cut-off packet body. + + The capture array itself ends partway through the protocol packet (no + trailing zero padding), so the demodulated bit stream is genuinely + shorter than PHY LENGTH implies. This must be reported as a structured + FAIL (TRUNCATED_PACKET) with CRC NOT_REACHED -- never as an exception + and never as a fabricated packet/payload. + """ + + plan = lab044.build_transmission_plan(deterministic_payload()) + full = lab044.signed_iq_bytes_to_complex64(plan.tx_cs8) + header_symbols = lab044.PREAMBLE_BIT_COUNT + lab044.PHY_HEADER_BIT_COUNT + # Preamble and PHY header fully present; only ~100 of 2144 packet bits follow. + partial_symbol_count = header_symbols + 100 + cut_sample = partial_symbol_count * lab044.SAMPLES_PER_SYMBOL + truncated = full[:cut_sample] + capture = np.concatenate(( + np.zeros(60_000, dtype=np.complex64), + truncated.astype(np.complex64), + )) + + reception = lab044.analyze_capture_independently(capture) + analysis = reception.analysis + + assert analysis["phy_sync_pass"] is True + assert analysis["frame_header_pass"] is False + assert analysis["packet_received"] is False + assert analysis["frame_header"]["reason"] == "TRUNCATED_PACKET" + assert analysis["crc"] == lab044.CRC_NOT_REACHED + assert analysis["classification"] == "C" + assert reception.packet_bytes is None + assert reception.payload_bytes is None + + +def test_capture_intervals_present_for_independently_decoded_frame() -> None: + """Gap B: BEFORE/DURING/AFTER diagnostics, bounded by the independent start.""" + + capture, plan = synthetic_capture() + reception = lab044.analyze_capture_independently(capture) + intervals = reception.analysis["capture_intervals"] + + assert intervals["frame_start_sample"] == 60_000 + assert intervals["frame_end_sample"] > intervals["frame_start_sample"] + for region in ("before", "during", "after"): + stats = intervals[region] + assert stats is not None + assert "sample_range" in stats + assert "duration_seconds" in stats + assert "mean_i_code" in stats + assert "mean_q_code" in stats + assert "rms_after_dc_removal_code" in stats + assert "amplitude_p95_code" in stats + assert "amplitude_p99_code" in stats + assert "amplitude_p999_code" in stats + assert "peak_complex_code" in stats + assert "clipping_count" in stats + assert "clipping_fraction" in stats + assert "декодированной PHY LENGTH" in intervals["note"] + + +def test_capture_intervals_undefined_when_no_independent_start( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Gap B: with no independent preamble, DURING must stay undefined, not guessed.""" + + def never_detects(samples: np.ndarray) -> dict: + return { + "preamble_detected": False, + "method": "stub", + "payload_used": False, + "peak": 0.0, + "median": 0.0, + "over_median_db": 0.0, + "over_second_db": 0.0, + "coarse_start_sample": 0, + "refined_start_sample": 0, + } + + monkeypatch.setattr(lab044, "_detect_preamble", never_detects) + capture = np.zeros(50_000, dtype=np.complex64) + reception = lab044.analyze_capture_independently(capture) + intervals = reception.analysis["capture_intervals"] + + assert intervals["before"] is None + assert intervals["during"] is None + assert intervals["after"] is None + assert intervals["whole_capture"] is not None + assert "не найден" in intervals["note"] + + +def test_receiver_does_not_read_payload_before_diagnostic(tmp_path: Path) -> None: + events: list[str] = [] + payload_path = tmp_path / "tx_payload.bin" + packet_path = tmp_path / "tx_packet.bin" + payload_path.write_bytes(deterministic_payload()) + packet_path.write_bytes(b"packet") + reception = lab044.IndependentReception({"crc": lab044.CRC_FAIL}, b"rx", b"payload") + + def analyzer(samples: np.ndarray) -> lab044.IndependentReception: + del samples + events.append("independent_decode") + return reception + + def comparer( + result: lab044.IndependentReception, + tx_payload_path: Path, + tx_packet_path: Path, + ) -> dict: + assert result.analysis["crc"] == lab044.CRC_FAIL + events.append("diagnostic_read") + assert tx_payload_path.read_bytes() == deterministic_payload() + assert tx_packet_path.read_bytes() == b"packet" + return {"stage": "AFTER_INDEPENDENT_CRC_DECISION"} + + lab044.analyze_then_compare( + np.zeros(1, dtype=np.complex64), + payload_path, + packet_path, + analyzer=analyzer, + comparer=comparer, + ) + assert events == ["independent_decode", "diagnostic_read"] + + +class FakeRunner: + def __init__(self) -> None: + self.commands: list[list[str]] = [] + + def __call__(self, command: list[str]) -> subprocess.CompletedProcess: + self.commands.append(command) + return subprocess.CompletedProcess(command, 0, "", "") + + +class FakeProcess: + def __init__(self, startup_failed: bool = False) -> None: + self.startup_failed = startup_failed + self.terminated = False + + def poll(self) -> int | None: + return 1 if self.startup_failed else (0 if self.terminated else None) + + def wait(self, timeout: float) -> int: + del timeout + return 0 + + def terminate(self) -> None: + self.terminated = True + + def close_logs(self) -> None: + pass + + +class FakeStarter: + def __init__(self, startup_failed: bool = False) -> None: + self.startup_failed = startup_failed + self.processes: list[FakeProcess] = [] + + def __call__(self, command: list[str], stdout_path: Path, stderr_path: Path) -> FakeProcess: + stdout_path.write_text("0 overruns\n", encoding="utf-8") + stderr_path.write_text("", encoding="utf-8") + raw_path = Path(command[command.index("-r") + 1]) + raw_path.write_bytes(bytes(2 * lab044.CAPTURE_SAMPLE_COUNT)) + process = FakeProcess(self.startup_failed) + self.processes.append(process) + return process + + +def fake_reception() -> lab044.IndependentReception: + return lab044.IndependentReception( + { + "classification": "A", + "crc": lab044.CRC_PASS, + "phy_sync_pass": True, + "frame_header_pass": True, + "packet_received": True, + "protocol_header_pass": True, + }, + b"packet", + b"payload", + ) + + +def allow_preflight( + runner: lab043.CommandRunner, + guard: lab043.TransmitGuard, +) -> dict: + del runner + guard.allow() + return {"passed": True} + + +def test_common_clock_failure_means_zero_tx(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + runner = FakeRunner() + monkeypatch.setattr( + lab043, + "common_clock_preflight", + lambda runner, guard: (_ for _ in ()).throw(RuntimeError("clock")), + ) + with pytest.raises(RuntimeError, match="clock"): + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + payload_factory=lambda count: bytes(count), + ) + assert not any("-t" in command for command in runner.commands) + + +def test_frame_validation_failure_means_zero_tx(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + runner = FakeRunner() + monkeypatch.setattr( + lab044, + "validate_transmission_plan", + lambda plan: (_ for _ in ()).throw(ValueError("frame")), + ) + with pytest.raises(ValueError, match="frame"): + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + payload_factory=lambda count: bytes(count), + ) + assert runner.commands == [] + + +def test_rx_startup_failure_means_zero_tx(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(lab044, "CAPTURE_SAMPLE_COUNT", 16) + monkeypatch.setattr(lab043, "common_clock_preflight", allow_preflight) + runner = FakeRunner() + starter = FakeStarter(startup_failed=True) + with pytest.raises(RuntimeError, match="RX завершился"): + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + process_starter=starter, + sleep_function=lambda seconds: None, + payload_factory=lambda count: bytes(count), + ) + assert not any("-t" in command for command in runner.commands) + assert starter.processes[0].terminated is True + + +def test_one_tx_and_exception_cleanup(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(lab044, "CAPTURE_SAMPLE_COUNT", 16) + monkeypatch.setattr(lab043, "common_clock_preflight", allow_preflight) + cleanup_calls: list[bool] = [] + monkeypatch.setattr( + lab043, + "_disable_clock_outputs", + lambda runner: cleanup_calls.append(True) or [], + ) + runner = FakeRunner() + starter = FakeStarter() + + def analyzer(samples: np.ndarray) -> lab044.IndependentReception: + del samples + raise RuntimeError("analysis") + + with pytest.raises(RuntimeError, match="analysis"): + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + process_starter=starter, + sleep_function=lambda seconds: None, + payload_factory=lambda count: bytes(count), + analyzer=analyzer, + ) + assert sum("-t" in command for command in runner.commands) == 1 + assert starter.processes[0].terminated is True + assert cleanup_calls == [True] + + +def test_transmit_guard_rejects_second_tx() -> None: + guard = lab043.TransmitGuard() + guard.allow() + guard.register_transmit() + with pytest.raises(RuntimeError, match="Повторная передача"): + guard.register_transmit() + + +def test_artifact_sha256_recorded_for_every_real_rx_and_tx_file( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """Gap C: every artifact that actually exists on disk gets a real SHA-256.""" + + monkeypatch.setattr(lab044, "CAPTURE_SAMPLE_COUNT", 16) + monkeypatch.setattr(lab043, "common_clock_preflight", allow_preflight) + runner = FakeRunner() + starter = FakeStarter() + reception = fake_reception() + + def analyzer(samples: np.ndarray) -> lab044.IndependentReception: + del samples + return reception + + def comparer( + result: lab044.IndependentReception, + tx_payload_path: Path, + tx_packet_path: Path, + ) -> dict: + del result, tx_payload_path, tx_packet_path + return {"stage": "AFTER_INDEPENDENT_CRC_DECISION"} + + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + process_starter=starter, + sleep_function=lambda seconds: None, + payload_factory=lambda count: bytes(count), + analyzer=analyzer, + comparer=comparer, + ) + + metadata = json.loads((tmp_path / "metadata.json").read_text(encoding="utf-8")) + hashes = metadata["artifact_sha256"] + for name in ( + "tx_payload.bin", + "tx_packet.bin", + "tx_bpsk_cs8.iq", + "rx_cs8.iq", + "rx_packet.bin", + "rx_payload.bin", + ): + path = tmp_path / name + assert path.is_file(), name + assert hashes[name] == hashlib.sha256(path.read_bytes()).hexdigest() + + +def test_no_fake_rx_artifacts_or_hashes_when_packet_not_decoded( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """Gap C: never fabricate rx_packet.bin/rx_payload.bin, never fabricate their hash.""" + + monkeypatch.setattr(lab044, "CAPTURE_SAMPLE_COUNT", 16) + monkeypatch.setattr(lab043, "common_clock_preflight", allow_preflight) + runner = FakeRunner() + starter = FakeStarter() + reception = lab044.IndependentReception( + { + "classification": "D", + "crc": lab044.CRC_NOT_REACHED, + "phy_sync_pass": False, + "frame_header_pass": False, + "packet_received": False, + "protocol_header_pass": False, + }, + None, + None, + ) + + def analyzer(samples: np.ndarray) -> lab044.IndependentReception: + del samples + return reception + + def comparer( + result: lab044.IndependentReception, + tx_payload_path: Path, + tx_packet_path: Path, + ) -> dict: + del result, tx_payload_path, tx_packet_path + return {"stage": "AFTER_INDEPENDENT_CRC_DECISION"} + + lab044.run_hardware_experiment( + tmp_path, + runner=runner, + process_starter=starter, + sleep_function=lambda seconds: None, + payload_factory=lambda count: bytes(count), + analyzer=analyzer, + comparer=comparer, + ) + + assert not (tmp_path / "rx_packet.bin").exists() + assert not (tmp_path / "rx_payload.bin").exists() + metadata = json.loads((tmp_path / "metadata.json").read_text(encoding="utf-8")) + hashes = metadata["artifact_sha256"] + assert hashes["rx_packet.bin"] is None + assert hashes["rx_payload.bin"] is None + assert hashes["rx_cs8.iq"] is not None + assert hashes["tx_payload.bin"] is not None + + +def test_direct_invocation_smoke() -> None: + project_root = Path(__file__).resolve().parents[1] + result = subprocess.run( + [sys.executable, str(project_root / "experiments" / "lab044_hackrf_bpsk_packet.py"), "--help"], + cwd=project_root, + capture_output=True, + text=True, + timeout=30, + ) + assert result.returncode == 0, result.stderr + assert "--hardware" in result.stdout