18 OpenSynaptic Standard Data Format Definition (Normative)

This document provides strict definitions for the standard data formats used in the current OSynaptic-FX implementation. Unless otherwise stated, all multi-byte integers are in big-endian byte order.

A.1 Data Frame (DATA_*) Wire Format

Applicable commands: DATA_FULL(63), DATA_DIFF(170), DATA_HEART(127) and their secure variants (64/171/128).

Offset	Length	Field	Definition
0	1	cmd	Command byte
1	1	route_count	Fixed to 1 in current implementation
2	4	source_aid	Sender AID (u32)
6	1	tid	Transaction/channel identifier (u8)
7	6	timestamp_raw	Raw timestamp (u48)
13	N	body	Business payload
13+N	1	crc8	CRC8 over body
14+N	2	crc16	CRC16/CCITT over all preceding bytes (including crc8)

Receiver must verify crc16 first and reject on failure; if timestamp protection is enabled, must also perform replay/out-of-order checks.

A.2 Command Byte Definition

Category	Command	Value
Data	DATA_FULL	63
Data	DATA_FULL_SEC	64
Data	DATA_DIFF	170
Data	DATA_DIFF_SEC	171
Data	DATA_HEART	127
Data	DATA_HEART_SEC	128
Control	ID_REQUEST	1
Control	ID_ASSIGN	2
Control	HANDSHAKE_ACK	5
Control	HANDSHAKE_NACK	6
Control	PING	9
Control	PONG	10
Control	TIME_REQUEST	11
Control	TIME_RESPONSE	12
Control	SECURE_DICT_READY	13
Control	SECURE_CHANNEL_ACK	14

A.3 Single Sensor Business Payload Specification

Body specification generated by osfx_core_encode_sensor_packet*:

sensor_id|unit|b62_value

Constraints:

• sensor_id: Sensor identifier string.
• unit: Standardized unit (e.g., Pa, K).
• b62_value: Base62-encoded result of round(standardized_value * 10000) as int64.

A.4 Control Frame Minimal Structure (Common)

• ID_REQUEST: [cmd:1][seq:2] (minimum 3 bytes, optional JSON device-meta appended)
• ID_ASSIGN: [cmd:1][seq:2][assigned_id:4] (base 7 bytes)
  • Optional extension (backward-compatible): server MAY append [server_time:8] (u64 BE unix timestamp) making total 15 bytes. Old/simple clients read only the first 7 bytes and ignore the extension. Use osfx_parse_id_assign() to safely parse both variants.
• HANDSHAKE_NACK: [cmd:1][seq:2][reason:utf8-bytes]
• TIME_REQUEST: [cmd:1][seq:2]
• TIME_RESPONSE: [cmd:1][seq:2][unix_ts:8] (u64 BE)

A.5 Secure Semantics

• Secure data cmd must map to corresponding base data cmd for semantic processing.
• If session is not established and secure data is received, receiver should reject (NO_SESSION semantics).
• Timestamp check returns: ACCEPT / REPLAY / OUT_OF_ORDER; latter two must be rejected.

A.6 Implementation Consistency Requirements

• Sender must frame according to order and encoding rules in this section.
• Receiver must verify frames in this section's verification order (length/CRC/sequence).
• When any field violates constraints, must return error or NACK, should not silently drop.

A.7 Required Fields for Data Transmission (Sender MUST)

Sender must provide the following fields when constructing DATA_* frames, none can be omitted:

• cmd: Must be one of defined data commands (63/170/127 or secure variants).
• route_count: Current implementation must write 1.
• source_aid: Must be valid device ID (u32).
• tid: Must be provided (u8).
• timestamp_raw: Must be provided (u48, recommended monotonically increasing).
• body: Must exist and be encoded per business payload specification.
• crc8: Must be calculated based on body before writing.
• crc16: Must be calculated based on all preceding bytes before writing.

A.8 Field Values and Boundaries

• cmd: Only values defined in command table allowed; unknown values must not be sent.
• route_count: Fixed to 1.
• source_aid: 0..4294967295 (engineering practice recommends avoiding 0).
• tid: 0..255.
• timestamp_raw: 0..2^48-1 (wire format); recommended to use unified second-level or millisecond-level calibration.
• body_len: Must satisfy implementation buffer limit; excess must be rejected.

A.9 Frame Construction Order (Sender's Unique Order)

1. Complete business data standardization and body encoding first.
2. Calculate crc8 over body.
3. Write header fields according to A.1 definition (cmd/route_count/source_aid/tid/timestamp_raw).
4. Concatenate body + crc8.
5. Calculate crc16 over current segment (header + body + crc8) and append.
6. Obtain final sendable binary frame.

Any deviation from this order may cause receiver's CRC verification to fail.

A.10 Pre-Send Checklist

Execute at least the following checks before sending:

• Command legality check (whether cmd is in defined set).
• Timestamp validity check (empty/rollback/abnormal jump).
• Buffer capacity check (avoid truncated write).
• crc8/crc16 self-check (optional secondary calculation comparison).
• Secure scenario session check (whether dict_ready/key_set are satisfied).

Any check failure must abort sending and return error.

A.11 Error Handling Specification (Sender Side)

• Missing/invalid fields: Return parameter error (recommend mapping OSFX_GLUE_ERR_ARG).
• Encoding failure: Return codec error (recommend mapping OSFX_GLUE_ERR_CODEC).
• Runtime dependency failure (session/plugin/routing unavailable): Return runtime error (recommend mapping OSFX_GLUE_ERR_RUNTIME).
• Prohibition on "sending with errors": Upon error, must reject sending, must not send incomplete or invalid frames.

A.12 User Input Data Contract (User Input Contract)

This section defines "what business/user side needs to provide", avoiding mistaking all wire format fields as user-provided.

A.12.1 osfx_glue_encode_sensor_auto(...) Path (Recommended)

User must provide:

• tid
• timestamp_raw
• sensor_id
• input_value
• input_unit

User does not need to provide (auto-generated or internally maintained by system):

• cmd (auto-determined by fusion_state to FULL/DIFF/HEART)
• route_count (fixed to 1)
• source_aid (from glue.local_aid)
• crc8 / crc16 (auto-calculated internally)

A.12.2 Low-level osfx_packet_encode_full/ex Path (Advanced)

If you directly call low-level packing API, cmd/source_aid/tid/timestamp/body are explicitly provided by caller; this mode is recommended only for protocol gateways or test tools, business side defaults to osfx_glue_encode_sensor_auto(...).

A.12.3 User Input Minimum Legal Set (Single Sensor)

Field	Required	Example	Notes
sensor_id	Yes	TEMP	Recommended short identifier, avoid delimiter |
input_value	Yes	23.5	Valid floating-point number
input_unit	Yes	cel / kPa / %	See docs/18-standardized-units.md
timestamp_raw	Yes	1710001000	Recommended monotonically increasing
tid	Yes	1	0..255

A.12.4 User Input Maximum Legal Set (Business Side)

Maximum legal set refers to "all fields that business side can completely provide and consumed by encoding chain under current implementation and configuration".

Single sensor path (osfx_glue_encode_sensor_auto(...)) maximum legal set:

Field	Required	Example	Notes
sensor_id	Yes	TEMP	Sensor identifier
input_value	Yes	23.5	Raw measurement value
input_unit	Yes	cel / kPa	Standardization input unit
timestamp_raw	Yes	1710001000	Recommended monotonically increasing
tid	Yes	1	Sub-template/transaction slot

Multi-sensor path (osfx_core_encode_multi_sensor_packet_auto(...)) maximum legal set:

Field	Required	Example	Notes
node_id	Yes	NODE_A	Device identifier
node_state	Yes	ONLINE	Device status
timestamp_raw	Yes	1710001234	Timestamp
tid	Yes	2	Sub-template/transaction slot
sensor[i].sensor_id	Yes	TEMP1	Required for each sensor
sensor[i].sensor_state	Yes	OK	Required for each sensor
sensor[i].value	Yes	23.5	Required for each sensor
sensor[i].unit	Yes	cel	Required for each sensor
sensor[i].geohash_id	Optional	wx4g0ec1	Encoded in template when GeohashId=true
sensor[i].supplementary_message	Optional	msg_temp1	Encoded in template when SupplementaryMessage=true
sensor[i].resource_url	Optional	https://r/1	Encoded in template when ResourceUrl=true

A.12.5 Items Requiring Both Server and Client Simultaneous Enablement (Strong Consistency)

If the following switches are inconsistent between the two ends, field semantic inconsistency will result (even misinterpretation after decoding), recommend server and client simultaneously enable or disable:

• DeviceId (node/source identity semantics)
• DeviceStatus (node_state semantics)
• Timestamp (time sequence semantics)
• SubTemplateId (tid semantics)
• SensorId (sensor mapping semantics)
• SensorStatus (status semantics)
• PhysicalAttribute (unit field semantics)
• NormalizedValue (value field semantics)

Security and control plane related items also recommended consistent on both sides:

• Secure session enablement strategy (secure_session related processes)
• Timestamp anti-replay/out-of-order strategy
• ID allocation control frame handling strategy (ID_REQUEST/ID_ASSIGN/NACK)

A.12.6 Extended Field Wiring Status (Current)

The following fields have been integrated into multi-sensor template encoding chain, controlled by payload_switches compile-time switches:

• GeohashId
• SupplementaryMessage
• ResourceUrl

Recommend server and client maintain consistent switches to avoid "field exists on sending side, receiving side processes as empty field" semantic divergence.

A.12.7 OpenSynaptic Strict Compatibility Recommendation (Important)

If target is strict compatibility with original OpenSynaptic send API, recommend using standard 4-tuple input calibration:

[sensor_id, sensor_status, value, unit]

And disable extended field switches in compatible chain:

• GeohashId = false
• SupplementaryMessage = false
• ResourceUrl = false

This avoids parser incompatibility on receiving side caused by extended template fields.

Related Documentation

• Integration explanation: docs/17-glue-step-by-step.md sections 0-11
• Data processing pipeline: ../DATA_FORMATS_SPEC.md
• Standardized units: docs/19-input-specification.md or corresponding standardized units documentation
• Implementation examples: docs/16-examples-cookbook.md