1. The checklists that kept them alive

Mary Robinette Kowal’s The Calculating Stars is full of checklists. The astronauts train until the procedures are reflex, because in an emergency there is no time to think from first principles. The ground controllers have their own checklists, and the two teams work through them together. The desktop has no crew, but it does have software and operators, and both need checklists for when things go wrong.

Entry 267 defined flight software and autonomy. This entry writes the procedures and anomaly responses that the software and operators will follow.

2. Normal procedures

Normal operations are scripted in advance. A few examples:

  • Daily schedule upload: verify the schedule, check constraints, uplink, confirm receipt.
  • Imaging campaign: point, configure the imager, expose, store, downlink metadata.
  • Momentum dump: monitor wheel speeds, enable magnetorquers, confirm rate reduction, resume normal pointing.
  • Ground pass: acquire signal, downlink telemetry, uplink commands, verify execution, release signal.
  • Eclipse entry and exit: reduce loads before entry, monitor battery, restore loads after exit.

These procedures are boring by design. Boring procedures are the ones that work.

3. Anomaly classification

Anomalies are sorted by severity:

  • Class C — informational: a parameter is outside its nominal range but within acceptable limits. Log and monitor.
  • Class B — warning: a parameter is approaching a limit or a redundant unit has failed. Investigate and plan a response.
  • Class A — critical: a parameter has exceeded a limit or a function is lost. Respond immediately, often by entering safe mode.

Class C anomalies can wait for the next ground pass. Class B anomalies need attention within the current contact or the next one. Class A anomalies trigger automatic onboard action and an immediate ground alert.

4. Common anomaly procedures

A small set of procedures covers most of what can go wrong in early operations:

Loss of signal

  1. Wait for the predicted pass time.
  2. If no signal, try backup frequencies and ground stations.
  3. Check spacecraft health from the last telemetry received.
  4. If the spacecraft is likely in safe mode, wait for the beacon.
  5. If still no signal, review recent commands for an inadvertent configuration change.

Safe mode entry

  1. Confirm safe mode via telemetry or beacon.
  2. Let the spacecraft stabilise in Sun acquisition.
  3. Assess power and thermal state.
  4. Retrieve event logs to identify the trigger.
  5. Plan recovery commands; do not rush.
  6. Uplink recovery sequence during the next contact.

Reaction wheel saturation

  1. Check wheel speeds and momentum trend.
  2. Verify magnetorquers are enabled and functioning.
  3. Command a momentum dump if not already in progress.
  4. If magnetorquers cannot dump momentum, consider a small thruster firing.

Power bus undervoltage

  1. Identify the cause: eclipse overdraft, high load, or battery degradation.
  2. Shed non-essential loads.
  3. Enter safe mode if voltage continues to drop.
  4. Review power budget and update schedule.

Thermal limit exceeded

  1. Determine which component is out of limits.
  2. Adjust attitude or heaters to bring it back into range.
  3. If a radiator is blocked or a heater stuck on, enter safe mode.

5. Command verification

Before any command is sent, the operator should confirm:

  • The command is intended for this spacecraft.
  • The command is valid in the current mode.
  • The command will not violate a constraint.
  • The expected result is clear.
  • A rollback path exists if the command fails.

Flight software should enforce its own checks: command checksum, sequence number, mode validity, and parameter limits. The ground and the spacecraft both act as guards.

6. The anomaly log

Every anomaly should be logged with:

  • Timestamp.
  • Affected subsystem and parameter.
  • Class and trigger condition.
  • Onboard response, if any.
  • Ground response and outcome.
  • Root cause, once known.

This log becomes the basis for improving procedures and software. A spacecraft that does not learn from its anomalies is more likely to repeat them.

7. Operations rehearsal

Procedures are only useful if people remember them. The operations team should rehearse:

  • Launch and early operations.
  • Safe mode recovery.
  • Loss of signal.
  • Payload anomaly.
  • Propulsion anomaly.
  • End-of-life disposal.

Rehearsal can be done with a simulator, a hardware-in-the-loop testbed, or tabletop exercises. The desktop’s compute attachment can run a software simulator for operator training.

What this changes

  • Normal operations are scripted as repeatable procedures.
  • Anomalies are classified by severity: informational, warning, critical.
  • A small set of common procedures covers loss of signal, safe mode, wheel saturation, undervoltage, and thermal limits.
  • Commands are verified both on the ground and onboard.
  • Every anomaly is logged for future learning.
  • Operations rehearsals are part of mission readiness.
  • The next entry will close the operations arc.