Troubleshooting
Best Practices
CC Errors on Failover Channel (No-Merge/Stream Switch)
8 min
this applies to failover channels configured in no merge mode in this mode, the channel carries a single active source at a time and performs a hard cut when switching cc errors on the failover output in this configuration are generally expected and fall into two distinct categories root causes 1\ switch correlated cc errors (expected behavior) in a no merge failover, any source switch adds a discontinuity in the mpeg ts continuity counter the outgoing stream transitions abruptly from one source's counter state to another when this happens the counters will not be in sync, and a cc error will occur on each switch event note you may see several cc errors as each pid has its own counter this is not a defect it is the inherent nature of hard cut switching there is no way to eliminate this without moving to a merge/redundancy (2022 7) configuration, which can stitch streams together at the packet level 2022 7 can only be used with binary identical rtp streams mitigation error concealment (ec) if the cc error at switch time is causing downstream issues (e g alerting systems, decoders), the following ec options can mask the problem continuous timeline makes the output stream appear as a single continuous timeline across switch events, reducing visible discontinuity to downstream equipment renumber cc renumbers continuity counters on output to suppress cc error reporting effectively masks counter breaks induced by the switch error concealment configuration menu these are cosmetic fixes only, the underlying switch event still occurs but downstream detection is suppressed 2\ source p1 errors passing through (configuration note) when the channel is not configured to switch on p1 error events this means if a cc error exists in the active source feed, it will be passed through to the failover output without triggering a switch if the goal is to fail away from a source that is generating cc errors, enable p1 based switching note this should only be used to switch when there are excessive cc errors which can be defined in the configuration options be aware that the switch itself will also introduce a cc error (see above) options when failover on p1 is switched on 3\ jitter introducing cc errors (transport layer) stream inputs can introduce packet timing jitter that does not manifest on the source monitoring points, but does appear after the failover channel's processing path this explains cases where cc errors appear on the failover output but not on the individual stream inputs mitigation increase the search window (latency buffer) on the failover channel it needs to be set at >50ms at all times, ideally be more in the 500ms 1s range unless latency is a major decision point a wider window gives the receiver more room to reorder and recover packets before they're passed downstream, which can reduce or eliminate jitter induced cc errors decision tree cc errors seen on failover channel output do errors correlate with source switch events? → yes → expected behavior for no merge failover enable ec with continuous timeline + renumber cc to suppress are cc errors also present on the stream source inputs? → yes → source feed issue consider enabling p1 error based switching to fail away from a bad source cc errors on failover output but not on stream inputs? → yes → likely jitter introduced in the failover path increase the search window/latency buffer no clear cause found → capture tr 101 290 data at both stream input and failover output and escalate to zixi support escalation checklist when escalating to zixi support, include the following zen master channel link and broadcaster version(s) involved channel name and failover mode (confirm no merge vs 2022 7 merge) stream input names and connection details (listener/caller, latency settings, search window size) whether ec is enabled and which options are active (continuous timeline, renumber cc) whether p1 error based switching is enabled or disabled timestamp and time zone of cc error events tr 101 290 error counts at stream inputs vs failover output source event log and channel event log export covering the error window any recent changes to channel config, stream settings, or source routing configuration summary setting current state recommendation failover mode no merge (hard cut) acceptable; 2022 7 merge eliminates switch cc errors entirely switch on p1 errors disabled enable if you want automatic failover on source cc errors error concealment not confirmed enabled enable with continuous timeline + renumber cc to suppress switch errors stream search window default/narrow widen if jitter induced cc errors are suspected notes moving to 2022 7 merge mode is the only way to fully eliminate switch correlated cc errors ec options suppress reporting but they do not prevent the underlying counter discontinuity if alerting thresholds are being hit by switch correlated cc errors, ec with renumber cc is the fastest resolution to mitigate for jitter investigation, compare tr 101 290 p1 cc error timestamps on the source inputs versus the failover output to confirm possible jitter causes before widening the search window
