Adding a Source – Failover

you can combine two or more sources to create a unified failover source whenever one of the sources is disrupted (or interrupted completely), zen master will seamlessly merge the streams, retrieving the missing packets from the alternate source the streams can be merged using one of the following methods stream switch zen master treats the streams as different and does not try to merge them if packets are missing from one stream, zen master will switch to the alternate stream content hitless merge if the sources do not have rtp headers, zixi will merge the streams using the mpeg ts payload content itself this method uses zixi's patented dna like sequencing algorithm to fill in the missing packets smpte 2022 7 hitless if the sources use rtp, and the rtp headers were maintained, zixi can use the smpte 2022 7 to compare and merge the two streams this option uses the sequence and the timestamp in the packets to compare which packets are missing, providing optimal results if the streams are not identical, the failover process will not be seamless and may take up to a second you can manually select one of the streams to be active by clicking on the lock button in the sources tab of the source details page the source designated as locked will be used whenever it is available until you click unlock zero bitrate and offline events always trigger failover for v15+ broadcasters, you can configure additional rules to failover for other content quality errors (e g p1 transport events, frozen video etc ) that occur beyond a specified threshold the following procedure describes how to add a failover source in zen master in addition, an explanation of each of the configuration parameters is given in the table below to add a failover source in the main navigation, click sources click + add the create new source selection window is displayed failover source settings click on failover the create new failover window opens failover source settings in the info section, in the name field, enter a unique name to identify this source in the system in the tags field, select one or more of the predefined tags from the drop down list tags are used for access control by selecting a tag, you are relating this source to that tag tags are associated with certain roles (through the users & roles screen), and roles, in turn, are associated with users and user groups if you would like to create a new tag, see creating a tag docid\ ypm9tw81ap6nq8stkv4rd in the events configuration profile field, select from the dropdown list the profile that you would like to apply to this source by default, the default profile is applied events profiles consist of a series of rules that determine precisely how various types of events are logged and what results they trigger the source's location is used to represent it in the proper position in the map display (see maps docid\ sfzm6jvorvuigo7x0dnue ) by default, the location is determined automatically based on the source's ip if you would like to manually set the location of the source (optional), use the following procedure click on the edit icon next to the location field a search box is shown below the location field enter identifying info for the location (e g street address, zip code, coordinates) search results are displayed below the search box as you type click on the desired location the selected location populates the location field in the input section, in the sources field, select two or more sources that you would like to include in the failover group if the selected sources are not binary identical then there will be a one second delay in the failover to the backup source as you select sources, they will appear in the table below the selector selected sources for each source, configure the following under priority to designate a source as a preferred source, select the preferred checkbox or to to designate a source as a slate, select the slate checkbox preferred source take precedence over non preferred source (i e , sources that were not marked as preferred) slate sources are sources that stream slate content, which is content that can be served when there are gaps in a live stream ad break that cannot be filled with a dynamically served ad under min bitrate (kbps) specify the bitrate threshold level in kbps to perform the failover (i e , if the bitrate drops under the threshold the failover will take place) if you would like to remove a selected source, click on the button next to the source in the max bitrate selector, leave the selection at auto to let zen master calculate the max bitrate for the failover source using the bitrates of the component sources or you can choose a max bitrate from the available options if you prefer for failover mode , select the desired mode for failing over/merging the streams options are stream switch – zen master treats the streams as different and does not try to merge them if packets are missing from one stream, zen master will switch to the alternate stream if the payload is not binary identical it will fall back to stream switch content hitless if the sources do not have rtp headers, zixi will merge the streams using the mpeg ts payload content itself this method uses zixi's patented dna like sequencing algorithm to fill in the missing packets smpte 2022 7 hitless if the sources use rtp, and the rtp headers were maintained, zixi can use the smpte 2022 7 to compare and merge the two streams this option uses the sequence and the timestamp in the packets to compare which packets are missing, providing optimal results if you have selected the content hitless or the smpte 2022 7 hitless option, in the merge search window \[ms] field, enter the time frame for which the system will search for the relevant packets in a matching stream (default 1000) if there is a latency difference between the streams, the search window value should be higher than the latency difference between the streams if merge mode is disabled, the time period specified in the search window field defines how long the stream will wait for packet before attempting to switch between streams if you have selected the stream switch option, in the zero bitrate/offline detection window \[ms] field, enter the time frame with zero bitrate or offline state after which the system will perform the failover failover rules allow you to set rules for what type of error and how often they occur to change to the failover stream if you would like to change the rules, use the following procedure (supported for broadcasters v15+) select the checkbox next to the error for which you would like to change the error rules options are p1 transport errors, frozen video, blank picture, silent audio, and low bitrate ( zero bitrate/offline is set as selected on a fixed basis ) set the parameters for sample duration the duration (in seconds) of the window of each sample samples the number of samples that you check in each sampling samples threshold the number of error results (within a sampling) that trigger the failover pid scopes the rule to a specific pid in the transport stream (e g , if you have multiple audio programs it will only apply the rule to the specified one) for example, if you set sample duration=10, samples=6, samples threshold=2 then if during a 60 second time period an error occurs during 2 out of the 6 ten second windows, failover will be triggered in the configuration section, in the ingest cluster field, select from the drop down list the ingest cluster on which the sources that you would like to group together are located in the target broadcaster/s field, specify which broadcaster/s within the cluster on which zen master can deploy the source either select a specific broadcaster (for manual clusters) or specify your preference for using primary vs backup broadcasters, options are prefer primary broadcasters, primary broadcasters only, backup broadcasters only or prefer backup broadcasters select the alert on pid changes checkbox if you would like to generate email notifications whenever a pid change is detected select the log scte 35 checkbox if you would like to log scte 35 markers for this stream to log scte 35 markers, scte 35 needs to be enabled both on the ingest cluster and on the source configuration scte 35 logging is only available for v13 1 40262+ broadcasters if log scte 35 was selected, you can choose to receive alerts for scte 35 markers by selecting the enable scte 35 alerts checkbox additional scte 35 options are available when you expand the scte 35 manipulation section scte 35 manipulation here you can enable scte 35 insertion set the scte 35 pid remove any existing scte 35 messages from the stream replace any existing scte 35 messages with null in the public output field, if you would like to allow access to the stream directly through a supported player, select the allow checkbox if you selected the allow checkbox, then enter a password in the password field to authenticate access to the stream under content analysis select one of the following options full select this option if you would like to enable full content analysis (cqa), including analysis of the stream's audio and video content, including audio levels, audio silence detection, frozen video detection, and more tr101 only select this option to perform tr101 analysis only (without video and audio content analysis) the analysis can be used to check the integrity of the ts and the most important elements of the stream the analysis includes priority 1 and 2 to learn more about tr101 analysis see tr 101 290 technical report none no content analysis is enabled it is generally recommended to enable full content analysis unless it is causing technical problems if content analysis is disabled, the system won't detect errors in the video and audio content of the stream and won't generate warnings for such events if you would like to monitor cea 608/cea 708 closed captions, select monitor cea 608/cea 708 checkbox monitoring cea 608/cea 708 closed captions requires using a version 14 4 or newer broadcaster if you would like to disable auto pull, select the disable auto pull checkbox disabling auto pull will lock channels using this source to the source's ingest cluster and targets to the source's current active broadcaster this setting only applies to pass through channels if auto pull is enabled , in the auto pull latency \[ms] field, specify the maximum latency for auto pull actions the broadcaster will update the latency as required, by using the maximal defined latency and then reducing the latency gradually (default 1000 ms) in the webrtc playback section, select the desired playback method webrtc enables you to monitor the stream via web browsers and mobile applications with real time communication (rtc) webrtc needs to be activated independently for each source in zen master (by default it is disabled) select one of the following options disabled users will not be able to view this source using webrtc (default) pass through passes the stream through to webrtc without transcoding this option can only be used if the input stream is already in a webrtc compliant format (video h 264 codec with baseline or main profile and no b frames, audio opus codec) auto zen master automatically transcodes the stream using the optimal profile for each specific user in this option zixi broadcaster will decode the video, transforming it into raw frames, which will be passed to the webrtc library, where it will be re encoded for each individual user best quality however, if monitored by multiple users this can consume extensive resources 1080p/720p/480p/360p zen master transcodes the stream to a webrtc compliant profile with the specified resolution the transcoding is done by zixi broadcaster's transcoder this method requires a zixi broadcaster with x264 or nvidia for transcoding this method allows many users to monitor the stream without encoding separately for each individual user, which saves cpu capacity since there is only one stream version, it is recommended to encode to a low resolution and low bitrate so that the stream will be viewable by all users monitoring the stream in the initial state section, select the disabled checkbox if you would like to disable this source upon initial configuration select the muted checkbox if you would like to disable error/recovery notifications for this source upon initial configuration you can enable/unmute the source at any time under error concealment, click the enable error concealment checkbox to enable the following error concealment mechanisms that can correct trasnaport stream for downstream irds 1\ continuous timeline corrects errors that are related to timeline jumps in the source this mechanism will normalize the timeline by creating a variant of the input (the input is doubled) and normalizing the timeline of the doubled input 2\ replace frames if there are frames with cc errors, it replaces them with skip frames this solves issues such as an ird that lost synchronization when it received broken frames this mechanism also creates a variant of the input (doubling the input) 3\ renumber cc the continuity counter is part of the ts encapsulation when there are cc errors in tr101 analysis it means that the counter has a gap due to the lost frames this mechanism can re write the cc counter for the downstream devices, so they will see a continuous increase of the counter 4\ delay (ms) enter the delay that the error concealment adds to the stream in miliseconds error concealment needs to buffer the stream to analyze it one second is a reasonable value lower values may cause the error concealment to not be able to fix some of the frames 5\ cbr padding enter a value for the rate at which padding will be added to maintain a specific output transport bitrate 6\ pcr interval select the frequency at which program clock reference is inserted into the outputs transport stream options include atsc standard 100 ms dvb standard 40 ms if you would like to adjust the advanced settings, click advanced the advanced settings fields are displayed advanced settings if you would like to track usage for this source, enter your billing code details see billing codes docid\ lalxj n928tegcpdpbirj in the auto pull mtu field, specify the maximum transmission unit (largest data packet that a network connected device will accept) in bytes for the auto pull to set the detection of frozen video, in the frozen video minimal duration field, specify the default minimal duration in seconds of frozen video after which the broadcaster will trigger a frozen video alert default 10 seconds this setting is supported for broadcaster version 16 3 and higher to set the detection of blank video, in the blank video minimal duration field, specify the default minimal duration in seconds of blank video after which the broadcaster will trigger a blank video alert default 10 seconds this setting is supported for broadcaster version 16 3 and higher click save hitless failover source configuration parameters table parameter description info name specify a name for the failover source in zen master tags select one or more of the predefined tags from the drop down list tags are used for access control by selecting an tag, you are relating this source to that tag tags are associated with certain roles (through the users & roles screen), and roles, in turn, are associated with users and user groups if you would like to create a new tag, see creating a tag docid\ ypm9tw81ap6nq8stkv4rd events configuration profile select from the dropdown list the profile that you would like to apply to this source by default, the default profile is applied note events profiles consist of a series of rules that determine precisely how various types of events are logged and what results they trigger location (optional) the source's location is used to represent it in the proper position in the map display (see maps docid\ sfzm6jvorvuigo7x0dnue ) by default, the location is determined automatically based on the source's ip enter identifying info for the location (e g street address, zip code, coordinates) and select the desired location from the suggestions that are shown input sources select from the drop down list two or more binary identical sources that you would like to include in the failover group if the selected sources are not binary identical then there will be a one second delay in the failover to the backup source if you would like to designate a source as a preferred source, select the preferred checkbox if you would like to designate a source as part of the slate, select the slate checkbox in the min bitrate\[kbps] field, specify the bitrate threshold level in kbps to perform the failover (i e , if the bitrate drops under the threshold the failover will take place) note if you would like to remove a selected source, click on the button next to the source failover mode select the desired mode for failing over/merging the streams options are stream switch – zen master treats the streams as different and does not try to merge them if packets are missing from one stream, zen master will switch to the alternate stream if the payload is not binary identical it will fall back to stream switch content hitless if the sources do not have rtp headers, zixi will merge the streams using the mpeg ts payload content itself this method uses zixi's patented dna like sequencing algorithm to fill in the missing packets smpte 2022 7 hitless if the sources use rtp, and the rtp headers were maintained, zixi can use the smpte 2022 7 to compare and merge the two streams this option uses the sequence and the timestamp in the packets to compare which packets are missing, providing optimal results merge search window \[ms] specify the time frame for which the system will search for the relevant packets in a matching stream (default 1000) note if there is a latency difference between the streams, the search window value should be higher than the latency difference between the streams zero bitrate/offline detection window \[ms] (for stream switch option) enter the time frame with zero bitrate or offline state after which the system will perform the failover failover rules failover rules allow you to set rules for what type of error and how often they occur to change to the failover stream if you would like to change the rules, use the following procedure (supported for broadcasters v15+) select the checkbox next to the error for which you would like to change the error rules options are p1 transport errors , frozen video , blank picture, silent audio, and low bitrate set the parameters for sample duration – the duration (in seconds) of the window of each sample samples – the number of samples that you check in each sampling samples threshold – the number of error results (within a sampling) that trigger the failover pid scopes the rule to a specific pid in the transport stream (e g , if you have multiple audio programs it will only apply the rule to the specified one) note for example, if you set sample duration=10, samples=6, samples threshold=2 then if during a 60 second time period an error occurs during 2 out of the 6 ten second windows, failover will be triggered configuration ingest cluster select from the drop down list one of the available ingest clusters as the destination for the source target broadcaster/s specify which broadcaster/s within the cluster on which zen master can deploy the source either select a specific broadcaster to be used or specify your preference for using primary vs backup broadcasters options are prefer primary broadcasters, primary broadcasters only , backup broadcasters only or prefer backup broadcasters note for managed clusters, it is not recommended to select a specific broadcaster, as the broadcaster’s availability may not remain stable alert on pid changes if you would like to generate email notifications whenever a pid change is detected, select this checkbox preserve rtp headers select this checkbox to preserve the rtp headers (which include the timestamps and sequences) and propagate them the output log scte 35 if you would like to log scte 35 markers for this stream, select this checkbox note to log scte 35 markers, scte 35 needs to be enabled both on the ingest cluster and in the source configuration note scte 35 logging is only available for v13 1 40262+ broadcasters enable scte 35 alerts (for log scte 35) if you would like to receive alerts for scte 35 markers, select this checkbox public output if you would like to allow access to the stream directly through a supported player, select the allow checkbox password (for public output) if you selected allow in the public output field, specify a password in this field to authenticate access to the stream content analysis select one of the following options full select this option if you would like to enable full content analysis (cqa), including analysis of the stream's audio and video content, including audio levels, audio silence detection, frozen video detection, and more tr101 only select this option to perform tr101 analysis only (without video and audio content analysis) the analysis can be used to check the integrity of the ts and the most important elements of the stream the analysis includes priority 1 and 2 to learn more about tr101 analysis see tr 101 290 technical report none no content analysis is enabled note it is generally recommended to enable full content analysis unless it is causing technical problems if content analysis is disabled, the system won't detect errors in the video and audio content of the stream and won't generate warnings for such events monitor cea 608/cea 708 closed captions if you would like to monitor cea 608/cea 708 closed captions, select this checkbox note monitoring cea 608/cea 708 closed captions requires using a version 14 4 or newer broadcaster disable auto pull if you would like to disable auto pull, select this checkbox note disabling auto pull will lock channels using this source to the source's ingest cluster and targets to the source's current active broadcaster this setting only applies to pass through channels auto pull latency \[ms] specify the desired latency for auto pull actions (default 1,000 ms) webrtc playback select the desired playback method note webrtc enables you to monitor the stream via web browsers and mobile applications with real time communication (rtc) webrtc needs to be activated independently for each source in zen master (by default it is disabled) select one of the following options disabled users will not be able to view this source using webrtc (default) pass through passes the stream through to webrtc without transcoding this option can only be used if the input stream is already in a webrtc compliant format (video h 264 codec with baseline or main profile and no b frames, audio opus codec) auto zen master automatically transcodes the stream using the optimal profile for each specific user in this option zixi broadcaster will decode the video, transforming it into raw frames, which will be passed to the webrtc library, where it will be re encoded for each individual user note best quality however, if monitored by multiple users this can consume extensive resources 1080p/720p/480p/360p zen master transcodes the stream to a webrtc compliant profile with the specified resolution the transcoding is done by zixi broadcaster's transcoder note this method requires a zixi broadcaster with x264 or nvidia for transcoding this method allows many users to monitor the stream without encoding separately for each individual user, which saves cpu capacity since there is only one stream resolution, it is recommended to encode to a low resolution and low bitrate so that the stream will be viewable by all users monitoring the stream scte 35 manipulation enable scte 35 insertion select chackbox to enable scte 35 insertion scte 35 pid set the scte 35 pid in which the messages will be inserted remove existing scte 35 remove any existing scte 35 messages from the stream replace with null replace any existing scte 35 messages with null initial state disabled select this checkbox if you would like to disable this source upon initial configuration muted select this checkbox if you would like to disable error/recovery notifications for this source upon initial configuration error concealment enable error concealment select this checkbox to enable the error concealment mechanisms listed below to correct transport stream for downstream irds delay (ms) enter the delay that the error concealment adds to the stream in miliseconds error concealment needs to buffer the stream to analyze it one second is a reasonable value lower values may cause the error concealment to not be able to fix some of the frames cbr padding (kbps) enter a value for the rate at which padding will be added to maintain a specific output transport bitrate pcr interval (ms) select the frequency at which program clock reference is inserted into the outputs transport stream options include atsc standard 100 ms dvb standard 40 ms advanced billing codes if you would like to track usage for this source, enter you billing code details, see (supported for broadcasters v14 8+) billing password enter the password that corresponds to this billing code auto pull billing code specify the maximum transmission unit (largest data packet that a network connected device will accept) in bytes for the auto pull auto pull billing password enter the password that corresponds to this auto pull billing code auto pull mtu (bytes) specify the maximum transmission unit (largest data packet that a network connected device will accept) in bytes for the auto pull frozen video minimal duration (seconds) specify the default minimal duration in seconds of frozen video after which the broadcaster will trigger a frozen video alert default 10 seconds this setting is supported for broadcaster version 16 3 and higher for existing sources, you may need to restart the source blank video minimal duration (sec) specify the default minimal duration in seconds of blank video after which the broadcaster will trigger a blank video alert default 10 seconds this setting is supported for broadcaster version 16 3 and higher for existing sources, you may need to restart the source ignore monitoring pids ignore pids for empty audio streams to prevent false warnings enter a comma delimited list of pid numbers that should not be monitored for null content