Unleashing Creativity at the Speed of Automation: Streamlining Story Creation Using AI and the Cloud

Introduction

The Media and Entertainment (M&E) industry is in a constant state of evolution due in part to the insatiable demand across all viewer platforms for captivating content. In this digital age, the ability to efficiently ingest and clip live or recorded video content and turn it into compelling stories ready for immediate distribution has become crucial for content creators.

In this blog, we will explore how TVU Search, a cloud-native AI-driven application from TVU Networks, is making it possible for broadcasters to find the content needed to create compelling stories for audiences across TV, web, and mobile platforms better and faster when compared to traditional methods.

Key Challenges

Broadcasters in the M&E industry have an enormous amount of recorded video content with more being added every single minute. Consisting of both live and archived footage, broadcasters have at their disposal the raw materials to create any story at any time. However, having access to a comprehensive library of assets doesn’t mean it’s simple or fast to find the right content.

Traditionally, finding relevant content requires laborious sifting through hours of video content to find the precise needed segments. This is a highly manual and time-consuming process which impacts the speed of story creation. In the era of social media, stories can unfold spontaneously, and it’s a requirement to report on them effectively in order to capture viewers’ attention.

What is TVU Search?

TVU Search is an advanced AI-powered live video ingest and discovery solution tailored for M&E professionals, revolutionizing their content utilization. Its robust search capabilities, driven by AI algorithms and automation, streamline content discovery, allowing users to effortlessly locate live feeds or clips for immediate playout, download, or sharing. Operating from a user-friendly web-based interface, TVU Search is infinitely scalable, accommodating an unlimited number of concurrent users and live sources from various devices.

With TVU Search, metadata generation is indexed with timecode including metadata generated using AI like speech transcription and facial recognition as well as other metadata embedded in the video like SCTE triggers and caption data.

Real-time monitoring and customizable multi-views add further convenience for users. The platform also presents valuable insights through intuitive charts and color-coded graphs, enhancing live and archived feed analysis with download counts, video clip data, and detailed usage records.

TVU Search’s cloud-native design leverages AWS infrastructure, providing scalability, flexibility, and resilience. The application is built as a set of loosely coupled microservices, enabling independent deployment, scaling, and management for optimized cloud performance, rapid development, and deployment.

The Timeline view in TVU Search provides an overview of content feeds.

Among its various features, TVU Search’s speech recognition feature allows users to easily find video content based on specific phrases or keywords, and highlight an area in the transcript to set the clip’s mark-in and mark-out timecode.

The speech recognition panel displays the transcript generated by TVU Search AI.

The Live Monitor page offers customizable multiview options of content sources based on a user’s chosen criteria and search parameters.

Users employ keywords, phrases, and images to do content searching. The “search by face” feature includes searching through images. By uploading an image, TVU Search finds content with matching people based on its facial recognition capabilities, such as in video clips.

Facial detection is another feature that facilitates rapid searching. AIML is used to detect and display faces as thumbnails within a designated or highlighted area in the video. Clicking on the thumbnail allows the user to navigate directly to the precise timestamp in the video where the person is visible.

What makes TVU Search so effective?

TVU Search enables users to manage a large number of live IP video streams including monitoring, ingesting and recording of live content from virtually anywhere with automatic AI-generated metadata. Users can quickly search for specific live or recorded content based on customizable criteria such as keywords, faces, or spoken words.

Users follow simple, intuitive steps when searching for content in TVU Search:

1. Automated or Hands-on Search: Users can take advantage of the AI-driven Active Search feature for live workflows, enabling time-saving automation of various actions, such as creating and exporting a video clip, notifications, and applying searchable tags, based on specific criteria within user-defined search profiles. Alternatively, users have the flexibility to search manually for content based on specified criteria.
2. Live Monitoring : TVU Search enables users to create personalized and practical multiviews of live sources, including the ability to customize the display type, search and filter sources, and make adjustments to the preview position. Additionally, users can select the sources to be displayed and effectively manage live recordings and outputs.
3. Thumbnail Preview and Real-Time Transcription: Once the search is performed, the system presents users with search results accompanied by visual thumbnails and relevant metadata, enabling users to quickly assess the content. Search results also come with a rolling transcription indexed to time, allowing users to preview the content alongside the synchronized transcription.
4. Content Download: Users have two options for setting the in/out points when downloading the search results: manual input or automated generation based on highlighted transcribed text. This flexibility ensures the user has complete control over when extracting the desired content.
5. Immediate Distribution: After extracting the desired content, users can seamlessly route it for immediate playout, download, or sharing for broadcast on multiple viewer platforms including social media.

Behind the scenes, TVU Search uses AWS services and serverless architecture to power its capabilities enabling users to capture, analyze, record, extract, transcode, and share content in real-time.

As video sources are uploaded into TVU Search, the platform processes the incoming streams using Amazon Elastic Container Services for Kubernetes (EKS) for content transcoding, restoration, exporting, and search. Rapid content delivery to the cloud is supported by leveraging Amazon Cloudfront alongside storage and database services including Amazon Simple Storage Service (Amazon S3), Amazon Relational Database Service (Amazon RDS), and others.

TVU Search also uses Amazon Rekognition – a service that makes it easy to add powerful visual analysis to your applications – to extract content metadata for facial recognition, facial tracking, and scene analysis, amongst other functions. The information from the video streams are then parsed and supplemented with additional metadata from various sources within the TVU ecosystem.

Transcoding of content, live and file, is achieved by using Amazon Elastic Transcoder and AWS Lambda, where AWS Lambda operations are orchestrated using AWS Step Functions.

TVU Search currently supports the following formats:

• Live IP stream input formats: TVU Grid, SRT, RTMP, RTMPS, UDP, HLS, RTSP
• Live IP stream output formats: TVU Grid, HLS, RTP-FEC, SRT, Zixi, RTMP(S), UDP, RTSP
• Exported video file formats: MPEG-4 (.TS, .MP4, .MOV, .MXF)
• Exported caption file formats: .SRT, .TXT, .DOCX

Summary

TVU Search, powered by AWS technology, is changing the media and entertainment landscape by transforming content discovery and creation processes. Instead of taking hours or days to search, locate, and clip video content, users can now find the content they need in minutes and immediately share it with viewers globally.

Learn more about TVU Search here.

About TVU Networks

TVU Networks is the premier global provider of SaaS and Cloud-based workflow solutions across multiple industries, including news, entertainment media, sports, corporate, streaming, houses of worship and government. Through the use of AI, microservices and automation-driven technology, TVU helps enterprises realize an efficient metadata and story-centric workflow through its ecosystem for live video acquisition, production and distribution. The company has been on the forefront of developing the tools needed to streamline and monetize the Media Supply Chain. TVU is a critical part of the operations of major media companies worldwide and is a Technology and Engineering Emmy® Award winner.