| The
systems have been used by NHK in live sports broadcasts. During sports broadcasts
in the current situation, tactics, rules, etc. are often visually described by
placing CGs on top of the pictures taken by the camera. In recent years, virtual
studio techniques using CGs linked to camera work have improved. For example,
by displaying the virtual K point of a ski jump, such techniques make it possible
for viewers of ski-jump competitions to get more visceral information about the
sport. Currently,
virtual displays during a live sports broadcast remain limited to information
known beforehand, such as world records, and to information that can be input
manually. Although manual CG manipulation during post-production is common for
sports competitions whose action varies moment-by-moment, it is difficult to do
so during a live broadcast. The
firm is researching image analyses that extract, track, and identify objects in
sports video. The aim is to make it possible to do real-time extraction by using
motion prediction techniques for efficient processing. Using image analyses and
motion prediction techniques, the firm has developed systems that can render in
real-time CGs linked to the movements of players, balls, and other objects. The
use of these systems in broadcasts can give viewers a more intimate feel of the
unfolding competition. The
systems were used during broadcasts to visualise the movements of baseball pitches,
golf tee shots and putts, and soccer offside lines. Even though the broadcasts
were live, the visualizations were able to be shown immediately after each play.
To give
an example if the trajectory of the baseball pitcher’s pitches can be displayed
during a live baseball broadcast, the aimed course and the drop of curve balls
can be clarified. Conventional systems can render such visual effects, but they
have difficulty displaying a clear trajectory in real-time. Also, they face problems
such as not being able to be used when the batter is left-handed, because the
image is con4 cluded. The pitch trajectory display system the firm has developed
extracts the ball region from the video in real-time, and superimposes the trajectory
on the relay video image. The
system is equipped with ball- motion prediction and interpolation algorithm, so
it can even be applied to pitches made to left-handed batters. The system does
not require special sensors or cameras. It can detect the ball from the video
taken by a camera used in the broadcast. Also, all processing is done on two-dimensional
image coordinates, so camera calibration is not necessary. The
system mainly consists of personal computers. The ball detection and CG rendering
processes are executed on two PCs. Although both processes cause a time delay,
a frame delay provides an adequate delay (a few frames) for the camera video to
match the timing of the CG. The extraction process mainly consists of moving object
extraction, ball object selection, and motion prediction sub-processes.
Although
the operator has to set the initial position of the search region on the screen,
after the first detection, the search region is updated based on the predicted
positions, and the ball is tracked automatically. To
speed up processing and to prevent mis-extraction of false objects outside the
ball’s path, the detection process is only performed in the search region.
The size of the search region is set to about 80×40 pixels. Because the
broadcast camera is not actively operated when the pitch is thrown, it can be
considered to be fixed. Therefore, the moving object extraction process can use
the frame difference method to extract ball region candidates. The
ball object selection process selects it from the candidates. By applying correction
filters that cover areas, colours, shapes, and predicted locations, only the ball
region is selected. | 
