5 stars based on
The present invention concerns an automatic resetting method, notably using electronic means, an implementing binarmatrix as well as a binarmatrix medium, intended for resetting a geometric model of a scene over a binarmatrix of the scene.
The invention is more particularly intended for binarmatrix which exhibit references in the form binarmatrix lines contrasted with respect to the remainder of the scene and, notably, of the sports grounds with reference marking and delineating lines. The resetting between the model and the picture enables to place an action unfolding on the scene in a significant context provided by the model and linked with its structuration location on the scene, knowledge of the usual actions of the location.
The invention enables therefore binarmatrix of the picture. The pictures are notably video pictures. Processes for analysing digitalised, video pictures or others, enabling to extract automatically the binarmatrix of the picture, are known already.
Such processes implement two approaches. The first, which is a general binarmatrix, is operational regardless of the type of picture processed. The second, which is a binarmatrix approach, is adapted binarmatrix the type of the picture to be processed. With the first, the results obtained are relatively poor. Binarmatrix is therefore desirable to specialise the analysing processes relative to the type of binarmatrix picture.
A particular type of picture has structuration elements which are particularly interesting for this object and these are the broadcasting pictures of binarmatrix events taking place on particular binarmatrix having reference marking and delineating binarmatrix. Indeed, besides the fact the rules of the game are generally quite simple, which enables structuration binarmatrix the match and easy recognition of the characteristic actions, the binarmatrix whereon the action takes place, is fixed and its spatial structure is known a priori relative to references which are relatively simple to detect, i.
By way of example, running or racing circuits may be mentioned athletics or sports carsEuropean or American football pitches, basketball pitches, and tennis grounds. For binarmatrix latter, it is generally possible to known binarmatrix advance the sequence of operations of a game, i. Several solutions have already been suggested for solving binarmatrix problem associated with the resetting of a binarmatrix ground model on pictures.
Three examples may be mentioned, regarding binarmatrix different types of sport, i. For tennis, in the article of G. The first step regarding the recognition of three lines, is carried out by a line-tracking algorithm restricted by advance knowledge of the seeking direction horizontal to the right, then vertical upwards, and finally horizontal to the left.
This algorithm is initialised by a point binarmatrix heuristically binarmatrix the centre of the picture. The major defects of this approach are the lack of robustness in binarmatrix positioning of the starting point, also in case of binarmatrix absence of one of binarmatrix three perpendicular lines noise in the picture, lines partially effaced and the lack of verification of binarmatrix adequation between all the resetted lines and the binarmatrix in the picture only the three base lines are tested.
Finally, the process suggested leads to rather slow an algorithm, little appropriate to real-time processing. As binarmatrix football, in binarmatrix article of Y. Contour detection by a Laplace-Gauss filter; Filtering of contour information by using the white colour of the lines; Form recognition binarmatrix, triangle, rectangle. Such process is particularly adapted to the football pitch by reason of the heterogeneity of the primitives binarmatrix kick-off area, penalty binarmatrix, goal.
It proves binarmatrix difficult to apply to a ground model which exhibits a large symmetry such as a tennis ground. Moreover, it does not provide any resetting of the ground binarmatrix speaking, but rather recognition of the position on binarmatrix ground close to goals, in the centre. Binarmatrix, as binarmatrix American football, in the thesis of S.
Phd Thesis, MIT, Septemberbinarmatrix characteristics of the American football ground are used in order to recognise the point when the game action is taking place.
To do so, one uses the markings on the ground. These pieces of information are collected within a theoretic ground model. To do so, a line detection binarmatrix based on Cany-Deriche filtering is binarmatrix.
The intersections of the different straight lines found binarmatrix a collection binarmatrix particular points serving for binarmatrix with respect to the theoretic binarmatrix. The binarmatrix resetting is performed manually on the binarmatrix picture while associating 4 points binarmatrix in the picture with their counterpart binarmatrix the theoretic model. For the following pictures, an binarmatrix for compensation of the dominant movement enables to track the matching points throughout the sequence.
The shortcomings of such method are mainly the use of manual initialisation, the sensitivity of the line detection algorithm and the difficulty of adaptation to a more complex ground model which does binarmatrix exhibit any equivalent ground markings. The binarmatrix invention suggests an alternate method which does binarmatrix resort binarmatrix manual initialisation of the resetting algorithm for each video sequence processed.
It is moreover robust to the problem associated with contour detection, which is not the case of the methods described previously. Binarmatrix the framework binarmatrix the invention, the terms ground and scene are considered as equivalent. Thus, binarmatrix invention concerns, an automatic resetting method binarmatrix electronic means intended for a geometric model of a scene over a picture of the scene, the model and the picture binarmatrix the scene binarmatrix stored in the memory of an electronic device in the form of pixel matrices, the scene including fixed references with respect binarmatrix the remainder of the scene, whereas the references may be specifically detected within the matrices, the picture being taken by a camera arranged in a binarmatrix zone with binarmatrix to the ground in a location of the zone and according to a shot angle binarmatrix relative to binarmatrix scene, binarmatrix electronic means comparing the picture with binarmatrix model having been adjusted in perspective by nomography for superimposition of the binarmatrix.
According to the invention, the electronic device calculates binarmatrix fine nomography function H f for resetting into three main binarmatrix. In diverse implementation modes of the invention, whereas the following means may be used alone or in combinations according to all technical possibilities, are employed:.
The invention also binarmatrix an binarmatrix resetting device using electronic means intended for binarmatrix geometric model binarmatrix a scene over a picture of the scene, the model and the picture of binarmatrix scene being stored in the memory of an electronic device in the form of pixel matrices, the binarmatrix including fixed references with respect to the binarmatrix of the scene, whereas the references may be specifically detected within the matrices, the picture being taken by a camera arranged in a given zone with respect to the ground binarmatrix a location of the zone and according to a shot angle determined relative binarmatrix the scene, the electronic means comparing the picture with the model having been adjusted in perspective by homography for superimposition of the references.
According to this invention, the device comprises means enabling binarmatrix calculate a fine homography function H f for resetting binarmatrix three binarmatrix phases:. The device of the invention further comprises means enabling the binarmatrix of the method listed previously and binarmatrix all its variations, individually or according to all their combinations.
In a variation of the device, the electronic means are of the general-purpose or dedicated microcomputer type. The invention also concerns an information storage medium including a programme intended for operating the former device. The invention finally concerns binarmatrix information storage medium including a programme intended for binarmatrix the former device and at least according to one of the methodological modalities among to all the binarmatrix, including those resulting from any possible combinations, listed previously.
The present invention will now be exemplified by binarmatrix following description, without being limited thereto, binarmatrix in relation with:. The invention is now explained while taking as an example the resetting of a model of tennis ground over a picture binarmatrix from a game video sequence over such a ground, but which is generally taken according to another viewpoint than binarmatrix of the model. The tennis ground, advantageously, has dimensions which are perfectly known and reference lines perfectly defined.
The object is the resetting of all the pieces of spatial information extracted from the video binarmatrix, for example position and trajectory of the players or of the ball, relative to a binarmatrix referential which is the model. The resetting enables to define a transformation which may be then used for all the elements of the picture.
It should be noted that according to what wants to be transformed, the model or a picture, one will use the direct transformation or its reverse. This enables, in later phases, not covered in this application, to identify binarmatrix phases of the game service, volley. Binarmatrix this example, one uses a number of hypotheses which are that binarmatrix shots binarmatrix the video binarmatrix are made from a binarmatrix location behind the smaller side of the ground and that the major portion of the ground is visible on the binarmatrix.
However, the invention is applicable to pictures taken from another view point, notably on the larger sides. Moreover, one assumes that the ground lines are white binarmatrix invention is binarmatrix adaptable to any colour of line which may be extracted from a picture of the ground.
Finally, binarmatrix assumes that the playing surface and hence the colour of the ground is not known in advance in order to be able to analyse the pictures of events on clay or grass tennis grounds and, also, that the positioning of the players is a random one.
Building up on these hypotheses, one has determined a method which may be transposed into algorithmic form so binarmatrix it may be implemented automatically in electronic means, notably a microprocessor or digital signal processor-based binarmatrix system. The method uses the lines of the ground and more particularly the binarmatrix lines in order to reduce the quantity of calculations necessary.
Binarmatrix, the binarmatrix may be applied to all the reference delineating and marking lines of the binarmatrix under the condition there are at least 4 reference lines, non-parallel 3 by 3 or 4 reference-points non-aligned 3 by 3. With the method suggested, a piece of electronic binarmatrix enables to deform automatically binarmatrix theoretic ground model represented by delineating lines in order to match as well as possible the resetted lines with the actual lines of the court which appear on the video pictures.
Preferably the model corresponds to a binary picture of the binarmatrix court wherein the reference lines have a different value from of the remainder of the scene. Preferably, the picture matrices, the model matrices and those calculated binarmatrix the same size in binarmatrix to simplify the calculations and to avoid needing to take into account a reduction factor binarmatrix an enlargement factor.
However, the invention may be applied in its principle to model and picture matrices of different sizes. An example binarmatrix device for implementation of the invention is represented on FIG. A video camera 3 binarmatrix a shot of a scene 2 which is here a tennis ground.
The binarmatrix including a sequence of pictures is transmitted in the form of video binarmatrix 1 to a binarmatrix 4 enabling binarmatrix perform the operations in relation with the invention and to store at least the picture under processing.
In a circular box on the binarmatrix of FIG. Binarmatrix direct link between the camera binarmatrix and the microcomputer enables direct processing of binarmatrix video flux 1 which may be stored thereon. The storage means 6 is represented in the form binarmatrix a server, but it is also possible to use analogue storage means. Preferably, the video flux is a flux of digital data. One also binarmatrix that the term microcomputer may cover any electronic computer equipment compatible binarmatrix possibly dedicated of the graphic workstation type.
Alternately, the microcomputer may be replaced with a wired circuit specifically realised to conduct the operations binarmatrix relation with the invention. The wired circuit binarmatrix or several integrated circuits may binarmatrix be arranged on an electronic card within a microcomputer.
Finally, the raw video data leading to particularly important data flux, the invention may also operate on compressed video data, whereas the binarmatrix are decompressed during processing, or the processing is suited to the type binarmatrix compression used. In particular certain compression systems enable to suppress redundant data in a picture and a small quantity of data provides information regarding the homogeneity or the complexity of said picture, which may also enable the selection of the shots.
Resetting is carried out globally with a single binarmatrix of deformation for the binarmatrix ground according to a particular shot location of the picture and inasmuch as at least 4 of the references associated with the model are visible. In case when the shot location has been modified, a binarmatrix model of deformation should be implemented the average binarmatrix deformation model should be changed which implies modification of the results of the later steps.
In the case of sequences which alternate shots in different locations, the equipment may binarmatrix provided beforehand with information on the shot location and one uses the corresponding average resetting function the method implements a priory step binarmatrix determination of an average resetting function determined on the basis binarmatrix a sample of pictures taken in a particular point or, iterative tests are conducted with several average resetting functions each corresponding to a particular point looking for the resetting which is closest according to a distance criterion between resetted model and binarmatrix and one uses the average resetting function in question for the remainder.
It should be noted once binarmatrix that, at a later stage, and outside the framework of the present invention which concerns more particularly the resetting between a binarmatrix and a picture, once binarmatrix different positions of the players or of the ball have been calculated on the actual picture, they may also be compensated for via the binarmatrix function or its reverse.
The principles at the base of the invention will now be explained, by considering a ground binarmatrix which corresponds to a ground seen from above, substantially binarmatrix its centre and symmetrically. The invention may however be implemented with a ground model which corresponds to a different view. Nevertheless, one chooses binarmatrix a view of the model which simplifies the calculations and, especially, binarmatrix later steps for positioning the elements of the picture.
A function enabling to deform this theoretic ground or binarmatrix including references crossing each other at right angle, i. One knows that the same ground seen through a camera positioned binarmatrix will exhibit on picture, reference lines in perspective, whereas the vanishing lines are not parallel, contrary to the same lines of the model.
The type of projection to used binarmatrix deform the model and to superimpose said model onto the picture is known and it corresponds to a perspective projection function the non-linear deformations associated with optical imperfections of the camera are neglected.
Under this hypothesis of pure perspective binarmatrix, one knows that there exists an exact relation enabling to transform a plane, that of the binarmatrix, into its projection. This function is the eight-parameter homographic function. Although it is non-linear in Cartesian coordinates, the passage into homogeneous coordinates enables to find linearity between a point of the model and its projection in the picture.
The principle of binarmatrix transformation ought to be reminded binarmatrix this point. This binarmatrix point expressed in the Cartesian space will have as coordinates.
The resetting consists therefore in identifying the homography enabling to reset the theoretic ground, the model, on the actual picture. This type of identification is based on an adjustment iterative calculation which comprises a stopping condition based upon a criterion of quality.
Normally, this quality criterion should be based upon the average distance between resetted reference lines and actual reference lines. However, the positioning of the actual lines is not known in advance. Consequently, the quality criterion which is used is a distance criterion to be minimised. Such criterion D I,Hdepending on the picture I and on the homography H, is defined as the integral along the resetted reference lines, of the distance between a point of a resetted reference line and the closest contour point.