JP4048779B2 - Distance image processing device - Google Patents

Description

【００５４】
求めた点Ｒの座標（Ｘｓｗ，Ｙｓｗ，Ｚｓｗ）に対応する世界座標系Ｃｗにおける検知対象物Ｏｂの代表点Ｐの座標（Ｘｐｗ，Ｙｐｗ，Ｚｐｗ）は、図１３に示す直角三角形ＯｃＰＱと、直角三角形ＯｃＲＴとの相似を利用して求めることができる。点Ｏｃと点Ｐとの距離は上述した代表距離Ｌであり、点Ｏｃと点Ｒの距離Ｌｃは直角三角形ＯｃＯｉＲの斜辺の長さであることに着目すれば、次式によって求めることができる。
Ｌｃ＝（Ｘｓｃ² ＋Ｙｓｃ² ＋ｆｃ² ）^1/2
したがって、点Ｐの高さＺｐｗは次式のように求められる。
Ｚｐｗ＝Ｌ（Ｚｃｗ−Ｚｓｗ）／Ｌｃ
つまり、あらかじめ３次元位置測定装置１の高さを定数として与えておけば、検知対象物Ｏｂの代表点Ｐについて床面Ｆからの高さ求めることができる。要するに、３次元位置測定装置１の物理仕様（受光面の物理サイズ、画素数、床面Ｆからの高さ、光軸の向きなど）と代表距離Ｌとを用いて画面座標系Ｃｉから世界座標系Ｃｗへの座標変換が可能になるのである。なお、３次元位置測定装置１および画像取込手段２の機能は第１の実施の形態と同様である。
【００５５】
（参考例２）
本例は図１０に示した参考例１と同様の構成を用いるが、参考例１では検知対象物の代表点として距離画像を２値化して抽出した白画素の領域の重心位置を用いたのに対して、本例では検知対象物について２個の代表点を求め、両代表点間の距離を求めることによって検知対象物の大きさを特徴量として抽出するものである。
【００５６】
すなわち、本例における代表距離計算手段８は、図１４（ａ）のような距離画像を領域選択手段７で２値化して得た図１４（ｂ）のような２値画像における白画素の領域において、図１４（ｃ）のように２個の代表点の座標（ｘｓ１，ｙｓ１）（ｘｓ２，ｙｓ２）を求める。両代表点間の距離が検知対象物の大きさに対応するように、両代表点は白画素の周縁の離れた２点として設定する。本例では白画素の領域の重心を通り距離画像の画面の水平方向（ｙ軸方向）に延長した直線と黒画素の領域との交点で白画素内の画素を代表点として抽出している。要するに、重心を求めた後に２値画像内を水平方向に走査し、画素値が黒画素（画素値＝０）になれば、１つ前の白画素を代表点の画素とする。このようにして求めた代表点の座標は距離画像に照合され、図１４（ｄ）のように各代表点に対応する画素の画素値が検知対象物までの代表距離Ｌ１，Ｌ２として求められる。
【００５７】
２個の代表点と各代表点に対応する代表距離Ｌ１，Ｌ２が求まると、領域特徴抽出手段９において参考例１と同様にして、各代表点の座標（ｘｓ１，ｙｓ１）（ｘｓ２，ｙｓ２）を装置座標系Ｃｃに変換して（Ｘｓｃ１，Ｙｓｃ１，Ｚｓｃ１）（Ｘｓｃ２，Ｙｓｃ２，Ｚｓｃ２）とし、さらに、世界座標系Ｃｗにおける検知対象物Ｏｂの代表点Ｐ１（Ｘｐ１ｗ，Ｙｐ１ｗ，Ｚｐ１ｗ）および代表点Ｐ２（Ｘｐ２ｗ，Ｙｐ２ｗ，Ｚｐ２ｗ）を、以下の形で求めることができる。
Ｘｐ１ｗ＝Ｌ１（Ｚｃｗ−Ｚｓ１ｗ）／ｘｓｃ１
Ｙｐ１ｗ＝Ｌ１（Ｚｃｗ−Ｚｓ１ｗ）／ｙｓｃ１
Ｚｐ１ｗ＝Ｌ１（Ｚｃｗ−Ｚｓ１ｗ）／Ｌｃ
Ｘｐ２ｗ＝Ｌ２（Ｚｃｗ−Ｚｓ２ｗ）／ｘｓｃ２
Ｙｐ２ｗ＝Ｌ２（Ｚｃｗ−Ｚｓ２ｗ）／ｙｓｃ２
Ｚｐ２ｗ＝Ｌ２（Ｚｃｗ−Ｚｓ２ｗ）／Ｌｃ
ただし、Ｚｓ１ｗは、参考例１において説明した数１を用い、（Ｘｓｃ，Ｙｓｃ，Ｚｓｃ）を（Ｘｓｃ１，Ｙｓｃ１，Ｚｓｃ１）に置き換えたときのＺｓｗの値であり、Ｚｓ２ｗは、（Ｘｓｃ，Ｙｓｃ，Ｚｓｃ）を（Ｘｓｃ２，Ｙｓｃ２，Ｚｓｃ２）に置き換えたときのＺｓｗの値である。他の値は参考例１において説明した通りである。
【００５８】
上述のようにして検知対象物Ｏｂの２つの代表点Ｐ１，Ｐ２の座標を求めることができるから、両代表点Ｐ１，Ｐ２の間の距離は以下の形で求めることができる。
（（Ｘｐ１ｗ−Ｘｐ２ｗ）^２ ＋（Ｙｐ１ｗ−Ｙｐ２ｗ）^２ ＋（Ｚｐ１ｗ−Ｚｐ２ｗ）^２ ）^１／２
上述のようにして検知対象物Ｏｂの２つの代表点Ｐ１，Ｐ２の間の距離によって検知対象物Ｏｂの大きさを推定することができる。また、距離画像内での代表点の求め方は規則的であるから、上述のようにして求めた距離は検知対象物Ｏｂの大きさを評価する値として用いることができる。なお、本例では２値画像における白画素の領域の重心から水平方向に直線を延長して代表点を求めたが、重心から垂直方向に直線を延長して代表点を求めてもよい。他の構成および動作は参考例１と同様である。なお、本例では２個の代表点を用いているが、３個以上の代表点を用いる場合も同様であって、たとえば重心から水平方向と垂直方向とに延長した直線上での白画素の領域の端点を用いれば４点の代表点を用いて同様に処理することができる。
【００５９】
（参考例３）
本例は、図１５に示すように、図１０に示した参考例１の構成に加えて領域特徴抽出手段９の出力を受けて検知対象物の存否を判断する検知判断手段１０を設けたものである。３次元位置測定装置１、画像取込手段２、領域選択手段７は参考例１と同様の構成であって同様に機能する。
【００６０】
ところで、本例の代表距離計算手段８は、距離画像から得た２値画像における白画素の領域の代表点の座標として、参考例１と同様に重心の座標（ｘｓｉ，ｙｓｉ）を求めるとともに、参考例２と同様に白画素の領域の周縁の２点（端点）の座標（ｘｓ１，ｙｓ１）（ｘｓ２，ｙｓ２）を求める。つまり、本例では３個の代表点に対応する代表距離Ｌ，Ｌ１，Ｌ２を距離画像から求める。代表距離Ｌ，Ｌ１，Ｌ２の求め方は参考例１および参考例２の手順を用いる。すなわち、本例における代表距離計算手段８は、図１６（ａ）のような距離画像を領域選択手段７で２値化して得た図１６（ｂ）のような２値画像における白画素の領域から、図１６（ｃ）のように重心となる代表点の座標（ｘｓｉ，ｙｓｉ）と端点である２個の代表点の座標（ｘｓ１，ｙｓ１）（ｘｓ２，ｙｓ２）とを求め、各代表点の座標（ｘｓｉ，ｙｓｉ）（ｘｓ１，ｙｓ１）（ｘｓ２，ｙｓ２）を距離画像に照合することによって、図１６（ｄ）のように各代表点に対応する代表距離Ｌ，Ｌ１，Ｌ２を求める。
【００６１】
代表距離計算手段８において代表距離Ｌ，Ｌ１，Ｌ２が求まると、領域特徴抽出手段９において参考例１および参考例２と同様にして座標変換を行い、検知対象物Ｏｂの代表点Ｐについて床面Ｆからの高さＺｐｗを求める。この演算は参考例１と同様の演算になり、代表点Ｐの高さＺｐｗは次式で表される。
Ｚｐｗ＝Ｌ（Ｚｃｗ−Ｚｓｗ）／Ｌｃ
ただし、Ｌｃ＝（Ｘｓｃ^２ ＋Ｙｓｃ^２ ＋ｆｃ^２ ）^１／２
また、領域特徴抽出手段９では、検知対象物Ｏｂの代表点Ｐ１，Ｐ２について距離ｄ１２を求める。この演算は参考例２と同様の演算になり、距離ｄ１２（つまり、検知対象物の幅）は次式で表される。
ｄ１２＝（（Ｘｐ１ｗ−Ｘｐ２ｗ）^２ ＋（Ｙｐ１ｗ−Ｙｐ２ｗ）^２ ＋（Ｚｐ１ｗ−Ｚｐ２ｗ）^２ ）^１／２
上述のようにして、領域特徴抽出手段９によって検知対象物の高さＺｐｗおよび幅ｄ１２を求めることができるから、検知判断手段１０では、高さＺｐｗおよび幅ｄ１２がともに規定した判別範囲内であるときに想定した検知対象物が監視空間に存在すると判定する。
【００６２】
いま、検知対象物を人体と想定すると、代表点Ｐは検知対象物の中心付近に位置するから頭部または肩部と考えられ、代表点Ｐ１，Ｐ２の距離は肩幅に相当すると考えられる。そこで、高さＺｐｗに対する判別範囲を８０〜２００ｃｍ程度に設定するとともに、幅ｄ１２に対する判別範囲を２０〜５０ｃｍ程度に設定すれば、監視空間内における人体の存否を検知することが可能になる。なお、本例において代表点として３点を用いているが、４点以上であってもよく、また検知対象物の大きさを評価するための２点を代表点とし、高さについては一方の代表点の高さを用いることも可能である。
【００６３】
（参考例４）
本例は基本的には図１５に示した参考例３と同様の構成であるが、代表距離計算手段８において重心を代表点として求める代わりに、領域選択手段７により得られた２値画像の白画素の領域内で距離画像の画素値が最小になる画素の座標を求める。この画素は白画素の領域内であるから、検知対象物の範囲内の画素である可能性が高く、かつ白画素の領域内で距離画像における画素値が最小になる画素であるから、検知対象物において３次元位置測定装置１からの距離が最小になる部位と考えられる。すなわち、検知対象物においてもっとも高い部位に対応している可能性が高いといえる。
【００６４】
そこで、３次元位置測定装置１を視野の中心線が鉛直下向きになるようにして天井Ｃに設置し、３次元位置測定装置１の下を通過する人体の頭部を検出する目的に使用するとすれば、本例では検知判断手段１０において高さに関して設定する判別範囲を参考例３よりも狭い範囲に設定することが可能になる。これは、参考例３では距離画像から求めた２値画像における白画素の領域の代表点として重心を用いていることにより、代表点が検知対象物におけるもっとも高い部位になる可能性が十分に高いとは言えないのに対して、本例の構成では代表点として３次元位置測定装置１からの距離が最小になる部位を選択したことにより、検知対象物におけるもっとも高い部位に対応している可能性が高くなるからである。しかして、検知判断手段１０において高さに関して設定する判別範囲を、参考例３では８０〜２００ｃｍに設定しているとすれば、本例では、たとえば１２０〜２００ｃｍに設定することが可能になる。他の構成および動作は参考例３と同様であって、本例の構成を採用すれば、参考例３に比較して判別範囲を狭くしたことによって、誤検出の可能性を低減することができる。
【００６５】
（参考例５）
本例は、図１７に示すように、基本的には図１５に示した参考例３と同様の構成を有する。ただし、本例の検知判断手段１０には、人体である検知対象物の代表点の高さについて時間変化を追跡することによって人体である検知対象物の転倒を検知する転倒検知手段１０ａを備える点が相違する。追跡する代表点としては参考例３のように距離画像を２値化した２値画像における白領域の重心または参考例４のように白画素の領域内で３次元位置測定装置１からの距離が最小になる点を用いる。
【００６６】
本例の特徴である転倒検知手段１０ａでは、領域特徴抽出手段９において求めた代表点の床面からの高さを逐次記憶する。すなわち、３次元位置測定装置１において一定時間毎に距離画像を生成し、生成された距離画像に基づいて代表点の高さの変化を追跡する。ところで、人の頭部の高さの時間変化は、人が座るときには図１８にイで示す変化になり、人が転倒したときには図１８にロで示す変化になる。つまり、図１８によれば、人が転倒するときには短時間で頭部の高さが大きく減少することがわかる。ちなみに、図示例では人が座るときの１秒間の高さ変化は最大で７０ｃｍ程度であるのに対して、人が転倒するときの１秒間の高さ変化は１２０ｃｍ以上になっている。そこで、たとえば３次元位置測定装置１において距離画像を１秒毎に生成し、距離画像から得られる代表点の高さの変化が１００ｃｍを越えたときに検知対象物である人体が転倒したと判定するように転倒検知手段９を構成すれば、座る行為と転倒とを区別しながらも転倒を検知することが可能になる。
【００６７】
なお、上述した各例において示した数値は一例であって、目的に応じて適宜に設定すればよく、とくに参考例５においては、転倒の判断に１秒毎の高さの変化率を用いるとしても、距離画像を生成する時間間隔をより短い時間（たとえば、０．１秒）に設定することによって転倒の瞬間をより確実に捉えることが可能になる。要するに、転倒検知手段１０ａでは代表点の単位時間あたりの高さの変化幅が規定値を越えたときには検知対象物が転倒したと判断するのである。
【００６８】
【発明の効果】
請求項１の発明は、監視空間内の全領域について物体までの距離を測定する３次元位置測定装置と、監視空間を撮影した画面上の各画素の画素値が３次元位置測定装置により測定した距離である距離画像を記憶する画像取込手段と、検知対象物について３次元位置測定装置からの距離の分布パターンを基準パターンとして記憶するパターン記憶手段と、前記距離画像内で基準パターンを走査するとともに距離画像内での基準パターンの各位置において距離画像と基準パターンとの差分の画素値からなる差画像を抽出する差画像抽出手段と、差画像抽出手段により抽出した差画像の各画素値と一定値との差分を残差として求め、残差の絶対値の総和の大きさを距離画像と基準パターンとの一致度として求める一致度演算手段と、一致度演算手段により求めた一致度を閾値と大小比較し一致度が規定の閾値より小さいときに監視空間内で基準パターンの検知対象物が存在すると判定する判定手段とを備えるものであり、距離画像に対する基準パターンのパターンマッチングを行うことに相当するから、距離画像を用いながらも比較的少ない演算処理で検知対象物の存否を正確に判断することが可能になる。とくに、一致度には差画像の画素値のばらつきの程度を用いているから、四則演算程度の簡単な演算によって少ない演算量で一致度を求めることが可能になる。
【００６９】
請求項２の発明は、請求項１の発明において、前記パターン記憶手段には前記検知対象物について大きさの異なる相似形状の複数種類の基準パターンが登録されているものであり、監視空間内で検知対象物までの距離が比較的大きく変化したり相似であるが大きさの異なる検知対象物が監視空間に存在する場合であっても、監視空間内での検知対象物の存否を容易に抽出することができる。
【００７０】
請求項３の発明は、請求項１の発明において、前記距離画像内から前記検知対象物に対応する領域を抽出する領域選択手段が付加され、前記距離画像から領域選択手段により抽出された領域が基準パターンとしてパターン記憶手段に格納されるものであり、現実の検知対象物を用いて基準パターンを設定することになるから、検知対象物に合致した形状の基準パターンを設定することができ、特定の検知対象物が監視空間内を移動するような場合でも他の物体と区別して追跡することが可能になる。
【００７１】
請求項４の発明は、請求項３の発明において、前記監視空間内を移動する前記検知対象物に適用する距離画像処理装置であって、前記領域選択手段が、監視空間内に検知対象物が存在している時間内の異なる時刻の２つの距離画像から差分である差分距離画像を求める差分計算手段と、差分距離画像において画素値が０ではない規定の閾値範囲内に属する画素からなる領域を前記距離画像から抽出し基準パターンとしてパターン記憶手段に格納する変化領域抽出手段とを備えるものであり、監視空間内で検知対象物が移動する場合に、検知対象物の移動によって生じた２つの距離画像間の差を利用して基準パターンを設定しているから、監視空間内で移動した検知対象物について基準パターンを自動的に生成することができる。
【図面の簡単な説明】
【図１】 本発明の第１の実施の形態を示すブロック図である。
【図２】 同上の動作説明図である。
【図３】 本発明の第２の実施の形態の原理説明図である。
【図４】 同上の動作説明図である。
【図５】 同上の動作説明図である。
【図６】 本発明の第３の実施の形態を示すブロック図である。
【図７】 同上の動作説明図である。
【図８】 本発明の第４の実施の形態を示す要部のブロック図である。
【図９】 同上の動作説明図である。
【図１０】 本発明の参考例１を示すブロック図である。
【図１１】 同上の動作説明図である。
【図１２】 同上の原理説明図である。
【図１３】 同上の原理説明図である。
【図１４】 本発明の参考例２の動作説明図である。
【図１５】 本発明の参考例３を示すブロック図である。
【図１６】 同上の動作説明図である。
【図１７】 本発明の参考例５を示すブロック図である。
【図１８】 同上の原理説明図である。
【符号の説明】
１ ３次元位置測定装置
２ 画像取込手段
３ パターン記億手段
４ 差画像抽出手段
５ 一致度演算手段
６ 判定手段
７ 領域選択手段
７ａ 差分計算手段
７ｂ 変化領域抽出手段
８ 代表距離抽出手段
９ 領域特徴抽出手段
１０ 検知判断手段
１０ａ 転倒検知手段[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a distance image processing apparatus used for determining the presence or absence of a detection object in a monitoring space using a distance image or extracting a feature amount of a detection object.
[0002]
[Prior art]
Conventionally, various image processing techniques have been proposed in order to determine the presence / absence of a detection target in the monitoring space based on the image in the monitoring space and to extract the feature amount of the detection target. In this type of image processing technique, a method of separating a detection target object from a background based on a grayscale image or a color image is widely adopted.
[0003]
However, in the case of grayscale images and color images, due to the positional relationship between the imaging means for capturing an image of the surveillance space and the detection target object, the presence of disturbance light, etc., the same detection target object is affected by density changes and color changes in the image. Differences occur, and in particular, the shadow area around the detection object changes. Therefore, there is a problem that false detection is likely to occur in applications where the positional relationship of the detection object with respect to the imaging means and disturbance light are likely to fluctuate. Although various proposals have been made to solve this problem, the influence of disturbance light cannot be sufficiently removed in principle for grayscale images and color images.
[0004]
On the other hand, it has been proposed to use a distance image in which a pixel value of each pixel in the image is associated with a distance from a specified reference point to an object corresponding to the pixel in the monitoring space. The reference point is set on the basis of the installation position of the three-dimensional position measuring device for obtaining the distance to the detection target. There are two types of technologies for obtaining the distance from the reference point to the object in the monitoring space: an active type that projects detection light onto the monitoring space and a passive type that does not project detection light. The active type is linear or fan-shaped. A method of scanning a light beam in a monitoring space is widely known, and a stereo image method using a plurality of TV cameras is widely known as a passive type. In either method, the distance to the object in the monitoring space can be obtained by using the principle of triangulation. Since this type of distance image is not easily affected by disturbance light, if the distance image is used, information regarding the detection target can be accurately obtained, and erroneous detection due to the influence of disturbance light is less likely to occur.
[0005]
By the way, in the technique of generating a distance image by the stereo image method, it is necessary to find corresponding points of a plurality of images, and in order to find the corresponding points, an edge portion having a large density change is extracted from the image. Currently, the association is performed. That is, since the distance can be obtained relatively accurately for the edge portion, the three-dimensional information obtained from the edge portion is compared with the edge of the three-dimensional model of the detection object registered in advance, and the degree of coincidence between the two is obtained. Whether or not the edge of the detection target is detected.
[0006]
[Problems to be solved by the invention]
As described above, the technology that obtains information related to the detection target using the information of the edge part in the image is used in the stereo imaging method to change the density in the image like the plane part of the detection target. This is because it is difficult to directly determine the distance for a region with a small area, and this type of information cannot be effectively used even though information other than the edge portion is included in the image.
[0007]
On the other hand, if the above-described active technique is used, the distance can be directly obtained even if it is other than the edge part such as the planar part of the detection target. Information on the detection target can be obtained more accurately.
[0008]
However, with respect to the degree of coincidence between the pre-registered three-dimensional model and the information about the detection target obtained from the distance image, the same pattern matching method as the image processing technique employed in the gray image and the color image is used as it is. It cannot be adopted. That is, in the distance image, the pixel value of each pixel is the distance from the reference point, and pattern matching is performed only by positioning (parallel movement and rotational movement) in a two-dimensional plane, such as a grayscale image or a color image. Therefore, the registered three-dimensional model must be aligned in the three-dimensional space with respect to the detection target obtained from the distance image. That is, if the distance and orientation of the three-dimensional model are not adjusted in the three-dimensional space, pattern matching with the detection target cannot be performed, and enormous operations are required to adjust the distance and orientation in the three-dimensional space. It is necessary and difficult to process in real time.
[0009]
The present invention has been made in view of the above-mentioned reasons, and its purpose is to accurately detect the presence / absence of a detection object and the extraction of the feature amount of the detection object with relatively few calculations while using a distance image. It is an object of the present invention to provide a range image processing apparatus that can be used.
[0010]
[Means for Solving the Problems]
  According to the first aspect of the present invention, a three-dimensional position measurement device that measures the distance to an object for all areas in the monitoring space, and a pixel value of each pixel on a screen that images the monitoring space is measured by the three-dimensional position measurement device. An image capturing unit that stores a distance image that is a distance, a pattern storage unit that stores a distribution pattern of a distance from the three-dimensional position measurement device as a reference pattern for the detection target, and a reference pattern that is scanned in the distance image And a difference image extracting means for extracting a difference image composed of pixel values of a difference between the distance image and the reference pattern at each position of the reference pattern in the distance image, and a difference image extracted by the difference image extracting means.eachPixel valueThe difference between the value and the constant value is calculated as the residual, and the total sum of absolute values of the residuals is calculated.The degree of coincidence calculation means to obtain the degree of coincidence between the distance image and the reference pattern, and the degree of coincidence obtained by the degree of coincidence calculation means are compared with a threshold value.When the degree of coincidence is smaller than the specified thresholdIn the surveillance spaceReference patternDetection objectIf there isAnd determining means for determining.
[0011]
A second aspect of the invention is characterized in that, in the first aspect of the invention, a plurality of types of reference patterns having similar shapes with different sizes are registered in the pattern storage means.
[0012]
According to a third aspect of the present invention, in the first aspect of the present invention, a region selection unit that extracts a region corresponding to the detection target object from the distance image is added, and a region extracted from the distance image by the region selection unit is provided. It is stored in the pattern storage means as a reference pattern.
[0013]
The invention of claim 4 is the distance image processing device applied to the detection object moving in the monitoring space according to the invention of claim 3, wherein the area selection means has a detection object in the monitoring space. A difference calculation means for obtaining a difference distance image that is a difference from two distance images at different times within the existing time, and an area composed of pixels belonging to a prescribed threshold range in which the pixel value is not 0 in the difference distance image And a change area extraction unit that extracts the distance image and stores it in a pattern storage unit as a reference pattern.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
As shown in FIG. 1, the present embodiment shows an example in which a three-dimensional position measuring device 1 is attached to a ceiling C so that the room R is a monitoring space. The three-dimensional position measuring apparatus 1 is assumed to use the active type described as the prior art, and projects a linear or fan-shaped light beam from the light projecting means to the monitoring space and the light beam in the monitoring space. And monitoring the position of the bright spot or bright line formed by the light beam using a semiconductor position detection element such as PSD or a TV camera as the light receiving means, to the site where the bright spot or bright line is formed Is configured to determine the distance. The technique for obtaining the distance to the part where the bright spot or bright line is formed in the monitoring space is well known and will not be described in detail. However, the positional relationship between the light projecting means and the light receiving means is known, and the light beam The distance can be calculated geometrically based on the principle of triangulation using the relationship between the light projecting direction, the position of the light projecting means, and the position of the bright spot or bright line image formed on the light receiving means. Accordingly, by scanning the light beam in the monitoring space as described above, the distance to the object on which the bright spot or the bright line is formed can be obtained for the entire monitoring space.
[0020]
The distance obtained by the three-dimensional position measuring apparatus 1 is input to the image capturing unit 2 and temporarily stored in the image capturing unit 2. In the image capturing means 2, the area for storing the distance is associated with the light beam projection direction, and the data stored in the image capturing means 2 is a distance image. That is, the distance obtained by the three-dimensional position measurement apparatus 1 is stored at an address corresponding to each pixel in the image, and as a result, a distance image having the pixel value of each pixel as a distance is obtained.
[0021]
In the present embodiment, a pattern matching technique is used to determine whether or not the detection target is included in the distance image stored in the image capturing unit 2. Therefore, pattern storage means 3 for storing a reference pattern to be compared with the distance image stored in the image capturing means 2 and difference image extraction means 4 for extracting a difference image between the distance image and the reference pattern are provided. .
[0022]
The reference pattern stored in the pattern storage means 3 is three-dimensional data related to the detection target, and is created using CAD data in this embodiment. In other words, if CAD data is projected onto a projection plane in an appropriate direction, a reference pattern equivalent to a distance image having the projection plane as a screen and the distance from the three-dimensional position measurement apparatus 1 as a pixel value can be obtained. The reference pattern thus obtained is three-dimensional data in which a distance is associated with each position in the two-dimensional plane, and becomes a distribution pattern of the distance from the three-dimensional position measuring device 1 for the detection target. In the reference pattern storage unit 3, a reference pattern generated by projecting CAD data on a projection surface in any direction in which the detection target is expected to be projected in the monitoring space is registered. The difference image extraction unit 4 obtains a difference image using pixel values as differences between the distance image stored in the image capturing unit 2 and a plurality of reference patterns stored in the reference pattern storage unit 3. In obtaining the difference image, each reference pattern is scanned in the distance image stored in the image capturing means 2, and the reference pattern in the two-dimensional plane is scanned as in the case of pattern matching with respect to a grayscale image or a color image. Since scanning is performed, processing for one reference pattern is completed in a relatively short time.
[0023]
Now, if the reference pattern coincides with the part including the detection target in the distance image stored in the image capturing means 2, it is considered that the pixel values of the pixels in the difference image are all equal. That is, the direction of the detection target represented by the CAD data for generating the reference data in the reference pattern storage unit 4 matches the direction of the detection target in the monitoring space. It can be regarded as a state in which the images can be overlapped only by being translated without being rotated.
[0024]
Therefore, if the degree of variation in the pixel value of each pixel in the difference image extracted by the difference image extraction unit 4 as described above is evaluated, the degree of coincidence between the data of the detection target in the monitoring space and the reference pattern And the presence / absence of the detection target in the monitoring space can be determined, and the orientation of the detection target in the monitoring space can be known as a feature amount. As a method for evaluating the degree of variation in the pixel value of each pixel in the difference image, a general method for evaluating the degree of variation can be used. In this embodiment, the difference between each pixel value and a fixed value is obtained as a residual, and the magnitude of the absolute value of the residual (referred to as “residual absolute sum”) is evaluated as the degree of variation. This constant value can be set as appropriate, but it is desirable to use the average value of the pixel values of the difference image. The residual absolute sum is obtained by the degree-of-matching calculating means 5 and the obtained residual absolute sum is inputted to the judging means 6 to determine whether the detection target is present in the monitoring space by determining whether it is a predetermined threshold or not. can do. Further, the direction of the detection object in the monitoring space can be known from the reference pattern when the residual sum is minimized. Here, in the present embodiment, it is assumed that the reference pattern does not need to be enlarged or reduced with respect to the distance image stored in the image capturing unit 2, but the monitoring space is relatively wide and the distance image, the reference pattern, and the like. If the ratios are different, enlargement / reduction may be performed on one of the distance image and the reference pattern.
[0025]
Below, operation | movement of the structure mentioned above is demonstrated easily collectively. In order to simplify the description, the distance image will be described as one-dimensional data instead of two-dimensional data. That is, it is assumed that the distance is obtained for the position on the straight line. The distance image (indicated by a) stored in the image capturing means 2 and the reference pattern (indicated by B, B ′) stored in the pattern storage means 3 have a relationship as shown in FIG. The reference pattern (b) (b ′) is scanned with respect to the distance image (b), and moves from the position of the reference pattern (b) to the position of the reference pattern (b ′), for example. The difference image extraction means 4 obtains a difference image from the distance image by shifting the position of the reference pattern (b) (b ′) by a certain number of pixels. As shown in FIG. 2B, each pixel value in the difference image (that is, the difference between the distance image (b) and the reference pattern (b) (b ′)) varies with respect to the reference pattern (b). The variation is large with respect to the reference pattern (b ').
[0026]
In order to evaluate the degree of variation in the pixel values in the difference image, the coincidence calculation means 5 obtains an average value of the pixel values in the difference image, and obtains each pixel value in the difference image as shown in FIG. The difference from the average value is obtained as a residual. Further, the coincidence calculation means 5 obtains an absolute residual sum that is the sum of absolute values of residuals. The average value of the pixel values in the difference image is an offset corresponding to the difference between the distance between the three-dimensional position measurement device 1 and the detection target and the distance to the projection plane onto which the CAD data is projected when generating the reference pattern. The degree-of-matching calculation means 5 can be regarded as removing the offset from the pixel value of the difference image. The determination means 6 determines that the detection target represented by the reference pattern exists in the monitoring space when the residual absolute sum obtained as described above is smaller than a predetermined threshold value set in advance.
[0027]
As apparent from FIG. 2, the sum of the pixel values of the difference image may be smaller when the degree of coincidence with the reference pattern is lower than when the degree of coincidence is high, but it is not the sum of the pixel values of the difference image. By using the sum of the absolute values of the residuals obtained by subtracting a certain value corresponding to the offset from the pixel value of the difference image, the larger the variation of the pixel value, the greater the accuracy, and the degree of coincidence with the reference pattern can be accurately evaluated. it can.
[0028]
(Second Embodiment)
As described in the first embodiment, if the distance from the three-dimensional position measuring device 1 to the detection target changes, the size of the detection target changes in the distance image. When the distance changes relatively greatly, it becomes impossible to accurately obtain the degree of coincidence with the reference pattern. Therefore, in the present embodiment, a plurality of reference patterns having different sizes with respect to the same detection target are stored in the pattern storage unit 3.
[0029]
That is, as shown in FIG. 3, when the distances to the projection planes PL1, PL2, and PL3 on which the CAD data is projected are different, the sizes S1, S2, and S3 of the reference patterns obtained from the same CAD data are different. Therefore, as shown in FIGS. 4A, 4B, and 4C, a plurality of reference patterns B1, B2, and B3 having different sizes (three in the illustrated example) are registered in the pattern storage means 3 in advance. The degree of coincidence of each reference pattern B1, B2, B3 with respect to the distance image stored in the image reading means 2 is obtained.
[0030]
Accordingly, it is assumed that a distance image as shown in FIG. 5A is stored in the image capturing means 2, and reference patterns B3, B2, which are shown as lines a in FIGS. 5B, 5C and 5D, respectively. If B1 is registered in the pattern storage means 3, the residuals are shown as lines b in FIGS. 5B, 5C and 5D, respectively. That is, it can be said that the residual absolute sum is the smallest with respect to the reference pattern B2, and the degree of coincidence with the reference pattern B2 is the highest. In this way, even if the size of the area occupied by the detection target in the distance image changes, it is possible to accurately obtain the degree of coincidence with the reference pattern.
[0031]
In the configuration of the present embodiment, since a plurality of reference patterns having similar shapes are stored in the pattern storage unit 3, when the change in the distance to the detection target is relatively large or similar, a plurality of similar but different sizes are stored. Even if there are individual detection objects, the degree of coincidence between the distance image and the reference pattern can be obtained to extract the determination of the presence / absence of the detection object. Other configurations and operations are the same as those of the first embodiment.
[0032]
(Third embodiment)
In the present embodiment, as shown in FIG. 6, a region selecting means 7 is added to the configuration of the first embodiment. The area selecting means 7 binarizes the distance image stored in the image capturing means 2 to extract the area where the detection target exists from within the distance image, and uses the data of the area extracted from the distance image as a reference It has a function of registering it in the pattern storage means 3 as a pattern. The region selection unit 7 is used when registering a reference pattern in the pattern storage unit 3 and is used when determining the presence or absence of a detection target or extracting a feature amount by pattern matching between a distance image and a reference pattern. Not in.
[0033]
The operation of the area selection unit 7 when registering the reference pattern in the pattern storage unit 3 will be described in more detail. Two threshold values Lt1 and Lt2 (Lt1 <Lt2) are set for the distance in the area selecting means 7, and it belongs to the distance range between the threshold values Lt1 and Lt2 from the distance image stored in the image capturing means 2. Extract pixels. That is, if the coordinates of each pixel of the distance image are represented by (x, y) and the pixel value of each pixel is represented by D (x, y), the pixel satisfying Lt1 <D (x, y) <Lt2 A set of (x, y) is extracted and binarized so that pixels (x, y) belonging to the set are white pixels (pixel value is 1), and the remaining are black pixels (pixel value is 0). Since the mask can be formed by binarizing the distance image in this way, only the region corresponding to the white pixel is extracted from the distance image by superimposing the mask on the distance image. In this way, the region selection unit 7 can extract a set of pixels in a partial region from the distance image, and the set of pixels is stored in the pattern storage unit 3 as a reference pattern.
[0034]
That is, if a distance image as shown in FIG. 7A is stored in the image capturing means 2, the area selecting means 7 first binarizes the distance image as shown in FIG. 7B. A binary image is generated. When a white pixel region in the binary image is extracted from the distance image, a reference pattern as shown in FIG. 7C can be generated.
[0035]
The reference pattern registered in the pattern storage means 3 is the same as that in the first embodiment, and the difference image extraction means 4 extracts the difference image between the distance image stored in the image capture means 2 and the reference pattern. Then, the degree of variation of the pixel value of the difference image is evaluated by the coincidence degree calculating means 5, and when the degree of variation of the pixel value of the difference image is smaller than a prescribed threshold value, the determination means 6 It is determined that there is a region with a high degree of coincidence. Other configurations and operations are the same as those of the first embodiment.
[0036]
In the configuration of this embodiment, it is not necessary to set a reference pattern by CAD data for a detection target to be extracted, and a reference pattern is extracted from a distance image related to the detection target that is actually detected by the three-dimensional position measurement apparatus 1. Therefore, even if the shape of the detection target is complicated, the reference pattern can be easily generated and the reference pattern can be easily changed. Moreover, since the reference pattern is generated from the distance image obtained by the three-dimensional position measuring apparatus 1, it is possible to set a reference pattern having a shape that matches the detection target, and the specific detection target moves in the monitoring space. Even in such a case, it becomes possible to track the object clearly from other objects.
[0037]
(Fourth embodiment)
As in the third embodiment, the present embodiment is provided with the region selection means 7, but as the region selection means 7, as shown in FIG. 8, the difference between two distance images that are sequentially input is obtained. A device provided with a difference calculation means 7a and a change area extraction means 7b for extracting an area where a change has occurred in the distance image due to the difference between the distance images obtained by the difference calculation means 7a is used. The change area extraction unit 7b binarizes the difference of the distance image and performs expansion processing and contraction processing on the white pixel area obtained by the binarization to fill the white pixel area, A mask having a continuous region of white pixels is formed. Further, the change area extraction unit 7b extracts a region corresponding to the white pixel from the distance image as a reference pattern by overlaying a mask on the distance image input later from the two distance images input to the region selection unit 7. To do.
[0038]
The operation of the area selection unit 7 when registering the reference pattern in the pattern storage unit 3 will be described in more detail. Two distance images obtained at different times within the time during which the detection target exists in the monitoring space are input from the image capturing unit 2 to the difference calculation unit 7a provided in the region selection unit 7. The shorter the time difference between the two distance images, the better because the mask can be set smaller. However, the larger the time difference between the two distance images, the smaller the amount of calculation. Therefore, the time difference between the two distance images is set appropriately according to the desired amount of calculation. To do.
[0039]
The difference calculation means 7a obtains a difference distance image that is a difference between both distance images. If there is no moving object in the monitoring space, the pixel values of all the pixels in the difference distance image are 0, but if there is a moving object, a pixel whose pixel value is not 0 is generated. That is, if the detection target object moves in the monitoring space, the pixel value of the pixel corresponding to the region where the detection target object exists in the two input distance images becomes a non-zero value in the difference distance image.
[0040]
In the change area extraction unit 7b, two threshold values Lt3 and Lt4 (Lt3 <Lt4) are set for the distance in the same manner as the area selection unit 7 in the third embodiment, and both threshold values Lt3 and Lt4 are set from the difference distance image. Pixels belonging to the range between are extracted. That is, if the coordinate of each pixel in the difference distance image is represented by (x, y) and the pixel value of each pixel is represented by S (x, y), Lt3 <S (x, y) <Lt4 is satisfied. A set of pixels (x, y) is extracted, and binarized so that the pixels (x, y) belonging to the set are white pixels (pixel value is 1) and the rest are black pixels (pixel value is 0). If binarization is performed in this manner, a set of pixels whose pixel values are not 0 in the difference distance image can be extracted as white pixels. Therefore, there is an object moving in the surveillance space in the white pixel region. It represents the area that had been. Here, since the white pixel region is not necessarily one continuous region, the mask is generated by filling the hole so that one continuous white pixel region is formed.
[0041]
As described above, by superimposing a mask on the distance image input later from the two distance images and extracting the pixels in the region corresponding to the white pixels, the extracted pixels can be used as the reference pattern. become. That is, the extracted pixel is stored in the pattern storage unit 3 as a reference pattern.
[0042]
That is, if two distance images as shown in FIGS. 9A and 9B are sequentially stored in the image capture means 2, the difference calculation means 7a of the area selection means 7 uses each pixel of both distance images. The difference between the values is obtained to obtain a difference distance image as shown in FIG. Next, the change area extraction means 7b binarizes the difference distance image to generate a binary image as shown in FIG. In the binary image obtained in this way, it is considered that the detection object is included in the area where the white pixels are distributed. However, since the black pixel is mixed in the white pixel, it is considered that the detection object exists. A hole is filled with white pixels so that all the pixels in the region to be white pixels are generated, and as a result, a mask as shown in FIG. 9E is generated. When the mask thus obtained is superimposed on the distance image of FIG. 9B input later, the reference pattern shown in FIG. 9F can be generated.
[0043]
The reference pattern registered in the pattern storage means 3 is the same as that in the first embodiment, and the difference image extraction means 4 extracts the difference image between the distance image stored in the image capture means 2 and the reference pattern. Then, the degree of variation of the pixel value of the difference image is evaluated by the coincidence degree calculating means 5, and when the degree of variation of the pixel value of the difference image is smaller than a prescribed threshold value, the determination means 6 It is determined that there is a region with a high degree of coincidence. Other configurations and operations are the same as those of the first embodiment.
[0044]
Therefore, in the configuration of the present embodiment, when the detection object moves in the monitoring space, the reference pattern is set using the difference between the two distance images generated by the movement of the detection object. The reference pattern can be automatically generated for the detection object moved in the space, and the detection object can be easily detected and tracked after the reference pattern is set.
[0045]
  (Reference example 1)
  In each of the above-described embodiments, an example in which the presence / absence of a detection target in the monitoring space is mainly illustrated is shown.This exampleIs mainly intended to extract the feature amount of the detection object. Below, the example which calculates | requires the height dimension of the representative point of the detection target object from the floor surface F (refer FIG. 10) in the room R is demonstrated.
[0046]
  This exampleThen, since it is not necessary to determine the degree of coincidence of the detection target object with the reference pattern, it is sufficient to extract a region where the detection target object exists in the distance image. As shown in FIG. The same area selection means 7 as that of the embodiment is provided. Further, the configuration necessary for obtaining the degree of coincidence with the reference pattern in the third embodiment, that is, the pattern storage means 3, the difference image extraction means 4, the coincidence degree computing means 5, and the judging means 6 are not provided. A representative distance calculating means 8 and an area feature extracting means 9 described later are provided.
[0047]
The area selecting unit 7 generates a mask for extracting an area where the detection target exists from the distance image stored in the image capturing unit 2. That is, binarization is performed to extract a region having a pixel value between two threshold values for the distance image, and a white pixel region of the binary image obtained by binarization is set as a region where the detection target exists. .
[0048]
The representative distance calculation means 8 obtains the coordinates (xsi, ysi) of the center of gravity of the white area in the binary image obtained by the area selection means 7, and the pixel value D (xsi, ysi) of the coordinates (xsi, ysi) in the distance image. ysi) is obtained as a representative distance L from the reference point set according to the position of the three-dimensional position measuring apparatus 1 to the representative point of the detection object. Here, since the reference point is set based on the installation position of the three-dimensional position measuring apparatus 1, the position of the reference point is known, and the direction connecting the reference point and the representative point is also from the three-dimensional position measuring apparatus 1. Since the representative distance L is obtained, the region feature extraction means 9 can obtain the height of the representative point from the floor surface F.
[0049]
That is, if a distance image as shown in FIG. 11A is stored in the image capturing means 2, the area selecting means 7 first binarizes the distance image as shown in FIG. 11B. A binary image is generated. Next, as shown in FIG. 11C, the coordinates (xsi, ysi) of the center of gravity of the white pixel region in the binary image are obtained, and the coordinates (xsi, ysi) in the distance image are obtained as shown in FIG. The pixel value D (xsi, ysi) of ysi) is obtained as the representative distance L.
[0050]
Hereinafter, a method of calculating the height of the representative point based on the representative distance L in the region feature extraction unit 9 will be described with reference to FIGS. 12 and 13. Here, the two-dimensional coordinate system in the screen of the distance image is called a screen coordinate system Ci, and the coordinate system set in the real space is called a world coordinate system Cw. In the screen coordinate system Ci, the center of the screen is the origin Oi. The world coordinate system Cw sets the Zw axis direction vertically upward with respect to the floor surface of the room R, which is the monitoring space, and sets the origin Ow directly below the three-dimensional position measuring apparatus 1 on the floor surface. That is, if the height from the floor surface of the three-dimensional position measuring apparatus 1 is Zsw, the origin Oc of the apparatus coordinate system Cc is located on the Zw axis of the world coordinate system Cw and shifted by the height Zcw in the Zw axis direction. Will do. Further, it is assumed that the Zc axis of the apparatus coordinate system Cc forms an angle θ with respect to the Zw axis of the world coordinate system Cw. In the world coordinate system Cw, the Xw axis direction and the Yw axis direction are appropriately set according to the shape of the monitoring space. Further, in the field of view of the three-dimensional position measuring apparatus 1, the direction that coincides with the X-axis direction and the Y-axis direction of the screen of the distance image is the Xc-axis direction and the Yc-axis direction, and the front of the three-dimensional position measuring apparatus 1 is the Zc-axis The coordinate system for the direction is referred to as a device coordinate system Cc. The origin of the apparatus coordinate system Cc is set to a position on the center line of the field of view of the three-dimensional position measuring apparatus 1 and serving as a reference point for distance measurement. For example, if a TV camera is used as the light receiving means of the three-dimensional position measuring apparatus 1, the focal position of the photographing lens is set to the origin Oc of the apparatus coordinate system Cc. That is, if the focal length of the lens is fc, the origin Oi of the screen coordinate system Ci is located at a distance fc from the origin Oc of the apparatus coordinate system Cc.
[0051]
By the way, when the coordinates (xsi, ysi) of the representative point (the center of gravity of the white pixel region) of the detection object are obtained for the distance image, the coordinate value is obtained in units of the number of pixels, but it is representative in the screen coordinate system Ci. In order to obtain the height of the representative point from the floor surface F in the world coordinate system Cw based on the coordinates (xsi, ysi) of the point, it is necessary to convert the unit into a unit of length. For example, if a TV camera is used as the light receiving means of the three-dimensional position measuring means 1 and the physical size of the light receiving surface is Sxr × Syr [mm] and the number of pixels is Sxi × Syi [pixel], On the light receiving surface, the pitch between pixels in the X-axis direction is Sxr / Sxi [mm], and the pitch between pixels in the Y-axis direction is Syr / Syi [mm]. Accordingly, the coordinates (xsc, ysc) when the unit is millimeters on the light receiving surface of the representative point of the detection target can be obtained as follows.
xsc = xsi × (Sxr / Sxi)
ysc = ysi × (Syr / Syi)
Here, if the coordinates of the screen coordinate system Ci are expressed in millimeters, the device coordinate system Cc has the origin displaced by fc in the Z-axis direction, so the coordinates (xsc, ysc) in the screen coordinate system Ci are changed. The coordinates (Xsc, Ysc, Zsc) in the corresponding apparatus coordinate system Cc are (Xsc, Ysc, Zsc) = (xsc, ysc, fc).
[0052]
As described above, since the Zc axis of the apparatus coordinate system Cc forms an angle θ with respect to the Zw axis of the world coordinate system Cw, the coordinates (Xsc, Ysc, Zsc) in the apparatus coordinate system Cc and the world coordinate system Cw The relationship with the coordinates (Xsw, Ysw, Zsw) of In other words, the coordinate conversion from the apparatus coordinate system Cc to the world coordinate system Cw is performed according to Equation 1.
[0053]
[Expression 1]

[0054]
The coordinates (Xpw, Ypw, Zpw) of the representative point P of the detection target Ob in the world coordinate system Cw corresponding to the coordinates (Xsw, Ysw, Zsw) of the obtained point R are perpendicular to the right triangle OcPQ shown in FIG. It can be obtained using the similarity to the triangle OcRT. Focusing on the fact that the distance between the point Oc and the point P is the above-described representative distance L, and the distance Lc between the point Oc and the point R is the length of the hypotenuse of the right triangle OcOiR, it can be obtained by the following equation.
Lc = (Xsc²+ Ysc²+ Fc²)^1/2
Accordingly, the height Zpw of the point P is obtained as follows.
Zpw = L (Zcw−Zsw) / Lc
That is, if the height of the three-dimensional position measuring device 1 is given as a constant in advance, the height from the floor surface F can be obtained for the representative point P of the detection object Ob. In short, using the physical specifications (the physical size of the light receiving surface, the number of pixels, the height from the floor surface F, the direction of the optical axis, etc.) of the three-dimensional position measuring apparatus 1 and the representative distance L, the world coordinates from the screen coordinate system Ci. Coordinate conversion to the system Cw becomes possible. Note that the functions of the three-dimensional position measuring apparatus 1 and the image capturing means 2 are the same as those in the first embodiment.
[0055]
  (Reference example 2)
  This exampleIs shown in FIG.Reference example 1Use the same configuration asReference example 1In contrast, the center of gravity of the white pixel region extracted by binarizing and extracting the distance image is used as the representative point of the detection target.This exampleThen, two representative points are obtained for the detection object, and the size of the detection object is extracted as a feature amount by obtaining the distance between the two representative points.
[0056]
  That is,This exampleIn the area of white pixels in the binary image as shown in FIG. 14B obtained by binarizing the distance image as shown in FIG. As in (c), the coordinates (xs1, ys1) (xs2, ys2) of the two representative points are obtained. Both representative points are set as two points apart from the periphery of the white pixel so that the distance between the two representative points corresponds to the size of the object to be detected.This exampleIn this case, the pixels in the white pixel are extracted as representative points at the intersection of the straight line extending in the horizontal direction (y-axis direction) of the screen of the distance image through the center of gravity of the white pixel region and the black pixel region. In short, after the center of gravity is obtained, the binary image is scanned in the horizontal direction, and if the pixel value becomes a black pixel (pixel value = 0), the previous white pixel is set as the representative point pixel. The coordinates of the representative points thus obtained are collated with the distance image, and the pixel values of the pixels corresponding to the respective representative points are obtained as the representative distances L1 and L2 to the detection target as shown in FIG.
[0057]
  When the representative distances L1 and L2 corresponding to the two representative points and the representative points are obtained, the area feature extraction unit 9Reference example 1Similarly, the coordinates (xs1, ys1) (xs2, ys2) of each representative point are converted into the device coordinate system Cc to (Xsc1, Ysc1, Zsc1) (Xsc2, Ysc2, Zsc2), and the world coordinate system The representative point P1 (Xp1w, Yp1w, Zp1w) and the representative point P2 (Xp2w, Yp2w, Zp2w) of the detection object Ob in Cw can be obtained in the following form.
Xp1w = L1 (Zcw−Zs1w) / xsc1
Yp1w = L1 (Zcw−Zs1w) / ysc1
Zp1w = L1 (Zcw−Zs1w) / Lc
Xp2w = L2 (Zcw−Zs2w) / xsc2
Yp2w = L2 (Zcw−Zs2w) / ysc2
Zp2w = L2 (Zcw−Zs2w) / Lc
However, Zs1w isReference example 1Is the value of Zsw when (Xsc, Ysc, Zsc) is replaced with (Xsc1, Ysc1, Zsc1), using the equation 1 described above, and Zs2w is the value of (Xsc, Ysc, Zsc) (Xsc2, Ysc2, This is the value of Zsw when replaced with Zsc2). Other values areReference example 1As described in.
[0058]
  Since the coordinates of the two representative points P1, P2 of the detection object Ob can be obtained as described above, the distance between the two representative points P1, P2 can be obtained in the following form.
((Xp1w-Xp2w)²+ (Yp1w-Yp2w)²+ (Zp1w-Zp2w)²)^1/2
  As described above, the size of the detection object Ob can be estimated from the distance between the two representative points P1 and P2 of the detection object Ob. In addition, since the method of obtaining the representative points in the distance image is regular, the distance obtained as described above can be used as a value for evaluating the size of the detection object Ob. In addition,This exampleIn the binary image, the representative point is obtained by extending a straight line in the horizontal direction from the center of gravity of the white pixel region in the binary image, but the representative point may be obtained by extending the straight line from the center of gravity in the vertical direction. Other configurations and operations areReference example 1It is the same. In addition,This example2 uses two representative points, but the same applies when three or more representative points are used. For example, the end points of a white pixel region on a straight line extending from the center of gravity in the horizontal direction and the vertical direction are indicated. If used, the same processing can be performed using four representative points.
[0059]
  (Reference example 3)
  This exampleAs shown in FIG. 15, shown in FIG.Reference example 1In addition to the above configuration, a detection determination unit 10 that receives the output of the region feature extraction unit 9 and determines the presence or absence of a detection target is provided. The three-dimensional position measuring device 1, the image capturing means 2, and the area selecting means 7Reference example 1It has the same configuration and functions in the same way.
[0060]
  by the way,This exampleThe representative distance calculation means 8 includes the coordinates of the representative point of the white pixel area in the binary image obtained from the distance image.Reference example 1As well as obtaining the coordinates (xsi, ysi) of the center of gravity,Reference example 2Similarly, the coordinates (xs1, ys1) (xs2, ys2) of two points (end points) in the periphery of the white pixel region are obtained. That meansThis exampleThen, the representative distances L, L1, and L2 corresponding to the three representative points are obtained from the distance image. How to find the representative distances L, L1, L2Reference example 1andReference example 2The procedure is used. That is,This exampleThe representative distance calculation means 8 in FIG. 16 uses the area of white pixels in the binary image as shown in FIG. 16B obtained by binarizing the distance image as shown in FIG. As in (c), the coordinates (xsi, ysi) of the representative point that is the center of gravity and the coordinates (xs1, ys1) (xs2, ys2) of the two representative points that are the end points are obtained, and the coordinates (xsi) of each representative point are obtained. , Ysi) (xs1, ys1) (xs2, ys2) are collated with the distance image to obtain representative distances L, L1, L2 corresponding to the representative points as shown in FIG.
[0061]
  When the representative distance calculation means 8 finds the representative distances L, L1, and L2, the area feature extraction means 9Reference example 1andReference example 2The coordinate transformation is performed in the same manner as described above, and the height Zpw from the floor surface F is obtained for the representative point P of the detection object Ob. This operation isReference example 1The height Zpw of the representative point P is expressed by the following equation.
Zpw = L (Zcw−Zsw) / Lc
However, Lc = (Xsc²+ Ysc²+ Fc²)^1/2
Further, the region feature extraction means 9 obtains the distance d12 for the representative points P1 and P2 of the detection object Ob. This operation isReference example 2The distance d12 (that is, the width of the detection object) is expressed by the following equation.
d12 = ((Xp1w−Xp2w)²+ (Yp1w-Yp2w)²+ (Zp1w-Zp2w)²)^1/2
  As described above, since the region feature extraction unit 9 can determine the height Zpw and the width d12 of the detection target, the detection determination unit 10 determines that both the height Zpw and the width d12 are within the defined range. It is determined that the detection target that is sometimes assumed exists in the monitoring space.
[0062]
  Assuming that the detection target is a human body, the representative point P is considered to be the head or shoulder because it is located near the center of the detection target, and the distance between the representative points P1 and P2 is considered to correspond to the shoulder width. Therefore, if the discrimination range for the height Zpw is set to about 80 to 200 cm and the discrimination range for the width d12 is set to about 20 to 50 cm, it is possible to detect the presence or absence of a human body in the monitoring space. In addition,This example3 points are used as representative points in FIG. 2, but may be 4 points or more, and 2 points for evaluating the size of the object to be detected are used as representative points, and the height of one representative point is used as the height. It is also possible to use
[0063]
  (Reference example 4)
  This exampleIs basically shown in FIG.Reference example 3However, instead of obtaining the center of gravity as a representative point in the representative distance calculation unit 8, the pixel value of the distance image is minimized within the white pixel region of the binary image obtained by the region selection unit 7. Find the coordinates of the pixel. Since this pixel is within the area of the white pixel, it is highly likely that the pixel is within the range of the object to be detected, and the pixel value in the distance image is the smallest within the area of the white pixel. It is considered that the part of the object has a minimum distance from the three-dimensional position measuring device 1. That is, it can be said that there is a high possibility that the detection target corresponds to the highest part.
[0064]
  Therefore, it is assumed that the three-dimensional position measuring device 1 is installed on the ceiling C so that the center line of the visual field is vertically downward and used for the purpose of detecting the head of a human body passing under the three-dimensional position measuring device 1. IfThis exampleThen, the detection range to be set for the height in the detection determination means 10 isReference example 3It is possible to set a narrower range. this is,Reference example 3In this case, the center of gravity is used as the representative point of the white pixel region in the binary image obtained from the distance image, but the possibility that the representative point is the highest part of the detection target is not sufficiently high. for,This exampleThis is because, by selecting the part that minimizes the distance from the three-dimensional position measurement apparatus 1 as the representative point, the possibility of corresponding to the highest part of the detection target increases. Thus, the determination range to be set for the height in the detection determination means 10 isReference example 3Then, if it is set to 80-200cm,This exampleThen, it becomes possible to set to 120-200 cm, for example. Other configurations and operations areReference example 3LikeThis exampleIf you adopt the configuration ofReference example 3The possibility of erroneous detection can be reduced by narrowing the determination range as compared with FIG.
[0065]
  (Reference Example 5)
  This exampleIs basically shown in FIG. 15 as shown in FIG.Reference example 3It has the same configuration as. However,This exampleThe detection judgment means 10 includes a fall detection means 10a that detects a fall of a detection target that is a human body by tracking a change in time with respect to the height of a representative point of the detection target that is a human body. As a representative point to trackReference example 3The center of gravity of the white area in the binary image obtained by binarizing the distance image as shown in FIG.Reference example 4As described above, the point where the distance from the three-dimensional position measurement apparatus 1 is minimized within the white pixel region is used.
[0066]
  This exampleIn the fall detection means 10a which is the feature of the above, the height of the representative point obtained from the area feature extraction means 9 from the floor surface is sequentially stored. That is, the three-dimensional position measurement apparatus 1 generates a distance image at regular intervals, and tracks the change in the height of the representative point based on the generated distance image. By the way, the time change of the height of the person's head becomes a change indicated by a in FIG. 18 when the person sits down, and a change indicated by b in FIG. 18 when the person falls. That is, according to FIG. 18, it can be seen that the height of the head greatly decreases in a short time when a person falls. Incidentally, in the illustrated example, the maximum height change per second when a person sits is about 70 cm, whereas the maximum height change per second when a person falls is 120 cm or more. Therefore, for example, a distance image is generated every second in the three-dimensional position measurement apparatus 1, and it is determined that the human body that is the detection target has fallen when the change in the height of the representative point obtained from the distance image exceeds 100 cm. If the fall detection means 9 is configured as described above, it is possible to detect a fall while distinguishing between a sitting action and a fall.
[0067]
  As mentioned aboveExamplesThe numerical values shown in are examples, and may be set appropriately according to the purpose.Reference Example 5However, even if the rate of change in height per second is used for the judgment of the fall, by setting the time interval for generating the distance image to a shorter time (for example, 0.1 seconds) It will be possible to capture with certainty. In short, the fall detection means 10a determines that the detection target has fallen when the change width of the height of the representative point per unit time exceeds a specified value.
[0068]
【The invention's effect】
  According to the first aspect of the present invention, a three-dimensional position measurement device that measures the distance to an object for all areas in the monitoring space, and a pixel value of each pixel on a screen that images the monitoring space is measured by the three-dimensional position measurement device. An image capturing unit that stores a distance image that is a distance, a pattern storage unit that stores a distribution pattern of a distance from the three-dimensional position measurement device as a reference pattern for the detection target, and a reference pattern that is scanned in the distance image And a difference image extracting means for extracting a difference image composed of pixel values of a difference between the distance image and the reference pattern at each position of the reference pattern in the distance image, and a difference image extracted by the difference image extracting means.eachPixel valueThe difference between the value and the constant value is calculated as the residual, and the total sum of absolute values of the residuals is calculated.The degree of coincidence calculation means to obtain the degree of coincidence between the distance image and the reference pattern, and the degree of coincidence obtained by the degree of coincidence calculation means are compared with a threshold value.When the degree of coincidence is smaller than the specified thresholdIn the surveillance spaceReference patternDetection objectIf there isSince it corresponds to performing pattern matching of the reference pattern against the distance image, it is possible to accurately determine the presence or absence of the detection target with relatively little arithmetic processing while using the distance image. Is possible. In particular, since the degree of variation in the pixel values of the difference image is used as the degree of coincidence, it is possible to obtain the degree of coincidence with a small amount of calculation by a simple calculation such as four arithmetic operations.
[0069]
According to a second aspect of the present invention, in the first aspect of the invention, a plurality of types of reference patterns having different shapes with respect to the detection object are registered in the pattern storage unit. Easily extract the presence / absence of a detection object in the monitoring space even if there is a detection object in the monitoring space that has a relatively large change in the distance to the detection object or a similar but different size can do.
[0070]
According to a third aspect of the present invention, in the first aspect of the present invention, a region selection unit that extracts a region corresponding to the detection target object from the distance image is added, and a region extracted from the distance image by the region selection unit is provided. Since it is stored in the pattern storage means as a reference pattern and the reference pattern is set using an actual detection target, a reference pattern having a shape that matches the detection target can be set and specified. Even when the object to be detected moves within the monitoring space, it can be tracked separately from other objects.
[0071]
The invention of claim 4 is the distance image processing device applied to the detection object moving in the monitoring space according to the invention of claim 3, wherein the area selection means has a detection object in the monitoring space. A difference calculation means for obtaining a difference distance image that is a difference from two distance images at different times within the existing time, and an area composed of pixels belonging to a prescribed threshold range in which the pixel value is not 0 in the difference distance image Change area extracting means for extracting from the distance image and storing it in the pattern storage means as a reference pattern, and when the detection object moves in the monitoring space, two distances caused by the movement of the detection object Since the reference pattern is set using the difference between the images, the reference pattern can be automatically generated for the detection target that has moved in the monitoring space.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a first embodiment of the present invention.
FIG. 2 is an operation explanatory diagram of the above.
FIG. 3 is a diagram for explaining the principle of a second embodiment of the present invention.
FIG. 4 is an operation explanatory view of the above.
FIG. 5 is an operation explanatory diagram of the above.
FIG. 6 is a block diagram showing a third embodiment of the present invention.
FIG. 7 is an operation explanatory diagram of the above.
FIG. 8 is a block diagram of a main part showing a fourth embodiment of the present invention.
FIG. 9 is an operation explanatory diagram of the above.
FIG. 10 shows the present invention.Reference example 1FIG.
FIG. 11 is an operation explanatory diagram of the above.
FIG. 12 is a diagram for explaining the principle described above.
FIG. 13 is a diagram for explaining the principle described above.
FIG. 14 shows the present invention.Reference example 2FIG.
FIG. 15 shows the present invention.Reference example 3FIG.
FIG. 16 is an operation explanatory view of the above.
FIG. 17 shows the present invention.Reference Example 5FIG.
FIG. 18 is a diagram illustrating the principle of the above.
[Explanation of symbols]
  1 3D position measuring device
  2 Image capture means
  3 Pattern storage means
  4 Difference image extraction means
  5 Matching degree calculation means
  6 judgment means
  7 Area selection means
  7a Difference calculation means
  7b Change area extraction means
  8 Representative distance extraction means
  9 Area feature extraction means
  10 Detection judgment means
  10a Fall detection means

Claims (4)

A three-dimensional position measurement device that measures the distance to an object for all areas in the monitoring space, and a distance image in which the pixel value of each pixel on the screen that images the monitoring space is a distance measured by the three-dimensional position measurement device is stored. Image capturing means for performing detection, pattern storage means for storing the distribution pattern of the distance from the three-dimensional position measuring device as a reference pattern for the detection object, scanning the reference pattern in the distance image and reference in the distance image residual the difference between the difference image extracting means for extracting a difference image composed of pixel values of the difference between the distance image and the reference pattern at each position of the pattern, and each pixel value of the difference image extracted by the difference image extracting unit a constant value calculated as the difference, a matching degree calculation means for calculating the magnitude of the sum of the absolute values of the residuals as a match degree between the distance image and the reference pattern, the degree of matching calculated by the matching degree calculation means Distance image processing apparatus comprising: a determination means and the detection object of the reference pattern is present in the monitoring space when a threshold and compares match degree is smaller than the specified threshold value.   The distance image processing apparatus according to claim 1, wherein a plurality of types of reference patterns having similar shapes with different sizes are registered in the pattern storage unit.   A region selection unit for extracting a region corresponding to the detection target object from the distance image is added, and the region extracted by the region selection unit from the distance image is stored in the pattern storage unit as a reference pattern. The range image processing apparatus according to claim 1. A distance image processing apparatus to be applied to the detection object moving in the monitoring space, wherein the area selecting means has two distance images at different times within a time when the detection object exists in the monitoring space. A difference calculation means for obtaining a difference distance image that is a difference from the image, and a region composed of pixels belonging to a prescribed threshold range whose pixel value is not 0 in the difference distance image is extracted from the distance image and stored as a reference pattern in the pattern storage means distance image processing equipment according to claim 3, characterized in that it comprises a change region extraction means for.

Images