カテゴリー別アーカイブ: Kinect

Kinect v1-Win7 Unity

標準

kinect v1で動作確認→kinect v2へ応用、ということを考えて(v1は自宅置き)、v1で動かす方法をあれこれ思い出していました。

ひとつは、こちらでblog記事にした、Unity Wrapperを使う方法。playbackフォルダにモーションを保存でき、それを再生も出来るのですが、どうやって、汎用モーションの形にするのかがわからず、中座。

もうひとつは、Kinect with MS-SDK(スクリプト/アバターシステム RF Solutions)。kinect v2用のKinect Mocap Animator(10$)が出ているためか、こちらのバージョンは無料になって提供されています。

https://www.assetstore.unity3d.com/jp/#!/content/7747

Unity アセットストアにあるので、そのままインポートしてきます。

=====
<Kinect Gestureの使い方> 引用

2つの方法あり。

1.Main Cameraについている、KinectManagerを見てほしい。

There are two lists – “Player1 Gestures” (these are the gestures expected from player 1) and “Player2 Gestures” (expected from player 2).

The gestures in these lists will be detected during the entire game.

2.もうひとつは、KinectGestures.GestureListenerInterfaceを使ってコーディングする方法である。

For an example look at the KinectScripts/Extras/SimpleGestureListener.cs-script. Here is a short description of its methods:リスナースクリプトの例を載せてある。

UserDetected() can be used to start gesture detection programmatically.

UserLost() can be used to clear variables or to free the allocated resources. You don’t need to remove the gestures added by UserDetected()-method explicitly. They are removed automatically, before the invocation of UserLost().

GestureInProgress()-method is invoked when a gesture is started, but not yet completed or cancelled.

GestureCompleted() is invoked when the gesture is completed. You can add your own code there to handle the completed gestures.

GestureCancelled() is invoked, if the gesture is cancelled.

3.登録されているジェスチャーについて

①右手をあげる/左手をあげる RaiseRightHand / RaiseLeftHand – left or right hand is raised over the shoulder and stays so for at least 1.0 second.
②サイポーズ Psi – both hands are raised over the shoulder and the user stays in this pose for 1.0 seconds.
③Tポーズ Tpose – the hands are to the sides, perpendicular to the body (T-pose), for 1.0 seconds.
④ストップジェスチャー Stop – right hand is down and left hand is slightly to the side, but below the waist, or left hand is down and right hand is slightly to the side, but below the waist.右手または左手を手首位置より下で少し横に振る
⑤波 Wave – right hand is waved left and then back right, or left hand is waved right and then back left. 右手なら、左にウェーブしてから右に戻す。左手ならその逆。
⑥スワイプレフト SwipeLeft – right hand swipes left.右手を右から左にスワイプする
⑦スワイプライト SwipeRight – left hand swipes right.左手を右にスワイプする
⑧スワイプアップ/スワイプダウン SwipeUp / SwipeDown – swipe up or down with left or right hand
⑨クリック Click – left or right hand stays in place for at least 2.5s. Useful in combination with cursor control.右手または左手をその場所に少なくとも2.5秒止める。
⑩右手カーソルまたは左手カーソル RightHandCursor / LeftHandCursor – pseudo gesture, used to provide cursor movement with the right or left hand.
⑪ズームアウト ZoomOut – left and right hands are to the front and put together at the beginning, then the hands move in different directions.両手を自分の前であわせ、横に開く
⑫ズームイン ZoomIn – left and right hands are at least 0.7 meter apart and to the front at the beginning, then the hands get closer to each other. 70cmぐらい離した両手を自分の前で近づけること
⑬ホィール Wheel – left and right hands are shoulder size apart and to the front at the beginning, then the hands start to turn an imaginary wheel left (positive angle) or right (negative angle).肩幅に離した両手を前に出し、ホィールを回すように右回転させるか左回転させる
⑭ジャンプ Jump – the hip center gets at least 15cm above its last position within 1.5 seconds. 腰が少なくとも1.5秒以内に15cm以上浮くこと
⑮スクワット Squat – the hip center gets at least 15cm below its last position within 1.5 seconds 1.5秒以内に、腰が15cm以上沈むこと
⑯プッシュ Push – push forward with left or right hand within 1.5 seconds 1.5秒以内に右手または左手を前に押し出すこと
⑰プル Pull – pull backward with left or right hand within 1.5 seconds 右手または左手を1.5秒以内に後ろに引くこと

4.オリジナルジェスチャーの加え方

Here are some hints on how to add your own gestures to the Kinect gesture-detection procedure. You need some C# coding skills and a bit of basic understanding on how the sensor works. It reports the 3d-coordnates of the tracked body parts in the Kinect coordinate system, in meters.
To add detection of custom gesture, open Assets/KinectScripts/KinectGestures.cs. Then:
1. Find the Gestures-enum. First you need to add the name of your gesture at the end of this enum.
2. Find the CheckForGesture()-function. There is a long switch() there, and its cases process the detection of each gesture, defined in the Gestures-enum. You need to add a case for your gesture at the end of this switch(), near the end of the script. There you will implement the gesture detection.
3. For an example on how to do that, look at the processing of some simple gestures, like RaiseLeftHand, RaiseRightHand, SwipeLeft or SwipeRight.
4. As you see, each gesture has its own internal switch() to check and change the gesture’s current state. Each gesture is like a state machine with numerical states (0, 1, 2, 3…). Its current state along with other data, is stored in an internal structure of type GestureData. This data-structure is created for each gesture that needs to be detected in the scene.
5. The initial state of each gesture is 0. At this state, the code needs to detect if the gesture is starting or not. To do this, it checks and stores the position of a joint, usually the left or right hand. If the joint position is suitable for a gesture start, it increments the state. At the next state, it checks if the joint has reached the needed position (or distance from the previous position), usually within a time interval, let’s say within 1.0 – 1.5 seconds.
6. If the joint has reached its target position (or distance) within the time interval, the gesture is considered completed. Otherwise – it is considered cancelled. Then, the gesture state may be reset back to 0 and the gesture-detection procedure will start again.
To add detection of your own gestures, first try to understand how relatively simple gestures, like RaiseHand or Swipes, work. Then find a gesture similar to the one you need. Copy and modify its code to fit your needs. Hope this helps for a start 😉

Support, Examples and Feedback
E-mail: rumen.filkov@gmail.com; Web: http://rfilkov.com Skype: roumenf Twitter: roumenf

 

広告

Kinectアプリに挑戦です

標準

開発環境 Windows8 Kinectv1

SDKはKinect v1用に、

Kinect for Windows SDK v1.8

インストールが無事に終わると、そのままツールキットのダウンロードに誘導される

Kinect for Windows Developer Toolkit v1.8

Unity用Wrapper

http://wiki.etc.cmu.edu/unity3d/images/a/ac/Kinect1.7UnityPackage.new.zip
(いきなりダウンロードが始まるので注意)

http://www.naturalsoftware.jp/entry/2014/06/13/070623
「Kinect for Windows v1をUnityで使う(2014/6/3)」こちらが参考になります

===(引用)===

Kinect v1については公式なUnityパッケージがないのですが、(おそらく)有志の手によって作成されたUnityパッケージがあったので紹介します。

使い方の解説(簡易)

3Dモデルの操作、カラー、Depthの表示、スケルトンの位置表示ができます。

Kinectの機能はKinectPrefabにあります。本来はPrefabフォルダにあるようですが、見つからないので、他のプロジェクトで使うときにはこのPrefabごとコピペして使っています。

なお、エミュレーター機能も持っているようで、記録したKinectのデータを再生できます。

再生はDeviceOrEmulatorスクリプトのUseEmulatorにチェックをいれます。これでKinectが接続されていなくても動きを再生できます。再生するファイルはKinectEmulatorスクリプトのInputファイルになります。

記録はKinectで実行中にF10を押します。これで記録が開始されます。再度F10を押すと記録が終了します。なお、終了時にファイルへの書き込みが行われるので、2回目のF10を押す前にプログラムを終了させると記録されません。デフォルトでは上記InputファイルのAssets/Recordings/playback0が上書きされます。

動かしたいモデルに「Assets\Script\Kinect\KinectModelControllers」にある「KinectModelControllerV2.cs」スクリプトを追加します。続いてInspectorビューの「Kinect Model Controller V2」にモデルの設定をします。Swにプレファブを、それぞれのジョイントに対応するボーンを設定します。なおCollar(襟)というのがあるのですが、対応がよく分からず。とはいえ、設定しないと動かないので、Sholderを入れておきました。

続いてカラーデータの表示です。Planeなど表示させたいものに「Assets\Script\Kinect\KinectImgControllers」にある「DisplayColor.cs」を追加します。モデルと同じように「Dev Or Emu」にプレファブを追加します。

Depthデータの表示はカラーデータと同じです。Planeなど表示させたいものに「Assets\Script\Kinect\KinectImgControllers」にある「DisplayDepth.cs」を追加し、「Dev Or Emu」にプレファブを追加します。

====(引用2)

http://wiki.etc.cmu.edu/unity3d/index.php/Microsoft_Kinect_-_Microsoft_SDK

  • KinectSample – This scene shows you how a skeleton is generated / tracked by placing spheres at each of the bones tracked by the Kinect, and how to use kinect to control your model. Use this to get a feel for what the Kinect is capable of. It also shows you how to prepare your GameObjects

Prefabs

  • Kinect_Prefab– This prefab has an empty node which contains all of the necessary scripts for your world to start using the Kinect – this does NOT include the controller, which you will need to use to actually control your models. See setup
  • KinectPointMan – This prefab shows how KinectPointController works.

Kinect

  • KinectModelControllerV2 – (このスクリプトを動かしたいモデルにアタッチする)This is the script that you will attach to your model that you want to manipulate – you will drag each of the bones to be controlled into the appropriate slot, and determine which player controls which model.
  • KinectPointController – This script will place GameObjects that you define onto points that are tracked by the Kinect, generating a skeleton. The starting scene comes with an example of how this looks / should be done.
  • DisplayDepth – This script will get the depth image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
  • DisplayColor – This script will get the RGB image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
  • KinectRecorder – This script will record your movement and output playback files for the emulator.
  • KinectEmulator – This script will act as a virtual Kinect. It works with playback files. For now it only simulate the skeleton data.
  • KinectSensor – This script gets data from the physic Kinect.
  • DeviceOrEmulator – This script sets whether to use physic Kinect or the emulator.
  • SkeletonWrapper – This script grabs skeleton data.
  • DepthWrapper – This script grabs depth image data.
  • KinectInterop – This script grabs data from Microsoft Kinect SDK.
  • Recordings/playbackDefault – This is the default playback file for emulator. Do NOT remove this file.

Models and Materials

  • rainbowMan
This is an example model that you can use for testing / learning how to use the wrapper. You can drag the KinectModelControllerV2 script onto this model and attach the bones and immediately begin testing.

Using the Scripts

After importing the asset package, you have the KinectExample to test and get familiar with the KinectPointController and KinectModelControllerV2. You can use this as your base scene, or you can drag the appropriate GameObjects into a new one.

Setting up the Kinect_Prefab【シーンのどこかに置くこと】

As previously mentioned, the Kinect_Prefab GameObject is required to have Unity talk with / use the Kinect. This empty object needs to exist somewhere in your scene. However, if you are switching scenes, it will persist because of the DontDestroyOnLoad call in the KinectWrapper script, so you don’t need to instantiate a new Kinect_Prefab in each scene.【ただし、シーンを変更しても、残り続けるので、新しいプレファブをそれぞれのシーンにおく必要はない】

To get the best use out of the Kinect, you may need to tweak some settings within the Kinect_Prefab object itself – here is a breakdown of each option.

Device Or Emulator

  • Use Emulator – Check this to use emulator.

Kinect Sensor

NOTE: do NOT enable this script manually, it is controlled by DeviceOrEmulator.

  • Sensor Height – How high (in meters) off the ground is the sensor.
  • Kinect Center – This tells the Kinect where it should be looking for it’s 0,0,0 point (relative to the ground directly under the sensor). The default works pretty well.
  • Look At – Tells the Kinect how to orient the camera using the motor control.

NOTE: The following values allow you to smooth the skeleton data. Usually the default values work fine. Do NOT change them unless you find magic numbers for you project.

【デフォルトの数値でだいたいいいので、あまり変更しないように】

  • Smoothing – Default value 0.5.
  • Correction – Default value 0.5.
  • Prediction – Default value 0.5.
  • Jitter Radius – Default value 0.05.
  • Max Deviation Radius – Default value 0.04.

Kinect Emulator

NOTE:

  1. do NOT enable this script manually, it is controlled by DeviceOrEmulator.
  2. If you are in the emulator mode, it will automatically play the playbackDefault file.
  3. Press F12 to play file of the path Input File. Press F12 again to switch back.
  • Input File – The file you recorded for the emulator. The default value is “Assets/Kinect/Recordings/playback0”.
  • Sensor Height – How high (in meters) off the ground is the sensor.
  • Kinect Center – This tells the Kinect where it should be looking for it’s 0,0,0 point (relative to the ground directly under the sensor). The default works pretty well.
  • Look At – Tells the Kinect how to orient the camera using the motor control.

Kinect Recorder

NOTE:

  1. Press F10 to start recording, and press F10 again to stop.
  2. You can record multiple files in one run.
  3. You recording files will be created sequentially in the folder determined by Output File. The default value is “Assets/Kinect/Recordings/playback”, so the your recording files in one run will be named as “playback0”, “playback1”, etc.
  4. The recorder will overwrite existing files. Remember to save your file to another folder before you run your world again and do the recording; or you can change the value of Output File to save the new files to another place.
  • Output File – Where your recording files will go. The default value is “Assets/Kinect/Recordings/playback”.

Controlling your Character

Simply put, to get your character moving, you need to first attach either the KinectModelControllerV2 to control the bones of the model, or KinectPointController to control a series of GameObjects.

NewModelController【新しいモデルへの設定方法】

To set this up, do the following:

  1. Drag the KinectModelControllerV2 script on to your model.【モデルにKinectModelControllerV2スクリプトをアタッチする】
  2. Select the model, and find the Sw variable (it stands for Skeleton Wrapper). Drag your Kinect_Prefab from your CURRENT SCENE into this variable. This gives us a pointer to the main update script.【①シーンの上にひとつ置いたKinect_Prefabをドラッグして、Sw変数にアタッチする】
  3. Now, expand your model fully so that each bone is visible in the hierarchy.
  4. One by one, drag each bone you want to control onto the appropriate variable. Make sure to double-check the bones, as it’s very easy to place it incorrectly.【モデルのボーンをスクリプトにドラッグ(かダブルクリック)していく】
  5. After all the bones are placed, set the player that should control this model. 0 is player one, and 1 is player two.【何人がコントロールするか決める。0は一人、1は二人・・・】
  6. Next, determine whether you want the entire skeleton to be affected by the Kinect (or just some parts) and set the appropriate mask. If you don’t see one that you want, write your own!【身体のどの部分の動きをKinectでトレースするか、マスクを設定する)
  7. If you intend to animate your model while the player is controlling it, set “animated” flag and determine how much blending between the animation and the Kinect should occur – this is a range from 0 to 1.【プレイヤーのコントロール準備まで動かしていたいときは、animatedフラッグをたてて、最初のモーションと、キネクトモーションをどれぐらいの割り合いでブレンドするか、決める】

KinectPointController

To set this up, do the following:

  1. Create an empty object to store each of the objects you want to control.【コントロールするオブジェクトひとつにつき、ひとつエンプティオブジェクトを作る】
  2. Parent each of the objects into this empty node, and place them all at the same location.【オブジェクトは、このエンプティと同じ場所に置く。】
  3. Attach the KinectPointController to the parent node.【親ノードに、KinectPointControllerスクリプトをアタッチ。】
  4. Select the parent node, and find the Sw variable (it stands for Skeleton Wrapper). Drag your Kinect_Prefab from your CURRENT SCENE into this variable. This gives us a pointer to the main update script.
  5. Drag each of the objects you want to be controlled into the appropriate variable.
  6. Set the player number (0 = player one, 1 = player two) and the Mask.

For an example of how this should work, check the KinectExample provided in the package.

Updates (Aug. 2013)

  1. Updated wrapper to Kinect SDK 1.7 in Unity 4.2.
  2. Added in Joint Orientation data from Kinect sensor.
  3. Added seat mode.
  4. Fixed the color and depth image display bug.

 

====
BlenderでKinectデータを使う

https://github.com/asahiufo/AMK2B