Outerra forum

Outerra Engine => Ideas & Suggestions & Questions => Topic started by: Deejayseesee on April 13, 2011, 10:08:17 pm

Title: TrackIR
Post by: Deejayseesee on April 13, 2011, 10:08:17 pm
The implementation of TrackIR would be fantastic for an engine like Outerra. It has proven to work wonders in flight, racing and military situations and will immerse the gamer more into the experience.

http://www.naturalpoint.com/trackir/

[video]http://www.youtube.com/watch?v=9wXx3vMy_AQ[/video]

[video]http://www.youtube.com/watch?v=B5-wRIa9GDg[/video]


Regards,
David Charnock
Title: TrackIR
Post by: cameni on April 14, 2011, 05:38:35 am
It should not be a problem to implement a support for it, and we expect we'll integrate it in near future. I don't entirely like the NDA NaturalPoint want us to sign before being able to get our hands on the SDK though.
TrackIR support may come from a third party as a plugin, removing potential restrictions from everyone else (if it's possible).
Title: Re: TrackIR
Post by: Alcook on March 07, 2012, 04:50:38 am
Hi

+1 need Track IR support, it increases immersion.

@Outerra Team: I love you, I never seen a so realistic real time environnement.
Title: Re: TrackIR
Post by: [deleted] on March 07, 2012, 09:38:47 am
Very old thread :P
And I suggested a freetrack support too.
Title: Re: TrackIR
Post by: KelvinNZ on March 08, 2012, 07:47:54 pm
Ok, so for the time being why not use EZDok camera
Title: Re: TrackIR
Post by: Andrey on March 09, 2012, 05:19:51 am
Track IR would be cool feature. I'm looking forward for joystick "change view" lever support first... Or maybe it's already included just need to update drivers ?...

PS Picture to explain
(http://lh6.ggpht.com/_-0dh_1qD6CU/Szgr8CEt8dI/AAAAAAAAZbc/QtTGlo_UG_Q/thrustmaster_t_flight_hotas_x_joystick_main.jpg)
 - "change view"  leverlit
Title: Re: TrackIR
Post by: pico on March 09, 2012, 05:49:26 am
Freetracksoftware allows you to control the Mouse. I use it sometimes in Outerra. ;)
Title: Re: TrackIR
Post by: Andrey on March 09, 2012, 05:58:00 am
Freetracksoftware allows you to control the Mouse. I use it sometimes in Outerra. ;)

Mouse is working okay, that's true ) However is not very comfortable inside the cockpit.
Btw, not to create new topic - is any way to change camera focal length ?... Didn't find this feature..

I'm not great in Engligh so one more picture to explain
(http://www.cqrdigital.com/images/FOVchart.jpg)

Title: Re: TrackIR
Post by: cameni on March 09, 2012, 06:43:02 am
FOV in graphics settings.
Title: Re: TrackIR
Post by: [deleted] on March 09, 2012, 10:29:23 am
Freetracksoftware allows you to control the Mouse. I use it sometimes in Outerra. ;)

That function is a little buggy for me :>
Title: Re: TrackIR
Post by: Andrey on March 09, 2012, 10:44:35 am
FOV in graphics settings.

Thanks! Working! ) It would be cool to change it with ctrl+mouse wheel or something like this...
Title: Re: TrackIR
Post by: foobie42 on January 24, 2013, 05:07:00 pm
I can write you code and release it under public domain that implements TIR API. No NDA/SDK needed, no blobs either.

Best to send mail but I'll try to check the forum.
Title: Re: TrackIR
Post by: cameni on January 25, 2013, 04:19:47 am
That would be perfect. We are going to create an API for custom input devices, this could be one of the things supported this way.
Title: Re: TrackIR
Post by: foobie42 on January 25, 2013, 06:12:30 am
Disclaimer: I never owned NP software or hardware. Never signed any EULA, NDA or whatnot. This code might work but it's best if you test it on a TIR device. Tell me of bugs! I will self-flagellate in contempt...

Also I know no disassembly! And don't know where to find NPClient{64,}.dll!

Code: [Select]
/* This is free and unencumbered software released into the public domain.

 * Anyone is free to copy, modify, publish, use, compile, sell, or
 * distribute this software, either in source code form or as a compiled
 * binary, for any purpose, commercial or non-commercial, and by any
 * means.
 *
 * In jurisdictions that recognize copyright laws, the author or authors
 * of this software dedicate any and all copyright interest in the
 * software to the public domain. We make this dedication for the benefit
 * of the public at large and to the detriment of our heirs and
 * successors. We intend this dedication to be an overt act of
 * relinquishment in perpetuity of all present and future rights to this
 * software under copyright law.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
 * IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
 * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
 * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 * OTHER DEALINGS IN THE SOFTWARE.
*/

#include <stdlib.h>
#include <stdio.h>
/* this is unfortunately necessary as the API is Windows-only */
#include <windows.h>

#ifdef __cplusplus
#  define MY_EXTERN extern "C"
#else
#  define MY_EXTERN
#define MY_EXPORT(ret) MY_EXTERN ret __declspec(dllexport) __cdecl
#endif

#pragma pack(push, 1)
typedef struct {
  __int16 status; /* zero means success, it's easy to forget! */
  __int16 frame; /* if idempotent, device paused */
  __int32 padding;
  float roll, pitch, yaw; /* divide by 8191 when received, in radians */
  float x, y, z; /* between -16383 and 16383 */
  float padding2[9];
} tir_headpose_t;
#pragma pack(pop)

typedef struct {
HMODULE hlibrary;
__int32 (__stdcall* tir_reghwnd)(HWND hwnd);
__int32 (__stdcall* tir_regprogid)(__int16 progid);
__int32 (__stdcall* tir_queryver)(__int16* version);
__int32 (__stdcall* tir_reqdata)(__int16 bitmask);
__int32 (__stdcall* tir_stopcurs)(void);
__int32 (__stdcall* tir_startxmit)(void);
__int32 (__stdcall* tir_getdata)(tir_headpose_t* pose);
/* there are more functions, especially for cleaning up
* but the library is typically called on startup
* and unloaded on exit. if you really must add them,
* you can either extract them from NPClient.dll or
* google for it. for instance I once found symbols
* and some constants in Doxygen format...
* if you REALLY need them I can finish the damn thing
* but it's so much boilerplate :(
*/
} tir_state_t;

MY_EXPORT(tir_state_t*) mytir_init(void)
{
HKEY hkey = NULL;
tir_state_t* ret;
int i;
DWORD sz = 1900;
/* Windows paths are limited to 260 characters */
char buf[2048];

ret = malloc(sizeof(tir_state_t));
if (ret == NULL)
return NULL;
memset(ret, 0, sizeof(tir_state_t));
RegOpenKeyExA(HKEY_CURRENT_USER, "Software\\NaturalPoint\\NATURALPOINT\\NPClient Location", 0, KEY_QUERY_VALUE, &hkey);
if (!hkey)
goto error;
buf[0] = '\0';
if (RegQueryValueExA(hkey, "Path", NULL, NULL, buf, &sz) != ERROR_SUCCESS)
goto error;
(void) RegCloseKey(hkey);
hkey = NULL;
buf[2047] = '\0';
for (i = 0; buf[i]; i++)
if (buf[i] == '/')
buf[i] = '\\';
/* in the registry path there's normally a trailing slash already */
#ifdef MY_BUILD_AMD64
strcat(buf, "NPClient64.dll");
#else
strcat(buf, "NPClient.dll");
#endif
ret->hlibrary = LoadLibraryA(buf);
if (ret->hlibrary == NULL)
goto error;
if ((ret->tir_reghwnd = GetProcAddress(ret->hlibrary, "NP_RegisterWindowHandle")) == NULL)
goto error;
if ((ret->tir_regprogid = GetProcAddress(ret->hlibrary, "NP_RegisterProgramProfileID")) == NULL)
goto error;
if ((ret->tir_queryver = GetProcAddress(ret->hlibrary, "NP_QueryVersion")) == NULL)
goto error;
if ((ret->tir_reqdata = GetProcAddress(ret->hlibrary, "NP_RequestData")) == NULL)
goto error;
if ((ret->tir_stopcurs = GetProcAddress(ret->hlibrary, "NP_StopCursor")) == NULL)
goto error;
if ((ret->tir_startxmit = GetProcAddress(ret->hlibrary, "NP_StartDataTransmission")) == NULL)
goto error;
if ((ret->tir_getdata = GetProcAddress(ret->hlibrary, "NP_GetData")) == NULL)
goto error;
return ret;
error:
if (hkey)
  (void) RegCloseKey(hkey);
free(ret);
/* sorry but I don't know anything about unloading DLLs. unless you do, there's a handle leak */
/* if you unload it, no need to touch function pointers, they just become invalid */
return NULL;
}

MY_EXPORT(void) mytir_start(tir_state_t* state, HWND window_handle)
{
__int16 tmp;
/* while you have to call it, can pass anything valid, like desktop handle I suppose? */
(void) state->tir_reghwnd(window_handle);
(void) state->tir_regprogid(1901); /* Falcon 4, an old flight simulation. Need to pass valid Id */
(void) state->tir_queryver(&tmp); /* better to call it, even if we don't care about it! */
(void) state->tir_reqdata(119); /* specifies which elements of the pose structure to return */
(void) state->tir_stopcurs(); /* better to call it */
(void) state->tir_startxmit(); /* idem */
}

MY_EXPORT(void) mytir_get_pose(tir_state_t* state, tir_headpose_t* ret)
{
memset(ret, 0, sizeof(tir_headpose_t));
ret->status = 1; /* if the call fails for some reason, at least we shouldn't think we have a valid pose when we don't */
(void) state->tir_getdata(ret);
}

#ifdef MY_DEMO
int main(void)
{
tir_state_t* state;
int i;
tir_headpose_t ret;

state = mytir_init();
if (!state)
return 1;
mytir_start(state, GetDesktopWindow());
for (i = 0; i < 3600; i++)
{
mytir_get_pose(state, &ret);
if (ret.status)
return 1;
printf("%f %f %f ~ %f %f %f\n", ret.yaw * 180 / 3.14 / 8191, ret.pitch * 180 / 3.14 / 8191, ret.roll * 180 / 3.14 / 8191, ret.x, ret.y, ret.z);
Sleep(35);
}
return 0;
}
#endif
Title: Re: TrackIR
Post by: foobie42 on January 26, 2013, 05:07:32 am
Can someone try the damn thing on a TIR device and tell me if it works?
Title: Re: TrackIR
Post by: foobie42 on January 26, 2013, 12:36:24 pm
Confirmed working with software version 5 and TIR4. Will do some more tests with TIR5 device.
Title: Re: TrackIR
Post by: foobie42 on January 26, 2013, 04:20:21 pm
Confirmed working with TIR5 and latest software.

Deallocating the DLL address space/handle might leave some leaks anyway since NPClient might malloc. I'm not sure if it does but cannot discount the possibility.
Title: Re: TrackIR
Post by: Dino1981 on April 27, 2013, 04:09:07 pm
I love Outerra, and even with the simple Cessna cockpit I love to fly here. I have a TrackIR 4 device and would be happy, if I could use it. I don't understand the code, which is written here.

Can someone please tell us what to do, to get TrackIR working in Outerra, perhaps Cameni himself?


THX
Title: Re: TrackIR
Post by: cameni on April 27, 2013, 04:37:03 pm
We need to create an input plugin system where this can be plugged as a third party addon, to avoid any potential legal issues. Should be done soon as it's also needed elsewhere.
Title: Re: TrackIR
Post by: ZeosPantera on April 27, 2013, 09:20:56 pm
Should be done soon as it's also needed elsewhere.

Read... OCULUS RIFT BITCHES!!
Title: Re: TrackIR
Post by: cameni on April 28, 2013, 01:54:14 am
Oculus will be supported natively .. they aren't asking everyone to sign a NDA to get rid of competition.
Title: Re: TrackIR
Post by: foobie42 on April 28, 2013, 07:29:09 am
Can you support the FreeTrack interface natively or through a plugin?

https://sourceforge.net/p/facetracknoir/codegit/ci/posix/tree/ftnoir_tracker_rift/

Above are Rift plugin sources from FaceTrackNoIR.
Title: Re: TrackIR
Post by: cameni on April 28, 2013, 07:44:34 am
As long as the license is permissible we can support it natively. If FaceTrackNoIR provides an abstract interface that will then enable us supporting anything that FaceTrackNoIR supports - that would be nice.
Title: Re: TrackIR
Post by: foobie42 on April 28, 2013, 07:53:48 am
FaceTrackNoIR can communicate using "protocols" with games. As such, you may either "borrow" the tracker code itself for native support of some tracker, like Rift, -or- support one of the protocols we support, for supporting everything FaceTrackNoIR supports. If you're interested in FreeTrack itself, here's a C implementation, should be plenty of info how to interface with it:

https://sourceforge.net/p/facetracknoir/codegit/ci/posix/tree/freetrackclient/freetrackclient.cpp

And here's the data mapping used by FreeTrack:

https://sourceforge.net/p/facetracknoir/codegit/ci/posix/tree/ftnoir_protocol_ft/fttypes.h

Please keep in mind not to access the FTMemMap stuffies unless you want to limit your FT support to NoIR, original FreeTrack doesn't have those data members. They're postioned *after* the original struct, as such, there's limited compatiblity between these 2 programs.

Finally, here's the Linux mapping code:

https://sourceforge.net/p/facetracknoir/codegit/ci/posix/tree/ftnoir_protocol_wine/

Note: Despite the confusing name, the Wine protocol is useful for native games, NoIR has an X-Plane plugin using the same protocol.

Here's the architecture of the program:

- trackers provide tracking data from a sensor
- filters remove jitter from input
- protocols supply data to games

If you need clarifications about licensing from original authors of some code parts, can contact them directly.
Title: Re: TrackIR
Post by: cameni on April 28, 2013, 09:12:30 am
Thanks, we'll take a look at it.
Title: Re: TrackIR
Post by: foobie42 on April 29, 2013, 10:24:30 am
Linux platform currently lacks a standardized way to access headpose data from a sensor.

Are you interested in taking part of informally creating a de-facto standard? I've been in contact with other developer and he proposed we do something about it. Last thing we all want is another platform under crippling monopoly? Of course, same standard can be applied to other platforms.

cheers,
-sh
Title: Re: TrackIR
Post by: cameni on April 30, 2013, 01:27:46 am
Is the TIR support included somewhere in there? Would be good to have a single interface.
We may create an input plugin that supports FaceTrackNoIR using one of its protocols, I guess FT or FTN. The configuration should remain outside, there will be only a mapping in OT controls.

As for Linux - yes, it would be good to use a common intelligently designed API with other developers. But then it would be good to have a multiplatform lib from the start.
Title: Re: TrackIR
Post by: foobie42 on April 30, 2013, 01:46:43 am
There's no TIR support in FTNOIR because of Windows requirement for signed drivers, because the company manufacturing the device is known to be litigious, and finally, because someone would have to do the work.

There's a TIR driver in the linux-track project however, but only for Linux and Mac.

If you're also speaking of NPClient reverse-engineered sources, they're kept private, to help avoid legal harassment by the aforementioned company.

FreeTrack can actually work on Linux, with use of the 'ftnoir_compat' library in the posix branch. It provides a uniform way to access mutexes and shared memory mappings between platforms. After that, all that's required is changing windows.h non-standard stuffies to their ANSI/POSIX equivalents, maybe also some calling convention preprocessor macros and some other implementation details.

PS Linux support most likely won't make it into FaceTrackNoIR due to original author's objections, therefore we've forked the project as https://github.com/opentrack/opentrack
Title: Re: TrackIR
Post by: zaelu on May 07, 2013, 04:14:01 am
I also wish for head tracking support in Outerra.

I use Freetrack for other games and freetrack can export data with more than one interface, TIR, Freetrack, Mouse, MSFX. In Outerra I used it as mouse export but is very limited, has no proper reset to center and drifts around and... no 6DoF of course.

 Freetrack is supported by many games, unfortunately I am not a programmer and I don't know exactly how it is done. DCS World has a "headtracker.dll" file that makes the link between the game and the program.

Facetracknoir has now took the freetrack development and further updates to the tracker interface will be made by the team behind "Facetracknoir" http://facetracknoir.sourceforge.net/ (http://facetracknoir.sourceforge.net/)

However the original Freetrack source code can still be found on the site of freetrack: http://www.free-track.net/english/ (http://www.free-track.net/english/)

direct link would be this:

https://camil.dyndns.org/svn/freetrack/tags/V2.2/ (https://camil.dyndns.org/svn/freetrack/tags/V2.2/)

If needed I try to gather cross-reference on DCS forums on how the new updated dll was build or something.

Title: Re: TrackIR
Post by: foobie42 on May 07, 2013, 04:26:40 am
No worries, we'll get the interface done once Outerra exports 6DOF API. There's already a plugin for X-Plane for Linux where there's no head-tracking protocol of any kind, other than for Wine games. I'll try to backport it in time for FaceTrackNoIR 1.8!

As for the FreeTrack confusion, FaceTrackNoIR supports the same protocol as FreeTrack, but doesn't supersede the original software in any way. The '2.0' number perhaps is the source of confusion :(

DCS DLL talks the FreeTrack protocol on Windows, but ideally we'd get it working for all platforms.
Title: Re: TrackIR
Post by: zaelu on May 07, 2013, 04:38:33 am
I see. Is good that it will not be let out :) .

I understood that facetracknoir team had taken over freetrack interface development when they (facetracknoir) had released their latest version of facetracknoir. I tested it but I don't like it as it is more complicated in some parts and simplified in other parts... maybe later.

Here is a hint that they got freetrack legacy to further development:

http://sourceforge.net/p/facetracknoir/wiki/FreeTrack20/ (http://sourceforge.net/p/facetracknoir/wiki/FreeTrack20/)
Title: Re: TrackIR
Post by: foobie42 on May 07, 2013, 04:47:09 am
Only thing that changed is a few fields at the end of the memory-mapped structure. So that all games that support FreeTrack can read previous fields with head-pose data, while the new ones can serve as a bridge between TIR and FT proper.

As for the rest of the statements, I don't appreciate piggybacking on the name. It hasn't been discussed with FT team, so it's unauthorized.

Shameless plug: https://github.com/opentrack/opentrack is a fork for all platforms and bugs fixed at that.
Title: Re: TrackIR
Post by: zaelu on May 07, 2013, 04:56:47 am
I understand. I didn't knew they didn't talked with FT team. I hope they'll fix this and they cooperate with each other.
Title: Re: TrackIR
Post by: Pyroman31FF on May 11, 2013, 07:33:30 am
Disclaimer: I never owned NP software or hardware. Never signed any EULA, NDA or whatnot. This code might work but it's best if you test it on a TIR device. Tell me of bugs! I will self-flagellate in contempt...

Also I know no disassembly! And don't know where to find NPClient{64,}.dll!

Code: [Select]
/* This is free and unencumbered software released into the public domain.

 * Anyone is free to copy, modify, publish, use, compile, sell, or
 * distribute this software, either in source code form or as a compiled
 * binary, for any purpose, commercial or non-commercial, and by any
 * means.
 *
 * In jurisdictions that recognize copyright laws, the author or authors
 * of this software dedicate any and all copyright interest in the
 * software to the public domain. We make this dedication for the benefit
 * of the public at large and to the detriment of our heirs and
 * successors. We intend this dedication to be an overt act of
 * relinquishment in perpetuity of all present and future rights to this
 * software under copyright law.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
 * IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
 * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
 * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 * OTHER DEALINGS IN THE SOFTWARE.
*/

#include <stdlib.h>
#include <stdio.h>
/* this is unfortunately necessary as the API is Windows-only */
#include <windows.h>

#ifdef __cplusplus
#  define MY_EXTERN extern "C"
#else
#  define MY_EXTERN
#define MY_EXPORT(ret) MY_EXTERN ret __declspec(dllexport) __cdecl
#endif

#pragma pack(push, 1)
typedef struct {
  __int16 status; /* zero means success, it's easy to forget! */
  __int16 frame; /* if idempotent, device paused */
  __int32 padding;
  float roll, pitch, yaw; /* divide by 8191 when received, in radians */
  float x, y, z; /* between -16383 and 16383 */
  float padding2[9];
} tir_headpose_t;
#pragma pack(pop)

typedef struct {
HMODULE hlibrary;
__int32 (__stdcall* tir_reghwnd)(HWND hwnd);
__int32 (__stdcall* tir_regprogid)(__int16 progid);
__int32 (__stdcall* tir_queryver)(__int16* version);
__int32 (__stdcall* tir_reqdata)(__int16 bitmask);
__int32 (__stdcall* tir_stopcurs)(void);
__int32 (__stdcall* tir_startxmit)(void);
__int32 (__stdcall* tir_getdata)(tir_headpose_t* pose);
/* there are more functions, especially for cleaning up
* but the library is typically called on startup
* and unloaded on exit. if you really must add them,
* you can either extract them from NPClient.dll or
* google for it. for instance I once found symbols
* and some constants in Doxygen format...
* if you REALLY need them I can finish the damn thing
* but it's so much boilerplate :(
*/
} tir_state_t;

MY_EXPORT(tir_state_t*) mytir_init(void)
{
HKEY hkey = NULL;
tir_state_t* ret;
int i;
DWORD sz = 1900;
/* Windows paths are limited to 260 characters */
char buf[2048];

ret = malloc(sizeof(tir_state_t));
if (ret == NULL)
return NULL;
memset(ret, 0, sizeof(tir_state_t));
RegOpenKeyExA(HKEY_CURRENT_USER, "Software\\NaturalPoint\\NATURALPOINT\\NPClient Location", 0, KEY_QUERY_VALUE, &hkey);
if (!hkey)
goto error;
buf[0] = '\0';
if (RegQueryValueExA(hkey, "Path", NULL, NULL, buf, &sz) != ERROR_SUCCESS)
goto error;
(void) RegCloseKey(hkey);
hkey = NULL;
buf[2047] = '\0';
for (i = 0; buf[i]; i++)
if (buf[i] == '/')
buf[i] = '\\';
/* in the registry path there's normally a trailing slash already */
#ifdef MY_BUILD_AMD64
strcat(buf, "NPClient64.dll");
#else
strcat(buf, "NPClient.dll");
#endif
ret->hlibrary = LoadLibraryA(buf);
if (ret->hlibrary == NULL)
goto error;
if ((ret->tir_reghwnd = GetProcAddress(ret->hlibrary, "NP_RegisterWindowHandle")) == NULL)
goto error;
if ((ret->tir_regprogid = GetProcAddress(ret->hlibrary, "NP_RegisterProgramProfileID")) == NULL)
goto error;
if ((ret->tir_queryver = GetProcAddress(ret->hlibrary, "NP_QueryVersion")) == NULL)
goto error;
if ((ret->tir_reqdata = GetProcAddress(ret->hlibrary, "NP_RequestData")) == NULL)
goto error;
if ((ret->tir_stopcurs = GetProcAddress(ret->hlibrary, "NP_StopCursor")) == NULL)
goto error;
if ((ret->tir_startxmit = GetProcAddress(ret->hlibrary, "NP_StartDataTransmission")) == NULL)
goto error;
if ((ret->tir_getdata = GetProcAddress(ret->hlibrary, "NP_GetData")) == NULL)
goto error;
return ret;
error:
if (hkey)
  (void) RegCloseKey(hkey);
free(ret);
/* sorry but I don't know anything about unloading DLLs. unless you do, there's a handle leak */
/* if you unload it, no need to touch function pointers, they just become invalid */
return NULL;
}

MY_EXPORT(void) mytir_start(tir_state_t* state, HWND window_handle)
{
__int16 tmp;
/* while you have to call it, can pass anything valid, like desktop handle I suppose? */
(void) state->tir_reghwnd(window_handle);
(void) state->tir_regprogid(1901); /* Falcon 4, an old flight simulation. Need to pass valid Id */
(void) state->tir_queryver(&tmp); /* better to call it, even if we don't care about it! */
(void) state->tir_reqdata(119); /* specifies which elements of the pose structure to return */
(void) state->tir_stopcurs(); /* better to call it */
(void) state->tir_startxmit(); /* idem */
}

MY_EXPORT(void) mytir_get_pose(tir_state_t* state, tir_headpose_t* ret)
{
memset(ret, 0, sizeof(tir_headpose_t));
ret->status = 1; /* if the call fails for some reason, at least we shouldn't think we have a valid pose when we don't */
(void) state->tir_getdata(ret);
}

#ifdef MY_DEMO
int main(void)
{
tir_state_t* state;
int i;
tir_headpose_t ret;

state = mytir_init();
if (!state)
return 1;
mytir_start(state, GetDesktopWindow());
for (i = 0; i < 3600; i++)
{
mytir_get_pose(state, &ret);
if (ret.status)
return 1;
printf("%f %f %f ~ %f %f %f\n", ret.yaw * 180 / 3.14 / 8191, ret.pitch * 180 / 3.14 / 8191, ret.roll * 180 / 3.14 / 8191, ret.x, ret.y, ret.z);
Sleep(35);
}
return 0;
}
#endif


How/ where do i have to implement this?
Title: Re: TrackIR
Post by: foobie42 on May 11, 2013, 07:37:01 am
Implement what, sorry?

Edit: The code's pretty self-explanatory and commented. Try running it when compiled as an executable. It even comes with a 'main' function. Given my interest of helping the NaturalFail ship sink, I can help you implementing support for their product in your own software. Down with NDAs and anticompetetive behavior!
Title: Re: TrackIR
Post by: Pyroman31FF on May 11, 2013, 07:44:34 am
The lines of code you posted. I thught i would have to put them in somewhere thus enabeling TIR.
Title: Re: TrackIR
Post by: foobie42 on May 11, 2013, 07:55:54 am
Is this for Outerra? There needs to be an API for controlling the pose for plugins.
Title: Re: TrackIR
Post by: exadeci on May 13, 2013, 05:12:47 am
The major flaw of this system is that is not as natural as claimed because the screen doesn't move with your head so you end up seeing less of the screen when you have your head turned.
A good immersive system would be the Oculous Rift http://www.oculusvr.com/ (http://www.oculusvr.com/) we can hope they lower the price when it start being used in more games and this system is tested and approved by several renowned developers http://www.youtube.com/watch?feature=player_embedded&v=DhcOMOWRMnA# (http://www.youtube.com/watch?feature=player_embedded&v=DhcOMOWRMnA#)!
Title: Re: TrackIR
Post by: foobie42 on May 13, 2013, 05:16:35 am
Head tracking, either pure image processing, or with some kind of 'aid' like FreeTrack, while not perfectly natural, after some getting used to is better than you claim.

Bigger problem is accuracy, given how one degree of rotation input gets mapped to several degrees output.

I see no problem with supporting both VR devices and 'traditional' head/face tracker. opentrack does both! The developer of the Rift plugin uses the software with the GRID racing game.
Title: Re: TrackIR
Post by: Pyroman31FF on May 13, 2013, 11:13:40 am
The major flaw of this system is that is not as natural as claimed because the screen doesn't move with your head so you end up seeing less of the screen when you have your head turned.

Have you ever used it? I takes less than a month to get used to it and i use it absolutely flawless for years now. It is second nature for me, going that far my brother noticed me moving my head lightly flying the outerran MiG29. Next thing is my axis are so tuned (like everyone else does btw) that minor headmovements already turn the FoV ingame multiple times 360°. So you never lose focus of the screen, it still is centered completely. Don't just quote what some clueless good for nothings wrote somewhere in the internet. They got it all wrong.
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 02:47:13 pm
So I've tested connection to the FaceTrackNoIR, used the file mapping + mutex to get the data from the tracker using a 3-point cap. It generally works, but I'm having some issues there too.

Here's what I'm doing: initially the named file mapping is opened and named mutex created. Acquiring the mutex doesn't seem to prevent FT from changing the values for some reason.

When the tracker doesn't have good info (one IR LED goes out of the screen), I get values reset to zero, is there a way to discern this case of bad input from the centered view, so I can freeze the camera at the last position?

Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 03:00:58 pm
Most unusual, please see:

https://sourceforge.net/p/facetracknoir/codegit/ci/master/tree/FTNoIR_Protocol_FT/FTNoIR_Protocol_FT.cpp

Code: [Select]
// Check if the pointer is OK and wait for the Mutex.
//
if ( (pMemData != NULL) && (WaitForSingleObject(hFTMutex, 100) == WAIT_OBJECT_0) ) {
//
// Copy the Raw measurements directly to the client.
//
[...]

Code: [Select]
static const char* FREETRACK_MUTEX = "FT_Mutext";
In particular, the return value of WaitForSingleObject is checked for, if mutex wasn't initialized properly, values wouldn't be updated properly. Are you sure that the typo in mutex name is there (it was like this in FT written in Delphi), and that both programs are running as the same user?

It sounds rather lame, but if you check for all values being 0, it would solve your latter issue. All trackers use imprecise numerical methods for the purpose of retrieving the rotation and translation, and chances of it ever being computed to all zeros are amazingly slim.

Unfortunately the beginning part of the FT memory mapping is set in stone and if a new data element was added, you'd lose compatibility with FreeTrack, the Pascal/Delphi software.

Edit: In case you find this helpful, here's the canonical way of accessing memory mappings: https://sourceforge.net/p/facetracknoir/codegit/ci/develop/tree/compat/compat.cpp
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 03:16:27 pm
Yes, both programs are running under the same user, and the mutex name is correct as in that file (I copied them from there).
Double checked if I didn't make a mistake, but I'm getting this:

Code: [Select]
if(prevDataID != pMemData->data.DataID) {
    ...
}
else {
    //here sometimes prevDataID != pMemData->data.DataID
}

Can I detect that the tracker provides some extra info after the FT part? Or the tracker type/version.
I saw some version/provider info methods as dll exports in freetrackclient.cpp, but since this here works through the shared memory mapping we aren't loading/exporting anything.
Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 03:24:27 pm
Yes, both on develop and master branches, DataID isn't protected by a mutex, even though rest of the data members are.

Paging Wim Vriend, hoping he'll step in and get this fixed in time for 1.8.

Only added data after the 'portable' part of the mapping is what's necessary to pass the NPClient 'api key' through the mappings. So it turns out tracker/filter data isn't passed in any way, no :(

I've fixed the DataID race in my fork ages ago: https://github.com/opentrack/opentrack/blob/master/ftnoir_protocol_ft/ftnoir_protocol_ft.cpp but chances of it finding its way back to the original project are pretty slim :(

I'd disregard the DataID, treat data as always fresh, except for the all-zeros case. Does that sound even remotely sensible?

Will try to get DataID fix into the develop branch, fix mismatched locking, and some logic errors introduced into the shared memory segment class, in worst case I'll lose commit privileges but if hadn't given up on the parent project, wouldn't have forked it in the first place.
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 03:35:07 pm
I'd disregard the DataID, treat data as always fresh, except for the all-zeros case. Does that sound even remotely sensible?
Okay, that sounds reasonable.

I have yet to test it with FreeTrack and OpenTrack, but with FTNoIR and with the 3-point cap the tracker often freezes and stops updating, or when one point vanishes (turning to the side too much) and then returns back, the angles are all wrong, yaw = 180° and such, and I have to stop/start it. Not sure if I'm doing something wrong there.
Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 03:39:20 pm
That's related to 3-point perspective-n-points problem. There are 2 equal solutions to the problem, additionally, sometimes software loses track of which point is which.

Patrick put safeguards against both issues when he first submitted the PT tracker, but they only help on continuous frames. If tracking is ever lost, both issues can occur.

There are also some thingies to adjust like minimum point radius, and binary thresholding strength. You can see it in PT tracker options without stopping tracker operation.
Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 03:51:19 pm
This is the least I can do https://sourceforge.net/p/facetracknoir/codegit/ci/b4bae5a6020c591be6f60d1d51e25ab2de631f91/

cheers
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 03:55:07 pm
Hmm, I see .. but this seems to be some extra problem atop of that, or it confuses it really badly - because in this case yaw becomes stuck at 180° most of the time, pitch changes only little, roll is usually at 49 ... and in this state it's prone to freezing for a couple of seconds too.
Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 04:08:30 pm
If your model looks like this:

Code: [Select]
__
\ /

And the down-center point is facing upward when tracking is lost, it'll confuse points with each other.

Is this the case?

Edit: At best can introduce a reprojection error check, if it gets big, switch back to init phase, to minimize breakage. But only for opentrack. If you're only evaluating the software for the purpose of implementing support, there's little point...
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 04:25:11 pm
My cap is similar to this http://www.nilojr.com.br/DIY-Trackir/adesivo/marcelo/trackir2.jpg (http://www.nilojr.com.br/DIY-Trackir/adesivo/marcelo/trackir2.jpg) except there are 3mm IR leds. When I turn the head sideways so that the camera sees only the top point and one of the bottom points, and then I turn back, then it happens.
Title: Re: TrackIR
Post by: foobie42 on July 20, 2013, 04:40:38 pm
Does the same happen when you have dynamic pose resolution off? Nevermind problems that dynamic pose resolution was supposed to solve.

In point_tracker.cpp -> PointTracker::track I'd change the call from reset_velocities() to reset() completely. Depends if you can compile the software on your own. Sadly opentrack's ABI is incompatible with FTNOIR plugins, due to VPTR changes, different CRT version, and more.

If you can provide a video of this happening, like a webcam feed with points visible, as well as model dimensions, I'm offering to get to the bottom of this and committing changes to opentrack.
Title: Re: TrackIR
Post by: cameni on July 20, 2013, 04:59:43 pm
With the dynamic pose off it behaves differently, it doesn't get stuck but pitch and roll change with yaw.

Here's what I get:
(http://i.minus.com/jbljroYuotmlok.png) (http://minus.com/lbljroYuotmlok)

I can make a video of the input, but as it seems to be a problem between my camera/cap and the tracker, it doesn't affect the general functionality. We'll see how the FT support works for others in OT first.
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 08:15:03 am
Outerra + FreeTrack IR cap test - MiG 29 (http://www.youtube.com/watch?v=dJFJiGZo6uw#ws)

I'd like to combine this better with the Oculus Rift tracker, to add the positional tracking into it. Since the Rift provides the orientation, the problem of detecting which point is which could be simplified and made more robust, and the position alone could be inferred just from two points. It would be cool to add two IR LEDs directly into the Rift chassis and add support for pure positional tracking into FTNoIR or OpenTrack. Actually, raw positions of two points would be enough I think ...

Or maybe this should be done by putting some stickers on it and use it with normal cameras.
Title: Re: TrackIR
Post by: foobie42 on July 21, 2013, 08:27:32 am
Actually, in opentrack, as well as ftnoir, there's support for using two trackers at once with selecting axes they provide.

In opentrack there's Rift support, you may want to try using it as a primary tracker, with positional data provided by secondary PT.

But it's understandable you want native Rift support inside the software, without requiring any third-party stuff.

Stickers would hardly work, unless someone trains a Haar classifier for them, which is robust but fast enough to run every frame... Of course there's the matter of solving a 2-point perspective-n-points equation and providing the relevant data.
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 08:36:14 am
I can combine Rift and FT already, but I think FT tracking could benefit from the info that Rift already has. If I could get just raw screen positions of detected points, a two-point positional tracking built on top of the Rift can be fast and robust.

With the stickers I meant something that can be easily detected and discerned from the rest, to get the screen positions.
Title: Re: TrackIR
Post by: Jagerbomber on July 21, 2013, 10:05:53 am
Outerra + FreeTrack IR cap test - MiG 29 (http://www.youtube.com/watch?v=dJFJiGZo6uw#ws)

Stupid question... So this tracking works with just a webcam and the LEDs?
Where would I get the LED cap or clip or whatever?
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 10:14:01 am
It's actually both positional & rotational position that is gotten from it, as in the video. And the LEDs are infrared, to avoid interference with normal light. The camera should have an IR filter as well.
Title: Re: TrackIR
Post by: Jagerbomber on July 21, 2013, 10:40:31 am
It would be sweet if they added positional LED tracking on the Rift.  Maybe a separate optional kit from them that works well with it.

I really gotta get into this IR tracking stuff..... (or something better, idk).
Title: Re: TrackIR
Post by: zaelu on July 21, 2013, 02:54:47 pm
Very nice Cameni! When it will be implemented in Outerra? :)
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 03:01:42 pm
In opentrack there's Rift support, you may want to try using it as a primary tracker, with positional data provided by secondary PT.
Tested with opentrack-20130716_1, but it always silently quits after a seemingly random number of seconds of normal function, after the tracking is started.

Btw either with FTNoIR or with OpenTrack and with the point tracker, there are no raw point values (X1-X4, Y1-Y4) except for X1. Could it be done so that they would be always present, even if there's not enough points to infer the pose data?
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 03:09:39 pm
Very nice Cameni! When it will be implemented in Outerra? :)
Next build, which is due .. soon, after we hide some naked protruding wires of other unfinished stuff :)
Title: Re: TrackIR
Post by: foobie42 on July 21, 2013, 03:15:34 pm
Any entries in the event log, as in, access violations?

There doesn't seem to be anything fishy in Rift tracker that might cause a crash.

mm0zct, who wrote the Rift tracker, has been using it and sharing the software among Rift users... Never heard of any reports.

Only thing that comes to mind is sharing PDB's in install directory...

Oh, if you could editbin (included with MSVC compilers) /SUBSYSTEM:CONSOLE, you'd get debugging standard output and error, there might be some messages from SDK itself, which *could* in theory be causing an error exit...

This is embarassing, because one of the biggest reasons for the fork was removing sloppy code, fixing bugs, and making the software more maintainable...

Edit: Paging mm0zct.

> Could it be done so that they would be always present, even if there's not enough points to infer the pose data?

I'm sorry but that'd require some architectural changes. In particular, the tracker -> protocol, protocol is agnostic of whether it's an optical tracker, a face tracker, or INS.

Edit 2: The aforementioned fields are kept for compatibility purposes only. As for PT's mode of operation, it short-circuits the tracking function if not enough points are found.

If you're willing to donate a positional 2-point algorithm, I can add branching to PT, allowing for selection of a smaller amount of points. But there's a bit of debugging, testing and hacking involved, and it's perfectly understandable if you wouldn't want to get involved.

Edit 3: If you're using Norton or Avira crapola it might cause the issue. Just saying.
Title: Re: TrackIR
Post by: cameni on July 21, 2013, 03:49:27 pm
Console doesn't tell anything when the crash happens. Hooked the debugger and got this trace:

Code: [Select]
ntdll.dll!_RtlReportCriticalFailure@8()
  ntdll.dll!_RtlpReportHeapFailure@4()
  ntdll.dll!_RtlpLogHeapFailure@24()
  ntdll.dll!_RtlFreeHeap@12()
  kernel32.dll!_HeapFree@12()
  msvcr100.dll!free(void * pBlock=0x0864a490)  Line 51 C
  opentrack-tracker-pt.dll!522316c3()
  [Frames below may be incorrect and/or missing, no symbols loaded for opentrack-tracker-pt.dll]
  opentrack-tracker-pt.dll!5238ea6d()
  msvcr100.dll!malloc(unsigned int size=138867956)  Line 89 C
  QtGui4.dll!5473db8a()
  QtGui4.dll!547c3b94()
  QtGui4.dll!547c44d7()
  QtGui4.dll!547c44df()
  QtGui4.dll!547c9a8f()
  QtGui4.dll!5476a995()
  QtGui4.dll!54bd8d4b()
  ntdll.dll!_ZwFreeVirtualMemory@16()
  ntdll.dll!_RtlpSecMemFreeVirtualMemory@16()
  ntdll.dll!@RtlpFreeHeap@16()

There doesn't seem to be anything fishy in Rift tracker that might cause a crash.
Not Rift tracker, that one is integrated in OT, but the point tracker.

Quote
If you're willing to donate a positional 2-point algorithm, I can add branching to PT, allowing for selection of a smaller amount of points. But there's a bit of debugging, testing and hacking involved, and it's perfectly understandable if you wouldn't want to get involved.
Surely I would, this should be universal, for use in other projects as well.
For our purpose, with the Rift tracking already integrated, we would need just a shortcut from the point tracker after it extracts the raw points, with no additional computation. Together with the data from Rift we can then extract the positional info. Once this runs, it can be pushed back into the Rift tracker in opentrack, so that it provides the full pose data. In theory I could do it directly on the Rift tracker code in opentrack, but it would take much more time that way I think.

So I'm thinking - a special version/branch of the point tracker that puts the raw positions into the main fields (X/Y/Z/angles) for the time being, I'd develop the code for getting the position out of it, and then donate it back to opentrack to support that mode out of the box (or that combined mode too, for games that already integrate Rift and want just the extra option for position). Plus a tutorial for the hardware mod required, of course, when it all works as planned.
Title: Re: TrackIR
Post by: foobie42 on July 21, 2013, 05:05:51 pm
For 2 points to be supported in any case dynamic pose resolution has to be conditionalized. That is, the 'velocity' part, as position alone doesn't describe a rigid body anymore...

Would you rather want a ripped-out version of PT, running as executable? Stuff that doesn't fit a 2-point model can be ripped out and later merged and branching added.

opentrack uses CMake as its build system, which can be configured to produce .sln files, as well as projects for various IDEs. In worst case, Makefiles can be created. Learning curve's pretty small anyway.

In any case you can just run the blob extractor and nothing else for the moment. If you'd rather go with stickers instead of thresholding, HSV detection could be used.

Because so far there's only been one point model configuration, the code isn't generic enough to just fill in a few virtuals. Some refactoring is needed.

So where do we go from here? If you insist on getting position info from the blob extractor through a shared memory mapping it's doable, though bit awkward. With 'compat' library can create a different mapping altogether just for blobs. Alternatively you could build PT on either Windows or Linux, which is pretty seamless provided you have the dependencies.

The crash bug turned out to be stale object files on the build box, or some linker snafu. Rebuilding everything with no code changes helped. Today's build won't have this issue. Also including PDBs. Shoulda done it a long time ago.

TL;DR version - don't you want just the blobs in another mapping to hack on it?
Title: Re: TrackIR
Post by: foobie42 on July 21, 2013, 05:36:48 pm
https://github.com/opentrack/opentrack/commit/2634e7faa57c53434a8677c88e0e6dbd8134d4e0

Good enough? Do you want premade DLL and/or instructions on how to build the damn thing?

Note that presently the point list is unsorted. Will you be able to deal with it? This is because find_correspondences() side-effects the tracker object, as well as making assumptions about point count. What next?
Title: Re: TrackIR
Post by: cameni on July 22, 2013, 03:05:49 am
Thanks. I guess a premade dll would save me some time initially, but I can also build it myself - might be more useful in the long term.
Unsorted is fine.
Title: Re: TrackIR
Post by: zaelu on July 22, 2013, 03:17:23 am
Next build, which is due .. soon, after we hide some naked protruding wires of other unfinished stuff :)

Thanks, looking forward to it!
Title: Re: TrackIR
Post by: foobie42 on July 26, 2013, 08:23:12 am
Thanks. I guess a premade dll would save me some time initially, but I can also build it myself - might be more useful in the long term.
Unsorted is fine.

http://www.mediafire.com/download/548mfp45l8xfzmb/opentrack-tracker-pt.dll (http://www.mediafire.com/download/548mfp45l8xfzmb/opentrack-tracker-pt.dll)

Can use this with 20130716 or later builds. While -16 PT crashes, this one is ABI-compatible and works. It's based on the github branch, and as such, passes extracted points without computing or exposing headpose data.

If you plan on building yourself, which is recommended, make sure you use cmake-gui on Windows or Linux, and not cmake, as there are quite a few options to fill out for the build! In any case if you run into problems, send a message anywhere instead of blindly drudging along with unfamiliar codebase/build system.

Sorry about mediafire host but ananke.laggy.pk is in the process of being transferred to a different box.
Title: Re: TrackIR
Post by: foobie42 on July 27, 2013, 12:34:50 am
Sup cameni

How do you feel about using an aruco-readable marker for 6DOF tracking? All it requires is a printer. User already suggested it as an alternative to face tracking and it does indeed sound interesting. Can be splatted to the back of the VR headset of course.
Title: Re: TrackIR
Post by: cameni on July 27, 2013, 02:15:38 am
I still have problems with that crashing. I have the build 20130716.

How do you feel about using an aruco-readable marker for 6DOF tracking? All it requires is a printer. User already suggested it as an alternative to face tracking and it does indeed sound interesting. Can be splatted to the back of the VR headset of course.
That's what I meant by the markers to stick on the Rift, instead of using IR cam and LEDs. No exps what's the performance though.
Title: Re: TrackIR
Post by: foobie42 on July 27, 2013, 03:10:17 am
for cameni (http://www.youtube.com/watch?v=w0aX59wR1sg#)

Should do 120 Hz in 640x480, if that was even possible by today's consumer-quality webcams, actually 6ms per frame on my i7. It's completely stateless, too.

http://ananke.laggy.pk/opentrack/opentrack-20130727_1.7z (http://ananke.laggy.pk/opentrack/opentrack-20130727_1.7z) - build with PT changes by Patrick and aruco!

Note that this build contains unmodified PT which doesn't export raw point data... But please do tell if it crashes for you! Because if it does, there's something seriously amiss here. Just gave you the 'install' directory that I'm running from myself!

Excuse the messy appearance, it's hot here and I haven't slept at night as usual :D

I believe that due to how reliable it is, there's no need to develop/port-from-Delphi the 2-point algorithm at all... Don't you agree?

Both aruco and opencv are liberally licensed. Are you mad enough to link them into Outerra's address space? :)

IMPORTANT: Singularity on the Y axis is solved by extrinsic guess from previous pose.
Title: Re: TrackIR
Post by: cameni on July 27, 2013, 06:21:13 am
LOL, nice video :)
This version seems to be stable, no crashes so far.

The problem with augmenting the Rift with extra positional sensing is twofold

I think ideally we would use the accelerometer in Rift to integrate the position, correcting it with the positional information from opentrack, obtained from one of the trackers. So I was just thinking how to get that positional info, the two-point (or more) algorithm should be easy to implement when it has auxiliary orientation info from the Rift, and also more robust as currently the detection is pretty unstable on my setup. With the raw values I should be able to get the position more reliably, and just 2 points are enough.

The simplest way would be to have the accelerometer integrating only when there's a good position info from the tracker, but just combining the positional info from the tracker would be enough for a start.

I'm going to test it with aruco, that would be a pretty simple way. However, since the orientation isn't needed, I guess something simpler could be done with opencv alone ... like again, getting raw values of some markers. But I need to check what opencv provides.
Title: Re: TrackIR
Post by: foobie42 on July 27, 2013, 07:50:55 am
You can write a rather simple complementary filter to avoid jumpiness when updating positions. Unless you want to go full EKF which is... unadvisable for sanity's sake.

Obtaining position only seems like a special case since it doesn't fully describe a rigid body in some space...

I'm having a rather big problem with the LevMarq solver for the PNP problem, on some marker-on-head alignments it treats both pitch signs as one... There are many more people with the same issue. Only other option is to use EPNP solver which has abysmal accuracy. Other than that, I've tried initializing LevMarq with EPNP extrinsic guess, but it 'optimized' it away...

Why all that? It delivers vastly superior performance to a face tracker, requiring little filtering, and being entirely stateless which is a great boon! Better than that, unlike an IR tracker, there's no need to distinguish individual points, since aruco markers have orientation built in...

If not for degenerate pitch case, I'd be seriously recommending this to everyone...
Title: Re: TrackIR
Post by: zaelu on July 27, 2013, 05:17:19 pm
http://www.outerra.com/forum/index.php?topic=70.msg22148#msg22148 (http://www.outerra.com/forum/index.php?topic=70.msg22148#msg22148)

:D
Title: Re: TrackIR
Post by: foobie42 on July 28, 2013, 08:09:30 am
opentrack + aruco AR marker tracker + Falcon BMS 4.32u6 (http://www.youtube.com/watch?v=PH5pv9Ou53M#)

All works fine, but marker has to be tilted backwards, due to aforementioned pitch issue.
Title: Re: TrackIR
Post by: cameni on August 03, 2013, 01:32:10 pm
Testing opentrack-20130731 with aruco - values shown in the tracker differ from what is sent via the FreeTrack protocol. The main problem is that Z/RawZ saturates at 999, while the tracker can go much farther than that (anything >60.0 shows as 999 in RawZ).

Two questions:
- how do I convert FT translation units to the values shown by the tracker?
- why are the filtered and raw values different even when no filter is used?

I've been using the filtered values from ftnoir and opentrack, and found that those are quite unstable, often sticking/saturating at odd angles and losing track, while the raw values seem to be much more usable.
Title: Re: TrackIR
Post by: foobie42 on August 03, 2013, 01:55:38 pm
1) Divide values by ten.
2) There's a spline applied mapping real-life values to game output. Check the 'curves' tab, you can drag the control point to upper-right corner if you wish.
3) Make a proper gain for the filter, Accela -> Preset -> rotation:2, translation:1 for starters. Reduction factor should be at 1k, max value, since this is polling interval. Accela splines are extrapolated linearly into infinity, unlike game mappings, which are bounded at last control point.

Alternatively you can take the BSD-licensed Aruco library and integrate it with your software. You might have to use sthalik/aruco if it fails to build, e.g. under the GNU compiler, or with OpenCV from git.

I don't know the dimensions of the VR headset, but if it's bulky to the sides, you can splat more markers. Each has an id, it works as long as they're unique. That can solve the ~70 deg yaw limit.

Thresholding is a pain, if any user has asymmetric lighting e.g. due to a window to the side, problems are bound to occur. That's why Otsu thresholding fails for me badly and have to use a less elaborate method of detecting marker's edges.
Title: Re: TrackIR
Post by: cameni on August 03, 2013, 02:32:00 pm
I don't think the filter section does anything there (with aruco and FT). Changing the gain did nothing to the translation values, nor making the curve linear y=x. I can see the raw value of Z going fine, but filtered Z always saturates at ~40cm from the camera, and that's what I'm getting with FT protocol.

Even setting the filter to none behaves as if there was the default Accela filter bound, and I'm definitely not receiving the raw translation values in FreeTrack protocol, they are the same as the filtered ones.
Title: Re: TrackIR
Post by: foobie42 on August 03, 2013, 02:40:13 pm
Do you have a profile selected in the upper-center box? If you have none whatsoever it might fail to work. Just saying.

Only the filter spline extrapolates. The game output spline needs to be dragged as much as you want output to the game. If max y = 40, it'll be bounded to that value.

Additionally, unless camera FOV is correct-ish and marker is exactly 7 cm in size starting from the black border, translation values won't be in real-life units :( Can add a marker size option again, didn't think anyone would care about that.

As for raw values, that's correct. And it's the same way as in FTNOIR, no one until you cared about that :-)
Title: Re: TrackIR
Post by: cameni on August 03, 2013, 02:50:13 pm
Yes I have saved it in a custom profile, restarted.
Output spline in the filter is set like this:
(http://i.minus.com/jWFRJE1JRkF6b.png) (http://minus.com/lWFRJE1JRkF6b)

Attached the file with my settings.
Title: Re: TrackIR
Post by: foobie42 on August 03, 2013, 02:56:22 pm
Set a linear spline in the curves window, activated by the bottom-right button. Right now it seems to use some nonsensical default-profile value, outside the window's bounds. Will provide access to raw FT data in the next binary build.
Title: Re: TrackIR
Post by: cameni on August 03, 2013, 03:08:39 pm
Ok, it's better, allows me to get further from the camera, like ~65cm instead of 40. However, the raw values continue beyond 100.0, the filtered ones saturate. Would it be possible to let them go beyond that? Or I might just use the raw ones once they are in in the next build.

With camera fov ~45° and a 10cm marker it's much more comfortable if I could be >1m away, especially with the Rift, it provides much larger operational area. I guess that eventually I'll integrate aruco directly, to save people the trouble, but I'd like to get it working with opentrack first.
Title: Re: TrackIR
Post by: foobie42 on August 03, 2013, 03:25:18 pm
http://ananke.laggy.pk/opentrack/opentrack-20130803_1.7z (http://ananke.laggy.pk/opentrack/opentrack-20130803_1.7z)

Do tell if there are any new problems with detecting the marker. I've removed gaussian blur prior to 'adaptive' thresholding, and increased the box size. You can easily check if it's detected by a red border around the black frame.

Lots of grain in the image will dramatically increase CPU usage though, due to contour finding being a mess :(
Title: Re: TrackIR
Post by: cameni on August 03, 2013, 04:11:15 pm
Perfect. The marker is detected without problems, now it also uses less CPU, but that may also be because it got dark outside :)

It works pretty nicely in combination with the Rift, with absolute positioning it's less dizzy, as the picture reacts to the full motion and brain doesn't get confused.
Title: Re: TrackIR
Post by: cameni on August 04, 2013, 12:53:37 pm
I've posted a blog about ArUco+opentrack+outerra: http://outerra.blogspot.sk/2013/08/absolute-positioning-with-oculus-rift.html (http://outerra.blogspot.sk/2013/08/absolute-positioning-with-oculus-rift.html)

The thresholding is indeed a problem there, when there's a brightly lit spot somewhere on the image it will fail. I had to add extra light source to to boost the marker brightness.
Title: Re: TrackIR
Post by: foobie42 on August 04, 2013, 01:03:53 pm
You may want to run 'aruco_test.exe live' to see how thresholding works in the first place.

For the first slider, the setting is 11 (box filter neighborhood), for the second one, it's 5 (thresholding sensitivity).

You may want to reduce camera gain to avoid overbright marker when rotated toward the light. It's unfortunately not totally light-invariant, as light on high gain totally blinds the marker, making it all white... Hard for the detector to see markings if there aren't any in the first place.

Brightly-lit spots ANYWHERE in the image won't break thresholding, as it works in a 11x11 neighborhood, provided the marker isn't nearly close to that spot. If you have like me natural light from the side however, it can cause problems with yaw.
Title: Re: TrackIR
Post by: cameni on August 04, 2013, 04:52:42 pm
Could the filtered positional values not be clamped at 100 in opentrack? I switched to raw values because only those are unclamped with aruco, but now it's shaky in FT.
Title: Re: TrackIR
Post by: foobie42 on August 06, 2013, 08:37:40 am
You can press the 'center' key defined in shortcuts window. That's how regular users do it.

In any case just make sure you configure the filter using the preset tab, valies rot:2 and trans:1 are fine for marker tracking.

Or you can rip out the Accela filter's guts, or make an direct elliptical filter or something. I wrote Accela so copyright isn't a problem.

The request is pretty singular, it'd be better to help you implement filtering in your own software, together with Aruco instead of making far-ranging changes in opentrack.
Title: Re: TrackIR
Post by: cameni on August 06, 2013, 09:21:59 am
Okay, I didn't know about the centering so I've implemented it in OT. Rift is centered there on a key, so it only made sense to sync the resets in the Rift+opentrack mode.

But basically I just wanted to test opentrack and Aruco without too much extra work first, to see how it works. For now, I find it quite sensitive to the lighting conditions and noise, and given that with Rift I only need the positional info, it's an overkill. Adding 2 LEDs to the Rift and using it with the point extractor would be more robust, but it's unlikely to find a widespread use given that consumer Rift will likely come with a built-in magnetic position tracking. So I guess it will remain only for my tinkering, when I get some time for it again.

Title: Re: TrackIR
Post by: Jagerbomber on August 12, 2013, 08:58:46 pm
Cameni, how well do you think overall tracking (especially positional) could work with a Rift and a single Playstation Move controller? (or maybe multiple idk) and a webcam (maybe the PS Eye or the new one coming with the PS4... or... something else.)
Title: Re: TrackIR
Post by: cameni on August 13, 2013, 02:26:39 am
I have no experience with the Move, but the technology seems to be quite elegant. So basically if you attached a glowing orb to the Rift, detection of the sphere on the camera is fairly simple and fast.
Title: Re: TrackIR
Post by: Jagerbomber on August 13, 2013, 05:41:15 pm
If you're using the Rift for rotation, is one enough, or would multiple be better?
Title: Re: TrackIR
Post by: cameni on August 14, 2013, 03:54:48 am
They are using the visible size of the orb to determine the distance, it is enough to find out the position. With multiple ones it could be more precise though.
Title: Re: TrackIR
Post by: Jagerbomber on August 14, 2013, 05:14:14 pm
Thought so.  Maybe someday I shall have glowing Playstation Move antlers!  ;D
Then the whole nerd herd will respect me!

I still wonder how smoothly it will work.  Only cam I have is the PS3 version of the PS Eye, and I'm pretty sure there's unofficial PC drivers for it, but I haven't bothered yet.
Title: Re: TrackIR
Post by: foobie42 on August 15, 2013, 04:55:33 pm
PS3 Eye is a superb sensor, CLEye drivers give kind of bad color correction, but framerates are awesome.
Title: Re: TrackIR
Post by: Jagerbomber on August 15, 2013, 05:09:04 pm
Cool  ;D
Title: Re: TrackIR
Post by: zaelu on August 28, 2013, 01:44:40 pm
I also use a modified PS3 Eye for my Freetrack. I modified 3 or 4 by now and they work great. Minus mine that has a little glitch at starting of the tracking but it is functioning with a "work around". :)
Title: Re: TrackIR
Post by: PytonPago on August 28, 2013, 02:38:10 pm
Heh, saw someone using those to make an click-glove - just an IR light at the tip of the finger whyte an button on top, so you can use it as a "touch-screen workaround" for classical LCDs - guy used that for the DCS Ka-50 cockpit buttons.
Title: Re: TrackIR
Post by: DivineSense on September 09, 2013, 07:00:33 am
Cool PytonPago! Do you have any more info on this? I tried googling but came up with empty hands.
Title: Re: TrackIR
Post by: foobie42 on September 17, 2013, 07:19:29 am
cameni, is there any Linux API we could standardize properly, instead of using nothing at all in your codebase, or evdev joysticks and manual cfg edits in case of the eurotruck2 sim?

Please see: <http://forum.scssoft.com/viewtopic.php?f=41&t=27> in the case of the trucker sim.
Title: Re: TrackIR
Post by: PytonPago on September 17, 2013, 07:32:01 am
Cool PytonPago! Do you have any more info on this? I tried googling but came up with empty hands.

https://www.youtube.com/watch?v=oO8l4bPTiHs (https://www.youtube.com/watch?v=oO8l4bPTiHs)

Except you can use normal full finger gloves and the button will be on the fingertip (lighting the IR diode whyte a slight push towards the screen) witch can be under a second layer of textile for making it invisible and not damaging the screen. Saw it in another video like that from a KA-50 sim-pit builder. Its a very interesting thing indeed.

The wii works under PC too, so dont mind the "Wiimote whiteboard" there ...
Title: Re: TrackIR
Post by: PytonPago on September 17, 2013, 07:35:50 am
Ahh ... here it is :

www.youtube.com/watch?v=aLkfx6QxLfg (http://www.youtube.com/watch?v=aLkfx6QxLfg)
Title: Re: TrackIR
Post by: DivineSense on October 28, 2013, 05:10:04 pm
Wow Python, that's brilliant!!! I want to make one of those!
Title: Re: TrackIR
Post by: Francous on January 12, 2016, 03:13:20 pm
Hi everyone.. OK this post is about Track IR into Outerra , do we have a fix to make it work or implementation into the engine ?  Thank you

  For track IR update ?