MENU

Part I : Gear Up Your Cam !

They are two main components in the system : the cameras and the gateway. The cameras are based CMUcam5 from Charmed Labs which is a position tracking camera which can output 2D coordinates.   Why this camera ? I have tested almost every alternatives, and the first prototypes were even based on Raspberry Pi camera, and the 2D position tracking was made using OpenCV image library on the Raspberry. I have also been working with the wiimote sensors and many standard HD webcam with FreeTrack. None of them did the job by a long shot ! You can order this camera on Amazon for 69$. What’s under the hood of a CMUcam5 ? Multi point tracking (great for multiple users) 640*480 @ 50FPS 1280*800 @ 30FPS Lens field-of-view: 75 degrees horizontal, 47 degrees vertical Voltage : 5 to 12V Power consumption : 140mA Open Source (hardware and firmware) Removable optics OK we found the camera, what do we “Gear Up” ? That camera is nice, but we have to add some stuff to make it great. Here is what we are going to do : 1. Get to know your camera 2. Remove infrared blocking to filter 3. Add infrared pass […]
Read More ›

Part II : Make the camera wireless with ESP8266

  Let’s sum up what we have done so far : we ordered a pair of color tracking cameras we modified them into Infrared tracking cameras we updated their firmware to track Infrared and tested them we built a small case to give them a nice look There is one last modification we have to bring to the cameras, and that modification is a game changer : we are going to make them wireless ! Of course we won’t send the video, but the coordinates of the points the camera is tracking. In this article we are going to connect an ESP8266 Wifi shield to the CmuCam to send coordinates over Wifi. Here is what we are going to do, step by step. All you need to know for this part is how to program an Arduino (I have seen 8 years old doing it so…I’ll give you a link just in case).   The basics Connect NodeMCU to CmuCam Program the ESP8266 The basics Here we’ll go through all you need to know to make it out of this tutorial alive. First if you don’t know what is Arduino, please read about it here : Arduino. We are not going […]
Read More ›

Part II : Build the Gateway and get your position ! (old)

  You have proudly assembled at least two wireless cameras and you finally want to test them. Good ! Because at the end of this post, your cameras will be talking with the Gateway, and we will get 3D coordinates from the two sets of 2D coordinates. The big step will be building the gateway, which is a Raspberry Pi 2, and installing all the software. Let’s have a look at the steps we are going to go through : 1. Prepare your Raspberry Pi 2. Make it talk with the cameras 3. Behind the curtains : calculating the 3D position by image resectionning 4. Send data back to the user with Websockets As you can see, we still have a lot on our plates. But once you have understood the concept, it gets easier. Prepare your Raspberry Pi We are going to use a Raspberry Pi 2 as a Gateway to receive 2D positions from the cameras, compute them into 3D positions with OpenCV and send them back to the user via WebSockets. Get a Raspberry Pi 2 and all the stuff necessary to go with (5V supply, Wifi dongle, SD card) and a Nrf24L01+ with long range antenna. Install Raspbian on your […]
Read More ›

Part IV : Send your position to Unity

In this part we will see how to retrieve the position in Unity and assign it to the main camera. We won’t go through the setup of Unity as you’ll find all the information you need online. You will have to install Websocket Sharp package (either the DLL or buy the Unity package for 15$ on the asset store). Here is the script you will have to use to connect to the Gateway websocket, get your position and send it to the camera. You just to add it into your project and call it somewhere as a component. using UnityEngine; using System.Collections; using WebSocketSharp;   public class WebSocketCameras : MonoBehaviour {   private WebSocket ws; private bool detecting = false; private Vector3 position; private Vector3 position2; private string GatewayIP = "192.168.1.101"; // Set your gateway IP here private string WebsocketPort = "9014"; Camera mainCam;     void Start () {   // Retrieve the main camera mainCam = Camera.main;   // Create and Open the websocket ws = new WebSocket("ws://"+ GatewayIP + ":" + WebsocketPort); ws.OnOpen += OnOpenHandler; ws.OnMessage += OnMessageHandler; ws.OnClose += OnCloseHandler;   ws.ConnectAsync();   }   private void OnOpenHandler(object sender, System.EventArgs e) { Debug.Log("WebSocket connected!"); }   private […]
Read More ›

Part III : Set Up Unity for a VR Experience

First, this isn’t going to be a long post showing you how to install and use Unity. In a few words, Unity is one of the most famous Game Engine. It allows you to develop games for PC, Android, iP Wii and almost every other platform. Unity is the most advanced Game Engine for VR (as opposed to Unreal Engine for example). Here I am just going to share some links and info to help you jumpstart your development of VR applications in unity.   The most important step is to integrate OpenCV in Unity. Then we need to communicate with the cameras using websockets. Finally we have to create a calibration sequence using OpenCV and add the tracking function. All of this had been done in the previous version using a Gateway, but I now have to translate all the code from python to C# and clean it of course. Get Unity Simply download and install the latest Unity Free version on your PC/Mac Communicate with the cameras (Websocket) The cameras communicates on your Wifi network using Websockets (each camera is a websocket server). To receive their 2D datas, your PC or phone has to be turned into a Websocket client. So we […]
Read More ›

VR Tracker: DIY position tracking system – Introduction

Oculus Rift, VR One, Samsung Gear, Sony Playstation VR… All those Virtual Reality are great, but they are missing something ! They do not have a position tracking system ! This means you can look up down right left, but can’t stand up and move around in your room. Too bad because the best experiences happens when you can actually move around by yourself in your virtual environment. How do I know ? Well I have been working with many of those headsets as a consultant for an augmented reality company, and I have developed a 3D position tracking system compatible with all those devices ! And let me tell you, I have tested this tracking system with Samsung Gears, and the result was AWESOME !!! I was able to move around in a 5 meters by 8 meters room and therefore in my virtual environment. What is so great about position tracking ? The range of applications is much wider ! You can for example create an experience in a museum where users could visit a pyramid using VR headsets in an Egyptian exposition. Or visit an appartement or a building even before it has been built ! You could also create […]
Read More ›

Class in a Case : Assemblage des ordinateurs

Class in a Case est un projet lancé par PC2, visant à fabriquer des micro ordinateurs portables, consommant peut d’énergie, qu’on peut facilement transporter pour donner des formations. Voici comment s’assemblent ces petits ordinateurs imprimés en 3D. Matériel requis : – Boitier à imprimer en 3D, 3 pièces principales (si vous les désirez, envoyez un mail à PC2) – 1 Raspberry Pi 2 – 1 Écran 7 pouces avec contrôleur – 4 vis M2 6mm – 10 vis M2 12mm – 4 vis M2 16mm     Assemblage :   1.Insérer l’écran dans son cadre                       2. Visser les clips sur le cadre de l’écran Il faut utiliser 6 vis de 12mm, avec 5 clips de 5mm et 1 clips de 2mm ATTENTION : Mettre le clips le plus fin au bon endroit !                         Les 5 clips de 5mm ont été placés, il reste le dernier, de 2mm, le plus petit.                           3. Fixer le boitier principal Il suffit de poser le boitier, en faisant passer le câble de […]
Read More ›

Class in a Case : 3D Printing cases

Lately I started the design of the cases for Class in a Case and here are some pictures ! As I am writing the V1 is printing. I already made a V0, mostly for testing, but the V1 should be much closer to a final version. It is going to need some improvements of course, but it will be ready very soon ! If you want the files, here are the V1 in STL and Solidworks 2015 format. I wouldn’t recommend to use them as is because those are still beta, but if you want to bring more features, feel free to use them ! 3D Files The cases are made for Raspberry Pi version B, and this screen : 7 inch hdmi I will make an assembly and demo video as soon as the case are OK.   UPDATE : The white case was V0, but here is V1, which is fully working and just needs minor corrections :  
Read More ›
Skip to toolbar