|Bob Mottram 38b2840830 list||6 years ago|
|.metadata||11 years ago|
|.settings||11 years ago|
|Debug||11 years ago|
|cppunitlite||11 years ago|
|deb||11 years ago|
|docs||11 years ago|
|images||6 years ago|
|ros||11 years ago|
|segment||10 years ago|
|unittests||11 years ago|
|.cproject||11 years ago|
|.project||11 years ago|
|LICENSE||6 years ago|
|Makefile||10 years ago|
|README.md||6 years ago|
|anyoption.cpp||11 years ago|
|anyoption.h||11 years ago|
|drawing.cpp||11 years ago|
|drawing.h||11 years ago|
|fast.cpp||11 years ago|
|fast.h||11 years ago|
|harris.cpp||11 years ago|
|harris.h||11 years ago|
|libcam.cpp||11 years ago|
|libcam.h||11 years ago|
|linefit.cpp||11 years ago|
|linefit.h||11 years ago|
|main.cpp||10 years ago|
|makepackage.sh||11 years ago|
|motion.cpp||10 years ago|
|motion.h||11 years ago|
|omni.cpp||10 years ago|
|omni.h||10 years ago|
|polynomial.cpp||11 years ago|
|polynomial.h||11 years ago|
This is a utility for dealing with images obtained from omnidirectional vision systems. It is intended for use on mobile robots equipped with omnidirectional vision sensors.
Here's how you can make an inexpensive omnidirectional vision system using a webcam and easily available materials.
Here's what you'll need:
Pull the top off of the Christmas baubel, then drill a 5mm diameter hole at the opposite end.
Push the aluminium tube through the holes on either side of the baubel, then crimp and bend the end of the tube so that it prevents the baubel from slipping off.
Using pliers, bend the tubing into a frame like this:
The height of the frame should be about 160mm.
Remove the outer casing from the webcam. You may need a precision screwdriver to do this, and how you do it really depends upon the specific model.
Cut a piece of cardboard larger than the triangle shape of the frame.
Use the bradawl or screwdriver to make holes in the cardboard, then fix the webcam onto the cardboard using tie wraps threaded through the holes.
Lay the cardboard over the triangle of the frame and make holes so that you can tie wrap it down. Make sure that the camera is pointing at the centre of the baubel. A good way to do this is to run the camera and adjust the position as needed.
Trim off any excess cardboard using scissors.
The final result should look something like this:
The under side looks like this:
It's also a good idea to tie wrap the camera cable onto the frame, so that it can't get pulled out as it's moved around.
For a more professional system you could use plywood or sheet aluminium rather than cardboard.
To install the software first you will need to ensure that OpenCV version 2 is installed. A package for this can be downloaded from the project home page or downloads tab. At the time of writing this is not the standard version of the OpenCV libraries available from within the package manager, and the main advantage is that the newer version allows images to be encoded for video streaming.
A Debian type Linux operating system is assumed here. To install from the terminal type:
sudo dpkg -i opencv-2.0.deb
Then the main utility can be installed as follows:
sudo dpkg -i omniclops-x.xx.deb
Any additional dependencies should be installed automatically.
Alternatively, once the OpenCV libraries have been installed you can also compile from source in the usual way with:
make clean make
If you make changes to the code you can also easily create a new deb package by running:
Configuring for use with Eclipse
To use with the Eclipse C++ IDE:
Create a new C++ project
Import the source from the Import/File System option
Within the project properties select C++ Build/Settings Under GCC C++ Compiler/Directories/Include paths (-l) enter
Under GCC C++ Compiler/Miscellaneous enter
-c -fmessage-length=0 -lcam -lcv -lcxcore -lcvaux -lhighgui `pkg-config --cflags --libs gstreamer-0.10` -L/usr/lib -lcv -lcxcore -lcvaux -lhighgui `pkg-config --cflags --libs glib-2.0` `pkg-config --cflags --libs gstreamer-plugins-base-0.10` -lgstapp-0.10
Under GCC C++ Linker/Libraries/Libraries (-l) enter
cv cxcore gstapp-0.10 highgui
Under GCC C++ Linker/Libraries/Library search path (-L) enter
Under GCC C++ Linker/Miscellaneous/Linker flags enter
/usr/lib/libcxcore.so /usr/lib/libcvaux.so /usr/lib/libcv.so /usr/lib/libhighgui.so
After applying those settings you should now be able to compile the project within Eclipse.
The most basic way of running the software is as follows:
omniclops --dev /dev/video0
where video0 is the camera device to be used. To find out what video devices are available just type:
One of the first things which you'll need to do is specify the inner and outer radii for the mirror. These are expressed as percentages.
In this example the outer radius is set to 90% and the inner to 15%. There are no limits on these figures, so potentially the outer radius could be set to greater than 100% if the camera is close to the mirror.
omniclops --dev /dev/video0 --radius 90 --inner 15
The purpose of the inner radius is to remove the camera itself from the image, since this is typically not of interest. The inner radius can be set to zero if you wish to include the camera.
To exit the program press the Escape key.
Unwarping is a particular way in which the omnidirectional image can be visualised, by projecting the raw image to a virtual camera located at the centre of the spherical mirror.
omniclops --dev /dev/video0 --unwarp
A useful feature is to be able to project the image from a spherical mirror onto the ground plane. This requires that the system know a few things about its geometry.
omniclops --dev /dev/video0 --diam 60 --dist 50 --focal 3.6 --elevation 400 --ground
where diam is the diameter of the spherical mirror, dist is the distance from the camera lens to the closest point of the mirror, elevation is the typical height of the camera above the ground plane and focal is the camera focal length. All distances are in millimetres.
Planar projection helps to remove the high degree of distortion around the periphery, and this results in the blocky appearance of the below image. Such images, although of low quality, can be useful for visual odometry purposes, since straight lines in the real world appear (mostly) straight within the image.
If you wish to create an image showing the paths of rays of light for your specified geometry, use something like the following:
omniclops --dev /dev/video0 --diam 60 --dist 50 --focal 3.6 --elevation 400 --raypaths rays.jpg
This will save an image called raypaths.jpg. At present only spherical mirrors are supported, although the system could be adapted for other types of mirror geometry.
Detecting lines within the image can be useful for localisation or visual odometry.
omniclops --dev /dev/video0 --lines
Optionally lines can be saved to file for subsequent analysis.
omniclops --dev /dev/video0 --saveradial lines.dat
The file contains 5 integer values per entry:
1. Orientation of the line in degrees 2. start x coordinate in pixels 3. start y coordinate in pixels 4. end x coordinate in pixels 5. end y coordinate in pixels
Optical flow can be shown by adding the --flow option to the command line.
For example, to show the optical flow in the raw image
omniclops --dev /dev/video0 --flow
or to show optical flow within the unwarped image
omniclops --dev /dev/video0 --unwarp --flow
When running the software on embedded systems, or if you wish to save a single image and then exit, it can be useful to run without displaying the image on the screen.
Headless mode can be enabled as follows:
omniclops --dev /dev/video0 --save test.jpg --headless
It's possible to stream video over a network using gstreamer. This can be useful for remotely operated vehicles or surveillance systems.
On the server:
omniclops --dev /dev/video0 --stream --headless
then on the client:
gst-launch tcpclientsrc host=[ip] port=5000 ! multipartdemux ! jpegdec ! autovideosink
To do a local test you can use localhost as the IP address.
One thing which you might typically wish to do is log images to file periodically. To have the program save an image and then exit use the following command:
omniclops --dev /dev/video0 --save test.jpg --headless
Alternatively you might wish to only save edge features to file, as follows:
omniclops --dev /dev/video0 --saveedges edges.dat --headless
This saves a binary file called edges.dat. This consists of pairs of unsigned short integers (16 bits x 2) giving the x and y image coordinates of each edge feature. To try to increase the accuracy the edge values are sub-pixel interpolated, so to obtain a floating point value for the x and y coordinates divide the values by 32.
You might also wish to save the 3D coordinates for rays of light corresponding to observed edge features. This feature can be useful for applications requiring bundle adjustment or synthesis of the data with that obtained by other sensors.
omniclops --dev /dev/video0 --diam 60 --dist 50 --focal 3.6 --elevation 400 --rays rays.dat --headless
The resulting binary file contains 6 signed short integers per ray, specifying the beginning and end of the ray. The first three values are a point on the mirror surface and the second three values are a point on the ground plane. All values are in millimetres.
Detection of edge features http://www.youtube.com/watch?v=2LEMFjGX1Bk
Planar projection http://www.youtube.com/watch?v=cjd6VbszaA4
Planar projection of edge features http://www.youtube.com/watch?v=AVZ5Osnde-U
Detecting radial lines http://www.youtube.com/watch?v=qH4ExS3XlH4
Image unwarping http://www.youtube.com/watch?v=Q_EmnGQpb2M
Optical flow http://www.youtube.com/watch?v=gWnDruAYCNQ