Difference between revisions of "Visual Systems for Robots"
From RoboWiki
(→Part 2: Training on 2D processing) |
(→Part 2: Training on 2D processing) |
||
| (14 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | __NOTOC__ | ||
by ''RNDr. Andrej Lúčny, PhD.'' (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010 | by ''RNDr. Andrej Lúčny, PhD.'' (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010 | ||
| + | * You can use Matlab [[Image:IconMatlab.png]] or Octave [[Image:IconOctave.png]] | ||
* download accompanying workfiles [[Media:visual.zip]] | * download accompanying workfiles [[Media:visual.zip]] | ||
| − | * unzip (to <TT>C:\Work</TT> | + | * unzip (recommended to <FONT Color="green"><TT>C:\Work</TT></FONT>) |
== Part 1: Training on input image mainpulation == | == Part 1: Training on input image mainpulation == | ||
| Line 18: | Line 20: | ||
* start Octave or Matlab | * start Octave or Matlab | ||
| − | * change directory to directory with this training using the <TT>cd 'C:\Work\part1'</TT> | + | * change directory to directory with this training using the <FONT Color="green"><TT>cd 'C:\Work\part1'</TT></FONT> |
| − | * launch <TT>tr1</TT> | + | * launch <FONT Color="green"><TT>tr1</TT></FONT> |
== Part 2: Training on 2D processing == | == Part 2: Training on 2D processing == | ||
| + | |||
| + | * Install additional Plug-Ins for ImageJ. [[Media:ij-plugins_toolkit.jar|Download Plugin]], copy it to the <FONT Color="green"><TT>cd 'C:\Work\ImageJ\plugins\ij-plugins_toolkit.jar'</TT></FONT> | ||
| + | |||
* ''start'' ImageJ | * ''start'' ImageJ | ||
| Line 27: | Line 32: | ||
* ''turn it grayscale:'' Image / Type / 8bit | * ''turn it grayscale:'' Image / Type / 8bit | ||
* ''descrease noise by threshold:'' Image / Adjust / Threshold | * ''descrease noise by threshold:'' Image / Adjust / Threshold | ||
| − | ** ''select'' Black & White ''from menu | + | ** ''select'' Black & White ''from menu'' |
| + | ** ''move with low and high end of range to emphasize the seen objects'' | ||
| + | ** Apply ''and close the threshold window'' | ||
* ''make the image binary:'' Process / Binary / Make binary | * ''make the image binary:'' Process / Binary / Make binary | ||
* ''if the picture contains now black object on white background, change it to white object on black background:'' Image / Lookup Tables / Invert LUT | * ''if the picture contains now black object on white background, change it to white object on black background:'' Image / Lookup Tables / Invert LUT | ||
* ''try to eliminate holes in object by'' Process / Binary / Dilate ''and'' Process / Binary / Erode | * ''try to eliminate holes in object by'' Process / Binary / Dilate ''and'' Process / Binary / Erode | ||
* ''try'' Process / Binary / Skeletonize | * ''try'' Process / Binary / Skeletonize | ||
| + | |||
* ''restart'' ImageJ | * ''restart'' ImageJ | ||
* ''open'' pic1.jpg | * ''open'' pic1.jpg | ||
* ''start segmentation to 2 colors by'' Plugins / Segmentation / k-means clustering ''(select two colors)'' | * ''start segmentation to 2 colors by'' Plugins / Segmentation / k-means clustering ''(select two colors)'' | ||
| − | * ''return to the opened'' pic1.jpg ''and three times apply'' Process /Smooth | + | * ''return to the opened'' pic1.jpg ''and three times apply'' Process /Smooth ''then again perform the same segmentation'' |
| − | ''then again | ||
* ''compare the two segmented images and select better'' | * ''compare the two segmented images and select better'' | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | start Octave or Matlab | + | * ''restart'' ImageJ |
| − | change directory to directory with this training by cd | + | * ''open'' pic2.jpg |
| − | launch tr2 | + | * ''apply'' Process / Filters / Gaussian Blur / 1.0 |
| + | * ''apply'' Process / Find edges | ||
| + | * ''apply'' Process / Binary / Make Binary | ||
| + | * ''apply thinning by'' Process / Binary / Skeletonize | ||
| + | |||
| + | |||
| + | * start Octave or Matlab | ||
| + | * change directory to directory with this training by <FONT Color="green"><TT>cd 'C:\Work\part2'</TT></FONT> | ||
| + | * launch <FONT Color="green"><TT>tr2</TT></FONT> | ||
| + | |||
| + | == Part 3: Training on 3D processing == | ||
| + | |||
| + | * ''start'' ImageJ | ||
| + | * ''open'' pic3.jpg, depthX.jpg ''and'' depthY.jpg | ||
| + | |||
| + | There is box and figure on the image pic3.jpg -- you can see that these images has the same resolution.<BR> | ||
| + | <TT>depthX</TT> represents X coordinate of surface point visibile at the particular pixel<BR> | ||
| + | <TT>depthY</TT> represents Y coordinate | ||
| + | |||
| + | How many times further is the box than the figure ? | ||
| + | |||
| + | (Hint: use ImageJ to display coordinates of the proper surface points | ||
| + | and find distance from camera from depth images) | ||
| + | |||
| + | (Hint2: The correct answer is half of very known and famous number) | ||
| + | |||
| + | |||
| + | |||
| + | == Part 4: Training on recognition == | ||
| + | |||
| + | * start Octave or Matlab | ||
| + | * change directory to directory with this training by <FONT Color="green"><TT>cd 'C:\Work\part4'</TT></FONT> | ||
| + | * launch <FONT Color="green"><TT>tr4</TT></FONT> | ||
| + | |||
| + | This program recognizes orange color of ball and turn the image pic4.jpg to pic5.jpg | ||
| + | |||
| + | |||
| + | * Copy <FONT Color="green"><TT>Hough_Circles.class</TT></FONT> and <FONT Color="green"><TT>Hough_Circles.java</TT></FONT> into <FONT Color="green"><TT>C:\Work\ImageJ\plugins\HoughTransform</TT></FONT> (or similar path) | ||
| + | |||
| + | |||
| + | * ''start'' ImageJ | ||
| + | * ''open'' pic5.jpg | ||
| + | * ''turn it to binary:'' Process / Binary / Make Binary | ||
| + | * Process / Binary / Dilate | ||
| + | * Process / Binary / Skeletonize | ||
| + | * Plugins / Hough Cricles / ''ask for 1 circle from 30 to 100 and the program should caclulate center of the circle'' | ||
| + | |||
| + | What will happen if we do not decrease number of white points | ||
| + | by skeletonization ? | ||
| + | |||
| + | == Software used == | ||
| + | |||
| + | * Matlab or [http://www.octave.org Octave] | ||
| + | * [http://rsbweb.nih.gov/ij/ ImageJ] (we assume it installed in C:\Work\ImageJ) | ||
Latest revision as of 11:45, 6 July 2010
by RNDr. Andrej Lúčny, PhD. (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010
- You can use Matlab
or Octave 
- download accompanying workfiles Media:visual.zip
- unzip (recommended to C:\Work)
Part 1: Training on input image mainpulation
- start ImageJ
- open pic1.jpg
- Image / Color /
RGB SplitSplit Channels - apply Image / Lookup Tables / Red to pic1.jpg (red)
- apply Image / Lookup Tables / Green to pic1.jpg (green)
- apply Image / Lookup Tables / Blue to pic1.jpg (blue)
- close everything
- open pic1.jpg
- apply Image / Type / 8bit to get gray image
- apply Process / Binary / Make Binary to get binary image
- start Octave or Matlab
- change directory to directory with this training using the cd 'C:\Work\part1'
- launch tr1
Part 2: Training on 2D processing
- Install additional Plug-Ins for ImageJ. Download Plugin, copy it to the cd 'C:\Work\ImageJ\plugins\ij-plugins_toolkit.jar'
- start ImageJ
- open pic1.jpg
- turn it grayscale: Image / Type / 8bit
- descrease noise by threshold: Image / Adjust / Threshold
- select Black & White from menu
- move with low and high end of range to emphasize the seen objects
- Apply and close the threshold window
- make the image binary: Process / Binary / Make binary
- if the picture contains now black object on white background, change it to white object on black background: Image / Lookup Tables / Invert LUT
- try to eliminate holes in object by Process / Binary / Dilate and Process / Binary / Erode
- try Process / Binary / Skeletonize
- restart ImageJ
- open pic1.jpg
- start segmentation to 2 colors by Plugins / Segmentation / k-means clustering (select two colors)
- return to the opened pic1.jpg and three times apply Process /Smooth then again perform the same segmentation
- compare the two segmented images and select better
- restart ImageJ
- open pic2.jpg
- apply Process / Filters / Gaussian Blur / 1.0
- apply Process / Find edges
- apply Process / Binary / Make Binary
- apply thinning by Process / Binary / Skeletonize
- start Octave or Matlab
- change directory to directory with this training by cd 'C:\Work\part2'
- launch tr2
Part 3: Training on 3D processing
- start ImageJ
- open pic3.jpg, depthX.jpg and depthY.jpg
There is box and figure on the image pic3.jpg -- you can see that these images has the same resolution.
depthX represents X coordinate of surface point visibile at the particular pixel
depthY represents Y coordinate
How many times further is the box than the figure ?
(Hint: use ImageJ to display coordinates of the proper surface points and find distance from camera from depth images)
(Hint2: The correct answer is half of very known and famous number)
Part 4: Training on recognition
- start Octave or Matlab
- change directory to directory with this training by cd 'C:\Work\part4'
- launch tr4
This program recognizes orange color of ball and turn the image pic4.jpg to pic5.jpg
- Copy Hough_Circles.class and Hough_Circles.java into C:\Work\ImageJ\plugins\HoughTransform (or similar path)
- start ImageJ
- open pic5.jpg
- turn it to binary: Process / Binary / Make Binary
- Process / Binary / Dilate
- Process / Binary / Skeletonize
- Plugins / Hough Cricles / ask for 1 circle from 30 to 100 and the program should caclulate center of the circle
What will happen if we do not decrease number of white points by skeletonization ?
