Difference between revisions of "Visual Systems for Robots"

From RoboWiki
Jump to: navigation, search
(→‎Part 2: Training on 2D processing)
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
__NOTOC__
 
by ''RNDr. Andrej Lúčny, PhD.'' (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010
 
by ''RNDr. Andrej Lúčny, PhD.'' (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010
 +
 +
* You can use Matlab [[Image:IconMatlab.png]] or Octave [[Image:IconOctave.png]]
 +
* download accompanying workfiles [[Media:visual.zip]]
 +
* unzip (recommended to <FONT Color="green"><TT>C:\Work</TT></FONT>)
  
 
== Part 1: Training on input image mainpulation ==
 
== Part 1: Training on input image mainpulation ==
  
−
* download [[Media:pic1.jpg]] and [[Media:tr1.m]]
+
* ''start'' ImageJ
 +
* ''open'' pic1.jpg
 +
* Image / Color / <S>RGB Split</S> Split Channels
 +
* ''apply'' Image / Lookup Tables / Red ''to'' pic1.jpg (red)
 +
* ''apply'' Image / Lookup Tables / Green ''to'' pic1.jpg (green)
 +
* ''apply'' Image / Lookup Tables / Blue ''to'' pic1.jpg (blue)
 +
* ''close everything''
 +
* ''open'' pic1.jpg
 +
* ''apply'' Image / Type / 8bit ''to get gray image''
 +
* ''apply'' Process / Binary / Make Binary ''to get binary image''
 +
 
 +
* start Octave or Matlab
 +
* change directory to directory with this training using the <FONT Color="green"><TT>cd 'C:\Work\part1'</TT></FONT>
 +
* launch <FONT Color="green"><TT>tr1</TT></FONT>
 +
 
 +
== Part 2: Training on 2D processing ==
 +
 
 +
* Install additional Plug-Ins for ImageJ. [[Media:ij-plugins_toolkit.jar|Download Plugin]], copy it to the <FONT Color="green"><TT>cd 'C:\Work\ImageJ\plugins\ij-plugins_toolkit.jar'</TT></FONT>
 +
 
 +
 
 +
* ''start'' ImageJ
 +
* ''open'' pic1.jpg
 +
* ''turn it grayscale:'' Image / Type / 8bit
 +
* ''descrease noise by threshold:'' Image / Adjust / Threshold
 +
** ''select'' Black & White ''from menu''
 +
** ''move with low and high end of range to emphasize the seen objects''
 +
**  Apply ''and close the threshold window''
 +
* ''make the image binary:'' Process / Binary / Make binary
 +
* ''if the picture contains now black object on white background, change it to white object on black background:'' Image / Lookup Tables / Invert LUT
 +
* ''try to eliminate holes in object by'' Process / Binary / Dilate ''and'' Process / Binary / Erode
 +
* ''try'' Process / Binary / Skeletonize
 +
 
 +
 
 +
* ''restart'' ImageJ
 +
* ''open'' pic1.jpg
 +
* ''start segmentation to 2 colors by'' Plugins / Segmentation / k-means clustering ''(select two colors)''
 +
* ''return to the opened'' pic1.jpg ''and three times apply'' Process /Smooth ''then again perform the same segmentation''
 +
* ''compare the two segmented images and select better''
 +
 
 +
 
 +
* ''restart'' ImageJ
 +
* ''open'' pic2.jpg
 +
* ''apply'' Process / Filters / Gaussian Blur / 1.0
 +
* ''apply'' Process / Find edges
 +
* ''apply'' Process / Binary / Make Binary
 +
* ''apply thinning by'' Process / Binary / Skeletonize
 +
 
 +
 
 +
* start Octave or Matlab
 +
* change directory to directory with this training by <FONT Color="green"><TT>cd 'C:\Work\part2'</TT></FONT>
 +
* launch <FONT Color="green"><TT>tr2</TT></FONT>
 +
 
 +
== Part 3: Training on 3D processing ==
 +
 
 +
* ''start'' ImageJ
 +
* ''open'' pic3.jpg, depthX.jpg ''and'' depthY.jpg
 +
 
 +
There is box and figure on the image pic3.jpg -- you can see that these images has the same resolution.<BR>
 +
<TT>depthX</TT> represents X coordinate of surface point visibile at the particular pixel<BR>
 +
<TT>depthY</TT> represents Y coordinate
 +
 
 +
How many times further is the box than the figure ?
 +
 
 +
(Hint: use ImageJ to display coordinates of the proper surface points
 +
and find distance from camera from depth images)
 +
 
 +
(Hint2: The correct answer is half of very known and famous number)
 +
 
 +
 
  
−
* start ImageJ
+
== Part 4: Training on recognition ==
−
* open pic1.jpg
 
−
* Image / Color / RGB Split
 
−
* apply Image / Lookup Tables / Red to pic1.jpg (red)
 
−
* apply Image / Lookup Tables / Green to pic1.jpg (green)
 
−
* apply Image / Lookup Tables / Blue to pic1.jpg (blue)
 
−
* close everything
 
−
* open pic1.jpg
 
−
* apply Image / Type / 8bit to get gray image
 
−
* apply Process / Binary / Make Binary to get binary image
 
  
 
* start Octave or Matlab
 
* start Octave or Matlab
−
* change directory to directory with this training by cd
+
* change directory to directory with this training by <FONT Color="green"><TT>cd 'C:\Work\part4'</TT></FONT>
−
* launch tr1
+
* launch <FONT Color="green"><TT>tr4</TT></FONT>
 +
 
 +
This program recognizes orange color of ball and turn the image pic4.jpg to pic5.jpg
 +
 
 +
 
 +
* Copy <FONT Color="green"><TT>Hough_Circles.class</TT></FONT> and <FONT Color="green"><TT>Hough_Circles.java</TT></FONT> into <FONT Color="green"><TT>C:\Work\ImageJ\plugins\HoughTransform</TT></FONT> (or similar path)
 +
 
 +
 
 +
* ''start'' ImageJ
 +
* ''open'' pic5.jpg
 +
* ''turn it to binary:'' Process / Binary / Make Binary
 +
* Process / Binary / Dilate
 +
* Process / Binary / Skeletonize
 +
* Plugins / Hough Cricles / ''ask for 1 circle from 30 to 100 and the program should caclulate center of the circle''
 +
 
 +
What will happen if we do not decrease number of white points
 +
by skeletonization ?
 +
 
 +
== Software used ==
 +
 
 +
* Matlab or [http://www.octave.org Octave]
 +
* [http://rsbweb.nih.gov/ij/ ImageJ] (we assume it installed in C:\Work\ImageJ)

Latest revision as of 12:45, 6 July 2010

by RNDr. Andrej Lúčny, PhD. (MicroStep-MIS, Ltd., Slovakia) presented at the Robotic Summer School 2010

  • You can use Matlab IconMatlab.png or Octave IconOctave.png
  • download accompanying workfiles Media:visual.zip
  • unzip (recommended to C:\Work)

Part 1: Training on input image mainpulation

  • start ImageJ
  • open pic1.jpg
  • Image / Color / RGB Split Split Channels
  • apply Image / Lookup Tables / Red to pic1.jpg (red)
  • apply Image / Lookup Tables / Green to pic1.jpg (green)
  • apply Image / Lookup Tables / Blue to pic1.jpg (blue)
  • close everything
  • open pic1.jpg
  • apply Image / Type / 8bit to get gray image
  • apply Process / Binary / Make Binary to get binary image
  • start Octave or Matlab
  • change directory to directory with this training using the cd 'C:\Work\part1'
  • launch tr1

Part 2: Training on 2D processing

  • Install additional Plug-Ins for ImageJ. Download Plugin, copy it to the cd 'C:\Work\ImageJ\plugins\ij-plugins_toolkit.jar'


  • start ImageJ
  • open pic1.jpg
  • turn it grayscale: Image / Type / 8bit
  • descrease noise by threshold: Image / Adjust / Threshold
    • select Black & White from menu
    • move with low and high end of range to emphasize the seen objects
    • Apply and close the threshold window
  • make the image binary: Process / Binary / Make binary
  • if the picture contains now black object on white background, change it to white object on black background: Image / Lookup Tables / Invert LUT
  • try to eliminate holes in object by Process / Binary / Dilate and Process / Binary / Erode
  • try Process / Binary / Skeletonize


  • restart ImageJ
  • open pic1.jpg
  • start segmentation to 2 colors by Plugins / Segmentation / k-means clustering (select two colors)
  • return to the opened pic1.jpg and three times apply Process /Smooth then again perform the same segmentation
  • compare the two segmented images and select better


  • restart ImageJ
  • open pic2.jpg
  • apply Process / Filters / Gaussian Blur / 1.0
  • apply Process / Find edges
  • apply Process / Binary / Make Binary
  • apply thinning by Process / Binary / Skeletonize


  • start Octave or Matlab
  • change directory to directory with this training by cd 'C:\Work\part2'
  • launch tr2

Part 3: Training on 3D processing

  • start ImageJ
  • open pic3.jpg, depthX.jpg and depthY.jpg

There is box and figure on the image pic3.jpg -- you can see that these images has the same resolution.
depthX represents X coordinate of surface point visibile at the particular pixel
depthY represents Y coordinate

How many times further is the box than the figure ?

(Hint: use ImageJ to display coordinates of the proper surface points and find distance from camera from depth images)

(Hint2: The correct answer is half of very known and famous number)


Part 4: Training on recognition

  • start Octave or Matlab
  • change directory to directory with this training by cd 'C:\Work\part4'
  • launch tr4

This program recognizes orange color of ball and turn the image pic4.jpg to pic5.jpg


  • Copy Hough_Circles.class and Hough_Circles.java into C:\Work\ImageJ\plugins\HoughTransform (or similar path)


  • start ImageJ
  • open pic5.jpg
  • turn it to binary: Process / Binary / Make Binary
  • Process / Binary / Dilate
  • Process / Binary / Skeletonize
  • Plugins / Hough Cricles / ask for 1 circle from 30 to 100 and the program should caclulate center of the circle

What will happen if we do not decrease number of white points by skeletonization ?

Software used

  • Matlab or Octave
  • ImageJ (we assume it installed in C:\Work\ImageJ)
Retrieved from "https://wiki.robotika.sk/robowiki/index.php?title=Visual_Systems_for_Robots&oldid=5248"