Related YoLinux Tutorials:
°Web Display of Panoramic Photos
°Online Photo Album
°Linux Video: OpenShot
°YoLinux Tutorials Index
|Digital Images and Formats:
Color is the human perception of various frequencies/wavelengths and intensity
of light (electromagnetic radiation) based on an object's physical properties
such as light absorption, reflection, or emission spectra.
Colors are defined and quantified by the degree to which they stimulate human cone cells in the retina.
For humans, this is known as "visible light".
Digital display of color:
Digital monitors: The representation of Red, Green and Blue (RGB) combined (additive coloring) to radiate light in various quantities to achieve the appropriate color.
Each color can be represented by individual numeric quantities.
The combination of all colors renders white.
Digital printers: Cyan, Magenta and Yellow (CMY) combined (subtractive coloring) to absorb light in various quantities to achieve the appropriate color.
The combination of all colors renders black.
Primary colors are red, blue and yellow which can be combined to any other color (used in the arts and painting world).
Digital representation of color:
Hex representation of the digital representation of colors:
The more bits which are assigned to represent a color, the more shades that can be represented.
Twenty four bit color uses eight bits to represent each of red, green and blue.
Eight bits can represent a number from 0 to 255 (0 to FF in hex).
|White (All colors)||FF||FF||FF|| |
|Black (No colors)||00||00||00|| |
See color picker
|TIFF||Tagged Image File Format: supports a wide range of options. Most applications only support a subset of TIFF called baseline TIFF. Lossless LZW compression. TIFF also supports Zip lossless and JPEG Lossy compression. Expired Adobe ownership. Used by scanners and professional photographers. Not a good web format.|
|GIF||Graphics Interchange Format: 8 bits/pixel, colormap of 256 colors from a 24 bit RGB colorspace. Lossless compression (LZW). Good for simple images. Patents were owned by Unisys and Compuserve. Patents have expired. Supports interlacing, animations and transparency. GIF uses only 256 colors. Best for non photographic images with fewer colors, like icons.|
Little endian byte order
|JPEG||Joint Photographic Experts Group: preferred for photographs as it supports more colors (16 million colors, 24 bit) simultaneously. Compressed (10:1) with quality of 1 to 100 (full quality) using Huffman encoding of 8x8 pixel blocks. Starts to have obvious pixelation after about 70 due to reduced color availability. Lossy compression (acceptable for web). Various formats available eg 12 bit version for greyscale medical imaging, EXIF and JFIF. Uses YCC (not RGB unless at quality 100%) to store brightness (luma) and color (chroma B & chroma R). Patents overturned due to prior art or have expired (20 yrs). (.jpg .jpeg .jpe .jif .jfif .jfi)|
Big endian byte order
|PNG||Portable Network Graphics: 8 bit (256 colors) and 24 bit (16 million colors) color palettes. Lossless MNG compression thus tend to be larger files than equivalent JPEG. Supports multiple representations for color palette index (color map). Supports transparency, interlacing, animations. Comparable to GIF without the 256 color palette limitation. JPEG better choice for web images. Supports alpha channels (transitions from opaque to completely transparent). Great for use with transparent backgrounds while not having jagged boundaries.|
|XPM||ASCII Text - good for embedding images in C/C++ and Python source code|
|BMP||Microsoft bitmap image file: Supports multiple compression methods or none and multiple bits per pixel varieties.|
|ICO||Icons (web and Microsoft desktop)|
|RAW||no compression. Full color (no translation to color map). Direct dump from camera. No compression thus large files. Hundreds of formats due to variations in cameras CCD, resolution and sensitivity.|
|SVG||Scalable Vector Graphics: coordinates for line segments and curves. Not an pixel raster image.|
|Lossy Compression ||yes||no ||no ||no ||no |
|Lossless Compression ||no ||yes||yes||yes||yes|
|Optional Uncompressed||yes||no ||no ||no ||yes|
|Grayscale ||yes||yes||yes||yes||no |
|RGB color ||yes||yes||yes||no ||yes|
|8 bit color support ||yes||yes||yes||no ||yes|
|16 bit color support ||no ||yes||yes||no ||yes|
|CMYK/LAB color ||yes||yes||no ||no ||yes|
|Index Colormap ||yes||yes||yes||no ||no |
|Transparency ||no ||yes||yes||no ||yes|
|Animation ||no ||no ||no ||yes||no |
|Layers ||no ||yes||no ||yes||yes|
Also see the YoLinux image mime type example tests.
High Dynamic Range (HDR) Photography:
HDR Photography is an attempt to even out extreme contrast ranges in light exposure.
Extreme contrast ranges can have the effect of clipping dark regions to areas of the photograph which are all black and bright regions to areas of all white.
HDR photography is accomplished by taking several images of the same subject and varying the exposure (fraction of a second), the aperture (f-number) and sensitivity (iso value).
This is called 'bracketing'.
The bracketed images are combined to form a "tone mapped" image by taking only the properly exposed pixels of each image which are best suited for display or print. The tone mapping reduces the dynamic range of the image while retaining the local contrast between pixels.
Exposure fusion is a similar and more simple technique creating the resulting tone mapped image directly from the bracketed exposures without generating the intermediate HDR image.
HDR Image Formats:
Generate tonemap HDR image files (OpenEXR, RGBE, TIFF or PFS) from a set of JPEG, TIFF or Raw images with QtpfsGui or the Gimp plugin Exposure-blend.
- OpenEXR is a high dynamic-range (HDR), high color fidelity image file format (.exr) developed by Industrial Light and Magic for digital imaging.
The OpenEXR format was used in the movies Harry Potter and the Sorcerers Stone, Men in Black II, Gangs of New York, and Signs and is ILM's primary image format.
Also supports transparency.
There are also open source libraries available: IlmImf, lmath and an image display application "exrdisplay".
Source code: https://github.com/openexr/
- RGBE HDR Radiance (.hdr) source images use a four bytes (a full 32 bits) to store pixel information. It avoids clipping by offering a higher range of values by using the fourth byte to store exponent information for the quantity of light per pixel.
The first three bytes are RGB color data.
- 32 bit floating point TIFF supported in current COTS imaging programs.
- Photoshop PSD and PSB large document format supports 32 bits, 300,000 pixels in any dimension. Support limited to Photoshop.
Also see Linux and HDR workflow.
|Linux Monitor Calibration:
To edit photos correctly and to perform color manipulation properly, one must calibrate the monitor so that when the software displays red, red is what is displayed.
This requires a "Colorimeter" which can measure the exact frequency and magnitude of light being displayed by the monitor.
Any variance can be adjusted with the video card gamma table and monitor controls.
Calibration settings include:
- White Point: native white for the display. Typical values: 5000 to match print media (number refers to a temperature in degrees Kelvin).
- White Level: measured in cd/m2
- Black Level: target brightness of black in cd/m2
- Tone curve/gamma: default is 2.2.
- Black output offset:
- Black point correction:
The "Colorimeter" and affiliated software are used to generate an International Color Consortium (ICC) color profile which can be used to adjust your graphics card and monitor to display the correct colors.
The ICC profile of a device (monitor, camera, scanner, printer, ...) describes the color reproduction characteristics for the device.
The device profile is stored in a display.icc file representing the ICC color profile for the device, in this case a monitor.
The ICC profile is a table describing the properties of a colorspace (range of colors). A device with a large colorspace is preferred.
To view your current color profile and settings start the Gnome systems setting tool (/usr/bin/gnome-control-center) and choose color:
Select "View details". If a colorimeter is connected, the "Calibrate..." option will not be grayed out.
The CIE 1931 colorspace gamut tab
TRC: Tone Reproduction Curve tab
The following software can calibrate, generate profiles and set the video card gamma table with corrective parameters.
The software also supports verification of settings.
|dispcalGUI||Part of the Argyllcms suite of tools. Python program which works with a variety of color meters to define adjustments which need to be made to correct color, brightness/darkness, contrast, etc|
Install: apt-get install dispcalgui
Install dependencies: apt-get -f install
(note location of Argyll: /opt/Argyll_V1.5.1/bin)
Requires Argyll: download Argyll_Vx.x.x_linux_x86_xx_bin.tgz
|KDE digiKam||Photo manager and camera image offload program with built-in monitor calibration interface. Select "Color Management".|
Colorimeters, also known as Spectro-Colorimeters or Spectrophotometer, are color measurement instruments which are required to measure the color output by the monitor. The products shown are USB devices which are manually placed against the screen to measure display color from known spectral image samples.
DispcalGUI will work with the following models:
- X-Rite DTP92 and DTP94
- X-Rite/GretagMacbeth i1 Display 1 and Display 2/LT
- X-Rite i1 Display Pro
|Datacolor - Spyder
DispcalGUI will work with the following models:
- Datacolor Spyder 2, 3 and 4
|ColorHUG||Monitor calibration based on Linux and open source. They ship a USB based colorimeter and a Linux LiveCD.|
|Linux Imaging and Graphics Design Software:
Pixel based image processing allows for editing and enhancement of photos and photo image based file formats like JPEG.
Vector graphics images are not stored on a pixel basis but on a geometric basis of points, lines and curves and are more popular with graphics designers. Thus the editing must be performed on the geometry.
Hybrid editors are becoming popular which allow the use of vector graphics and pixel based images.
Linux imaging and editing software falls into two categories:
- # pixel based graphics
- # vector based graphics
|Software applications for image and photo viewing and pixel editing:
Pixel based image viewers and photo editors:
Note that while Xara Xtreme can edit and manipulate pixel images, Xara is primarily a vector graphics tool and as such is listed below.
|eog||Eye of Gnome: Supplied with most distributions of Linux as the default desktop image browser for systems using the Gnome desktop. Mostly used for viewing image files. Editing is limited to scaling, rotation and flipping of images.|
|Shotwell||Gnome camera image import, viewing and photo organization program. Supports simple image editing such as rotation, crop, red eye reduction, exposure adjustment, color saturation and tint as well as web sharing to popular photo sharing web sites. Very time and date centric. I still prefer organization by directory.|
|digiKam||KDE camera image import, viewing and photo organization program. Supports JPEG as well as RAW camera imports and tweaking. Supports geo-coding and processing of negatives. Supports display monitor calibration.|
Organization by directory with storage of meta information makes it a little more flexible than Shotwell's mandated time stamp directory organization. DigiKam has strong photo search capability due to meta-tagging photo information.
|Gwenview||Image viewer for the KDE desktop. Viewing and image interrogation. Has sharing interface for most of the popular image sharing web sites. Editing is limited to image resizing, rotation, flipping, cropping and red eye reduction of images.|
|xv||Legacy viewing and image manipulation tool. Edit image with full view of inner workings of the image file such as Gif colormap, color saturation, etc. One of the first FOSS image viewing and editing software applications available on Linux/Unix.|
xv manual [pdf]
|KolourPaint||Paint program, simple image raster annotation (add lines, arrows, curves, circles, boxes, text). Simple graphic generation. Not a photo editor although it can import photographic images for markup.|
Ubuntu Installation: apt-get install kolourpaint4
|XPaint||Paint program, simple image raster annotation (add lines, arrows, curves, circles, boxes, text). Simple graphic generation. Not a photo editor. Supports PPM, XBM, XPM, XWD, LXP, TIFF, PNG, GIF, JPG, BMP, ICO, PS |
Ubuntu Installation: apt-get install xpaint
|Gimp||Supplied with most distributions of Linux as the default desktop image tool. Fully featured editing and pixel image manipulation tool. Many plugins available. Photoshop "like" image editing supporting layers, color editing, perspective and distortion correction, colorizing, image retouching, transparency, sizing, rotation, format conversion, text support, etc.|
Gimp users manual (2.8)
Notice dust/hair like artifact in upper right hand portion of image
Circle identifies pixels to clone over artifact identified by point of arrow.
|Image processing tool to remove small dust artifacts from picture scans.|
(similar to the Gimp "clone stamp" tool)
Ubuntu deb package generation and installation:
Start application: /usr/bin/tamanoir
- apt-get install libtiff-dev
- apt-get install libhighgui-dev
- apt-get install libqt4-dev
- apt-get install dpkg-dev
- apt-get install libopencv-core-dev
- apt-get install libopencv-video-dev
- apt-get install libopencv-features2d-dev
- apt-get install libopencv-objdetect-dev
- apt-get install libopencv-ml-dev
- apt-get install libopencv-contrib-dev
- apt-get install libopencv-legacy-dev
- svn checkout http://tamanoir.googlecode.com/svn/trunk/ tamanoir
- cd tamanoir
- dpkg -i /tmp/tamanoir_1.svn20130305-1_amd64.deb
Auto identify artifacts or manually move green box to artifact. Clone mode will identify source pixels to clone and arrow will show artifact (pixel clone destination). Left window shows original image while right window will preview the changes.Artifact Categories:
- Positive: artifacts to detect are darker than image
- Negative: artifacts to detect are lighter than background
Tamanoir users manual
File browsing and selection
Image rotation menu
|Commercial donation based freeware.|
Simple editing: rotation, crop, resize, adjust brightness, contrast, saturation, filter blur, sharpen, watermarking, vignetting (fix optical edge effects), ...
Strength: Capable of converting between hundreds of different image formats. Use other tools for editing.
|FixPicture||Online web tool. Upload photos and simple edit online. Format conversions. Very limited editing capability.|
Command line/Batch Imaging:
|Software applications for Scalable Vector Graphics (SVG):
Vector and wire-frame graphics:
Note that Kara Xtreme and Krita have combined support for pixel and vector graphics.
|Open Office/Libre Office Draw||Drawing, diagramming, charting and sketching program, simple image editing. Simple business and presentation graphics generation. Can output image formats (BMP, GIF, JPEG, PNG, TIFF, ...)|
|Xara Xtreme||Desktop photo image manipulation, image editing and vector graphics design software. Claims to be very fast with live interactive image updates (not command then wait, fully interactive image adjustments and changes). Based on wxWidgets GUI framework. Photo image editing (red eye reduction, color balance, tones, shadows, color blending, special effects, fades, frames, feathering, lighten/darkening, transparency support). Although it supports so many image editing features, it is not a pixel editing tool. Supports the generation of graphics, text, complex polygons and Scalable Vector Graphics (SVG). Supports most Adobe Illustrator drawing features. Supports Adobe Photoshop and Illustrator files as well as Photoshop plugins.|
GNU open source with impressive professional graphics capabilities.
Download: RecomXaraLX0.7_rev1692.tar.bz2Software demonstration videos of Xara Xtreme:
Install (eg to /opt/): tar xjf RecomXaraLX0.7_rev1692.tar.bz2
|Inkscape||Generate and edit Scalable Vector Graphics (SVG). Native file format is SVG.|
Install: apt-get install inkscape
Inkscape user's manual
|Karbon14||KDE desktop tool to generate and edit Scalable Vector Graphics (SVG). Support for ODG, SVG, WPG, WMF, EPS/PS vector formats. Drawing and path editing support. |
Ubuntu Installation: apt-get install karbonAlso see Xara Xtreme
|Krita||Derived from Gimp, KOffice/Calligra KImageShop. Painting and image editing. Combines pixel and vector layers to support paint (brushes, patterns, textures and gradients) and illustrator (lines, curves and polygons) features. Supports HDR (Open EXR) and ColorIO for visual effects work.|
File import support: Photoshop PSD, PNG, BMP, GIF, JPEG, JPEG2000, EXR, PNM, EPS. PDF, OpenDocument ODG, OpenRaster, Camera RAW, TIFF, XCF and many more ...
Ubuntu Installation: sudo apt-get install krita
tutorial and examples
|Medical Imaging and DICOM
Digital Imaging and Communications in Medicine (DICOM) is a medical imaging format used for X-ray, CT scans and MRI images.
DICOM is a wrapper file format encapsulating JPEG images (lossy or lossless variants).
Here are a few DICOM file viewers for Linux: (many also can also view volumetric CT and MRI 3D scans)
|AMIDE||Supports DICOM and volumetric medical image viewing. Features include support for multiple colormaps, thresholding, 3D and volume rendering, filtering, fly-through, image dataset alignment,...|
GTK+ based GUI. GPLv2
|Ginkgo CADx||Advanced 3D tools, worklist support, multiplanar reconstruction, fusion, ...|
Commercial application but open source. Free and supported versions (which also has extra capabilities).
Ginkgo CADx Sourceforge page
|MIPAV||MIPAV is a medical image viewer for PET, MRI, CT and microscopy 2D and 3D datasets. Includes analysis tools to support biomedical data research (Talairach space brain mapping, FRAP, FRET), filtering (spatial, noise, thresholding, Isotropic diffusion), ... |
Java based cross platform application. Source and binaries available.
Funded and developed by the National Institute of Health Center for Information Technology (CIT)
|Aeskulap||This medical image viewer can query and fetch DICOM images from network archive PACS nodes. Open source based on gtkmm, glademm and gconfmm.|
Download not currently available??
|MRIcron||Supports multiple image layers, image intensities, greyscales and color schemes with threshold control. Also supports volume rendering (voxels) with various viewing angles and slices. GTK2 based GUI.|
(Follow-on to MRIcro)
|XnView||Commercial application. A general image viewer which also happens to support viewing DICOM images.|
CCD/Camera "raw" images:
Support for raw image formats: CRW, NEF, RAF, CR2, DNG, ...
Digital Camera Storage:
I have always been able to read digital cameras with Linux either using the USB interface or removing the SD card and putting it in a USB attached SD card reader.
No special drivers are required. Using the Gnome Nautilus file browser I have always been able to download photos and video.
Tip: The SD memory cards used in cameras are typically formatted using the Microsoft DOS FAT16 or FAT32 format.
When using an SD memory card in your camera which is greater than 32 Gb, Linux will require additional support drivers to read the "Extended FAT" device:
No reboot required.
- Ubuntu: 14.04
- sudo add-apt-repository universe (typically there by default)
- sudo apt-get update
- sudo apt-get install exfat-fuse exfat-utils
Note the difference between the package names "exfat-fuse" and "fuse-exfat" used for the older 12.04 release.
Using the incorrect package and repository name may result in the following error:
fuse-exfat : Depends: fuse-utils (>= 2.7) but it is not installable
- Ubuntu: 12.04
- sudo add-apt-repository ppa:relan/exfat
- sudo apt-get update && sudo apt-get install exfat-utils fuse-exfat
Tethered Camera: (trigger camera shutter remotely from PC)
Satellite and Multispectral Imagery:
Satellite sensors often gather data in frequency bands invisible to the human eye or in ranges too sensitive for humans to distinguish as different.
Image file formats cater to human vision and are usually not valid formats for the frequency spectrum gathered by the sensor.
This is true for infrared sensors, chlorophyll and organic carbon studies, etc.
For this reason, satellite sensor data is not stored in standard image formats but more typically in HDF format which hosts an internal hierarchical data schema specific to a given sensor.
Sensor data can be mapped to various colors for human display and study.
Tools like HDFView can generically view and plot graphs of the raw numerical data.
Image tools like SeaDAS can generate image displays of the data.
GDAL and GMT are typically used when earth projection and warping of data to the globe and vice versa are required.
- NASA Satellite sensor ocean color data and image downloads - publicly available raw satellite data (HDF5) for various satellite sensors, various parts of the planet and for various dates and times.
- HDF5 - Hierarchical Data Format. Site has libraries and tools for use with HDF5 files.
Supports the ability to organize, store, discover, access, analyze and share data. Developed by NCSA, LBNL and DOE.
HDFView - Java file viewing GUI for HDF4 and HDF 5 files.
Tree view on left shows data schema while right hand side shows the data. Select the graph icon to plot data.
- SeaDAS - SeaWiFS Data Analysis System (SeaDAS) to process, display and analyze satellite ocean color data.
Supports statistics and plotting functions, map re-projection and geo-coding.
(screenshot of sensor data mapped to colors)
- GDAL - Geospatial Data Abstraction Library - GIS tools from the "Open Source Geospatial Foundation".
Raster geospatial data format translation library and command line tools (translate, warp, etc for raster and vector data).
Released with FWTools (compiled binary downloads [alt]) includes GDAL, NetCDF and other library dependencies. FWTools is a set of GIS tools for raster/vector viewing, projection and mapping/cartography. Includes OpenEV geospatial data analysis viewer for raster and vector data.
- GMT Generic Mapping Tools (140 command line tools for manipulating geographic and Cartesian data sets in support of mapping)
School of Ocean and Earth Science and Technology University of Hawaii Manoa
GUI front-end to GMT command line tools: Mirone, iGMT (Tcl/Tk)
Steganography: Hidden Messages and Invisible Watermarks
Steganography is the technology of embedding a message within an image.
It is not visible and typically requires a password and decryption to extract.
Steganography can also be used to mark images with digital watermarks for identification.
Linux Steganography Software:
Image stitching software (Linux and other platforms):
- Hugin - GUI photo stitching application (premier FOSS tool for photo stitching)
(includes PanoTools dependencies and Nona, a PT Stitcher equivalent)
- Pano tools Home - Pano tools on Sourceforge - pano tools Sourceforge project page
by Prof. Dr. Helmut Dersch (Hochschule Furtwangen University of Applied Sciences)
- PT Stitcher: GUI stitching program
- PT Mender: GUI stitching program to replace PT Stitcher
- PT Optimizer: optimizes control points
- Gimp plugin: Stitch Panorama (python plugin)
Image stitching libraries: (rotation-invariant interest point detector and descriptor)
Also see the YoLinux tutorial on displaying panoramic images on the web
|Hugin: Panorama Photo Stitcher
Hugin has matured greatly and is a solid panoramic photo stitching tool.
It supports a fair amount of automation including automatic generation of control points and stitching as well as very esoteric and advanced support for HDR, lens distortion, exposure optimization and horizon straightening.
Install: apt-get install hugin
Following the three step "assistant" is the easiest approach:
[Potential Pitfall]: Note that binary releases of Hugin for Red Hat/Fedora does not compile in automated control point generation due to fear of patent reprisal.
You will get this message if this capability is not built-in:
- Load images: select the images to load.
- Align images: select the "Align..." button and after processing, the preview panel will display.
- Create panorama: Using the defaults will generate a project file and a TIFF image.
|Spherical Panoramic imagery tools:
Spherical Panoramic Photography allows for 360 degree horizontal panoramic coverage as well as 180 degree vertical viewing.
The resulting image is an immersive all-around view from a single location.
Spherical Panoramic Photography is enabled by using a Spherical Panoramic tripod head which rotates the camera about its focal point to avoid parallax and allow seamless photo stitching of images.
Spherical Panoramic Cameras:
Spherical Panoramic cameras are available which take a single pano shot.
This is enabled by the use of multiple CCDs and wide angle lenses or curved mirrors which broaden the field of view to 360 degrees for a single CCD and lens.
Commercial Linux Spherical Panoramic Software:
See the YoLinux list of spherical panoramic viewers for
Spherical Panoramic Photography Display
Spherical Panoramic Image Manipulation Tools:
Sample Spherical Panoramic Photography:
Spherical Panoramic Tutorials:
Spherical Panoramic Video:
- VideoStitch - 360 GoPro video based hardware and stitching software (Linux, MS/Win and Mac)
Uses Hugin to generate a template. Requires NVidia CUDA GPU enabled graphics card.
- Kolor - 360 GoPro video based hardware and stitching software (MS/Win and Mac only)
- Bubl - consumer spherical panoramic video camera package
- Immersive Media - 360 video equipment and software, also demos
- GoPano - bell shaped mirror for video and photo for digital consumer cameras, GoPro and iPhone cameras. 360 degree panoramic, not spherical panoramic
- 360Heros.com - GoPro 360 video mounts for 6 or more cameras. Stereo 3D spherical panoramic mounts also available.
- Elmo QBiC 360 degree rig + Elmo QBiC MS-1 video camera (Full rig: Panoramic-X) - 360 mount for 4 compact QBiC video cameras
[360 Freadom - tutorials]
- JAUNT - 3D stereo spherical panoramic video hardware package (for Oculus or Samsung VR)
Spherical Panoramic Photography is the 360 degree view of a single point, looking outwards and is enabled by stitching complete coverage of the sphere as formed by that point.
Structure from Motion (SfM) generates an imaging experience where the view is on the outside looking in at the object.
This allows for an unlimited number of view points and thus it is impossible to have 100% photographic coverage.
Thus photographs are used to generate points to define a 3D point model from which one can generate a faceted 3D model upon which photographic texture maps can be applied.
This 3D CAD model can then be viewed from any direction.
At first custom viewers were used but now that most browsers support WebGL, it has become the leading web format.
Microsoft is building the Microsoft Photosynth.net software and cloud service to generate SfM models.
The Microsoft ICE program stitches photos and Microsoft Photosynth is used to generate and publish the SfM model to the Microsoft cloud. The client not available for Linux.
The first SfM software was developed by universities and public institutions in both the USA and Europe and made available to the open source community.
SfM "Structure from Motion" tool chain and software:
Multiple images / multiple views
Generate Point Cloud/Vectors
Texture mapped model
Step 1: Generate the point cloud from photographs.
- Import Images
- Photo Point Matching (SIFT)
- 3D Reconstruction and PBA (Sparse Point Cloud) generates the Dense Point Cloud (.mat and .sift files generated)
- Dense Point Cloud generation
- Generate Surface Mesh Model
- Texture Mapping
Generate sparse point cloud from multiple unordered photos.
Images must have EXIF focal length info.
Outputs bundle.out and “.ply” camera information (view with Scanalyze).
Bundler uses SIFT key point and feature recognition libraries and Ceres large nonlinear least squares solver.
- PMVS2 / CMVS: Clustering Views for Multi-view Stereo: PMVS2 generates a dense point cloud. Uses Bundler to obtain camera parameters. Run Bundle2PMVS to generate input to PMVS2 from Bundler “.out” files. CMVS used to substructure photos and model so that it is manageable. Uses Bundler output. PMVS2 included with CMVS download.
- IFP: SURE: image pairs (stereo) are selected and used to generate a dense point cloud
Also see LibTSgm
- DLR: SGM Computer Vision Toolkit
- libmv: unordered images and dense point cloud
- OpenSynther: FOSS version of MS/Photosynth. Uses OpenSurf/ParallelSurf (image matching using FLANN libraries)
- CMP: SfM cloud service (source available) to generate point cloud from images. Various software solutions available:
- Large Scale Sequential Processing (sfmseqv1 and sfmseqv2) eg ordered data if your source is Google Streetview.
- Large Scale Unordered Data Processing (sfm3drv1 and sfmdrv2)
- Classical Unordered Data Processing (using Bundler)
- Multi-View Dense 3D Reconstruction: colored or textured surface models (final step)
Step 2: Generate a triangular faceted mesh model from a point cloud
- MeshLab: mesh generation from point cloud, edit points and 3D triangular mesh models. GUI program.
- PoissonRecon: Screened Poisson Surface Reconstruction. Command line tool to convert ply formated oriented points into a surface mesh.
- CMP-MVS: mesh/model generator (NVM file >> CMP-MVS format) Command line executable. Newer versions are only available as an MS/Windows executable. No other platforms and no source. Requires CUDA GPU support.
Step 3: View
- Verold: Web GL based viewing of hosted 3d models
- SketchFab: hosted publishing of 3d models viewable by web browser
Integrated FOSS SfM Applications and tool chains:
[source on github]
Separate tools includes:
- Bundler: convert images to point cloud
- SFMToolkit Bundler support tools: BundlerMatcher, BundlerFocalExtractor, BundlerViewer
- BundlerFocal Extractor: manage image EXIF info
- BundlerMatcher: extract and match features. Performs Bundler function but uses GPU for compute improvement. (based on SiftGPU)
- BundlerViewer: Bundler and PMVS2 output viewer based on Ogre
- BundlerToPly: generate 3D point cloud (and normals) “.ply” file
- BundlerCleaner: remove pts
- CMVS: takes SfM image input and generates sets of image clusters of a manageable size. Subdivides image models so processing and use is manageable.
- MeshLab: Display point cloud and generate triangle mesh model (.stl file) from the point cloud
- Insight3d: integrated GUI application. (uses SBA)
- VisualSFM: all-in-one integrated tools and GUI (CMP, SURE, CMVS)
- Point Cloud Generation:
- IFP SURE: image pairs (stereo) are selected and used to generate a dense point cloud
- SiftGPU/PBA: key point and feature recognition libraries
- CMP-MVS: mesh/model generator (NVM file >> CMP-MVS format)
- CMVS/PMVS: takes SfM image input and generates sets of image clusters of a manageable size. Subdivides image models so processing and use is manageable.
- osm-bundler: 3D geometry reconstruction for OpenStreetMap
- Arc3d: cloud web service for non-commercial use
Integrated Commercial Products:
Photo Tips for SfM:
- Every part to be reconstructed should be visible on at least 3 images from three distinct but not radically different viewpoints
- There should be a two thirds overlap of images from one image to the next and limited change in camera angle (15-20 deg) from photo to photo.
Acute 3D recommends: "For aerial photography, a longitudinal overlap of 80% and lateral overlap of 50% or more are recommended. To achieve best results, acquire both vertical and oblique photographs ..."
- Disable camera auto-rotate and photo stitching features (if available or enabled)
- Disable any post-processing feature which may adjust saturation, hue, denoising or brightness
- Focused (not blurred) images are best
- Lots of details and textures help in automated image matching
- Fully exposed (no large dark underexposed shadows) images are best.
Photos taken at noon may reduce shadows.
Indoor photos with fixed light is preferable to flash.
Manual exposure settings help reduce the color discrepancies in the texture map.
Auto mode can lead to variations if the exposure is changed from photo to photo.
- Use a fixed focal length for all of the photos. Avoid optical and digital zoom changes between photos.
- Avoid fisheye lenses and lenses with extreme distortions
- Planar scenes are hard to work with as the focal length can not be determined
- Reflective surfaces (mirrors), surfaces with no color variation and transparent objects can be problematic for object (non-terrain) models.
This often leads to holes in the model.
- Subject must be stationary. Movement changes the positions of the definition points from photo to photo
- Keep photos less that 3200 pixels in length or width (VisualSFM/Sift limitation)
- Retain camera EXIF data in JPEG photos (focal length, sensor size, etc)
- Points too close to (or behind) the camera plane are problematic
- Precision: projected pixel size (m/px) x focal length (mm) x photo's largest dimension (px) = sensor width (mm) x distance to the subject (m)
- PBA: Parallel Bundle Adjustment
- MVS: multi-view stereo
- PMVS: an implementation of a patched based MVS
- SGM: Semi-Global Matching
- DSM: digital surface model
- DTM: digital terrain model
Feature detection, recognition, processing, analyzing, and understanding images and video. This field has been expanded to include tracking and event detection.
- OpenCV: image recognition, image tracking, images/video. Integrated with CUDA/OpenCL to use the GPU.
- ROS: Robot Operating System: Includes lots of Computer Vision algorithms for robotic operation and automation.
- SIP: SCILAB Image Processing: filtering, blurring, edge detection, thresholding, histogram manipulation, segmentation, mathematical morphology, color image processing, etc.
OCR: Optical Character Recognition
- Tesseract OCR: convert images to text in over 60 languages. Sponsored by Google.
- Leptonica: image processing library used by Tesseract OCR.
- OCRopus: use of large scale machine learning for addressing problems in document analysis.
- GNU Ocrad: reads images in pbm (bitmap), pgm (greyscale) or ppm (color) formats and produces text in byte (8-bit) or UTF-8 formats
- Panoramic hosting
- GD Library - graphics library
- XMP Manager - GUI to manage XMP metadata. XMP is a schema for EXIF, IPTC, JFIF,.. metadata within JPEG, PSD, TIFF, AVI, WAV, MPEG, MP3, MOV file formats
- GpicSync - auto geocode photos from GPS track log
- GPS Babel - GPS conversion and transfer
Download tracks from a USB connected Garmin: gpsbabel -t -i garmin -f usb: -o gpx -F GpsTrackLog.gpx
"Fundamentals of Digital Imaging"
by H. J. Trussell and M. J. Vrhel
ISBN # 052186853X Cambridge University Press
Introduction to digital imaging covering core techniques of image capture and display of monochrome and color images. Presents fundamental tools within a powerful mathematical framework.
Suitable for advanced undergraduates and graduates in electrical engineering and computer science, and practitioners in industry.
"Gimp 2.8 for Photographers: Image Editing with Open Source"
by Klaus Goelker
ISBN # 1937538265 Cambridge University Press
Guide to the user interface and tools
"Xara Xtreme 5: The Official Guide"
by Gary David Bouton
ISBN # 0071625593 McGraw-Hill Osborne Media; 1 edition (October 15, 2009)
How to use QuickShapes, draw vector paths, use fills for photo-realistic effects, and compose professional desktop publishing documents. Concept to completion with examples that span a range of art styles--using the vector drawing program that thinks it's a bitmap editor.
"Inkscape Beginner's Guide"
by Bethany Hiitola
ISBN # 1849517207 Packt Publishing (May 24, 2012)
Graphics design guide to generate complex shapes, stylized text, use filters and work with images.
"The Book of Inkscape: The Definitive Guide to the Free Graphics Editor"
by Dmitry Kirsanov
ISBN # 1593271816 No Starch Press; 1 edition (October 6, 2009)
From the basics of using the program: drawing, working with objects, transformations and styling, adding text and shapes to how to create business cards, animations, and technical and artistic drawings.
"Complete Guide to High Dynamic Range Digital Photography"
by Ferrell McCollough
ISBN # 1600591965 Pixiq; 1 edition (May 6, 2008)
Thorough, easy-to-follow, and visually spectacular guide.