Big Deac

Big Deac is up, running, and fully operational. System specifications are as follows:

• Dual Intel Xeon E5-2620 CPUs (6 cores each at 2.0 GHz, 24 total virtual cores)
• 64 GB RDIMM ECC DDR3 RAM
• AMD Radeon HD 7970 GPU (32 OpenCL cores, 3 GB DDR3 RAM)
• SuperMicro X9-DAI motherboard (4 PCI-E x16 slots, 16 DIMM slots, 10 SATA ports)
• 2X 2TB Western Digital Black HDDs (RAID 1, for long-term storage)
• 128 GB Samsung 840 Pro Series SSD (Soon to be 2 in RAID 0, for working with data)

As the specs above might betray, Big Deac is a beast and has laughed at most things we have thrown at it so far. While using Photoscan we have now found that we are software limited and no longer hardware limited. We are capable of processing image sets of up to 1000 uncompressed TIFFs on the highest quality settings,
but the software crashes during the 3D model generation. I suspect a memory overflow. Displaying these large 3D models is also difficult, and the software does not do a good job of resampling the model. When displaying the model, the attempt by the software to display all 25+ million vertices brings the program to a crawl or totally freezes it.

We are attempting to work around these issues by processing images in chunks. Photoscan allows you to have multiple chunk within a project. Each chunk has its own images, does its own model generation, and exports its own products. The chunks can be combined at the end of the process. The hope is that by processing the images in chunks we can process the data sets at full quality in separate parts and then merge them later.

Another problem we have found with the software is that while the core functionality (model generation) is very well parallelized using traditional CPU parallelization at some points and OpenCL at other points, accessory functionality like orthomosaic generation (which is quite important to us) is not parallelized at all. This means that the software carries out all of the orthomosaic generation using only one out of 24 available CPU cores. This means that during orthomosaic generation the processing power available to the Photoscan is roughly the same as a middle-of-the-road laptop from 2003 (so not great). Our hope is that Agisoft will resolve this issue with time, as the way of the world now is multicore; single-core high clock speed processors simply aren't available.