International Journal of Computer Science Research and Application 2013, Vol. 03, Issue 01(Special Issue), pp. 16-21 ISSN 2012-9564 (Print) ISSN 2012-9572 (Online)

INTERNATIONAL JOURNAL OF COMPUTER SCIENCE RESEARCH AND APPLICATION

© Author Names. Authors retain all rights. IJCSRA has been granted the right to publish and share, Creative Commons 3.0

www.ijcsra.org

Methodology for 3D reconstruction of objects for teaching virtual restoration Silviu Butnariu1, Florin Gîrbacia2, Alex Orman3 1

Transilvania University of Brasov, [email protected] Transilvania University of Brasov, [email protected] 3 Transilvania University of Brasov, [email protected] Author Correspondence: 29, Eroilor Blvd., RO-500036, ROMANIA, +40.268.418967, [email protected] 2

Abstract In this paper, we propose a methodology for 3D virtual reconstruction of objects that can be applied to virtual restoration. The methodology is based on an image-based modelling technique and allows generating a textured 3D mesh from a set of images. The proposed methodology consists in the following actions: obtain images of the object, processing of the images, 3D reconstruction of the object, finishing and completing details, restoration of the 3D virtual model. First, we review several frameworks and toolkits that can be used for image-based modelling and then a detail example of 3D reconstruction is presented. The advantage of this methodology for 3D virtual reconstruction is the use of inexpensive equipment, because only common video and computing devices are needed.

Keywords: 3D virtual reconstruction, image-based modelling, virtual restoration.

1. Introduction Reconstruction of a three dimensional models needed for various applications using various techniques and technologies is a complex process. The reconstructed 3D models can be used in virtual environments for visualization applications, classification and analysis. The most common method to obtain a 3D model of an existing object is 3D scanning. Three dimensional scanning can be used to gather spatial location of points rapidly and obtain 3D coordinates of the target surface. This is a new technical method for the rapid creation of 3D image model of the object. The problem addressed 3D models of various sizes and complexity: from existing small objects to buildings. Recently, virtual reconstruction based on Virtual Reality (VR) technologies has been proposed as an improved interface for a lot of specialty areas, such as: ancient architecture reconstruction (Gaianai et al, 2001; Mehta, 2001; Lu, 2008; Popovici et al, 2008; Yao et al, 2008; Issini et al, 2009), monumental paintings of the church (Petrova et al, 2011), completion of facial image in ancient murals (Lanitis and Stylianou, 2009; Wang et al, 2011), restoring content from distorted documents (Brown et al, 2007), dentistry (Martorelli and Ausiello, 2012), tracking of human movement (Quah et al, 2005), large-scale scenes such as urban structures (Yao et al, 2010). Three dimensional scanning can be performed with equipment ranging from the most expensive to some very simple. The operation can be done with equipment based on direct contact (on mechanical principles: MicroScribe, Romer) or noncontact (ultrasound, x-ray, laser). This equipment is very expensive. In case of less demanding applications, the use of these technologies may be prohibitive. Therefore, we aimed to identify techniques and technologies that use common of the shelf equipment that is available to all users, and able to return high quality virtual objects. Based on this identified technologies we propose a methodology that can be used for teaching virtual restoration at Transilvania University from Brasov.

17

International Journal of Computer Science Research and Application, 3(1): 16-21

2. Methodology for 3D reconstruction using photogrammetry In principle, the proposed methodology for image-based 3D virtual reconstruction of an object could be formulated as to cover the following steps (Fig. 1): Acquire a set of images. The first step essential in geometric reconstruction using principles of photogrammetry is to acquire a set of images of the object that is intended to be restored. The way in which photographs are acquired is a very important aspect greatly reflected in the quality of the final reconstruction. The object is photographed from different angles, spatially arranged in a circle or arc around the object and each photo must contain about 70-80% of previously captured image content. Generate a point cloud data from the images. The images from the data set will be used to identify the interest points and obtain the 3D coordinates of the object. The process is carried out using a specific framework. Because is a computer intensive process, using a standalone software framework require a powerful computing system. A better solution is the usage of a cloud of web service solution. A review of the actual image-based solutions will be presented in the next section. Generate a 3D mesh model. On this step the resulted point cloud will be used to generate a textured 3D mesh. For these step a dedicated library can be used (for example the free software Meshlab). A detail example of using Meshlab to generate a 3D textured mesh of the object will be presented in the section 3.

Figure 1: The proposed methodology for virtual reconstruction

3. Review of the image-based 3D reconstruction frameworks In recent years, due to the development of cloud computing, several frameworks for image based reconstruction have been developed. We review the most actual frameworks and compared their features and capabilities. In the following paragraphs we provide a brief description of each framework and we propose a framework for the implementation of the proposed methodology.

3.1 ARC3D ARC3D (Vergauwen, 2006) is a free web service that provides a standalone software application for uploading photos to the server system. The application delivers good results when a number of about 70 photographs are used. After processing the images, the results are received by a notification e-mail containing a links to the files generated. It contains two types of reconstruction data: pixel maps in *. v3d format that can be used for the manual reconstruction of 3D mesh from a generated point cloud using MeshLab software of a 3D textured mesh in *.obj format processed by the ARC3D server application (but not accurate).

3.2 Hypr3D Hypr3D (www.hypr3d.com) is a free web service similar to ARC3D that provides only a web interface to upload and download the files. Optimal results are obtained when 70-80 images are used. The advantage of this solution, compared with ARC3D, is the reduced time of delivering the reconstruction results (about 3-4 hours). The web service delivers two types of geometry: low resolution mesh in *.dae 3D format with *.jpg textures and a point cloud in *.ply format. Also a high resolution mesh in *stl format can be obtained, but without textures applied.

18

International Journal of Computer Science Research and Application, 3(1): 16-21

3.3 My 3D Scanner My 3D (www.my3dscanner.com/) Scanner is also a free web service comparable to Hypr3D. This service can process up to 100 photos (but in a long time up to 25). The advantage of this application is the high resolution mesh delivered (up to 1000000 triangles). The drawback is the missing of textures applied to the mesh, because it contains only color information. The result delivered by the application contains a *.obj file for the 3D mesh and a *.ply for the point cloud.

3.4 Autodesk 123D CATCH 123d CATCH is a free service offered by Autodesk that uses cloud computing for the reconstruction of 3d objects from images. The management of data is made trough a standalone software application installed on the user PC. This application allows uploading images, downloading the results, rendering the processed 3D point cloud or mesh, export the result using several 3D formats (*.dwg,*.fbx,*.rzi,*.obj,*.ipm, *.las ). This service can process automatic up to 70 photos in a short time (about 1-2 hours). The result of the reconstruction process contains the point cloud, 3D mesh and texture. After reviewing the available image based reconstruction software, the usage of ARC3D and Meshlab framework was selected for the methodology that was used for teaching virtual restoration, because this framework allows manual processing of point cloud which conduct to better results and understanding of 3D processing algorithms.

4. Case study In this section is presented the methodology described in the previous paragraph used to reconstruct in a virtual environment the exterior surfaces of a monument based on a set of pictures and the software ARC3D and Meshlab. The reconstructed object is a monument from the old center of Brasov city. The ARC3D software was used to obtain a cloud of 3D points from a set of 2D pictures and Meshlab software was used for the filtering commands and the reconstruction of the 3D surfaces from the point clouds.

Price User interface Processing time Cloud processing Maximum number of images Point cloud output 3D mesh output Texture output Maximum number of mesh triangles Apply automatic texture Native 3D files format

Table 1: Comparison between photogrammetry services My 3D ARC3D Hypr3D Scanner Free Free Free Web-based + standalone Web-based Web-based software 4 – 5 hours 3 – 4 hours 10 – 24 hours Yes Yes Yes

Autodesk 123D CATCH Free Software standalone 1 – 2 hours Yes

70

200

100

60 – 70

Yes Yes Yes

Yes Yes Yes

Yes Yes Yes

Yes Yes Yes

100000

200000

1900000

150000

No *.obj/*.jpg or -/*.v3d/*.jpg

Yes *.ply *.stl,*.dae *.jpg

– *.ply *.obj –

Yes *.3dp

The first step was to take 16 pictures of the considered object from different angles (smaller than 30 degrees) in order for the ARC3D software to be able to apply the reconstruction algorithms. There were taken other sets of pictures of different monuments, but because the angle between two pictures was too large, the ARC3D software sent an error message. In order to load the pictures in the ARC3D server an account has to be created.

19

International Journal of Computer Science Research and Application, 3(1): 16-21

The next step was to open in Meshlab the model.v3d file. Then the most significant pictures were selected and the Export PLY button was pressed. After a few minutes, the files with the *.ply extension were generated accordingly to the selected pictures. Every file was imported in a different Layer. All the Layers were united to create a single point cloud using command Filters / Layer and Attribute Management / Flatten Visible Layers.

Figure 2: Upload the images on the server using ARC3D software The points that were not part of the reconstructed monument were deleted by using the following commands: Select Vertexes for selecting the points and Delete current set of selected vertexes for deleting them.

a) b) Figure 3: Clean-up the point cloud (a) Sub sampled point cloud (b) After obtaining the point cloud that contains only reconstructed monument data, the Mesh Element Subsampling command was applied in order to increase the number of points from about 8000 to 60000 using command Filters / Sampling / Mesh Element Subsampling / Number of Samples 60000 / Apply.

Figure 4: Poisson reconstruction 3D mesh In this way a new point cloud was created with the name Sampled Mesh. From the Layer area the Sampled Mesh point cloud was selected and the Surface Reconstruction: Poisson command was applied in order to obtain a 3D mesh from the point cloud.

20

International Journal of Computer Science Research and Application, 3(1): 16-21

The triangles generated by the previous command that didn't belong to the model were selected using the command Filters / Selection / Select faces with edges longer than and then they were deleted by applying the command Delete current set of selected faces. The final step was to apply the command Vertex Attribute Transfer and the 3D model of the monument was obtained.

Figure 5: The final result of the proposed methodology for 3D reconstruction

5. Conclusions Reconstruction of 3D virtual objects of various sizes is an important step for development of virtual restoration applications. In this paper actual techniques and technologies used for 3D reconstruction of objects were presented. For teaching of virtual restoration we identified a 3D reconstruction technique based on photogrammetry. Several software applications (ARC3D, Hypr3D, My 3D Scanner, Autodesk 123d Catch) were studied and a comparative study which highlight strengths and weaknesses of each software was conducted. Based on this review a methodology using this virtual reconstruction software was subsequently detailed. For the validation of the proposed methodology a particular application - reconstruction of statuary located on a building facade was presented. The advantage of this methodology for 3D virtual reconstruction is the use of inexpensive equipment, because only common video and computing devices are needed.

6. Acknowledgements The first author’s work was supported by research project PN-II-PT-PCCA-2011-3 Robotic assisted brachytherapy, an innovative approach of inoperable cancers – CHANCE and second author’s work was supported by the Sectoral Operational Programme Human Resources Development, financed from the European Social Fund and by the Romanian Government (POSDRU 89/1.5/S/59323).

References Baker, S. and Matthews, I., 2004, Lucas-Kanade 20 years on: A unifying framework, International Journal of Computer Vision 56, pp. 221. Brown, M.S., Mingxuan, S., Ruigang, Y., Lin Y. and Seales, W.B., 2007, Restoring 2D Content from Distorted Documents, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1904. Butnariu, S., Gîrbacia, F., 2012, High quality restoration of demolished church “Sfânta Vineri” from Bucharest using V.R. technologies. In Proceedings of The IVth European Symposium on Religious Art, Restoration and Conservation, “Al.I. Cuza” University, Iasi, 2012. Cruz-Neira, C., 1995, PhD thesis: Virtual Reality Based on Multiple projection Screens: The CAVE and Its Apps to Comp Science and Engineering, Electronic Visualization Laboratory, EVL, Univ. of Illinois at Chicago. Gaiani, M., Gamberini, E. and Tonelli, G., 2001, VR as work tool for architectural & archaeological restoration: the ancient Appian way 3D web virtual GIS, Proceedings of Seventh International Conference on Virtual Systems and Multimedia, pp. 86.

21

International Journal of Computer Science Research and Application, 3(1): 16-21

Gîrbacia, F., Butnariu, S., Orman, A.P., Postelnicu, C.C., 2012, Virtual restoration of deteriorated religious heritage objects using augmented reality technologies. In Proceedings of The IVth European Symposium on Religious Art, Restoration and Conservation, “Al.I. Cuza” University, Iasi. Grussenmeyer, P., Hanke, K ., Streilein, A., 2002, Architectural photogrammetry. In M. Kasser and Y. Egels, Eds, Digital Photogrammetry, Taylor & Francis, pp. 300-339. Issini, G., Polverini, D. and Pugnaloni, F., 2009, Virtual Reconstruction and Real-Time Interactive Visualization of the Monumental Area between Thien Mu Pagoda and Van Thanh Temple in Hue City (UNESCO Site), Vietnam, 13th International Conference on Information Visualisation, 561. Orman, A.P., 2012, Reconstrucţia 3D şi vizualizarea colocalizată a unor statui de la Biserica Neagră (3D Reconstruction and co-localized visualization of Black Church’s gargoyles) MSc thesis: Transilvania University of Brasov, Department of Automatics, Electronics and Computers. Lanitis, A. and Stylianou,G., 2009, e-Restoration of Faces Appearing In Cultural Heritage Artefacts, 15th International Conference on Virtual Systems and Multimedia VSMM '09, 15. Lu, S.Z. (2008, Virtual Reconstruction of FouGuang Temple Based on Virtual Reality, International Conference on Management of e-Commerce and e-Government ICMECG '08, 208. Martorelli, M and Ausiello, P., 2012, A novel approach for a complete 3D tooth reconstruction using only 3D crown data, International Journal on Interactive Design and Manufacturing, ISSN 1955-2513, June 2012. Mehta, M., 2001, Virtual reality applications in the field of architectural reconstructions, Proceedings of Seventh International Conference on Virtual Systems and Multimedia, 2001, 183. Petrova, Y.A., Tsimbal, I.V. , Laska, T.V. and Golubkov, S.V., 2011, Practice of Using Virtual Reconstruction in the Restoration of Monumental Painting of the Church of the Transfiguration of Our Saviour on Nereditsa Hill, 15th International Conference on Information Visualisation (IV’2011), 389. Popovici, D.M., Talaba, D., Canciu, E., Voinea, V., Bogdan, C.M. and Popovici,N., 2008, Using virtual reality in 3d multi-modal reconstruction of historical sites, Workshop on Virtual Reality in Product Engineering and Robotics: technology and applications; Special issue of: Bulletin of the Transilvania University of Braşov, ISSN 1221-5872. Quah, C.K., Gagalowicz, A., Roussel, R., and Seah, H.S., 2005, 3D Modeling of Humans with Skeletons from Uncalibrated Wide Baseline Views, Computer Analysis of Images and Patterns Lecture Notes in Computer Science, 3691, 379-389. Vergauwen, M. and Van Gool, L., 2006, Web-Based 3D Reconstruction Service, Machine Vision Applications 17, 411. Styliadis, A.D., 2008, Historical photography-based computer-aided architectural design: Demolished buildings information modeling with reverse engineering functionality, Automation in Construction 18, 51–69. Styliadis, A.D., Sechidis, L.A., 2011, Photography-based facade recovery & 3-d modeling: A CAD apps in Cultural Heritage, J. of Cultural Heritage 12, pp. 243–252. Talaba D., Horvath, I., Lee, K.H., 2010, Special issue of Computer-Aided Design on virtual and augmented reality technologies in product design, Computer-Aided Design 42, 361-363. Vergauwen, M. and Van Gool, L., 2006, Web-Based 3D Reconstruction Service, Machine Vision Applications, 17, pp. 411426. Wang, Q., Lu, D. and Zhang,H., 2011, Virtual Completion of Facial Image in Ancient Murals, Workshop on Digital Media and Digital Content Management (DMDCM 2011), 203. Yao, J., Zhang, H. and She, F., 2008, Research on Method of 3D Reconstruction of Ancient Architecture (Nanputuo Temple), International Conference on Cyberworlds, 627. Yao, J., Ruggeri, M.R., Taddei, P., Sequeira, V., 2010, Automatic Scan Registration Using 3D Linear and Planar Features, 3D Research, Vol. 1, No. 3, pp. 2-18. Yilmaz, H.M., Yakar, M., Yildiz, F., 2008, Documentation of historical caravansaries by digital close range photogrammetry, Automation in Construction 17, pp. 489–498. Autodesk Image Modeler ver. 2009: http://usa.autodesk.com/ ARC3D Automatic Reconstruction Cloud ver. 2.2: http://www.arc3d.be/ Mesh Lab ver. 1.3.2: http://meshlab.sourceforge.net/ MicroScribe: http://www.revware.net/products/microscribe/ Romer, portable measuring arm: http://www.romer.eu/press_86.htm?id=2056

A Brief Author Biography Silviu BUTNARIU, PhD – lecturer at Transilvania University of Brasov, Faculty of Mechanical Engineering. Fields of interest: Mechanical, Computer Aided Design, Finite Element Analysis, Mobile Robots, Virtual Reality, Virtual Manufacturing Systems, Tracking Systems, Scanning and Reconstruction 3D. Florin GIRBACIA, PhD – postdoctoral researcher at Transilvania University of Brasov. Fields of interest: Virtual Reality Technologies, Augmented Reality Applications, Human Computer Interfaces, Virtual Reality Programming, 3D Modelling of Virtual Environments, Computer Aided Design, Virtual Reality 3D Immersive systems, Haptic systems, Virtual Reality Alex ORMAN, MsC – graduated at Transilvania University of Brasov. Fields of interest: Virtual Reality Technologies, 3D reconstruction.

Copyright for articles published in this journal is retained by the authors, with first publication rights granted to the journal. By the appearance in this open access journal, articles are free to use with the required attribution. Users must contact the corresponding authors for any potential use of the article or its content that affects the authors’ copyright.