R-CNN based PolygonalWedge Detection Learned from Annotated 3D Renderings and Mapped Photographs of Open Data Cuneiform Tablets

Simple item page
dc.contributor.authorStötzner, Ernsten_US
dc.contributor.authorHomburg, Timoen_US
dc.contributor.authorBullenkamp, Jan Philippen_US
dc.contributor.authorMara, Huberten_US
dc.contributor.editorBucciero, Albertoen_US
dc.contributor.editorFanini, Brunoen_US
dc.contributor.editorGraf, Holgeren_US
dc.contributor.editorPescarin, Sofiaen_US
dc.contributor.editorRizvic, Selmaen_US
dc.date.accessioned2023-09-02T07:44:24Z
dc.date.available2023-09-02T07:44:24Z
dc.date.issued2023
dc.description.abstractMotivated by the demands of Digital Assyriology and the challenges of detecting cuneiform signs, we propose a new approach using R-CNN architecture to classify and localize wedges. We utilize the 3D models of 1977 cuneiform tablets from the Frau Professor Hilprecht Collection available as pen data. About 500 of these tablets have a transcription available in the Cuneiform Digital Library Initiative (CDLI) database. We annotated 21.000 cuneiform signs as well as 4.700 wedges resulting in the new open data Mainz Cuneiform Benchmark Dataset (MaiCuBeDa), including metadata, cropped signs, and partially wedges. The latter is also a good basis for manual paleography. Our inputs are MSII renderings computed using the GigaMesh Software Framework and photographs having the annotations automatically transferred from the renderings. Our approach consists of a pipeline with two components: a sign detector and a wedge detector. The sign detector uses a RepPoints model with a ResNet18 backbone to locate individual cuneiform characters in the tablet segment image. The signs are then cropped based on the sign locations and fed into the wedge detector. The wedge detector is based on the idea of Point RCNN approach. It uses a Feature Pyramid Network (FPN) and RoI Align to predict the positions and classes of the wedges. The method is evaluated using different hyperparameters, and post-processing techniques such as Non-Maximum Suppression (NMS) are applied for refinement. The proposed method shows promising results in cuneiform wedge detection. Our detector was evaluated using the Gottstein system and with the PaleoCodage encoding. Our results show that the sign detector performs better when trained on 3D renderings than photographs. We showed that detectors trained on photographs are usually less accurate. The accuracy on photographs improves when trained, including 3D renderings. Overall, our pipeline achieves decent results, with some limitations due to the relatively small amount of data. However, even small amounts of high-quality renderings of 3D datasets with expert annotations dramatically improved sign detection.en_US
dc.description.sectionheadersAI methods for Manuscripts and Documents
dc.description.seriesinformationEurographics Workshop on Graphics and Cultural Heritage
dc.identifier.doi10.2312/gch.20231157
dc.identifier.isbn978-3-03868-217-2
dc.identifier.issn2312-6124
dc.identifier.pages47-56
dc.identifier.pages10 pages
dc.identifier.urihttps://doi.org/10.2312/gch.20231157
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/gch20231157
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Object detection; Machine learning; Applied computing → Archaeology
dc.subjectComputing methodologies → Object detection
dc.subjectMachine learning
dc.subjectApplied computing → Archaeology
dc.titleR-CNN based PolygonalWedge Detection Learned from Annotated 3D Renderings and Mapped Photographs of Open Data Cuneiform Tabletsen_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
047-056.pdf
Size:
3.38 MB
Format:
Adobe Portable Document Format
Download
Collections
GCH 2023 - Eurographics Workshop on Graphics and Cultural Heritage