Programme
Here you can read the detailed programme and abstracts or download a simplified version by clicking on the blue button below.
Session Programme
FRIDAY OCT 4 – Theatre 1
DAY 1 THEME: REFORMATTING AND RESTORING
9:00 – 9:30 WELCOME AND INTRODUCTIONS
9:30 – 10:30 KEYNOTE – WHAT THE ARCHIVE CAN DO FOR AI…AND WHAT AI CAN DO FOR THE ARCHIVE by Martha Larson
Artificial Intelligence (AI) has the ability to automate tedious tasks in workflows needed for the creation, maintenance, and disclosure of audiovisual archives. Martha’s talk will argue that we should work to improve the knowledge flow from the archive to the groups who design and develop AI-based solutions. She will discuss examples of projects that have been carried out as collaborations between archivists and AI researchers over the past two decades. A common thread throughout these projects has been that AI research depends on the archive to succeed. Without a thorough understanding of what archivists need to support their work, AI research and development lack direction as well as ways of measuring how well they are solving underlying problems. The way forward is to expand the opportunity to collaborate with archivists to a wider range of researchers working on AI systems. The examples covered in this talk suggest that on its own, without the archive, AI overlooks certain types of data and certain tasks, and also misses out on important opportunities to create algorithms that provide direct support for archivists.
10:30 – 11:00 MORNING BREAK
11:00 – 11:30 A Methodology for Digitizing Wax Cylinders by Nick Bergh
Optical and stylus playback are often considered to be two opposing approaches to groove media playback. However, optical playback can be a powerful companion to typical stylus transfer workflows. While stylus transfer excels in sound quality and throughput, optical can handle media with missing grooves or cracks that are impossible to transfer safely with a stylus. Optical transfer also allows non-invasive auditioning of unlabeled media during cataloguing. Unfortunately, most current optical playback approaches tend to be expensive, slow, or exhibit poor sound quality. When they are used, they are often only used for select items within a collection because of the cost and complication of use. In order to help with this problem, the latest Endpoint cylinder machine has implemented a powerful yet low-cost optical playback system using an on-board laser module. Real-time optical tracking is made possible by using a separate high-resolution video camera traveling with the laser. This image-based groove tracking along with the already sophisticated stylus tracking systems on the machine allows a cylinder to be transferred with stylus only, laser only, or stylus and laser simultaneously. In other words, a complete integration of the typically disparate stylus and optical approaches. Although capturing stylus audio and laser audio together may seem redundant, it can help both quality assurance and quality control since the two systems have little in common. For example, an anomaly caused by debris on the stylus can be confirmed by checking against a simultaneous optical recording. Likewise, any concerns about laser position or optical behavior can be quickly checked against the stylus audio. Performing the optical playback in real-time also provides the unique ability to optimize optical variables such as angle and focus while listening directly to the audio. This presentation will discuss the optical pickup and tracking technology used for real-time playback on the Endpoint cylinder machine as well as how the quality compares to stylus playback on the same machine. Along with technical results, practical audio samples will be presented.
11:30 – 12:00 Have You Ever Tried…? An exploration of myth, queries, and neat things to try in audiovisual preservation by George Blood
Over many years in audiovisual preservation practitioners have proposed questions about practice, potential efficiencies, and generally asked “Have you ever tried…?” Well, we finally have. This paper explores topics ranging from reverse play of audio tapes to the impact on electromagnetic shielding of drilling holes in head shields to make it easier to adjust azimuth; from whether cleaning video tape affects the signal to the repeatability of film scans. How long do diamond styli last (and is it the biggest concern in the life of a disc playback system)? On each topic we’ll define the problem, describe our test procedure, summarize our data, and share our results (and some detours and surprises).
12:00 – 12:30 How to achieve authentic results with modern soundtrack scanners by Oliver Danner
During JTS 2010 in Oslo state-of-the-art soundtrack scanning technologies were discussed with the inventors of Sound Direct, Resonances and COSP. What has happened in the field during the past nine years? What are the strengths and weaknesses of the optical soundtrack scanning technologies available today? I am a professional audio media preserver who has exclusively worked with optical soundtracks since 2010 for the Bundesarchiv and I will answer the two questions above briefly. Mainly I will reason for a new parameter “slit height” that should be incorporated by modern soundtrack scanning systems. The scanning geometries of modern soundtrack scanning systems compare badly with the slit images of a historic soundtrack scanners and this directly affects the reproduction characteristic in terms of nonlinear distortion, signal-to-noise ratio and frequency response. Based on academic research and experience I want to elaborate on my conclusion that only a numeric definition of the height of the scanning slit would enable us to transfer optical soundtracks in a comprehensible and authentic manner and that the “slit height” parameter would clearly bring us towards the realization of a historic reproduction characteristic.
12:30 – 14:00 LUNCH BREAK
14:00 – 14:45 MXF Repair Flow: How to plan a major asset migration by Marjolein Steeman & Tristan Zondag
Recently a new mam-system was installed at Sound and Vision. Before installation we had to create new proxies for all our MXF-files. This gave us the opportunity to do a full Quality Check on all MXF-files of our archive. So now we have more detailed metadata on the actual quality of our MXF-files. When in september 2018 a client complaint came up about not being able to render an MXF because of some quality issue, alarm bells rang. It turned out that new software updates caused a blocking issue on a technical defect that had not even been noticed earlier, rendering the same file. This triggered an analysis on the quality metadata of all the files. As part of our working standards Sound and Vision has implemented the international standard of Open Archival Information System (OAIS). Migrations are planned within the function Preservation Planning. Preservation planning gives our management the opportunity to make well informed choices on preservation. And we document those choices. Each migration plan has a fixed table of contents, covering four sections: the context of the plan, the collection that is at stake, requirements that are to be met and what options we have to do this. And we end the document with an advice. We would like to illustrate this with the MXF-repair flow as a use case. We also would like to give insight on the actual outcome of the analysis of the Quality metadata. We will outline the scenarios on carrying out this substantial migration of 30% of our MXF-archive, in order to repair the issues that we found. We will tell about the used software, technical infrastructure, and our experiences with throughput as opposed to what was expected.
14:45 – 15:30 Using computer vision technology to accurately and objectively determine motion picture film condition by James Lindner
CINEQUAL has developed a collection management tool that provides a complete, objective, and precise view of motion picture film condition using a standardized language, and does it all at 24 frames per second. Using computer vision technology to capture film’s condition, the system measures every aspect of film, determining shrinkage, stretch, perforation integrity, and structural issues frame by frame. This is a new system that is cost-effective, fast, accurate, and far surpasses the capabilities of legacy tools and workflows. The system can objectively and repeatably assess collection holdings, identify issues, and determine the time and resources required to implement a restoration strategy—all with complete confidence. The system allows a totally new way to manage large collections and determine preservation priority objectively.
15:30 – 16:00 AFTERNOON BREAK + Poster Presentations
Preserving Culturally and Historically Important Collections via Community Archiving Workshop (CAW) (Sub: The CAW: Manila AsiaVisions AV Collection Project) by Rosemarie Roque
The J.H. Kwabena Nketia Archives: African Archives Best Practices — A Case Study for In-House Digitization in the Developing World by Nathaniel Kpogo Worlanyo and Audra V. Adomenas
Open Access & Intellectual Property Rights at the National Cultural Audiovisual Archives of India by Irfan Zuberi
16:00 – 16:30 Q&A on Conservation and Restoration Ethics in Digital Times: Refining Ethical Requirements and Ease their Communication by introducing Visual- and Mathematical Models by Jörg Houpert & Lars Gaustad
For all professional archivists, ethical requirements in media conservation and restoration are the foundation and guidelines for any practical handling of media. For this reason, archiving organisations like IASA, AMIA, FIAT/IFTA and FIAF began early to document in their Committees/Commissions the latest concepts and best practices for their members working with archive sound recordings, broadcast videos, or cinematographic films. In order to spread the knowledge about ethical principles worldwide, a translation into different languages was promoted. For example IASA-TC 03 – The Safeguarding of the Audiovisual Heritage: Ethics, Principles and Preservation Strategy – has official translations into ten languages. But even the latest version does not contain a visual illustration to facilitate the understanding and abstraction of the underlying concepts. The purpose of this session is to raise awareness on how important it is that these ethical rules become even more precise and better accessible to technically-minded individuals who prefer mathematical abstraction in order to internalize new concepts. Especially as some of these engineers are developing the preservation tools to process and reformat legacy media carriers. These tools are getting more and more software defined and getting more complex all the time. It is almost impossible for end-users in the archive to verify that ethical constraints are fully respected by a tool under consideration.Transforming textual descriptions into mathematical expressions provide interesting insides.Since mathematics is a high-precision language, this has the advantage that some of the intended and some of the undesirable ambiguities and inaccuracies in the description of ethical requirements become more explicit. This can lead to more technical and ethical discussions about the future construction of media migration and reformatting tools. This presentation, structured in a Q & A format, aims to fathom additional ways to develop guidelines that answer more precise questions about conservation and restoration ethics, especially for analogue content that will only be at our disposition in a digital format in the long-term. This presentation will utilize visual illustrations and some simple mathematical expressions to explain the proposed concepts. This session also pays homage to the great accomplishments of the first authors of IASA-TC-03, especial Dietrich Schüller, who directed Jörg’s professional focus on archival principles more than 20 years ago.
16:30 – 17:00 Saphir : Digitizing broken and cracked or delaminated lacquer 78 rpm records using a desktop optical scanner: Special focus on cracked lacquer discs by Jean-Hugues Chenot
Conventional playback of 78rpm audio disc records is usually the preferred method for digitizing the records. It does require skills but is relatively fast and usually delivers good quality. But as far as broken discs and cracked or delaminated lacquer records are concerned, conventional playback is not an option, because the groove cannot be tracked reliably, and the risk of destroying the record or the equipment becomes unacceptable. We present an optical alternative for recovering the audio signal from such records. The new desktop-sized Saphir system uses simple components and allows scanning optically a 78rpm disc side in less than 30 minutes, still keeping a good signal-to-noise ratio over all the frequency range, up to 20kHz, thanks to reflective principles. The decoding of the scanned pictures takes less than one hour in the easiest cases, and allows an experimented user to reconstruct completely the available signal in the correct order, even when lacquer flakes are distorted or missing. The thousands of available connections between tracks fragments are decided by a best-path solver, with guidance from the user. The recovered quality depends on the surface condition: exudates are a nuisance we are still working on, but reflective shining discs can be reconstructed up to a fair quality level. We will present a number of playbacks for damaged records in diverse conditions. Amongst the presented examples, on some disc sides, lacquer flakes have been re-positioned physically before being scanned under a glass pane. Discoloured and cracked lacquer discs can also be scanned. Our objective is to be able to provide to audio archives and service providers an affordable may of recovering those records that cannot be played using conventional means. The finalization of our new fast desktop-sized scanner is an important step towards this goal. Our presentation will be completed by a hands-on demonstration during the conference in the exhibition room, where we will digitize records for the willing participants.
17:00 – 17:30 Quality Control Experiences and Effectiveness in a Large-Scale Film Digitization Project by Peter Schallauer
In 2017, Indiana University (IU) launched the film phase of the Media Digitization and Preservation Initiative (MDPI), which will result in over twenty-five thousand large and small gauge film reels digitally preserved by the university’s 2020 bi-centennial. While partnering with a service provider has resulted in a very high throughput digitization workflow to meet its ambitious timeline IU has additionally created an extensive automatic and manual quality control workflow to guarantee specifications for aural and visual quality and structural standards are met. This presentation will provide an overview of the MDPI film phase, focusing on quality control needs and its implementation with IU’s practical experience in using the film scanning quality control software VidiCert to efficiently and cost effectively check thousands of files. We will discuss the innovative approach to quality control for film which automates error detection for the QC operator to quickly review files and make decisions that guarantee quality preservation and restoration. Examples and long-term statistics of human verified, critical scanning issues, such as over/under exposure, dust/dirt, framing error, freeze frame and unsteadiness will be shared along with experiences made in the first 18 month of operations where more than 3000 hours have been QC’d and consequently partially be re-scanned. The effectiveness of quality control integrated with the film scanning process will be evaluated on a long-term statistical basis.
SATURDAY OCT 5 – Theatre 1
Theme Room 1: BIG COLLECTIONS, BIG ASSETS, BIG DATA
9:00 – 9:30 Historical film colors and digital cinema by Giorgio Trumpy
In the framework of the ERC Advanced Grant FilmColors at the University of Zurich, we investigate new approaches for transferring the aesthetics of historical film color processes into digital form. For the Joint Technical Symposium we propose selected topics sustained by our spectroscopic analyses: 1. Reference images for automatic color correction – A film digitization must aim to recreate the appearance produced in cinema theaters by means of analog projectors. In the effort to achieve this original look, the color grading of the raw scan can benefit from a digital image that contains the target colors (reference image). With this aim in mind, we developed an automatic tool to perform the color correction. 2. Chromatic Callier effect – All of the most popular film scanners available on the market to date create digital files illuminating the film with a light diffuser. This type of illumination, while having the advantage of attenuating dust and scratches, differs completely from the way films are illuminated in a projector. Due to this fact, digitized early film colors (especially toned film) can significantly deviate from the original projected colors. 3. Color separation and non-standard dyes – Several scanners capture film colors with narrow spectral bands that considerably deviate from humans’ chromatic response. They exclude spectral regions where the typical dyes of chromogenic film do not exhibit their absorption peaks, in order to reduce the dyes’ cross-talk and increase “color separation”. This feature has important benefits when scanning color negatives and faded chromogenic prints. However, the selectivity of this spectral arrangement can represent a severe limiting factor for color rendition, especially when scanning early color films with non-standard dyes. 4. ERC Proof of Concept “VeCoScan” – There is no professional high-end film scanner on the market that is able to properly digitize the whole range of film colors. The proof of concept VeCoScan tackles this lack of technical solutions by developing and testing a multi-spectral and versatile scanning unit, whose design will adapt to the specific film type to be scanned. An outlook will be given on the activities for the development of a new generation of film scanners.
9:30 – 10:00 Requirements and new technologies for the inspection of photochemical film by Jörg Houpert
The diagnostics of the current condition of a preserved film stock is a necessary prerequisite before a film can be reformatted (scanned) into a digital master file. The human expert effort to perform these diagnostics is typically higher than the effort for the high resolution film scanning itself. With the huge amount of film material that needs to be inspected within the next decade, there is probably no other commercially viable route than to research and develop new assistive tools to streamline this challenging inspection task. The manual inspection process is often split into three categories: physical condition (mechanical incl. dirt, chemical, biological), photographic state (resolution, density, etc.), content and provenance related inspection These three categories can again be subdivided into aspects relative to the image or the audio track. If you even have to inspect multiple versions of a film print, this becomes a multidimensional type of problem. Film archives do not only have to deal with obsolescence of photo-chemical equipment and material, but also with the dramatically quick decrease of expertise and hard to find specialists. In addition, manually conducted film inspections typically include a significant subjective rating component. The objectivity and the comparability of evaluations can be significantly improved by automation. The automated inspection process can be driven to a very detailed documentation level, which in practice is not achievable by human experts. The presentation will give a status report, in which areas new technology can be expected to support the inspection process. Examples are: an improved selection of materials; an improved preparation of materials for digitization; a prediction of frame positions that can be problematic during a scan process; an improved selection and control of digitizing devices; an improved control for the automated generation of restoration copies The presentation should also foster an understanding on how a partly automatic generated inspection report can be used to ease the prediction on restoration cost and labour effort and whether such a report can help to decide which cleaning/washing procedure and which film scanner will be adequate for the job. But there are also pitfalls, where state-of-the-art inspection technology stays completely blind for very simple defects.
10:00 – 10:30 HDR, 4K UHD : What future for archives? The archives and the new color spaces are they friends or enemies ? by Thierry Delannoy and Benjamin Alimi
New mutation in the digital world : 2K and HD formats give way to 4K UHD with new color spaces: HDR, Rec 2020, P3, Aces… What are the contributions and the dangers of these new tools ? What technical advances for archives in these new formats ? What are the best practices to follow in order to have a faithful reproduction of the archive ? In our presentation, we will show you a panorama of examples: black and white, tinted, toned but also color archives, worked in these new standards.
10:30 – 11:00 MORNING BREAK
11:00 – 11:30 Video Reuse Detector—Reappropriating Trump by Pelle Snickars, Filip Allberg, Johan Oomen
The project “European History Reloaded: Circulation and Appropriation of Digital Audiovisual Heritage” (CADEAH)—funded by the European Union’s Horizon 2020 Research and Innovation programme—will shed light on how online users engage with Europe’s audiovisual heritage online. The project is a follow up on the EUscreen projects, and particularly looks at online circulation and appropriation of audiovisual heritage via the usage of algorithmic tracking and tracing technologies. The project brings together scholars and developers from Utrecht University, the Institute of Contemporary History (Czech Republic) and the digital humanities hub, Humlab at Umeå University—and also includes the Netherlands Institute for Sound and Vision as a cooperation partner. From a technical perspective CADEAH will use (and in a way reverse engineer) forensic video fingerprinting techniques. Within the media content industry video fingerprints are used to track copyrighted video material. But whereas the industry seeks to police and control piracy, CADEAH will use similar digital methods to analyse, research and discuss the cultural dynamics around re-use and remix of audiovisual heritage and archival footage. Building on the open source video fingerprinting technology Videorooter, CADEAH will develop an up-to-date and cutting edge system—the Video Reuse Detector—enabling users to upload a specific video to a system which will match the video against a set of fingerprints known to the same system. Implemented in Python and using the OpenCV machine learning framework, the Video Reuse Detector will compute a video fingerprint as a sequences of images hashes, store those fingerprints and associated metadata (title, source, thumbnails, frames etcetera) in a fingerprint database, and match a fingerprint or image hash against the current set of fingerprints in the database. In a collaboration with the Internet Archive, videos from the current Trump Archive— which collects TV news shows containing debates, speeches, rallies, and other broadcasts related to Donald Trump, before and during his presidency—will serve as a test case for the video reuse detector system currently being developed at Humlab. The system will have a capacity to compute and store fingerprints for several videos (or images) in an offline batch process, a feature that will enable easy load of reference video collections to be used in future matching. The open source system will also have an online (web) capacity to accept and match a video (or image) against the current set of fingerprints stored in the database, a feature that will be available as a web service in the form of a user interface with drag and drop capabilities for potential video and image matching. The Video Reuse Detector will hence address the current shortcomings of the Videorooter technology, foremost regarding the way in which hash sequences are generated but also avoiding the fixation to one specific hashing algorithm.
11:30 – 12:00 ReTV: bringing broadcaster archives to the 21st-century audiences: How ReTV solutions can optimise audiovisual content for online publication and maximise user engagement by Lyndon Nixon
Broadcaster archives today are faced with the challenge to keep up with the needs of 21st century online audiences who want to easily find content that is tailored to their personal preferences. In a competitive online media landscape, broadcasters and heritage professionals lack tools that would enable them to reach their target audiences with materials from archival collections. To remain competitive in the digital realm, broadcaster archives need to adapt their content to online media consumption and find new routes to effectively reach the right audiences. ReTV (http://retv-project.eu/) is a research project (funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 780656) that builds a novel infrastructure to help broadcasters and audiovisual heritage professionals address these challenges. ReTV tools automate the planning and publication of content that is tailored for online consumption and iteratively monitor and enhance this workflow, beginning with a visual analytics dashboard to enable editorial teams to easily identify and predict topics that will be trending with their target audiences. With the help of machine learning technologies for concept extraction and annotation, editors can generate thematic video summaries from archival content based on these topics. These video summaries are then automatically tailored for different online channels, personalised for specific audiences and published at an optimal time to maximise audience engagement. In the presentation, we will demonstrate how these technological solutions can bring the legacy of broadcaster archives to 21st century online environments and digital audiences.
12:00 – 12:30 Structural signatures: Using source-specific format structures to identify the provenance of digital video files by Bertram Lyons & Dan Fisher
Every complex digital file format requires the presence of self-describing and predictable internal binary structures. This internal structure is responsible for framing the stored content within the file so playback software can acquire the correct configuration details to reconstitute this encoded information. This applies to videos (e.g., mp4, mov, avi), audio files (e.g., wav, mp3), still images (e.g., jpg, tif, png), serialized packages (e.g., zip, tar), database files (e.g., sql), and file systems (e.g., FAT32, HFS), among many other content types. Traditional approaches to digital file forensics focus on the content of the file itself. Signal analysis takes the output of the reconstructed payload of the file and processes this output to identify traces that are targets of the particular investigation method. Semantic analysis identifies recognizable text that can be examined for further meaning. Signal and semantic analysis of digital media are both necessary and fruitful approaches. Our research indicates that an additional approach, structural analysis, which targets the internal structure (or, the syntax) of digital file formats, is a viable method to identify provenance, to detect tampering, and to propose approaches to file repair or reconstruction.
12:30 – 14:00 LUNCH BREAK
14:00 – 14:30 DeepRestore – AI techniques for Digital Film Restoration by Franz Hoeller
DeepRestore is a research project driven by HS-ART Digital (Austria) together with the TU-Graz Institute of Computer Graphics and Vision. (Austria), running from 01/2018 until end of 2020. The goal of the project is to evaluate AI technologies in the sector of digital film restoration. A focus is on applying modern machine learning techniques, in particular convolutional neural networks, to remove dust and scratches in archival footage. In the presentation we will show current results of the DeepRestore prototype and we will compare the speed and quality of the restoration with the classic dust & scratch filters from the DIAMANT-Film Restoration Software. We will discuss the advantage and disadvantage of the AI approach versus the classical approach in digital film restoration. The problematic of generating good training data for the particular problem to solve will be shown and possible solutions will be presented.
14:30 – 15:00 Bring New Life to Media Assets with Artificial Intelligence by Li Ang
At present, there are a large number of audio and video media assets stored in digital form all over the world. Traditional media management organizations usually classify and file audio and video data manually, which is not only expensive but also difficult to discover the value of media assets. In the era of artificial intelligence, big media data is an important fuel for deep learning and media management can also benefit from it. Using Deep Learning technology, the label information of media data can be extracted from multiple dimensions, and the media data can be fine managed according to the structured information. With Deep Learning, a neural network can be trained according to the use of media data, so as to predict and guide the production of media. While using media data more conveniently, deep learning can provide data support for a subarea according to the type of media data with some state-of-the-art technologies. For example, sports video, through motion recognition technology to analyze the motion pictures in the video, thus providing data support for assistant training or tactical analysis services. Shanghai Audio-Visual Archives （SAVA） will explore the cross-border application of traditional media assets based on Shanghai Media Group (SMG) Intelligent Media Management Platform.
15:00 – 15:30 Moving Image MetaData Based Finding Aids using Artificial Intelligence: Northeast Historic Film will create an Archival Moving Image Content Database for AI Research by James Lindner
While great progress has been made in the area of facial recognition for still and moving images, issues such as context are still cutting edge. Beyond context there is the issue of film language itself which is entirely new territory. For example is the famous shower scene from “Psycho” about a shower or about a murder? This paper will discuss new frontiers for AI research using archival databases in order to develop automatic metadata extraction that is cognizant of film language for contemporary as well as Archival moving images.
15:30 – 16:00 AFTERNOON BREAK
16:00 – 17:30 PLENARY SESSION AND OPEN DISCUSSIONS; Technical Awards
17:30 – 18:30 CLOSING PARTY
SATURDAY OCT 5 – Theatre 2
Theme Room 2: STATUS AND IMPACT OF TECHNOLOGY & PROGRAMMES AND SOLUTIONS
9:00 – 9:30 If I Knew Then What I Know Now: Evolution of MDPI’s Post-digitization Processing by Brian Wheeler
Indiana University announced the Media Digitization Preservation Initiative (MDPI) in October 2013 with the goal of digitally preserving and providing access to all significant audio, video, and film recordings on all IU campuses by the IU Bicentennial in 2020. Digitization began in mid-2015 and has now digitized more than 320,000 objects using more than 10 petabytes of storage. After digitization, every object in MDPI has to be verified to be stored correctly, checked for format conformance, processed into derivatives, and finally, distributed to a streaming video server. Conceptually, the process is straightforward, but like many things, the devil is in the details. The post-digitization processing has continually evolved since its inception in early 2015. Initially implemented to handle a couple of audio formats and processing a few terabytes of data per day, over the last few years it has been enhanced to handle peak transfers of more than 35 terabytes daily with more than 20 formats across audio, video, and film. This paper details how some of the implementation decisions have held up over time, such as using a tape library as primary storage and using an object state machine for object tracking, as well as some of the growing pains encountered as the system was scaled up. In addition, there is a discussion covering some of the surprises that have been encountered along the way.
9:30 – 10:00 Mass digitization systems and open-source software: A viable combination? by Etienne Marchand
Twenty years ago, usage of highly specialized equipment at every step was mandatory to achieve a successful digitization plan. Today, while playback equipment remains very specific, alternate solutions exist for encoding, processing and checking content. Nowadays’ CPU processing capabilities allow building encoders and processors with a more generic approach. Open-source software products offer an unprecedented flexibility and support a wide array of formats. Self-designing custom solutions has never been more accessible. Are those solutions as reliable and high quality as commercial products made by expert companies? Do they require specific skills to be operated and maintained? What are the limits, if any? We’ll present a feedback of our own experience at INA, and have an overview of some other applications at INA that rely on open-source software, including in-house developments accessible to anyone. Finally, we’ll talk about the benefits for a media management entity to be more and more involved within the open-source world.
10:00 – 10:30 New Phonograph: Enthusiasm and Inspiration as a Driving Force: How to leverage your content to overcome budgetary constraints by Anthony Allen & Martin Mejzr
In May of 2018, the National Museum of the Czech Republic set in motion a project aimed at preserving the diverse collections of mechanical audio carriers strewn across the nation. While funding for the project proved generous – in context of other national cultural grants – the scope of the problem at hand could not easily be accounted for when project planning was underway. As such, it has given way to a project that depends on an agile, resourceful approach as a matter of course. We have come to thrive from a cross-disciplinary team united in enthusiasm for audio. A mix of part-time workers and contractors, whose knowledge is not limited to one aspect of the work, has allowed us to address a multitude of problems that have risen, mainly, from a lack of funding. From building our own holders for cleaning records to taking a ‘develop-it-yourself’ approach to software tools, the team has drawn from each others’ knowledge and interests to navigate budgetary constraints. Developing relationships with experts and suppliers has been instrumental in overcoming our limitations. Those who visit, assess and grow to understand the purpose of their product within the archival environment have been invaluable to our progress. One strong partnership alone has helped us design custom carts and trolleys for transporting our audio objects, as well as sourcing appropriate labels, sleeves, boxes for long term storage and many other essential supplies at a fraction of the cost. We also secured a generous donation of equipment from Radio Free Europe, allowing us to overcome the ever-thinning market for used playback machines. In an attempt to rethink the way methodologies are used, we have taken inspiration from language books, infographics and more, to design the methodologies included in our research outcomes in such a way that they are accessible and offer a user journey which can be followed by those with only a basic understanding of audio documents. Furthermore, we have begun designing a ‘first aid kit’ for audio, a simple handbook which offers an at-a-glance overview of how to handle the most common at-risk audio formats. With a clean slate to start from, we are taking advantage of the breadth of technology, design and other resources now so easily available to ensure the project is a success despite financial limitations. To ensure our learning process is carried on to the next generation of archivists, the project has also begun to drive education across multiple trajectories. This has seen the launch of university courses and has given rise to discussions with collection owners across the nation. The legacy of this research project will be a newfound concern for historical media within the Czech Republic, and a customised centre for audio document preservation in which to do it.
10:30 – 11:00 MORNING BREAK
11:00 – 11:30 Automated Creation of Descriptive Metadata for Large Media Archives: Creating open source tools and workflows with the experts by Karen Cariani & James Pustejovsky
WGBH is confronted with the need to enhance descriptive data to improve online discoverability for large-scale digital indexing and analysis of media collections. There is a strong need to use computational tools to create more scalable and more accurate descriptive metadata for audiovisual archives. In addition, the potential use of these collections as a dataset for quantitative analysis by digital humanists and social scientists is significant if the data and access to the dataset can be improved. This work specifically addresses the following JTS areas of interest 1) Artificial intelligence in media management tools and 2) Open source and other development models for sustainable AV archive tooling. This presentation will demonstrate the collaborative work between archivists at WGBH and Computational Linguists (CL) at Brandeis University Computer Science Department to create open source tools and workflows for archivists to automate metadata creation for large digital media collections. Panelists will share work to date on the creation of a computational dataset from the American Archive of Public Broadcasting collection, including the policies and frameworks developed for providing access to the dataset. Creating transcripts of the audio is an obvious solution to describe the content and expose the text to search engines. Using speech-to-text tools can be adapted or “trained” for use to achieve greater degrees of accuracy, but they do not generate perfect transcripts, and the data is not always clean enough for consistent indexing. Within the context of CL, we can also explore tools and capabilities that allow for OCR of text on screen: being able to capture descriptive data on a slate, credit roll or lower third and embed that data into a transcript would enormously help with access. Other capabilities include video clean-up (removing bars and tone), categorization of different audio elements (music, applause, external sounds), distinguishing language types, categorizing scene types, and creating named entities that are linked to existing authority records from DBPedia, Wikidata, LCNAF and LCSH, and using forced alignment to sync text transcripts with media. These tools, to be most efficient and provide cleanest output, need to be pipelined, allowing each tool to perform a specific task and be trained. However, training such tools and refining their output requires human oversight, quality control, and expertise beyond most archivists. Collaborating with computational tool-making experts – linguists, computer scientists – to increase accuracy of the tools, allow easier use, and enhance outputs, would greatly benefit both communities. The challenge is to make the tools easy and intuitive – tools that anyone could implement at scale with large volumes of files. In addition, the data output needs to conform to archivists’ needs and metadata schemas. Such open source analytical tools have not been extensively used on large digital media collections to help with the laborious task of creating metadata. Historians, archivists, and computational scientists have seldom collaborated to further develop tools to help enhance archives and libraries. The collaborative opportunity fostered by this project can be adapted as a model that will greatly benefit the humanities and social sciences.
11:30 – 12:00 IIIF for AV: Improving access to audio archives by Adam Tovell and Andy Irving
IIIF (International Image Interoperability Format) has emerged in recent years as a powerful framework to enable the viewing, comparing, manipulation and annotation of images online. As a community driven, standardised set of technologies, IIIF is attractive to memory institutions in providing access to digital materials and their supporting metadata (e.g. descriptive and structural) without the limits imposed by bespoke, locally-developed applications and solutions. By describing content in a standardised way, digital collections can be rendered by a single ‘viewer’, reducing technical overheads, creating a unified, ‘content agnostic’ experience for the end-user, providing more opportunities for collaboration and opening up collections between institutions, globally. Access to digital collections is fundamental to research and scholarship, and to the promotion of collections to non-academic audiences. But providing access to digitised sound collections in a way that supports the needs of researchers and the curious public presents several challenges. Archival materials and ‘field recordings’ held on legacy physical formats can be very complex, often composed of several distinct recordings on multiple items, by several sequences of audio files, each of which are frequently non-sequential. To deliver these materials online with the metadata required to navigate the resource and to understand its contents requires rich but easily understandable metadata. Over the past two years and supported by a grant from the Andrew W. Mellon Foundation, the IIIF Presentation API has been further developed by a consortium of international partners including the British Library to support the delivery of audiovisual materials. IIIF v3 aims to support the various needs of audio creators and custodians. Use cases for IIIF for audiovisual materials have been collected from the across the IIIF community, including requirements such as the ability to: • Deliver complex archival resources to a research audience with rich metadata • Deliver complex audio resources in a simple way to the public • Create and display annotations as they apply to an audio or audiovisual recording • Compare multiple resources alongside one another • Play sound recordings of a piece of music or speech while following a musical score or transcript • Highlight or specify a region or point of interest in an audio or audiovisual file This work has also involved the development of the ‘Universal Viewer’ – an open-source viewer for IIIF image sequences – to support audio and to deliver all audio materials created through the British Library’s eight-year Save our Sounds programme. This paper will explore the challenges that delivering audio resources online presents; how IIIF metadata can be used to describe complex audio collections; how harnessing this metadata can open up possibilities for research and dissemination and how the Universal Viewer will provide access to audio collections online for the Library’s Save our Sounds programme.
12:00 – 12:30 What Good is an Ontology Anyway? by Raymond Drewry
Data sources that exist on their own are much less powerful than ones that connect to other data sources. One of the problems with these connections is that very few sources in any industry share a common model, much less a common format. The original idea of Linked (Open) Data proposed some technical solutions but failed to deal with the reality of current and legacy practices. Linked identifiers (as implemented by EIDR, other DOI systems, and Wikidata) go a long way to solving the connectivity problems but do not address models and formats. An ontology – basically a general and precise model that covers a particular domain (e.g. audiovisual works, rights statements, or legal opinions) – provides the infrastructure for reconciling (or at least being aware of) differences in format and meaning in the data. MovieLabs, working with several studios and experts from other communities (ontologists, librarians, archivists, and researchers), has defined an ontology for audiovisual works and done a prototype implementation, aggregating data from multiple sources and making it available in multiple formats. The ontology makes it fundamentally easier to deal with big data from multiple sources and data that has to be acquired from multiple sources to get good coverage. Current and proposed applications include: • Analysis of genre assignments for films across multiple genre systems, giving insights into how explicit genres can be used to find implicit genres • Commercial analytics based on metadata that is not often used because it is hard to find or not always present, exists in separate databases, etc. • Machine learning based on trailers, synopses, and other ancillary material • Gender analysis of contributors based on genres, release years, and countries of production • Finding groups of works that ‘go together’ in non-obvious ways, to support both research and commercial activities • Connecting archives and other institutions, building on the linked IDs in EIDR, including links to information about related non-audiovisual items such as source material (books, characters, theme park rides), locations (landmarks, real and fictional settings, etc.), and ephemera (scripts, reviews, artwork, and so on.) • Mapping of terms and vocabulary across multiple systems, simplifying communications and joint projects • Investigating a variety of localization problems (sub vs dub, local censorship ratings, local rankings and reviews, regionalized vs original titles, etc.) Most of these can be done in an ad hoc way using a few databases, but the ontology makes it much easier to go beyond individual sources and look at the data in different ways time after time, rather than creating one-off solutions for each instance of each task and constantly searching for and reintegrating the required information. We are publishing the ontology as an open source project, and are working on open source software that converts from common and useful data sources to the ontology, which will gave a base for both a ‘union catalogue of data’ and local projects that want to supplement their own data with data from public sources.
12:30 – 14:00 LUNCH BREAK
14:00 – 14:30 Evolution of data management for new uses of Ina’s collections: construction of a data lake by Eleanore Alquier and Gautier Poupeau
The French National Audiovisual Institute has been responsible since 1974 for the preservation of the audiovisual heritage produced by national broadcasts. The massive digitalization of these collections in the 1990s, the digital capture of 169 channels since 2001 in them frame of the legal deposit of television of radio, the opening of a “general public” (Ina.fr) website in 2006, the extension of legal deposit to web objects (sites, social networks, videos online, …) in 2009, are some of the steps taken by the Institute to take into account the digital technologies to benefit the audiovisual preservation. However, because of a historically “fragmented” IT architecture, the Institute has been suffering these last years from an under-utilization of its millions metadata collected in support of audiovisual collections, in particular in the frame of the legal deposit of radio and television. This is why since 2014 the Institute has been remodeling its documentary IT, in close coordination with the construction of a “data lake”, which will allow merging the metadata from all enterprise tools (documentary, legal, and commercial). The adoption of a data oriented strategy tends to develop new ways of structuring, modeling, storing and using metadata, guaranteeing their quality and their coherence, whether they come from external sources, from internal activities of documentary description or from tools based on artificial intelligence to generate automatically metadata. The presentation will describe this multiyear and multilayer project, which aims to deploy a new technical infrastructure and new innovative tools, and to harmonize metadata from various provenances with a new data model as well as to promote new uses of these contents: online editorialisation, new offers to Ina’s clients for footage or complete programs, exploration and analysis of Ina’s collections and metadata by researchers and students, data mining… The construction of the data lake will help to develop more relevant and accurate offers to Ina’s partners, without forgetting the legal framework in general, and more particularly intellectual property rights. While Ina’s collections go on growing ever faster (more than 1 million hours collected each year in the frame of radio and television legal deposit, more than 10 billion new URLs collected each year in the frame of web legal deposit), this project also raises the question of new technologies impact on expected skills for information professionals.
14:30 – 15:00 AMP: An Audiovisual Metadata Platform to Support Mass Description by Jon W. Dunn and Bertram Lyons
In recent years, concern over the longevity of physical AV formats due to media degradation and obsolescence, combined with decreasing cost of digital storage, have led libraries and archives to embark on projects to digitize recordings for purposes of long-term preservation and improved access. Beyond digitization, in order to facilitate discovery, AV materials must also be described, but many items and collections lack sufficient metadata. In 2014 Indiana University (IU) began an effort to digitize hundreds of thousands of hours of audiovisual materials from across campus through its Media Digitization and Preservation Initiative (MDPI). In 2015, with the support of consulting firm AVP, the IU Libraries conducted a planning project to research, analyze, and report on technologies, workflows, staffing, timeline and budgets to address the challenge of quickly and efficiently creating metadata for these materials. One of the outcomes of this planning project was identifying a need for a technology platform to support the incremental application of both automated and human-based processes to create and augment metadata. While there have been several open source and commercial efforts to date that demonstrate the possibilities for computationally assisted metadata generation and improved discovery, they have generally been narrow in focus and developed as standalone solutions. In truth, access to audiovisual objects at scale will require a variety of these analysis mechanisms, and these will need to be linked together with human tasks in a recursive and reflexive workflow engine that is compiling, refining, synthesizing, and delivering metadata to be used by any number of target systems. Recognizing that the need for such a platform extends well beyond any single institution, Indiana partnered with the University of Texas at Austin and AVP, with funding from the Andrew W. Mellon Foundation, to bring together a group of experts in the fall of 2017 to provide input into the technical design of a system to meet these needs, which we refer to as Audiovisual Metadata Platform, or AMP. This planning effort, outcomes of which are documented in a public white paper (https://go.iu.edu/ampreport), has led to a subsequent grant from the Mellon Foundation to support initial implementation and testing of the AMP platform beginning in 2019. In this new phase of AMP, known as AMP Pilot Development, or AMPPD, IU and AVP are working to develop an initial version of AMP as an open source system that will enable the creation and execution of workflows that link together both automated and human analysis activities to generate metadata for AV resources. The AMP system will then be pilot-tested against representative samples from collections both at Indiana University and at the New York Public Library to assess its feasibility for further development. In this session, we will describe the architecture of AMP, discuss the use cases and technical considerations that informed its design, and discuss the results so far from its implementation and testing.
15:00 – 15:30 Coexistence of (Asynchronous) Preservation Processes in Archive Asset Management by Silvester Stöger
In times of immediate availability of media through digital and networked dissemination paths, new and exciting approaches for archive use and completely file based media production, the vast repertoires of physical archive content are being inevitably unlocked in the time to come. Although this being one of the major quests of any archive owner, many archives still struggle with providing access to their legacy content and, equally important, with developing a long term digital preservation strategy. Even though challenging enough on its own, mass migration of physical collections to the digital domain is only the first big step; it takes more to build a state of the art media repository. Organizing and describing the sounds and images within the archive is a most vital prerequisite, let alone storage and essence management. The presenter will talk about the multiplicity of tasks that are necessary in preservation projects, pointing out that not all of them are synchronous processes, and how their coexistence can lead to efficient orchestration and labour separation. With a main focus on structured cataloguing processes, it will be described how to bridge the annotation gap between digitization and access.