Repository technology
The SDR is developed and maintained by Stanford Libraries’ Digital Library Systems and Services. The system uses a combination of open source technologies, including:
- PostgreSQL (“Postgres”) - serving as a robust, scalable repository back-end for storing and managing digital object metadata. Postgres’s design lends itself well to establishing relationships between objects in the digital library.
- Solr - used to index information and offers "blazing fast" search of the repository contents. Solr's flexibility can accommodate a diversity of data sources.
- Blacklight - builds on the Solr index to provide faceted searching, browsing, and tailored views of objects.
In addition to these technologies, Stanford Libraries developed and implemented:
- Cocina, a digital object data model that works with Postgres as the successor to Fedora 3, the SDR’s original object store.
- Moab for versioning archived digital objects. Details about this novel approach are available in this 2013 article originally published in the Code4Lib Journal. Stanford is now actively involved in the specification of the Oxford Common File Layout, which is based on Moab, and plans to implement OCFL in the future.
Storage
In 2018, we retired our local LTO-6 tape library and put into operation the ability to replicate all SDR content to cloud storage services, with content stored in three geographic zones by two different service providers. The aim is many copies, geographically dispersed, with multiple vendors and no single point of failure.
In 2020 we completed a 3-year effort to design, plan, and execute a full migration of our preservation storage system from NetApp to Ceph, an open source, distributed storage system. In addition to the access copies readily available online, one preservation copy of SDR content is stored locally on spinning disk. We have twelve local storage servers running Ceph, yielding ~2.2 PB of usable storage.
For more information about the SDR's underlying technology, please contact the SDR team.
