The Tasmanian Partnership for Advanced Computing (TPAC) is part of the Australian network of advanced computing facilities and specialise in Earth Systems Science. TPAC hosts a vast amount of research and model data for use by partner organisations, including the Intergovernmental Panel on Climate Change. TPAC required a data discovery and access system to provide a central view of many millions of data files hosted amongst its data storage sites.
Insight4 developed a OPeNDAP search engine and catalog library provides an integrated GridSphere portlet to search and navigate OPeNDAP datasets. The portlet provides a single point of entry to remote OPeNDAP data sets, and displays them in a common format which is searchable and easy to navigate.
The search engine has been developed by Insight4 in partnership with TPAC to facilitate the sharing of OPeNDAP earth systems science data sets amongst TPAC's research partners.
The software is freely distributable under the GPL. For detailed technical information download our white-paper which was published at the APAC '05 Conference in Brisbane, Australia.
The software is packed full of useful features to make it easy to administer and manage large organisational OPeNDAP datasets. These features include:
- GridSphere portal framework
- Searching / Sorting of dataset files
- Compatible with OPeNDAP standard for file meta-data
- Consolidated display of geographically dispersed data
- Powerful search-engine to search research web sites
- Highly scalable, performance optimised database
- Web based administration of categories and datasets
The system is developed using a three tier model, comprising of:
- Front-end JSR-168 portlets
- Java Data API and MySQL Database
- Java Search Crawler
The search crawler periodically checks specified locations (local directories or external websites) for updated or modified data sets. It then updates the portal with the most current state of every data set, and allows new data sets to be discovered and automatically added to the system.
The portal was implemented within the GridSphere portal framework - an open-source framework developed in Java. GridSphere is a leading JSR-168 portal framework which combines features that make it ready-enabled for Grid integration.
